Errors and timeouts
How to end a run as failed with a useful reason, and what AMC does when your endpoint is slow, down, or wrong.
Fail a run
When your agent can't finish, report run status: "failed" with a failureReason, mark the step that broke failed with its own failureReason, and keep the tokens it spent.
{
"orchestratorRunId": "run_5f2c0e8a41b94c55",
"status": "failed",
"currentStep": 2,
"totalSteps": 3,
"failureReason": "The invoice service returned 404 for INV-1042.",
"steps": [
{ "stepId": "fetch", "name": "Fetch invoice", "status": "completed", "tokenCount": 0 },
{ "stepId": "summarize", "name": "Summarize", "status": "failed", "tokenCount": 900,
"inputTokens": 820, "outputTokens": 80, "model": "<model-id>",
"failureReason": "LookupError: invoice INV-1042 not found" },
{ "stepId": "approve", "name": "Approve summary", "status": "pending", "tokenCount": 0 }
]
}- AMC stores the run-level
failureReasonwhen you send one. Without it, the first failed step'sfailureReasonis used. - Write the reason for the person reading it: what failed and, if you know, what to do. Bound its length, and keep confidential task data out of it.
- If the installation has failure-cause suggestions turned on, AMC classifies the reason and shows a suggested cause next to it.
- Report
failedfor your own aborts, timeouts and limits.cancelledmeans a person cancelled the run. - When the run produced a usable result but part of the work wasn't done, report
completedwithoutcome: "partial"instead. See Completed with issues.
When AMC can't reach you
| Call | On failure |
|---|---|
GET /health at registration | One attempt, 5 second timeout. The registration is refused as Unreachable or Access not granted. |
POST /runs | One retry (on a transport fault or a non-2xx status), then the caller gets 502 and no run is created. If your first attempt started work, stop it: AMC never learned its run id. |
Status poll (/status, /tokens) | The run keeps its last recorded state and is retried after 30 s, 2 min, 8 min, 30 min, then every 2 h. After 5 failures in a row AMC records an Orchestrator unreachable audit entry and sends a notification. A successful poll resets the count. |
POST /runs/{id}/messages | The reply isn't recorded and the intervention stays open, so the person can retry. |
POST /runs/{id}/cancel | The cancel fails with 502 and the run carries on. |
The poll cadence, retry schedule and alert threshold are AMC defaults an operator can tune.
Timeouts on your side
- Answer every call quickly.
POST /runsmust return once the run is accepted, not when it finishes. - AMC has no deadline for a custom-adapter run that keeps reporting
running. Enforce your own run time limit, and when it trips, end the run asfailedwith a reason that says so. - A waiting run can wait for days. Decide whether a question should expire, and fail the run with a reason if it does.
Restarts
AMC keeps polling a run until it ends. If your service restarts and forgets a run, every poll and every cancel gets 404, the run stays live in AMC, and the unreachable notification above follows. If runs must survive a restart or a redeploy, store run state outside the process. At the least, persist the run ids you issued, and answer /status for a run you lost with failed and a reason such as "The agent restarted and lost this run."
Status traps
- An unknown run status is treated as waiting and shows as Intervention. Stick to the documented values.
- Once a run is recorded as finished, AMC ignores later reports for it. Don't report
completedbefore your last step's tokens are in. - A malformed JSON body on a poll counts as a poll failure, with the same retry schedule as an outage.
Related
Last updated 2026-10-21