Troubleshooting

Run Troubleshooting

View as Markdown

Use this page when a test run stops, times out, or fails before producing a clear product issue.

completed with FAILED

The agent finished the run and found a product, configuration, or test-step issue. Open the run details, review failed steps, screenshots, and reported issues.

completed with BLOCKED

The agent could not produce a product verdict because an environment or setup problem stopped the test, or because the agent hit its own automation limits. For example, the target environment was down, credentials were missing or rejected, the test depends on seed data that does not exist, or the agent ran out of actions or time on a single step. Blocked runs are not product failures: they are excluded from failure counts and conclude GitHub checks as neutral.

The run’s output.blockedReason carries a category (environment, seed_data, credentials, test_setup, or automation), a plain-language summary, and the step errorCodes that produced the verdict. For environment and setup categories, fix the problem it describes, then retry the run. For automation, the QA agent exhausted its per-step action or time budget without reaching a verdict — split the step into smaller steps, or retry.

Environment unavailable

ENVIRONMENT_UNAVAILABLE means the target environment was down or unreachable before the tested journey could start: persistent server errors (5xx), a maintenance page, DNS or connection failures, or an app that never loads.

Restore the target environment (deploy, restart, or fix the outage), then retry the run.

Seed data missing

SEED_DATA_MISSING means a step depends on pre-existing test data (records, fixtures, or account state such as existing orders, subscriptions, or saved items) that does not exist in the environment. It also covers a required third-party integration (payments, email, CRM, calendar) that is disconnected, unconfigured, or in an error state, because the data the step needs can never exist while the connection is broken.

Seed the environment with the data the test depends on, connect any integration it requires, then retry the run.

failed

The worker or runtime failed before normal completion. Common causes include provider capacity, browser/device startup failures, AI provider interruptions, or invalid run setup.

Provider session unavailable

Browser and device providers can temporarily run out of capacity. Retry the run after a short wait.

Run timeout

If a run times out:

  1. Retry once to rule out transient slowness.
  2. Split long tests into smaller flows.
  3. Make steps more specific so the agent has less ambiguity.
  4. Check whether the target environment is slow or unavailable.

Step tool limit exhausted

STEP_TOOL_LIMIT_EXHAUSTED means one test step required too many agent actions before it could finish. This usually happens when a step combines multiple workflows or assertions.

The failed step summary states the concrete blocker the agent hit (for example an element that never appeared or a control that did not respond).

Split broad steps into smaller focused steps with one intent each.

Instead of:

“Log in, create a project, invite a teammate, run a test, and verify the result.”

Use:

  1. Log in.
  2. Create a project.
  3. Invite a teammate.
  4. Run a test.
  5. Verify the result.

UI target not found

If a failed step says the agent could not find a UI target:

  1. Confirm the element exists in the target environment.
  2. Update the step with clearer user-visible labels.
  3. Check whether the page requires auth or setup before that step.

Wrong target URL

Tests use the run target URL when provided, otherwise the project URL. For PR and staging runs, confirm the webhook, deployment, or dashboard trigger points to the expected URL.

Agent step error codes

API responses can include steps[].errorCode on failed steps. These codes are machine-readable setup, configuration, environment, or runtime hints; the dashboard uses them to show a suggested fix and documentation link. When every failed step carries a blockable code (setup, configuration, environment, or an agent-owned runtime guard) and no product issue was reported, the run result is BLOCKED instead of FAILED.

CodeMeaningSuggested fix
AUTH_CREDENTIAL_UNAVAILABLEThe step needed login/OAuth/provider credentials that were not available to the run.Select or add the project credential required by the login step.
AUTH_CREDENTIAL_INVALIDThe target app rejected the provided credential, such as an incorrect username or password.Confirm the credential works, update the project credential, then retry.
AUTH_BASIC_REQUIREDThe target site is blocked by HTTP Basic Auth before the app loads.Add HTTP Basic Auth credentials in site protection settings.
VERCEL_BYPASS_REQUIREDVercel deployment protection is blocking the preview URL.Add a Protection Bypass for Automation token for the project.
MOBILE_RELEASE_BUILD_REQUIREDThe uploaded mobile app is a React Native / Expo dev build that needs a Metro dev server.Upload a release build with the JS bundle embedded.
VIEWPORT_RESIZE_UNSUPPORTEDThe step asked the agent to resize the browser, page, screen, window, or viewport mid-run.Configure the viewport in test settings.
ENVIRONMENT_UNAVAILABLEThe target environment was down or unreachable before the tested journey could start.Restore the target environment, then retry.
SEED_DATA_MISSINGA step depends on pre-existing test data that does not exist in the environment.Seed the environment with the required data, then retry.
STEP_TOOL_LIMIT_EXHAUSTEDOne step required too many agent actions before it could finish.Split the broad step into smaller focused steps.

If errorCode is absent, use the failed step error or summary as the source of truth.