Local Development

Use the CLI to validate changes against your local dev server before pushing.

Set your target URL

Set the target once for the session:

export TESTERARMY_TARGET_URL="http://localhost:3000"

Or pass it per-run:

ta tests run <testId> --url http://localhost:3000

Run tests

Start your dev server, then run dashboard-defined tests locally:

# Run a single test
ta tests run <testId>

# Run all tests in a group
ta tests run --group <groupId> --project <id>

Headed mode

Watch the AI agent work in a visible browser:

ta tests run <testId> --headed

Useful for debugging flaky tests or understanding agent behavior.

Debug mode

Save detailed logs and transcripts to .testerarmy/:

ta tests run <testId> --debug

Check .testerarmy/<timestamp>/debug-run.json for the full stream/tool transcript.

Inspect artifacts

Every run writes to .testerarmy/<timestamp>/:

run-meta.json - run metadata
result.json - pass/fail results
debug-run.json - full transcript (with --debug)

List recent runs:

ta list

Typical workflow

# 1) Start dev server
pnpm dev

# 2) Set target
export TESTERARMY_TARGET_URL="http://localhost:3000"

# 3) Make changes

# 4) Run targeted test
ta tests run <testId>

# 5) Check results, iterate
ta list

CI usage

Set TESTERARMY_API_KEY and TESTERARMY_TARGET_URL as environment variables in your pipeline:

ta tests run --group <groupId> --project <id> --parallel 3 --json

Exit code 1 on any test failure - your pipeline fails automatically.

- name: Run QA tests
  run: npx testerarmy tests run --group <groupId> --project <id> --parallel 3 --json
  env:
    TESTERARMY_API_KEY: ${{ secrets.TESTERARMY_API_KEY }}
    TESTERARMY_TARGET_URL: ${{ steps.deploy.outputs.url }}

See Dashboard Tests for the full guide on running dashboard-defined tests.