CLI and Coding Agents

Local Development

View as Markdown

Use the CLI to validate changes against your local dev server before pushing.

ta tests run queues remote dashboard runs by default. Local execution is the exception for dev-server loops: pass --local to run dashboard tests in a local browser on your machine.

Set your target URL

Set the target once for the session:

$export TESTERARMY_TARGET_URL="http://localhost:3000"

Or pass it per-run:

$ta tests run <testId> --local --url http://localhost:3000

Run tests

Start your dev server, then run dashboard-defined tests locally:

$# Run a single test
$ta tests run <testId> --local
$
$# Run all tests in a group
$ta tests run --group <groupId> --project <id> --local

Add project memory

Use memories to give the agent durable project context before it runs tests:

$echo '{"category":"site_structure","title":"Auth route","content":"Login is at /login","importance":"high"}' | ta memories create --project <projectId>
$ta memories list --project <projectId>
$ta memories delete <memoryId> --project <projectId>

Headed mode

Watch the AI agent work in a visible browser:

$ta tests run <testId> --local --headed

Useful for debugging flaky tests or understanding agent behavior.

Debug mode

Save detailed logs and transcripts to .testerarmy/:

$ta tests run <testId> --local --debug

Check .testerarmy/<timestamp>/debug-run.json for the full stream/tool transcript.

Inspect artifacts

Every run writes to .testerarmy/<timestamp>/:

  • run-meta.json - run metadata
  • result.json - pass/fail results
  • debug-run.json - full transcript (with --debug)

List recent runs:

$ta runs list --project <projectId>

Typical workflow

$# 1) Start dev server
$pnpm dev
$
$# 2) Set target
$export TESTERARMY_TARGET_URL="http://localhost:3000"
$
$# 3) Make changes
$
$# 4) Run targeted test
$ta tests run <testId> --local
$
$# 5) Check results, iterate
$ta runs list --project <projectId>

CI usage

Prefer remote runs in CI so results land in the dashboard — see ta ci or ta tests run --group <groupId> --project <id> --wait --json.

To execute on the CI machine’s own browsers instead, set TESTERARMY_API_KEY and TESTERARMY_TARGET_URL and pass --local:

$ta tests run --group <groupId> --project <id> --local --parallel 3 --json

Exit code 1 on any test failure - your pipeline fails automatically.

1- name: Run QA tests
2  run: npx testerarmy tests run --group <groupId> --project <id> --local --parallel 3 --json
3  env:
4    TESTERARMY_API_KEY: ${{ secrets.TESTERARMY_API_KEY }}
5    TESTERARMY_TARGET_URL: ${{ steps.deploy.outputs.url }}