Using Claude Code and Codex to Create and Run E2E Tests

A coding agent can implement a feature, write the unit tests, open the pull request, and move on in the time it used to take to read the ticket. What has not sped up in most teams is the part where somebody opens a browser and checks that the feature works for a user. That gap is why we built the TesterArmy CLI and MCP server the way we did: so that Claude Code, Codex, or Cursor can hand the browser check to a specialised system and read the verdict back in the session where it wrote the code. This is a walkthrough using the exact commands from our docs. We build TesterArmy, so weigh the enthusiasm accordingly and check the boundaries at the end.

Two ways in: the CLI and the hosted MCP server

The CLI is published on npm as testerarmy, with ta as an alias, and the docs describe it as an agent-first control plane for projects, environments, credentials, saved tests, and runs that execute in TesterArmy cloud. Setup for an agent is four commands, from the agentic usage guide:

ta auth asks for an API key, generated in the dashboard under Profile then API Keys. For non-interactive sessions and CI the docs use an environment variable:

If your agent supports skills, the docs recommend the official one, with structured instructions for running tests, reading results, and iterating on failures:

The second way in is the hosted MCP server at https://tester.army/mcp. It follows the authenticated remote MCP specification over Streamable HTTP, and you sign in with your TesterArmy account through OAuth 2.1 with dynamic client registration, so there is no API key to create or rotate. For Claude Code the docs give one command, then /mcp inside the session to finish sign-in:

For Codex:

followed by codex mcp login testerarmy. Cursor and VS Code take a JSON entry pointing at the same URL, and stdio-only clients can bridge through mcp-remote. The CLI's former ta mcp local server is no longer available, so replace any stdio entry with the hosted configuration. A terminal or CI workflow suits the CLI and its JSON output; an editor-native agent suits MCP; both expose the same capabilities.

Let the agent discover the workflow itself

The CLI is self-discoverable. Rather than scraping long help output, an agent starts with ta agent init and ta docs, where ta docs prints task-oriented command routing and --json returns structured output. The prompt our docs suggest is short:

The error.hint instruction matters. Errors come back as JSON envelopes, and the agent decides its next action from the hint instead of parsing prose, the difference between an agent that recovers and one that guesses. The case for stable commands, predictable JSON, and non-interactive execution is in building the testing interface for agents.

Project, environment, memory, test, run

The docs lay out the usual path an agent takes once authenticated:

Environments are named, saved URLs, so a preview deployment or a tunnel to a dev server becomes a target the agent selects with --env. Memories are durable testing context, such as where login lives, saved once and used by every later run. Tests are a title plus plain-English steps, each typed as an action or an assertion, with no selectors and no code. ta tests run queues the run in TesterArmy cloud and --wait holds for the verdict.

A session: the password reset change

Claude Code is asked to fix a bug in the password reset flow, edits the reset page and the token handling, and pushes a branch. A preview deployment comes up at its own URL. The agent saves that URL as a project environment with the environments-create command above, preview URL in place of the staging one, and drafts a test for the flow it changed:

The inbox is an agent mail inbox saved on the project, the docs' recommended path for anything sent by email, password resets included: the agent reads the message, extracts the link, and continues. The agent pipes those steps through ta tests create in the JSON shape shown above, then queues the run with ta tests run <testId> --env staging --wait --json, substituting the environment name it created.

What comes back is a run with an execution status and a test outcome, and the docs are explicit that completed on its own is not a pass. PASSED means the flow worked. FAILED means the agent found a product, configuration, or test-step issue, and the coding agent's next move is to open the first failed step, its screenshot, and the issue summary, fix the code, and re-run. BLOCKED means an environment or setup problem stopped the test before a product verdict, and here the agent should leave product code alone. The run's output.blockedReason carries a category of environment, seed_data, credentials, test_setup, or automation, a plain-language summary, and the step errorCodes that produced it. In a preview workflow the codes you meet most are VERCEL_BYPASS_REQUIRED, when deployment protection sits in front of the preview and a Protection Bypass for Automation token is needed, and AUTH_CREDENTIAL_UNAVAILABLE, when the login step had no saved credential. Blocked runs are excluded from pass rates, so a misconfigured preview never reads as a broken product.

The same session over MCP

Over the MCP server the agent works in prose, and the docs list prompts that work well. To triage: "The last run of my checkout test failed. Read the transcript and the browser telemetry and tell me whether it's a bug in our app or a problem with the test." The agent goes from verdict to transcript to console and network logs, usually enough to name the failing request or the step where the UI diverged. Two details matter. Console and network telemetry is captured for web runs only and becomes available once the run finishes; mobile runs have the transcript instead. And when the agent fetches the run video, it returns a signed link that expires in 15 minutes; the video itself is never pulled into the conversation.

The server also ships built-in guides the agent fetches with its get_guide tool, covering reliable steps, test sizing, groups, and debugging, so a fresh agent follows TesterArmy conventions unprompted. A prompt such as "Create a test in the Web project that signs up a new user with a temporary email, verifies the welcome email arrives, and asserts the dashboard loads" turns a bug report into a saved test in one turn.

From one session to every pull request

The session above verifies one change once. The durable version is to connect the GitHub App so the exploration agent reads every pull request, writes a plan for that change, and posts the result as a check and a comment. Since coding agents increasingly write PR descriptions, the docs include an AGENTS.md excerpt that teaches your agent to end every description with executable testing instructions: entry route, numbered steps by visible label, expected behaviour, and the test account by label rather than credential. The planner reads the full description, so agent-written PRs arrive ready to test. What the exploration agent produces, and how it differs from a generic generated test case, is in can AI generate good test cases.

What is published, and the boundaries

The CLI is open source under MIT alongside two sibling tools, Scout for API testing driven by an OpenAPI spec and unbox-ai for reading agent traces; the open source page lists them. Scout and unbox-ai work without an account; the CLI needs a TesterArmy API key, because the runs happen in our cloud.

The boundaries. An agent connected this way can delete projects, tests, and groups; the destructive tools are marked as such, good clients confirm before running them, and the CLI docs ask the agent to check before deleting. Use scoped API keys and keep secrets out of shell history. Stored passwords are never returned by any tool, and credential-bearing steps are redacted from transcripts. Every run consumes a test run from your plan, which matters when an agent iterates on a failure; the free tier gives 5 runs to try the loop. If the app your agent is building is itself an AI product, the assertions need a particular shape, covered in how to test AI apps. And if you have been giving your coding agent a raw browser through Playwright MCP, the comparison with what is described here is in Playwright MCP vs an AI testing agent.

FAQ

Can Claude Code run end-to-end tests by itself? It can drive a browser with MCP tools and check a page once. A repeatable test with saved steps, run history, a video, and a PASSED, FAILED, or BLOCKED verdict needs a testing system behind it. The TesterArmy CLI and MCP server let Claude Code hand that work off and read the result back into its context.

Do I need an API key for the TesterArmy MCP server? No. The hosted MCP server authenticates with your TesterArmy account over OAuth, and the client opens a browser for sign-in on first use. The CLI uses an API key, which suits terminals and CI, and both expose the same functionality.

Does this work with Codex and Cursor as well as Claude Code? Yes. The CLI works as a skill for Claude Code, Codex, and OpenCode, and the MCP server is documented for Claude, Claude Code, Cursor, Codex, and VS Code, with an mcp-remote bridge for clients that only support stdio.