Green check to merge
When every planned step passes, the check goes green and the pull request is ready to merge.
[02] On the pull request
What lands on
the pull request
TesterArmy posts a check and a comment on every pull request, updated on every push.
The verdict where you review
The comment holds the verdict, each planned step's result, screenshots and a recording.
Red check with reasons
A failed step turns the check red, and Details opens the run with its screenshots and recording.
One verdict per step
The comment lists every planned step with its own result, so reviewers see exactly where a flow broke.
Same run in Results
Every pull request run also lands in the Results table, filtered by the PR Preview environment.
Test your next pull request before it merges
Connect the GitHub App, open a pull request and read the verdict where you review.
[03] PR settings +
Settings that fit your setup
Pick the test groups and the exploration agent that run on each pull request, add a quiet period for busy branches and block merges on failed tests.
Block merges
on failure
Turn the setting on and a failed planned step fails the check, so the pull request cannot merge.
Delay automatic test runs
A quiet period means busy pull requests test only the newest commit, and older runs are cancelled.
Your preview deployment
Runs open the preview deployment for the pull request, whether Vercel, Netlify or Coolify built it.
A real Test Account
Each run signs in with a Test Account you define, so every flow starts from a real session.
Test groups per pull request
Pick the saved test groups and the exploration agent that run on each pull request.
[04] Proof
Novu merges about 30% faster,
by its CTO's own estimate
Novu runs TesterArmy against every pull request before merge, and its CTO estimates merges land about 30 percent faster.
TesterArmy runs against the live staging environment, not a local copy of the codebase, which gives a different angle of confidence.
Dima Grossman
Co-founder & CTO, Novu
TesterArmy is a core part of my software factory. Exploration tests for every PR is a killer feature.
Ethan Shea
Co-founder & CTO, Regbase
~30% faster
Time to merge
Cut in half
Flaky tests, across Novu's saved suite
Read how Novu moved from an outsourced QA service to tests on every pull request.
Novu case studyBoth figures are Dima's own estimates from the interview of 2026-08-26; Novu does not formally track them.
FAQ
Frequently asked questions
What is pull request testing?
TesterArmy runs saved regression tests, an exploration agent, or both against the pull request's preview deployment. Results appear in GitHub checks and comments with planned steps, screenshots and recordings.
Does it block merges?
Checks are advisory by default. Enable Block merges on failed tests in the PR Testing tab and require the TesterArmy checks in your GitHub branch protection or ruleset to block merges when tests fail.
Which pull requests trigger a run?
Automatic runs start when a supported preview deployment is ready for a connected repository. An optional target-branch filter limits which pull requests run. Busy branches can use a quiet period to test the newest deployment once.
Do we have to write tests first?
No. The exploration agent reads the pull request and generates a targeted test plan. You can also select saved test groups to protect critical flows, and run both kinds of tests on the same pull request.
Does it work with GitHub Enterprise or GitLab?
The native GitHub App supports GitHub.com. GitHub Enterprise Server and GitLab pipelines can trigger tests through the API or group webhooks. Without a supported GitHub connection, results remain in TesterArmy and exploration planning uses the PR metadata supplied by the caller.
What happens on a docs-only pull request?
A shared skip judge reviews the changed files and can skip pull requests that do not need application testing, including documentation-only changes. Both regression testing and the exploration agent use this decision.
How long does a run take?
It depends on the flows and the size of the change. The check shows the run's status and duration, and the result is posted to the pull request when it finishes. You can configure a quiet period for subsequent deployments on a busy branch.
How is this different from running Playwright in CI?
Playwright runs the test scripts you maintain. TesterArmy runs plain-language saved tests and can generate a test plan from each pull request. It manages the browser and reports the steps, screenshots and recording back to the pull request.
