Is the Testing Pyramid Still Relevant in the Age of AI Agents?

The test pyramid is the most durable diagram in software testing. Mike Cohn put it in print in Succeeding with Agile in 2009, having sketched it years earlier with Lisa Crispin, and Martin Fowler's bliki entry plus Ham Vocke's practical guide made it the default mental model for a generation of teams. We build an agent that lives at the top of that pyramid, so we have an interest in whether the shape still holds; the honest way to answer is to check each of the pyramid's original arguments against what an agent changes.

What the pyramid was an answer to

Fowler's account of the test pyramid is worth reading in his own words, because the diagram is usually quoted without its reasons. For much of his career, he writes, test automation meant tools that drove an application through its user interface, often by recording an interaction and playing it back. That turned into what he calls an ice-cream cone: a portfolio dominated by UI tests that were "brittle, expensive to write, and time consuming to run." An enhancement could break dozens of recorded tests, the tests slowed the build, and they were prone to non-determinism, which undermined trust in them.

Cohn's original picture, as Vocke lays it out in The Practical Test Pyramid, has three layers from bottom to top: unit tests, service tests, and user interface tests, with fewer tests the higher you go.

The three arguments hidden inside the shape

The pyramid resolves into three separate claims that usually travel together.

The first is a maintenance argument. UI tests encode the current structure of the interface, so every interface change touches the encoding. Vocke states it plainly: end-to-end tests require a lot of maintenance and run pretty slowly, so reduce them to a bare minimum, ideally the one or two user journeys that define the core value of the product.

The second is a speed argument. Vocke expects thousands of unit tests to complete within a few minutes, while UI tests take far longer, which is why he orders pipeline stages by speed and scope rather than by test type, with the slow broad tests last.

The third is a diagnostic-precision argument, and it is the one that gets lost. Fowler describes high-level tests as a second line of defence: a failure there means a bug in the functional code and also a missing or incorrect unit test, so replicate the bug with a unit test before fixing it. Vocke's rule of thumb is the same: if a higher-level test spots an error and no lower-level test fails, write the lower-level test.

Fowler also leaves a footnote that matters here. The pyramid, he writes, "is based on the assumption that broad-stack tests are expensive, slow, and brittle compared to more focused tests, such as unit tests. While this is usually true, there are exceptions. If my high level tests are fast, reliable, and cheap to modify - then lower-level tests aren't needed." The shape was never a law. It was a response to the economics of one kind of top layer.

What changes when the top layer is an agent

Here is TesterArmy's top layer, concretely. A test is a list of plain-English steps written the way you would brief a teammate: "Log in with the saved shopper account", "Verify the cart shows the Pro plan with the correct monthly price". There are no selectors and no test code. On each run the agent looks at the rendered page, decides what to click from what a user would see, and returns a video, per-step screenshots, a step trace, and a verdict. Runs happen in cloud browsers and mobile simulators, in parallel, on every pull request.

The maintenance argument weakens substantially. Fowler's re-recording problem assumed the test stored a description of the interface. An agent stores a description of the intent and resolves it against whatever the interface looks like today. What remains is keeping the flow description in sync with what the product is supposed to do. We sized that residual in the hidden tax of test maintenance, and explained why "self-healing" is the wrong word for it in self-healing tests: what is real and what is marketing, since there is no script to heal.

The speed argument weakens only in part. The infrastructure half goes away: no local browser to install, no macOS runner to keep alive, and journeys run in parallel rather than serially. What stays is that a journey through a real browser takes minutes and a unit test takes seconds. Vocke's advice to order stages by speed applies as written; agent runs belong after the unit tests, on the preview deployment, answering the one question unit tests cannot: does the deployed branch work for a user.

The brittleness argument is the interesting one. Vocke notes that end-to-end failures are quite often false positives from timing, browser quirks, and popups. An agent that reads the page absorbs most of that noise, but a different class remains: the preview is down, credentials were rejected, seed data is missing. TesterArmy gives that class its own verdict. A completed run ends as PASSED, FAILED, or BLOCKED, where BLOCKED means an environment or setup problem, or an agent automation limit, stopped the test before a product verdict was possible; blocked runs carry an output.blockedReason and are excluded from failure counts and pass rates. The false-positive complaint does not vanish, but red stops meaning "go find out whether the app or the environment is at fault."

What does not change

The diagnostic-precision argument stands untouched. When an agent reports that checkout failed at the discount step, with a recording, you know a user-visible promise broke; you do not know which function broke it. Fowler's rule applies as it did in 2012: replicate the failure with a unit test, then fix it. His other warning holds too: a rich JavaScript UI should still have most of its behaviour covered by JavaScript unit tests.

And the ice-cream cone is still a failure mode. In 2012 it meant a portfolio of recorded UI tests with almost nothing underneath. In 2026 the equivalent is a team that turns on an agent, watches it catch bugs, and quietly stops writing unit tests because the journeys are green. That team has traded a precise, seconds-scale inner loop for a minutes-scale outer loop, and when a failure arrives nothing will point at a function. Against Fowler's footnote: an agent makes the top layer cheap to modify and reasonably reliable. It does not make it fast in the sense a unit test is fast, and it never makes it precise.

A revised shape, without a cute name

Unit tests for logic, as many as the logic requires. This layer is unchanged, and Vocke's advice to test observable behaviour rather than internal structure is the part most worth rereading.

Narrow integration tests at the boundaries, and fewer broad ones. Vocke's narrow tests, one integration point at a time where you serialise and deserialise data, stay. What can shrink is the layer of broad, half-end-to-end integration tests built because journey coverage was too expensive to own; once journeys run on every pull request, those mostly duplicate a higher layer, and Vocke's rule applies: delete the test that no longer adds confidence.

Journey coverage by an agent on every pull request. Vocke's e-commerce example, the journey that defines the product, looks like this as TesterArmy steps:

Log in with the saved shopper account.
Search for "bicycle" and open the first result.
Add the bicycle to the basket.
Verify the basket shows the bicycle with its price.
Complete checkout with the saved test card.
Verify the confirmation page shows an order number.

Saved as a regression group, that runs against every preview deployment through pull request testing, with its own GitHub check and PR comment. Alongside it, the exploration agent reads each PR and writes a plan for what changed; how it plans is in can AI generate good test cases, and what reviewers see is in how automated PR testing speeds up code review. Splitting flows into a short list that gates deploys and a longer list that runs on PRs is covered in smoke testing vs regression testing.

Exploratory testing by people, as Vocke prescribes. Whether the product is confusing, whether the refund path matters more than referrals this quarter, and what a hostile user would try are human questions, in-house or through an outsourced QA team.

We sell the third layer, so weigh that when reading this. The honest boundary is in autonomous QA: can an agent replace your test suite: an agent replaces the suite for journey-level coverage, and the layers beneath it are yours to keep.

FAQ

Who invented the test pyramid? Mike Cohn described it in his 2009 book Succeeding with Agile, where he called it the Test Automation Pyramid. According to Martin Fowler's etymology note, Cohn first drew it in conversation with Lisa Crispin in 2003-4 and presented it at a Scrum gathering in 2004, and Jason Huggins arrived at the same idea independently around 2006.

Does an AI testing agent replace unit tests? No. An agent tells you that a user journey broke and shows you where, with a recording; it does not tell you which function is wrong. Fowler's rule still applies: reproduce a high-level failure with a unit test before fixing it, so the lower layer keeps the bug from returning.

How many end-to-end tests should a team have? Under the old economics Vocke's answer was a bare minimum, the one or two journeys that define the product, because each one cost so much to maintain. With an agent, the useful count is the number of flows whose breakage you would fix urgently, since adding a flow costs a few sentences rather than a test file with an owner.