AI Agents

Devin Now Shows Its Work: What Self-Testing Agents Change for Review

Cognition uses GPT-6 Astra so Devin tests its own changes and returns evidence, not just a diff.

Devin Now Shows Its Work: What Self-Testing Agents Change for Review — article cover

The bottleneck in agentic coding is rarely writing the code. It is reviewing it. As engineering teams produce more changes than they can read line by line, the question shifts from “did the agent write this?” to “what proof do we have that it works?”

On September 11, 2026, OpenAI published a customer story on how Cognition is using GPT-6 Astra across its products, including Devin, its CLI, and its desktop tools. The interesting part is not raw code generation. It is that the agent is being pointed at its own output and asked to demonstrate that the output functions.

Testing as part of the agent’s job

Cognition co-founder Walden Yan describes the improvement in terms of proof: Astra’s ability to test and show that its work behaves as expected. In the published example, Devin uses Astra to test Otter Run, an iPhone game, and returns a recording of the game running in a simulator plus a report listing which checks passed and which areas were left untested.

That pairing matters. A recording shows behavior; a report shows scope. Together they give a reviewer two different kinds of evidence — what happened, and what was never examined. The second is usually the harder thing to get from an agent, and the more useful one when deciding whether to trust a change.

Bug reports come back as screenshots

Cognition also describes a faster loop for customer-reported issues. When a customer sends a screenshot of a bug, the team can hand it to Devin with Astra, which fixes the issue and returns a screenshot showing the result. Yan says this helps the team get back to customers much quicker.

The pattern is consistent: the agent’s deliverable is not a patch alone, but a patch plus an artifact a human can inspect. If you are building agent workflows, that is a design decision you can copy without any new model — require the agent to produce the evidence alongside the change, and make the evidence part of the review surface rather than an afterthought.

What this does to the review queue

Yan frames the goal as reducing manual code review: over time, engineers should have to look at less code manually and ship more. That is a claim about direction, not a measured result, and the supplied material does not include numbers on review time, defect rates, or how often the agent’s self-tests are wrong.

That gap is worth naming. An agent that tests its own work is still grading its own homework unless a human or an independent check validates the test itself. The recording and the report make that validation cheaper, but they do not remove it. Teams adopting this pattern should decide up front which artifacts count as sufficient evidence and which changes still require a human to reproduce the result.

Where this fits in a broader trend

The shift from “agent writes code” to “agent demonstrates code” is the same concern that shows up in review tooling more generally: giving a reviewer a trustworthy loop of diff, execution, and observed output rather than a wall of text. That is the thread running through Copilot’s diff, terminal, and browser panels, and it applies just as much when the author is an autonomous agent.

For builders, the practical takeaway is narrow and testable. Pick one workflow where an agent currently hands you a diff and nothing else. Add a requirement that it also produce a runnable demonstration and an explicit list of what it did not verify. Then watch whether that changes how long review actually takes — and whether the untested list turns out to be honest.

Sources

AI-assisted summary compiled from the sources above, reviewed by a human before publishing.

FOUND_THIS_USEFUL?

Support more practical AI articles, tutorials, and build notes.

BUY_ME_A_COFFEE
SHAREXEMAIL