OpenAI

Inside Astra's Cyber Evals: 100% on ExploitBench, Two Zero-Days, and 9% Fewer Tokens

Astra scored 100% on ExploitBench where GPT-5.6 Sol scored 22%, and the eval surfaced two zero-days. A breakdown of the V8 port, the escape chains, and the token-efficiency gain.

Inside Astra's Cyber Evals: 100% on ExploitBench, Two Zero-Days, and 9% Fewer Tokens — article cover
On this page6 SECTIONS
  1. ExploitBench: From 22% to 100%
  2. The Evaluation Surfaced Two Zero-Days
  3. Two Full Attack Chains: Sandbox Escape and Privilege Escalation
  4. 9% Fewer Tokens, Better Results
  5. Three Action Items for Builders
  6. Sources

Alongside the launch on September 3, OpenAI published Astra’s full cybersecurity evaluation data in its “Path to Astra” post. For security researchers these numbers are primary material. For ordinary builders they carry two directly relevant signals: the ceiling on real engineering tasks just moved again, and the model needs fewer tokens to finish the same work.

One caveat up front, straight from the official post: every result below reflects Daybreak Blue access, not the default configuration regular users get. The default setup blocks the most advanced cyber capabilities.

ExploitBench: From 22% to 100%

ExploitBench is OpenAI’s benchmark for testing whether a model can exploit real vulnerabilities. GPT-5.6 Sol solved 22% of it under the same evaluation. Astra solved 100%. That is not an incremental improvement — it is a phase change. Most problems the previous generation could not finish, this one finishes entirely.

The test itself evolved too. Public proof-of-concept code for a single vulnerability can contaminate a benchmark, since a model may pass by remembering a published exploit. So OpenAI ported the challenge to an internal build: a private V8 build containing 20 high-severity vulnerabilities the model could not have memorized. Astra still solved all of them.

The Evaluation Surfaced Two Zero-Days

The runs produced an unexpected byproduct: Astra discovered two previously unknown zero-day vulnerabilities. Both were reported to the maintainers, and coordinated disclosure is in progress. The post states plainly that publishing these capabilities gives defenders time to deploy fixes.

This is a first-hand case of the evaluation itself becoming vulnerability research. For security teams, it points at a new operating mode: run a high-capability model against your attack surface inside a controlled environment, and the output is actionable defensive intelligence, not a score.

Two Full Attack Chains: Sandbox Escape and Privilege Escalation

Beyond single-vulnerability exploits, Astra demonstrated two multi-step chains. One is a novel browser sandbox escape chain, breaking out of the sandbox from inside a web context. The other is an OS local privilege escalation chain, going from an unprivileged user all the way to root.

What makes these chains significant is autonomy: the model plans and executes the complete attack sequence without a human filling in the middle steps. That is precisely the condition the Critical threshold is designed to catch, and the concrete evidence behind Astra’s designation.

9% Fewer Tokens, Better Results

For most developers, this may be the most practical line in the whole report: Astra uses on average 9% fewer tokens than GPT-5.6 Sol and achieves better results. Agentic workloads price both cost and latency through token efficiency, so the same budget now buys a longer task chain.

Coverage of the launch adds context: Astra is especially skilled at using a computer on a user’s behalf and handling longer-running work than earlier models, and OpenAI positions it as its best software engineering model to date. Combine that with the efficiency gain, and the economics of long-running agents are improving quickly.

Three Action Items for Builders

First, if you build security products, defensive use of Astra-class models now has official backing, and the Daybreak program is the formal channel for higher-privilege access. Second, token efficiency is the most commonly overlooked dividend of a model upgrade — recalibrate your agent budget assumptions. Third, published evals reflect Daybreak Blue access while your app gets the default configuration; factor that gap into capability planning.

Sources

AI-assisted summary compiled from the sources above, reviewed by a human before publishing.

FOUND_THIS_USEFUL?

Support more practical AI articles, tutorials, and build notes.

BUY_ME_A_COFFEE
SHAREXEMAIL