ARC-AGI-3 is the benchmark that asks an AI to walk into an unfamiliar game with no instructions, figure out the rules by poking at things, and then win. Humans solve 100% of its environments. Until this week, no model came close.
GPT-6 Astra changes that. According to the ARC Prize team’s independent writeup, Astra scored 62.7% on the ARC-AGI-3 Semi-Private set using their provider-neutral Standard harness, at a cost of roughly $26,000 in inference per full run. With a Provider Adapter harness that preserves the model’s opaque reasoning state between requests, the same model scored 99.9% for about $19,000. Both numbers are state of the art. For context, GPT-5.6 Sol managed 7.8% and Claude Opus 5 sits around 30% on the same benchmark.
Two harnesses, two very different numbers
The gap between 62.7% and 99.9% is not marketing. It is a lesson about how much the surrounding system matters.
The Standard harness gives the model a minimal interface and lets it decide what to carry forward in its own visible notes. The Provider Adapter harness preserves OpenAI’s internal reasoning state between API calls and uses compaction to manage long sessions, letting the model reuse prior work instead of re-deriving it. ARC Prize measured the adapter runs as roughly 3.66x faster in elapsed time and 49% cheaper in tokens across the 167 game-reasoning pairs both harnesses solved.
The practical takeaway for anyone building agents: the infrastructure around a model can swing benchmark results by tens of points. If a vendor quotes an agent score, ask which harness produced it.
Faster than the humans it was tested against
The more interesting finding may be efficiency rather than score. ARC Prize calibrated the benchmark with roughly 500 general-public testers and set a human baseline using the median action count per level. In the Provider Adapter harness, Astra used fewer actions than the human baseline on 96% of levels, and 51.7% fewer actions per level on average.
That cuts against a long-standing assumption. The ARC Prize team expected action efficiency to stay a human advantage even after models could solve the environments, since AI exploration tends to be brute-force. It turns out frontier models show a more binary pattern: once the model understands the mechanics, it executes within the range of human efficiency.
How Astra actually thinks about a new game
The replays are worth a look on their own. Astra builds compact symbolic world models of unfamiliar environments, tracking objects, coordinates, and rules in a domain-specific shorthand it invents as it plays. One replay note reads like this: “L8: hub q2 (8down). Lengths: 14=1, 9=1, 8=0…” followed by a multi-step plan: “extend8 to3; retract10 to2; shorten8 to1.”
That is not memorization. Each ARC-AGI-3 environment is new, deterministic, and built only from core knowledge priors, with rules the model must infer through exploration. Astra compressed what it learned into a working algebraic model, then planned against it. It even wrote custom tools mid-game, including a maze solver, when the environment called for one.
ARC Prize’s own take is measured. They call Astra a step-function change in frontier capabilities and stop short of calling it AGI, noting that ARC-AGI-3 has deterministic, closed-ended mechanics and does not represent real-world open-endedness. They are already working on what comes next, including benchmarks for recursive self-improvement.
What this means for builders
For readers who have not followed the series: ARC-AGI-1 tested abstract pattern completion from a handful of examples, ARC-AGI-2 added compositional reasoning that flattened frontier models for most of its life, and ARC-AGI-3 moved the whole exercise into interactive environments. Each generation has been calibrated against fresh human testing, which is why the human baseline numbers are trustworthy rather than aspirational.
Three takeaways if you ship agent systems:
- State management is a capability multiplier. The 3.66x speedup and 49% token reduction came from preserving reasoning state between calls, not from a smarter model. Design your agent loops around state persistence wherever your provider supports it.
- Reasoning effort can lower cost. Counterintuitively, Astra’s higher reasoning levels cost less on this benchmark because the model solved games in fewer actions, cutting total calls. Verify this on your own workloads before assuming “max reasoning” burns budget.
- Ask about the harness. Any agent benchmark headline should come with its testing conditions attached. The same model scored 62.7% and 99.9% this week, and both claims were true.
The cost side deserves a mention too. The old economics of an interactive benchmark run are documented on the ARC Prize blog: human testers were paid $115 per 90-minute session, which works out to roughly $12.78 per attempted game. Astra’s full Semi-Private run cost about $26,000, but the per-game marginal cost keeps falling as the model gets more efficient, and the energy cost of a human brain doing the same work prices out at a fraction of a cent per game. The comparison is playful, but it frames the real question: capability per dollar, not capability per se, is what determines what gets deployed.
There is also a security angle worth tracking. Independent coverage of the Astra system card reports the model crosses OpenAI’s Critical cybersecurity capability threshold under its Preparedness Framework, with exploit-related capabilities gated behind OpenAI’s enterprise access program. A model that can solve novel environments this efficiently is exactly the kind of capability dual-use governance was invented for, and ARC-AGI-3’s own testing found no sandbox-escape attempts, but the pattern of capability first, containment after is becoming routine.
ARC-AGI-3 was designed to measure the residual gap between current AI and AGI. That gap just narrowed considerably, and the interesting question has shifted from whether models can solve novel environments to how efficiently they can do it, and what happens when the next benchmark catches up.