Here is a fact that should reorganize how you read agent benchmarks. In early July 2026, if you look up GAIA — the standard test for general AI assistants — you can find the top score listed as 44.8% or as 92%, on the same day, for the same benchmark. Neither is a typo. They're measuring different things, and almost nobody says which.
What GAIA is#
GAIA, introduced by Mialon and colleagues in 2023, is 466 questions that share a design goal: conceptually simple for a person, tedious or impossible for a model working alone. Answering one means chaining several reasoning steps, browsing the live web, using tools, and often reading a multimodal input — an image, a spreadsheet, a PDF. The questions come in three levels, graded by how many tool-use and reasoning hops they demand. Crucially, every answer is a single unambiguous string, so scoring is exact-match. There's no LLM judge to argue with; you got it or you didn't.
The number that made GAIA famous was the spread. Humans with a browser score about 92%. GPT-4 with plugins, in the original paper, scored about 15%. That six-to-one gap was the entire pitch: a benchmark that raw language ability couldn't paper over, because the hard part wasn't knowing things — it was doing a sequence of small, boring, correct steps in the world.
What changed isn't the model#
Fast-forward to now. Sort GAIA by base model — one model, minimal scaffolding — and the leader is around 44.8% (GPT-5 Mini), with the next model near 33%. That's real progress from 15%, and also nowhere near human. But sort the system leaderboards — the ones that rank a model plus its tools plus its orchestration policy — and the top entry reports about 92.36%. Human parity.
Same benchmark. The model in the winning system is not 47 points smarter than the model on the base-model board. Often it's a comparable model. The 47 points came from everything wrapped around it: the search loop, the tool selection, the verification pass, the retry logic, the answer-formatting step that makes exact-match forgiving. A controlled 2026 study of scaffold effects on GAIA makes the point directly — how you harness the model moves the score more than which model you pick.
GAIA started as a test of what a model can do. It ended as a test of what you built around it. The model became a component; the system became the subject.
That is the non-obvious thing, and it's easy to miss because the leaderboards all wear the same name. GAIA didn't get easier. It got re-pointed. In 2023 the bottleneck was the model, so GAIA measured the model. In 2026 the bottleneck is the scaffold — the agent architecture — so the same questions now measure the scaffold. The benchmark's difficulty didn't move; the thing it's sensitive to did.
Why this should change how you read the number#
If you're choosing a base model, read the base-model board and ignore the 92%. It was produced by an orchestration you're not buying. If you're deploying an agent, read the system board — but do not read the top number as a model capability. Read it as: this specific pipeline, on GAIA's specific task shape, hit this. The moment you ask "what harness produced 92.36%?" you're asking the right question, because the harness is what you'd have to reproduce.
And treat "GAIA is solved" with suspicion. Exact-match scoring rewards verification and answer-formatting — precisely the things a tuned system is engineered to nail. A 92% system score is strong evidence that a well-built agent loop handles multi-step web-and-tool tasks with unambiguous answers. It is weak evidence about anything with a fuzzy answer, a long horizon, or an adversary — which is most real work. GAIA measures a slice, cleanly. The slice just isn't the model anymore. (For where GAIA sits among the other agent tests, see our SWE-bench vs τ-bench vs GAIA breakdown.)
The general lesson#
GAIA is the clearest case, but the pattern is everywhere now. As base models converge, the variance in what an "AI system" can do migrates out of the weights and into the engineering around them — retrieval, tools, verification, control flow. Benchmarks built to isolate the model start, quietly, to measure the system instead, and they keep the old name while they do it. So when someone quotes you a benchmark score for an agent, the useful reflex isn't "how good is that model." It's "what, exactly, did they wrap around it — and is that the part I get to keep?"
