The 2026 AI Index · the chart everyone is quoting, and the one they're skipping

Stanford's 2026 AI Index landed this week, and one chart is doing all the rounds: SWE-bench Verified went from roughly 60% to saturation — effectively ~100% — inside a year. It's a good chart. It's also the least informative one in the report, because a benchmark hitting its ceiling tells you about the benchmark, not the models. When everyone passes the exam, the exam is over.

The chart we keep coming back to is quieter: the Foundation Model Transparency Index dropped 18 points, from 58 to 40. While capability lines go up and to the right, disclosure about training data, methods, and evaluation details is going down and to the right. Both lines are drawn by the same labs.

SWE-bench Verified      60% → ~100%   ▲ the chart in every deck
FM Transparency Index    58 →   40    ▼ the chart in ours

Why we care, concretely: our job is building evals people can trust. Falling transparency makes that harder in specific, annoying ways. You can't check whether your eval set leaked into training if nobody says what's in training. You can't interpret a vendor's reported score if the harness isn't published. And "trust us, it's better" is exactly the sentence an eval exists to replace.

So the practical takeaways we're acting on:

Treat every model as a black box, formally. Our harnesses assume nothing about what the model saw in training. Private task sets, rotated regularly, scored on process as well as outcome.

Own your baselines. If your only numbers are the lab's numbers, you have marketing, not measurement. We re-run everything on client stacks — the deltas from published scores are routinely double digits.

Saturated benchmark ≠ solved problem. SWE-bench saturating didn't end software engineering; it ended SWE-bench's usefulness as a discriminator. The 100-step ops workflows we test still fail plenty. That gap — between the exam and the job — is where we live.