Reading Mythos Preview's system card so you don't have to

From the reading group. The writeup is shorter than the card. You're welcome.

The headline is strange enough to repeat slowly: Anthropic built a model that scores 93.9% on SWE-bench Verified, measured it carefully, wrote it all down — and then decided not to make it generally available. The system card is the paper trail of that decision, and it's the most interesting document any lab has published this quarter.

We read it cover-to-cover in Tuesday's reading group. Three moves worth copying, whatever size your lab is:

1 · The welfare section. There's a section that takes the model's own experience seriously — carefully hedged, methodical, entirely unforced. Copying the humility is the point: they wrote about a thing they can't measure, said so plainly, and measured what they could anyway. Most teams do the opposite — measure nothing and hedge nothing.

2 · The refusal framing. Refusals are specified as behavior with test cases, not vibes — what the model declines, at what capability threshold, with worked examples of borderline calls. It reads like an API contract. We now write refusal specs for client agents the same way: enumerated, versioned, adversarially tested. "The agent shouldn't do bad things" is not a spec.

3 · The eval appendix. Task selection, harness configuration, sampling parameters, the failure taxonomy — written so someone else could re-run it. That's the tell of an honest eval: it's reproducible by people who don't like the result. Most vendor evals we audit fail this test on the first page.

The meta-lesson: the card documents a capability the company chose to hold, which means the document — not the launch — carries the credibility. For anyone shipping agents into production, that's the posture to steal. Write the system card first. If it's embarrassing to write, the system isn't ready; if it's easy to write, you probably haven't measured enough.