Harness engineering is a discipline now

Reading-group notes on the externalization survey: memory, skills, protocols, and the thing holding your agent together.

This week the reading group took on the 60-page unified review of externalization in LLM agents — memory, skills, protocols, and harness engineering, all treated as one research program. Three years ago "harness" meant the cable bundle behind your desk. In this survey it appears roughly two hundred times, capitalized with intent.

The core claim, compressed: an increasing share of agent capability doesn't live in the weights anymore. It lives in the stuff around them — the memory files an agent keeps, the skill library it pulls from, the protocols it speaks, and the harness that decides what it sees, when it acts, and how it fails. The checkpoint is the engine. The harness is the rest of the car, and most crashes are car problems.

What we underlined

Skills are externalized competence. A good skill.md is a senior engineer's playbook that survives the author leaving the room. The survey's evidence from large skill corpora matches what we see in our own library: skills fail to get reused for boring reasons — unclear preconditions, missing failure modes — not clever ones.

Most "reasoning failures" are harness mismatches. The paper's taxonomy of long-horizon failures reads like our incident channel: context starved at the wrong moment, tool output truncated mid-thought, memory that evaporates between steps. The model was fine. The car wasn't.

Verifiability decides where autonomy is safe. Give the agent more rope exactly where outcomes are checkable — tests, diffs, typed outputs — and keep the human gate everywhere else. This is the cleanest one-sentence policy for agent autonomy we've read all year.

We ran the numbers, obviously

You may remember our 50-step cliff post: four frameworks, one 100-step ops workflow, everything nose-diving between steps 40 and 60. This weekend we re-ran the whole suite — same model, same tasks, harness rebuilt along the survey's checklist: externalized memory with explicit refresh points, skills with stated preconditions, deterministic gates at the unverifiable steps.

baseline harness   → reliable through step ~45
rebuilt harness    → reliable through step ~82
weights changed    → 0

Not a controlled study — one workflow, one weekend, n=us. But a 37-step improvement for zero training runs is the kind of result that reorganizes your roadmap. Traces in the repo; kick the tires and file an issue if yours disagree.

The punchline

The bitter lesson taught everyone not to bet against the model. The 2026 corollary: don't bet against the harness either. The weights will keep getting better without your permission. The harness only gets better if someone treats it as a discipline — with reviews, evals, and a changelog.

The rebuilt harness checklist lives in our repo as harness.md. It's short. That's the point.