vs reading the raw design
Billed prompt tokens on identical tasks, at 10–100 screens. At 200 screens the raw design no longer fits at all.
THE CONTEXT BENCHMARK
We measured what shared context is worth.
Every AI tool claims to save your agent context. We built a benchmark, froze the tasks before running it, published the failures — including our own — and shipped the harness in the repo.
THE HEADLINE FINDING
At 200 screens, your agent cannot read the raw design at all.
A 200-screen product overflows a 1,000,000-token context window — the naive approach doesn't get expensive, it gets impossible. Below that size, it costs 18–52× more billed context than Airframe's shared design, task for task.
18–52×less context billed than reading the raw design, task for task.10–100 screens · B-173
THE NUMBERS
Three comparisons, no cherry-picking.
vs a well-played baseline
Against an agent using every targeted tool perfectly — no shared digest. Read-heavy work; about 1.5× when writes are mixed in.
live agents, by model
What real agents banked in live runs. Savings depend on whether the model actually opens the shared design; both models are published.
HOW WE KEPT OURSELVES HONEST
Built like a study, not a slogan.
Pre-registered tasks
11 tasks frozen before any measured run, with held-out probes added after the design was locked — including one the digest is expected to lose.
A steelman, not a strawman
The baseline agent plays every non-digest tool as well as it can be played. The number we publish is against that — not against the worst case.
Published failures
The measurement falsified our own folklore first: the old digest was worse than nothing on large projects. We redesigned it and re-ran everything.
RUN IT YOURSELF
The harness ships in the repo.
$ git clone https://github.com/airro/airframe
$ tsx benchmarks/digest-impact/bin/bench.ts --mode sessionDeterministic fixtures, seeded — the same numbers every run.
Read the full write-up: methodology, both accounting modes, and every unsolved task