The Last Generation of Checkers最后一代检查者

The 13.6% problem

Anthropic recently described a test it ran on its own coding agent. 1,053 paid users reviewed commands the agent wanted to execute, some of which were dangerous. The humans caught 13.6% of the dangerous ones.

This is a controlled study run by the company itself, so treat it as one data point. But it fits with something the company measured in ordinary use. Claude Code users approve 97% of the permission prompts they see. In March the figure was 93%. Anthropic has a name for it, approval fatigue, and in August it made automated review the default for new sessions on its Pro, Max and Team plans. The company says that automated mode catches 89%.

Now hold a second number next to the first.

In the US jobs most exposed to AI, employment of workers aged 22 to 25 is about 19% below where its trend says it should be. That figure comes from Stanford's Digital Economy Lab, working with ADP payroll records through June 2026. Experienced workers in the same jobs show no comparable gap. And almost none of the shortfall comes from layoffs. It comes from people who were never hired.

These look like two stories, one about software safety and one about the labor market. We think they are the same story, and that it is bigger than either.

The thesis. AI makes generating work almost free. That pushes the remaining value, and the remaining risk, onto checking: deciding whether an output is right, whether it is safe, and whether its consequences are acceptable. Checking is done by people with judgment. Judgment is mostly built through years of cheap, low-stakes, supervised practice, which is precisely the entry-level work AI absorbs first. We are raising the price of judgment and cutting off its supply at the same time. The bill arrives with a long delay, somewhere in the 2030s, when the people who should have become tomorrow's experts turn out never to have been trained.

What follows is the chain of evidence, the strongest objections we found, and what would change our mind.

1. What got cheap, and what didn't

Start with prices. Epoch AI estimates that since 2023 the cost of running a model at a fixed level of capability has fallen about 13-fold a year. While a capability is state of the art the decline is closer to 75-fold a year; two years later it slows to around 4.7-fold. Yesterday's intelligence becomes a commodity very fast.

Two things refused to follow that curve.

The first is the frontier. Top-tier list prices went up: the flagship input price rose from $1.25 per million tokens (GPT-5, August 2025) to $10 (GPT-6 Astra, September 2026). The newest capability is expensive because it is new. Tokens are cheap; the frontier is not.

The second is proof. Look at how slowly confidence moves wherever being wrong is expensive:

- To show with 95% confidence that a robotaxi causes fatalities no more often than a human driver, a RAND study calculated you need about 275 million miles without a fatal failure. To show it is 20% better than humans, more than 11 billion. Waymo had driven roughly 271 million miles by mid-2026: in miles alone, about 98.5% of the first bar and one-fortieth of the second.

- To show with 95% confidence that something works 99% of the time, you need about 299 trials without a single failure. For 99.9%, about 2,995. Each additional nine costs roughly ten times the evidence.

- No drug discovered from scratch by generative AI has been approved yet. The leading candidate's phase 3 trial enrolls 320 patients for 52 weeks. Biology keeps its own clock.

The most revealing result comes from medicine. In an Oxford randomized trial published in *Nature Medicine* in February 2026, 1,298 people worked through medical scenarios. The language models, answering on their own, identified the relevant conditions 94.9% of the time. People using the same models identified them less than 34.5% of the time, no better than a control group that had no AI at all. The models were 2024 versions, and newer ones are stronger. But the gap in that study had little to do with model strength.

The knowledge was in the machine. It did not survive the trip through a person who couldn't tell which parts of the answer mattered.

That is the shape of the economy we are moving into. Producing an answer is cheap. Knowing whether to trust it, and carrying the consequences when you act on it, is not. The second part is still done by people.

2. The checker is already tired

If checking is the bottleneck, the obvious question is how good people are at it. The early evidence is not encouraging.

Approval fatigue is the cleanest example. When a system asks you to confirm things all day, and nearly all of them are fine, you stop reading. Approving 97% of requests is a rational response to a stream that is mostly harmless. It also means the few requests that matter get handled by reflex. A check performed by reflex is not a check.

Software developers offer a second example. In a randomized trial METR published in July 2025, 16 experienced open-source developers worked on 246 real tasks. With AI tools they took 19% longer, yet they believed they had been about 20% faster. That is a forty-point gap between how the work felt and what was measured.

The follow-up is stranger. When METR ran a new round of the study, it hit a design problem: 30% to 50% of developers would not submit tasks they felt required AI. They no longer wanted to do the no-AI condition. The new numbers leaned toward a speed-up this time, and METR called them very weak evidence. But the design problem is the real finding. People had come to depend on the tool so completely that the controlled comparison could no longer be run cleanly. The dependency formed before the evidence did.

A third example shows what checking actually involves. FDE-Gym, a benchmark posted as a preprint in October 2026 and not yet peer reviewed, gave AI systems 95 delivery tasks adapted from real client projects. The best of 13 configurations solved 14. The authors named the main bottleneck: discovering unstated constraints. In a subset of 23 tasks where the systems were handed every key fact up front, only 10 reached a passing score.

Unstated constraints are the things nobody writes down because everyone already knows them. This client hates surprises on invoices. This database is shared with a team that runs reports at 3 a.m. This regulator reads the footnotes. The philosopher Michael Polanyi called this tacit knowledge, and summed it up in one line: "we can know more than we can tell."

Tacit knowledge is what a checker runs on. It is how a senior engineer looks at a command that seems correct and senses that something is off. It is also the kind of knowledge you cannot download. It gets absorbed slowly, by being close to the work.

3. Where judgment comes from

So how do people get it?

For most of modern economic history, through the bottom rungs. The junior analyst builds a thousand spreadsheets and learns which numbers lie. The new support agent answers the same ten questions until the day the eleventh one is different. The junior developer writes boilerplate, gets reviewed, breaks something in staging and slowly develops a feel for where danger lives.

That work has three properties. It is frequent. It is low-stakes. And it is describable: you can write down what a good result looks like. Those are the same three properties that make a task easy to hand to AI.

The prices are already being set in those terms. Intercom charges $0.99 per outcome for its AI support agent, roughly a conversation that ends without the customer needing more help. The median US customer-service representative earns $44,770 a year, according to the Bureau of Labor Statistics. Divide one by the other and the salary buys about 45,000 outcomes, around 180 per working day. Salesforce prices agent actions at about ten cents each. Once a task has a unit price, a manager can compare a person with an invoice.

This does not show up in layoff statistics, because it is not happening through layoffs. In August 2026 the US hiring rate was 3.3% and the layoff rate 1.0%: a low-hire, low-fire economy. The Bureau of Labor Statistics expects roughly 290,000 openings a year for customer-service representatives over the coming decade. Every one of them comes from someone leaving. None are new.

Our research notes put it in one line: you can't see it, because it is an offer that was never sent.

4. The same door, in another building

The same pattern appears somewhere few people think to look.

Google's search business is doing well. In the second quarter of 2026, Alphabet's search revenue was $63.3 billion, up 17% on the year. Meanwhile, between July 2025 and July 2026, the Google search traffic that news and media publishers receive fell 40.2%, according to Chartbeat's data. Google's share of publishers' pageviews dropped from 9% to 5%.

Look inside the publishers' numbers and you find the labor-market shape again. Pageviews from loyal readers were flat: 14.98 billion a month before, 15.02 billion after. New readers fell 14%. Returning readers fell 8%. Chartbeat's sample leans toward news sites, so treat the exact figures with some care. The direction is still clear.

Set this beside the Stanford result. Experienced workers: no gap. Young workers: 19% below trend. Loyal readers: flat. New readers: down 14%. In both cases the headline numbers look healthy. Unemployment is low; search revenue is growing. In both cases the damage is concentrated in the same place: the entrance. The people already inside are fine. The path in is narrowing.

This matters for the argument, because it suggests the mechanism is general. When an AI layer sits between people and the work, or between readers and writers, it absorbs the cheapest point of entry first. Incumbents keep their position. The door for newcomers closes so quietly that the aggregate figures don't register it.

5. The silent freeze

Now connect the pieces.

If you run a company in 2026, the logic is simple. You keep your experienced staff, because they hold the tacit knowledge and they are the ones checking the AI. You stop backfilling juniors, because the work juniors did is now cheaper as a service. Each decision is reasonable on its own. Stanford's data is consistent with what they add up to.

But the senior people you are relying on were made out of junior work. Expertise has a long lead time. In many professions it takes something like seven to ten years to go from beginner to the person who can be trusted to say "no, this is wrong." That range is our estimate, not a measured figure. A cohort that isn't hired between 2024 and 2027 is a cohort of experienced checkers that won't exist in the mid-2030s.

Demographers know this pattern. A missing birth cohort echoes for decades: first empty classrooms, then a thin workforce, then pressure on pensions. A missing hiring cohort echoes the same way, through skill instead of headcount. We call it the silent freeze. Nobody is fired. The ladder loses its bottom rungs, and the people higher up don't notice, because they stopped using those rungs years ago.

A second-order effect makes this worse. When much of the work runs through a few shared models, errors stop being independent. Human drivers make their own mistakes; software makes the same mistake everywhere at once. In 2026 Waymo recalled nearly its entire fleet twice, 3,791 vehicles in one recall and 3,871 in another. When failures are correlated, what you need most are checkers who think differently from the system, people whose judgment was formed somewhere other than inside the same tool. A workforce that learned its trade by approving AI output is the opposite of that.

So the system we are building has three moving parts:

1. Generation gets cheaper every year.

2. Independent, experienced checking gets more valuable, because errors scale and correlate.

3. The supply of experienced checkers shrinks, with a lag of most of a decade, because the training rungs were automated first.

That is the judgment gap. It won't show up in this quarter's numbers. It shows up when today's checkers retire.

6. The strongest case against us

Several facts push back on this argument, and they deserve full weight.

AI can be a ladder. The best-known study of AI in customer support, by Erik Brynjolfsson, Danielle Li and Lindsey Raymond, followed 5,179 agents. Productivity rose 14% on average and 34% for the newest and least-skilled workers, with little effect on experienced staff. The tool helped newcomers absorb what the best performers knew. Used as a tutor, AI shortened the ladder instead of removing it. The published version in the *Quarterly Journal of Economics* reports 5,172 agents and a 15% average gain. This is the most direct rebuttal to "AI took the ladder," and it points to the real variable: whether companies use AI to replace juniors or to train them faster.

The hiring slump has other causes. Interest rates are high. Some declines in job postings began before ChatGPT existed. Stanford's authors say plainly that their findings are descriptive patterns, not causal estimates of AI's effect, and the gap narrows when you control for education. A low-hire economy would squeeze young workers with or without AI.

Machines may check machines. By Anthropic's account, automated review catches far more dangerous commands than people did in its own study. If machine verification keeps improving, the need for human checkers may fall, and the gap may matter less than we think.

Developers may now be faster. METR's newer data, weak as it is, leaned toward a speed-up. Tools change quickly, and a 2025 trial is not a 2026 verdict.

These points change our conclusion from "this will happen" to "this is the default path unless something interrupts it." Here is what would change our mind:

- Stanford's next updates show the gap for 22-to-25-year-olds closing rather than widening.

- Junior hiring recovers in AI-exposed occupations even while rates stay high.

- Companies publish catch rates for their AI review processes, not approval rates, and those catch rates hold up under outside audit.

- Verification itself becomes cheap in a field: if formal methods and automated testing can establish correctness there, the need for tacit human judgment in that field really does fall.

7. Where the saved time goes

One more piece completes the picture. It explains why the people who still have jobs don't feel any lighter.

Medicine is the clearest case. AI scribes listen to a consultation and draft the notes. In a multi-site study, physician burnout fell from 51.9% to 38.8% within a month. It was a before-and-after comparison rather than a randomized trial, but the gain looks real. The healthcare press already has a name for what tends to follow: the productivity ratchet. Once a doctor can see patients faster, the expected number of patients goes up.

Hospitals are not special. Task-level speed-ups are real; role-level gains are often absorbed by more work. Who keeps the hour that AI saves depends on who sets the quota.

Capital markets have their own version. When Block cut about 40% of its staff in February 2026, its stock rose nearly 18% the next day. Markets are rewarding the swap of payroll for compute. Put plainly, your salary can turn into someone else's compute budget.

Now put this next to everything above. Experienced people are asked for more output, given more AI output to check and fewer juniors to delegate to. Approval fatigue is what that looks like at the level of a single click. The checker is checking more, resting less, and nobody is being trained to take over.

Our companion piece on the nervous system looks at the other half of this: why a person whose internal monitor is overloaded can't simply switch it off. Our finance piece follows the money that is paying for all this compute.

8. What to do with this

None of this calls for panic. It calls for a different kind of positioning. These are working proposals, not guarantees.

Own a denominator. When generation is cheap, trust goes to whoever can show a record that can be counted: projects shipped, calls audited, cases reviewed, errors caught. Waymo's edge is not only its model. It is 271 million miles. Find the equivalent in your field and start counting.

Keep some reps without the tool. If you want to stay able to check something, you have to keep practicing it. Do part of the work by hand on purpose: write your own first pass at the analysis, then compare it with the AI's. It is slower. It is also how your internal ruler stays calibrated.

Measure what you catch, not what you approve. For a team, the useful number is not how much AI output was accepted. It is how many errors were found before they reached a customer, and how they were found. If nobody knows that number, nobody is really checking.

If you hire, hire juniors as checkers in training. The Brynjolfsson result suggests AI works best for newcomers as a tutor. A junior who reviews AI output alongside a senior, and gets feedback, is learning exactly the skill the market will pay for in ten years.

Decide who keeps the saved hour. Before celebrating a productivity gain, decide where it goes: learning, rest or more volume. If you don't decide, the quota will.

The line underneath

The usual question about AI and work is which jobs it will take. We think that is the wrong frame. The more useful questions are which jobs trained the people we now depend on, and who is training their replacements.

Generating is getting cheap. Judgment, meaning checking, choosing and carrying the consequences, is getting scarce. A system that cuts off the supply of something it increasingly needs does not fail on the day it makes that decision. It fails years later, quietly, when it reaches for that thing and finds nobody there.

See the system. Build leverage.