The craft

It is a Tuesday afternoon and the feature works. You described it in a paragraph, a model wrote four files, the tests pass, the linter is quiet, and the thing does what you asked. The loop took nineteen minutes. The received view says the interesting question here is capability, whether the model is good enough yet, and the received view has a point, because capability is the thing that visibly keeps changing. It is also the thing that keeps failing to matter. Barry Boehm published the spiral model in 1986 with risk analysis at the center of every loop, and lighter methods displaced it because the analysis and its throwaway prototypes cost too much to run. The prototype is now free. Meanwhile the person holding the whole design together still runs on a working memory of roughly four to seven chunks, a figure that has barely moved since Miller estimated it in 1956.

Two costs that used to travel together came apart. Making things was the expensive half of nearly every knowledge craft, and it stopped being expensive. Checking things was the cheap half, mostly folded invisibly into the making and paid for by the same hours, and it is now the entire job. That is a different profession wearing the old job title, and most of the difficulty people report in working alongside machine intelligence is the friction of doing the new job with the reflexes of the old one.

The half that got cheap

The spiral model is worth a second look precisely because it lost. Each of its loops demanded objectives, a risk analysis, and often a prototype built to answer one question and then deleted. Teams could not afford to build things they intended to throw away, so they stopped, and the industry spent two decades optimizing for velocity instead. The economics that killed the spiral no longer hold. A throwaway prototype is an afternoon at most and frequently twenty minutes, which leaves the risk analysis it existed to serve as the only part of the loop still carrying a real cost. What was overhead became the work.

Boehm’s other well-known curve completes the picture. The cost of correcting a defect rises steeply with how late it is caught, cheap in specification and punishing in production. The exact multipliers he reported have been argued over ever since and we do not lean on them, but the shape has never been seriously disputed. Put the two together and the arithmetic is stark. Generation fell toward zero while the cost of an escaped defect did not move at all. When one input to a process becomes free and the others do not, the process reorganizes around what is still scarce, and what is still scarce here is knowing whether the thing is right.

Entropy is a context problem

Why does anything rot, though, given competent people? A practitioner talk in our research record makes the sharpest available answer, and it is not about capability at all. Architectural failures, the argument runs, are almost never caused by bad judgment. They are caused by context that exists somewhere but is spread across too many files, too many people, and too many moments in time to be present in one head at the moment of decision. The constraint underneath is not professional, it is anatomical. Working memory holds four to seven chunks whether you are in your first year or your thirtieth, and thirty years of expertise buys you bigger chunks rather than more slots. Codebases grow exponentially. Heads do not grow at all.

The talk lands on an image we did not expect to meet in an engineering context, because this program already runs on it. You cannot hold the design of the cathedral in your head while laying a single brick. We arrived at the kiln, the brick, and the cathedral from the question of how knowledge becomes durable; the talk arrives at the same cathedral from the question of why software decays, and the convergence is worth naming rather than smoothing over. Both are describing a builder whose reach exceeds their span and a structure that outlives everyone who touched it.

The failures this produces are small enough to be invisible and expensive enough to matter. An engineer adds one argument to a cached function, an options object where a plain value used to go. The code compiles, the types check, the tests pass, and the cache never hits again, because every call now constructs a fresh object and the cache is keyed on identity. Nothing about that change looks wrong at any point where a human or a linter examines it. Elsewhere, a tidy reusable hook for dismissing a popup attaches one listener to the document, which is correct, and a hundred instances of that popup attach a hundred of them, which is a different program. Elsewhere again, a coupon check goes into a checkout function long enough that nobody reads it end to end, and work that used to run in parallel now waits in line. In each case the information that would have prevented the mistake existed. It was simply somewhere else.

This is where the machine turns out to be structurally better rather than smarter. A model applies the same scrutiny to the ten thousandth file as to the first, holds a global and a local view at once, and does not tire at four in the afternoon on the day before a release. Those are not intellectual virtues. They are the absence of a body. The engineer at Vercel who spent years on performance work and, by the talk’s account, distilled some four hundred pull requests into a structured rules repository designed to be queried by agents was not writing documentation. They were manufacturing the presence of context at the moment someone would need it. The catch sits in the order of operations. A pattern has to exist before anything can enforce it, and a model cannot infer the rule from a codebase that applies it inconsistently, which is exactly the codebase most in need of the rule.

What passes every old test

The new failure mode is hard to catch because it satisfies every check built for the old one. A generated prototype arrives carrying all the signals a decade of practice taught us to trust. It runs. It is internally consistent. It is fluent and well organized and confident in a way that human first drafts rarely are. If a model builds an entire subsystem on the assumption that an endpoint returns a timestamp when it actually returns a string, everything downstream of that assumption will be coherent with it, because coherence is what generation optimizes for. The linters pass. The tests, often written by the same model from the same assumption, pass emphatically. Every conventional quality signal confirms the error instead of catching it, which is a strictly worse position than the one those signals were designed for, and it is why so much of the review effort that used to go into reading code now has to go into reading the tests.

A second thing degrades, more slowly and on the clock. Adherence to a complicated set of constraints thins as a session extends, and the model drifts back toward convention, toward the most common way of doing this rather than your way. Two of our engineering research documents reached that conclusion independently, and the practical response in both is the same and slightly undignified, which is to start a fresh session far more often than feels necessary. What that means for anyone in a long collaboration with a model is worth stating plainly, because it is not the intuitive risk. Long collaboration does not degrade toward error. It degrades toward the generic, which is much harder to notice, since the generic is exactly what a reviewer’s eye slides over.

The seat with the known defect

None of this is peculiar to software, and the oldest published account of it is not about software at all. In 1983 the psychologist Lisanne Bainbridge described what she called the ironies of automation. Automate the parts of a task that are easy to automate, and what remains for the human is monitoring, which is the activity humans perform worst; meanwhile the operator’s hands-on skill decays through disuse across exactly the years before the moment it is abruptly required. She was writing about process control rooms and cockpits. Four decades on, the description fits a lawyer reviewing a generated brief, a physician scanning a generated summary, an analyst signing off on a generated model. Everyone in those rooms has been promoted into the monitoring seat, and the monitoring seat has a known ergonomic defect.

The disciplines that answer it are unglamorous and they all do one thing, which is to move context back to the point where a decision is made. Writing down why rather than only what, so the reason a threshold is that number and the alternative that was rejected survive the person who knew them. Marking the load-bearing decisions, so that removing one means arguing with a sentence instead of deleting a line. Asking a second model to explain the work back rather than to approve it, since a critic and an author sharing a session share the author’s assumptions. Keeping the unit of work small enough that a person can still hold all of it, which is less a technique than the last honest defense. None of these are new practices. What is new is that they stopped being professional courtesies and became the load-bearing structure, because the thing they compensate for is no longer occasional.

Under all of it sits one question you can ask of anything you are about to ship, and it is not whether it works. It is whether anyone can say why it is safe. That question was always available and it was rarely the bottleneck, because a person who had written something line by line could usually answer it, and the answer came free with the labor. The labor is gone and the answer did not leave with it. It has to be bought deliberately now, and buying it is close to the whole of what we are paid for. That is a better trade than it sounds. The half of the work that just became cheap was never the half anyone loved, and the half that became scarce, holding the design in mind, knowing why, putting your name to the claim that this is sound, was always the craft. It has come back to the center of the job, and it is asking more of us than it used to.