Wise workflows

There is a serious research programme aimed at building the wise machine outright. Artificial phronesis, associated most closely with the philosopher John Sullins, asks for systems whose outcomes a wise person would recognize as wise, and it is deliberately functionalist, requiring the outputs rather than an inner life behind them. It is honest work. It is also not what a team ships this quarter, and while it proceeds, systems with no capacity for the thing keep getting installed at the precise moments where the thing was the job.

The alternative is to move the wisdom up a level. Wisdom is a property a workflow can hold without any single component holding it, so putting human judgment at the phases where the outcome turns on which rule applies buys the behavior the wise agent was supposed to supply. Nothing in the workflow has to be wise, which is fortunate, because nothing in it is.

Herbert Simon gave us the tool for finding those phases, and it has aged better than most things from the 1950s. He split decision-making into three. Gathering, scanning the environment for the conditions that call for a decision at all. Design, generating the possible courses of action. Choice, collapsing all that possibility into one committed act. Machine systems are extraordinary at the first two, which is exactly why they feel like they are doing the whole job, and the third phase is where discernment lives. The cognitive scientist John Vervaeke supplies the reason the third phase resists automation. Relevance is grounded in agency; an organism finds things relevant because it has something to lose, and a system with nothing to lose has no basis on which anything could matter to it, so it reproduces the relevance patterns in its training data rather than realizing relevance for itself.

Well-placed, the arrangement produces results that are not modest. In the Swedish MASAI trial, reported in 2023, mammography screening supported by a detection model was compared against standard double reading by radiologists across a population screening programme. The machine sorted the reading queue. Radiologists kept the reading and kept the decision. Screen-reading workload fell by forty-four percent with cancer detection holding steady, which is what correct placement looks like when somebody measures it.

The older proof needs no machine at all. In 2009 a team led by Atul Gawande published the results of introducing a nineteen-item surgical safety checklist across eight hospitals in eight cities, from Seattle to Ifakara to Manila. Deaths fell from 1.5 percent to 0.8 percent and complications from 11 percent to 7 percent. A checklist has no wisdom in it whatsoever. It is a piece of laminated card. What it did was restructure the moments at which the surgical team’s own judgment was forced to engage, and the workflow got wiser while every component stayed exactly as smart as it had been the week before.

So the discipline is placement, and it is decidable in an afternoon with two questions. Where in this workflow does the outcome turn on which rule applies, rather than on what the rules are? And at that point, who carries the consequence of being wrong? Wherever those two answers name different parties, the workflow has a hole in it, and no improvement in the model fills the hole. Note what does not count as placement. A person rubber-stamping a recommendation downstream is not at the choice point; they are a signature on somebody else’s choice, which is the arrangement that produces alerts on eighteen percent of admitted patients and clinicians who learn to click through them.

This is not a hedge against capability improving, and it holds better as the models get better. Every improvement in gathering and design raises the stakes riding on the choice, so a system that improves at everything except judgment concentrates the judgment rather than dissolving it. Build for that concentration and you get the thing everyone actually wanted from the wise machine, arriving early, out of parts that were never wise, wherever somebody bothered to put a person where the weight falls.