Reconstruction, not explanation

The complaint about automated decisions has settled into a familiar shape. The network is a black box, its reasoning is inaccessible in principle, and the best anyone can do is ask the system to explain itself afterward. The complaint is not wrong about the weights, and record-keeping and traceability obligations for high-risk systems are written into the European Union’s AI Act partly because of it. But go and try to audit an actual decision taken eighteen months ago, and the wall you hit is almost never the weights. It is that nobody recorded which release of a numerical library was installed, or what seed the sampler drew from, or which version of the time zone database was in force when a timestamp got converted.

Explanation and reconstruction are different obligations wearing the same word. An explanation is produced after the event, by or about the same system under question, and there is nothing to check it against; it is a plausible account of feature importance that would have sounded equally plausible had the decision gone the other way. A reconstruction is a rerun. You rebuild the computational state at the moment of inference, feed the same input through it, and see what comes out, which means the exercise can come out different and can therefore fail. Only the thing capable of failing is evidence.

What makes reconstruction hard is nearly always operational rather than neural, and the specifics are almost comically mundane. Floating-point addition is not associative, and a parallel reduction on a graphics processor makes no promise about the order in which it sums, so identical weights on a different card or a different driver can differ in the low bits and, at a threshold, in the answer. The time zone database changes several times a year, because jurisdictions keep moving their clocks and the database keeps recording it, so any computation conditioned on local time is silently pinned to a version nobody wrote down. Add the container image, the sampling parameters, the artifact hashes of whatever rubric or item bank was in force, and you have a list that is tedious rather than mysterious. Every item on it is recordable at the moment of the decision and unrecoverable a year later.

Which puts the honest system in an unusual position of having to grade itself in advance. Some decisions can be replayed exactly, given the captured environment. Some can be replayed only in distribution, matching within a stated tolerance, which is a real guarantee as long as the tolerance is published rather than discovered during the dispute. And some cannot be replayed at all, which is a fact about the system and not an accusation, provided it is declared before anyone asks rather than after. A product that labels a decision non-replayable has told a regulator something true and testable. An explanation has never once been able to do that.

The reason this belongs to measurement rather than to compliance is that it is the same commitment as the error bar, arriving in a different layer. An instrument that reports its own uncertainty is stating the limits of what its number can be asked to mean. A system that records its own envelope is stating the limits of what its decision can be asked to justify. Both are ways of refusing to be believed on charm. And a decision that can be run again in front of the person it was taken about is the only kind that ever really answered them.