Safety · research august 2026 · published 2026-08-03 · v2 · 2 min read · history
Below-threshold design
Autonomy is a capability to grant only where the work needs it
A constructive discipline for granting autonomous choice only where the work requires it, from machine-guarding interlocks to a tutor that withholds answers.
Below-threshold does not mean passive. A thermostat acts on a heating system, and a payroll service acts on accounts. The phrase names a design posture instead: the work does not require the system to choose its own route, initiate beyond a standing instruction, persist after the task, reach more tools, or resist interruption.
Initiation, persistence, tool access, and freedom to choose are often treated as a tide that rises on its own schedule. A system becomes more capable, then more autonomous, and the builder merely watches the water. Software does not acquire reach or persistence by weather. Someone connects the tool, grants the memory, widens the permission, or lets the process continue without a person present.
Industrial safety engineering settled the underlying principle long before software. A mechanical press can take a hand in a fraction of a second. Training and signage leave safety dependent on attention. A two-hand control changes the machine instead: it cannot cycle unless both of the operator’s hands are occupied on buttons away from the die. The dangerous state is unreachable. A capability withheld by construction needs no trust and carries no promise.
A study of machine tutoring makes the design difference measurable. The same model that harmed learning when it supplied answers largely stopped causing that harm when the product withheld the answer and scaffolded the student’s attempt. The model’s capability did not change. The shape of the product changed what that capability could do.
Below-threshold design applies the same discipline to autonomy. Work that needs a system to route around an obstacle may justify latitude over the route. Work that follows a known sequence does not. The builder grants only the freedom the task requires, then keeps initiation, persistence, tool access, reach, and resistance to interruption as narrow as the work allows.
Bounded autonomy is not a choice to fall behind. It preserves the ability to demand tool-grade predictability where the work does not need autonomous choice. A system can still act below the threshold. It simply acts within a course a person chose and remains easy to stop, inspect, and correct.
Evidence and lineage
Research trail
Follow the sources, inspect how the claims are graded, or propose a correction at the exact record it concerns.
Sources 2
-
US Occupational Safety and Health Administration (1974). Machine guarding standards for mechanical power presses (29 CFR 1910.217), two-hand control requirements
The engineering precedent: a dangerous state made unreachable by construction rather than governed by training or trust.
Comment on this source -
Hamsa Bastani, Osbert Bastani, Alp Sungu, Haosen Ge, Özge Kabakcı, Rei Mariman (2025). Generative AI without guardrails can harm learning, evidence from high school mathematics (PNAS)
The measured example of product shape changing the effect of an unchanged model capability.
Comment on this source
Claims and confidence 2
- verified
Two-hand control requirements in machine-guarding standards keep both of the operator's hands occupied away from the point of operation while the machine cycles, making the injury state unreachable.
29 CFR 1910.217 and the machine-guarding literature.
Respond to this claim - verified
A tutored variant of the same model, with prompts that withheld answers and scaffolded attempts, largely eliminated the learning harm.
The GPT Tutor arm of the same study. Restated verbatim from the crutch-effect apparatus and graded identically.
Respond to this claim