Almost every AI governance framework calls for human oversight, and almost every AI system design includes a point where a person reviews something. The two are not the same. Oversight is meaningful only when the person involved could realistically change the outcome. Where they cannot, the review step provides reassurance without control, and it can make accountability less clear rather than more.
Three conditions
- Time. A reviewer approving hundreds of items an hour is not exercising judgement on each one. Volume should be set so that review is possible, or the system should route only the cases that need a person.
- Information. The reviewer needs to see what the system saw: the evidence used, its confidence, and what will happen if they approve. A bare recommendation with an accept button invites agreement.
- Authority. The reviewer must be permitted to reject, and rejection must not carry a penalty. If overriding the system requires justification but accepting it does not, the design is biased towards acceptance.
The automation bias problem
People defer to systems that are usually right. That is a reasonable behaviour, and it is exactly what makes nominal oversight dangerous. As a system's accuracy improves, reviewers have fewer occasions to disagree, attention falls, and the rare error passes unchallenged. Good designs counter this by measuring how often reviewers override, by occasionally inserting known cases to confirm attention, and by making disagreement easy and visible.
Decide where the person sits
Not every decision needs the same kind of human involvement. Some actions can proceed automatically within clear limits, with a person reviewing a sample afterwards. Others should be held until someone approves them. A smaller set should never be automated at all. Setting these categories explicitly, in writing and in advance, is one of the most valuable governance decisions an organisation can make about an AI system, and one of the most often skipped.
Name the owner
Behind every review point there should be a named role that owns the outcome, whatever the level of automation beneath it. When something goes wrong, the question of who was accountable should already have an answer. Recent industry research has found that many organisations do not yet have that clarity for AI agents, which is a strong reason to settle it before deployment rather than after an incident.
Oversight that works is usually less visible than oversight that does not. It sits in a small number of well-chosen places, it is resourced properly, and it is measured. That is a design task, and it belongs at the start of a project.