Twelve decisions. The model has already made every one of them. Your job is the one the regulation describes and nobody funds: look at it, and override it if it is wrong.
The model is good. It is right about most of these, which is exactly what makes this hard.
01Each case has a clock, and the clock gets shorter as you go.
02If the clock runs out, the decision goes through approved. That is not a penalty. That is what happens.
03At the end you get a score, and you find out which ones the model got wrong.
Case 1 of 1211 still in the queue12s
Model decision
A to approve · O to override
Your score
0 / 4 caught
Errors you caught
Correct decisions you overrode
Approved because the clock ran out
Time spent
Automation bias is the documented tendency to accept a machine recommendation without the independent check you were put there to perform. It is not a character flaw. It is what a high accuracy system does to the attention of the person watching it, and it has been measured in aviation, in medicine and in screening for decades. In 2015, undercover teams carried prohibited items past airport screeners in 67 of 70 attempts.
So the question I keep asking is not whether a human is in the loop. It is whether that human was given the time, the reasoning and the standing to actually say no. That is the problem HumanChain exists to solve, and it is why I measure models at Modelometer instead of taking their word for it.