Four of Seven, and an Argument Nobody Won

She matched four. That is above what chance predicts and below the threshold everyone had agreed, and the argument that followed was long, public and bad-tempered on both sides. Her supporters said four of seven is unlikely enough to warrant the fuller study, that the sample was too small for a five-of-seven bar to be fair, and that the televised setting was hostile. The investigators said a threshold accepted in writing beforehand cannot be renegotiated once the number is known, and that the bar had been set high precisely because the sample was small. Both arguments are sound within their own terms, which is what makes the episode instructive.

The methodological lesson is that a threshold protects against post-hoc reinterpretation only if the parties genuinely accept its authority, and acceptance is easy to give in advance and hard to honour afterwards. It is worth adding, since this concerns a living person who was seventeen at the time, that the test established nothing about her honesty or her beliefs, and that nobody involved alleged she was cheating. No further formal test was arranged. A test with a small sample and a high bar can only ever produce a pass that means something and a failure that means very little, and the investigators accepted that asymmetry when they designed it.