The Natasha Demkina Test

Natasha Demkina, a teenager from Saransk, had become known in Russia and then in the British press for saying she could see inside people's bodies and identify what was wrong with them. The investigators designed a test intended to be decisive in a single sitting, which is difficult, and the design shows the effort. Seven volunteers were recruited, each with a documented condition that would be dramatic to anyone who could genuinely see inside a body: one had had a large organ removed, one had a metal plate in the skull, one had a surgically reshaped chest. Cards naming the seven conditions were laid out, the volunteers sat in a row, and she was asked to assign each condition to a person. Chance alone would give roughly one correct.

The threshold for passing to a fuller study was fixed in advance at five, a number chosen so that a pass would be hard to attribute to luck. All parties agreed to it before she began. She was given as long as she wanted and took several hours. Two further features of the design are worth recording. The conditions were chosen to be structurally distinct rather than merely different diagnoses, so that a genuine ability to see inside a body would not require fine discrimination: a missing organ and an intact one are not a subtle contrast. And the volunteers were seated together rather than examined one at a time, so that the task was assignment across a fixed set rather than a series of independent judgements, which is what allowed a chance expectation to be calculated at all.

She matched four. That is above what chance predicts and below the threshold everyone had agreed, and the argument that followed was long, public and bad-tempered on both sides. Her supporters said four of seven is unlikely enough to warrant the fuller study, that the sample was too small for a five-of-seven bar to be fair, and that the televised setting was hostile. The investigators said a threshold accepted in writing beforehand cannot be renegotiated once the number is known, and that the bar had been set high precisely because the sample was small. Both arguments are sound within their own terms, which is what makes the episode instructive.

The methodological lesson is that a threshold protects against post-hoc reinterpretation only if the parties genuinely accept its authority, and acceptance is easy to give in advance and hard to honour afterwards. It is worth adding, since this concerns a living person who was seventeen at the time, that the test established nothing about her honesty or her beliefs, and that nobody involved alleged she was cheating. No further formal test was arranged. A test with a small sample and a high bar can only ever produce a pass that means something and a failure that means very little, and the investigators accepted that asymmetry when they designed it.