Sometimes beliefs are stronger than statistics

One rainy afternoon I was watching a re-run of a detective show when I began to think about the phrase “beyond a reasonable doubt.” How much doubt, I wondered, is reasonable?

If you have an analytics background, you might fall prey to the temptation of trying to quantify the doubt. Using the language of statistics, you establish a confidence interval that a specific defendant is guilty. Given our legal system’s underlying principle of “innocent until proven guilty,” we don’t want to mistakenly send someone to jail. Therefore, we might set the confidence level as high as 99.9%, rather than the typical 95%.

However, Professor Gary Wells has shown this isn’t really how legal decisions work in practice. People are reluctant to assign liability when the plaintiff’s evidence is only based on statistical evidence – even if this evidence is compelling mathematically. Consider this example:

An application for child support has been dismissed despite a blood test showing it is 99.8% probable that the man being sued is the father of a four-year-old girl.

In a series of experiments, Prof Wells compared peoples’ willingness to believe so-called naked statistical evidence versus other forms of evidence such as expert testimony. Even when the probabilities of each being accurate were identical, the subjects were more than six times more likely to convict when the statistical evidence was presented as the rationale for an expert decision. In the above example, the plaintiff’s attorney might have been more successful if the expert had testified “based on a blood test that is 99.8% accurate, I conclude that the defendant is the father” rather than simply stating “based on a blood test, there is a 99.8% probability that the defendant is the father.”

To a statistician, there is no mathematical difference between the two statements. But psychologically, the experiments show people react more favorably to statements of belief which are based on probability than they do to the probabilities themselves. This is especially true in complex or ill-defined court cases in which attorneys are highly unlikely to be able to establish an accurate assessment of the probability of innocence or guilt.

You shouldn’t be surprised. In legal thrillers, colorful and charismatic attorneys are more likely to win cases. Persuasive employees seem to get their way more often analytical ones.

I’m not suggesting that facts don’t matter but sometimes beliefs are stronger than statistics.