J. R. Statist. Soc. A (1991)
154, Part 1, pp. 75-81
Can Jurors Understand Probabilistic Evidence?
By D. H. KAYEt and JONATHAN J. KOEHLER
Arizona State University, Tempe, USA University of Texas, Austin, USA
[Read at the International Conference on Forensic Statistics held in Edinburgh, April 2nd-4th, 1990]
SUMMARY
Some courts have been reluctant to admit testimony expressing probabilities because of
a concern that jurors will overweight it relative to other evidence. However, empirical
studies indicate a tendency to underweight statistical evidence when other sources of
evidence are available. This paper reviews recent studies with mock jurors.
Keywords: COGNITIVE ILLUSIONS; EVIDENCE; PROBABILITY
1. INTRODUCTION
For decades, attorneys, statisticians and psychologists have argued over the admis-
sibility of 'mathematical evidence' and the desirability of using Bayes's theorem to
quantify for the jury the probative value of trace evidence such as partial fingerprints
or bloodstains: Finkelstein and Fairley (1970, 1971); Tribe (1971); Fairley (1973);
Finkelstein (1978); Ellman and Kaye (1979); Saks and Kidd (1980); Fienberg and
Schervish (1986); Tillers and Green (1988). Fearing that jurors will be unduly
impressed by mathematical testimony, a few courts have held that the population
frequencies of incriminating traits and the conditional probability of an innocent
defendant's having an incriminating characteristic T are inadmissible: State versus
Carlson, 267 N.W.2d 170 (Minnesota 1978); State versus Boyd, 331 N.W.2d 480
(Minnesota 1983); State versus Kim, 398 N.W.2d 544 (Minnesota 1987); State versus
Schwartz, 447 N.W.2d 422 (Minnesota 1989). Although this is a minority view (Kaye,
1987), even courts allowing such testimony caution that they 'generally disfavor
admission of statistical evidence': Commonwealth versus Gomes, 403 Massachusetts
258, 526 N.E.2d 1270 (1988).
Given these views, it is important to know whether jurors can be trusted to evaluate
properly 'probability evidence', and what decision aids might assist them in this task.
For more than two decades, researchers have studied the ways that people process
probabilistic and statistical information, but only a small portion of these studies
focuses on the capacity of jurors to process explicitly quantitative probabilistic
evidence. This paper reviews this research. It concludes that the work has produced
several insights into the factors that affect the judgments of mock jurors, and that
it is valuable in devising optimal rules for the admission or exclusion of probability
evidence. At the same time, we do not believe that the experiments published to date
have been adequate in their design and implementation to demonstrate unequivocally
the extent to which jurors attend to trace evidence or to identify what decision aids,
if any, would promote an appropriate weighting of the evidence at trial. For another
review of these experiments, see Thompson (1989).
tAddress for correspondence: Center for the Study of Law, Science and Technology, Arizona State University,
Tempe, AZ 85287-0604, USA.
? 1991 Royal Statistical Society 0035-9238/91/154075 $2.00

76 KAYE AND KOEHLER [Part 1,
2. MAJOR THEMES OF RESEARCH INTO DECISION-MAKING
A large amount of research has been conducted on the ways in which people
process and employ statistical and probabilistic information. Although Bayes's
theorem provides a normative rule for revising probabilistic beliefs in the light of
new evidence, many studies indicate that within certain domains people do not
extract as much information from new evidence as the data warrant, that they are
slow to revise incorrect probabilistic hypotheses, that they attribute probative value
to diagnostically worthless information, that they underutilize statistical base rates
and that they confuse likelihoods and posteriors. For general reviews and collections,
see Kahneman etal. (1982), Nisbett and Ross (1980), von Winterfeldt and Edwards
(1986) and Slovic and Lichtenstein (1971); cf. Edwards and von Winterfeldt (1986)
and Saks and Kidd (1980). Recently, some research has appeared on how mock jurors
treat probabilities. In general, the results are consistent with the larger body of work
on statistical judgment, but some significant issues have yet to be explored fully.
3. RESPONSES OF MOCK JURORS TO PROBABILITIES
The experiments on probability assessments of mock jurors typically ask the
subjects to read a transcript of testimony concerning some incriminating trait T(such
as fibres or blood types left at the scene of the crime) that are said to occur in the
general population with some relative frequency F(T). The experiments probe the
extent to which mock jurors modify their assessments of guilt ('guilt' in the factual
sense that the defendant D committed the criminal acts) when they learn that D has
the trait T (an event that may be denoted T,D).
Before describing the results of such experiments, we should consider how a juror
whose partial beliefs behave like probabilities would react. In this Bayesian scheme,
a juror begins with some prior personal probability P(G) that the defendant
committed the acts as alleged. The juror then revises P(G) by conditioning on TD
according to Bayes's theorem:
P(GITD) P(TD I G) P(G)
P(G I TD) P(TDIG) P(G)(
Here, G denotes 'not guilty'. A juror who believes that the forensic test is error
free and that the guilty person does have the incriminating trait T, while only a
fraction F(T) of innocent people have T, will choose 1/F(T) for the likelihood ratio
P(TD I G)/P(TD I G). Such a juror therefore will conclude that the posterior odds are
P(G I TD) I P(G)
P(GI TD) F(T) P (()
When the sensitivity or specificity of the test is not unity, the likelihood ratio is
more complicated. (Thompson et al., 1990). Similarly, if there were a suspicion of
a frame-up (Tribe, 1971) or a selection effect in using the incriminating trait T, then
this ratio would be smaller than J/F(T).
3.1. Effect of TDand F(T)
In the first experiment to study whether mock jurors behaved according to
equation (2), Thompson and Schumann (1987) asked 144 university students to read a