Interpreting claims in offender profiles: the role of probability phrases, base rates and perceived dangerousness
- DOI: 10.1002/acp.1438
Abstract
Offender profilers use verbal and numerical probability expressions to convey uncertainty surrounding claims made about offender's characteristics. No previous research has examined how these expressions might affect the recipient's interpretation of the information. Seventy participants completed an online questionnaire and results showed a diverse range of interpretations of these uncertainty expressions. Moreover, characteristic base-rates and dangerousness affected the perceived likelihood of the profiling claim, such that increased base-rates and perceived dangerousness resulted in an increased perception of the claim being likely. Perceived likelihoods also depended on the framing of characteristics as well as the framing of the claim itself. Finally, where claims involved presenting a characteristic qualified by a low probability these claims were interpreted as more likely than not to be present. These findings have practical implications for profilers and more general theoretical implications for the Study of risk perception. Copyright (C) 2008 John Wiley & Sons, Ltd.
Author-supplied keywords
Interpreting claims in offender profiles: the role of probability phrases, base rates and perceived dangerousness
to a broader myriad of issues involved in investigating crime—such as interview strategies,
D .
A l
I t
th
in
APPLIED COGNITIVE PSYCHOLOGY
Appl. Cognit. Psychol. (2008)
Published online in Wiley InterScience
(www.interscience.wiley.com) DOI: 10.1002/acp.1438
*Correspondence to: Dr Louise Almond, Department of Psychology, Centre for Critical Incident Research,
University of Liverpool, Eleanor Rathbone Building, Bedford Street South, Liverpool, L69 7ZA, UK.
E-mail: lalmond@liverpool.ac.uk
yPortions of this research were presented at the Offender Profiling and Bad Character Conference, April 2007,
L
CNA intelligence led screens, risk assessments and geographical profiling—has emerged
s a result, offender profilers are now more broadly referred to as ‘Behavioura
nvestigative Advisors’ (BIA). Despite this, BIA reports still typically contain claims abou
e likely characteristics of offenders; expressed with varying uncertainty qualifiers to
dicate the extent to which the enquiry team can expect each claim to be true. UncertaintyTraditionally, profiling has been defined as the process of predicting the likely
socio-demographic characteristics of an offender based on the information available at
the crime scene (Alison, Mclean, & Almond, 2007). In the last 10 years, however, a change
of emphasis from the exclusive focus on the offender and his likely ‘psychological profile’SUMMARY
Offender profilers use verbal and numerical probability expressions to convey uncertainty surround-
ing claims made about offender’s characteristics. No previous research has examined how these
expressions might affect the recipient’s interpretation of the information. Seventy participants
completed an online questionnaire and results showed a diverse range of interpretations of these
uncertainty expressions. Moreover, characteristic base-rates and dangerousness affected the per-
ceived likelihood of the profiling claim, such that increased base-rates and perceived dangerousness
resulted in an increased perception of the claim being likely. Perceived likelihoods also depended on
the framing of characteristics as well as the framing of the claim itself. Finally, where claims involved
presenting a characteristic qualified by a low probability these claims were interpreted as more likely
than not to be present. These findings have practical implications for profilers and more general
theoretical implications for the study of risk perception. Copyright# 2008 John Wiley & Sons, Ltd.Interpreting Claims in Offender Profiles:
The Role of Probability Phrases, Base-Rates
and Perceived Dangerousnessy
GAE¨LLE VILLEJOUBERT1,
LOUISE ALMOND2* and LAURENCE ALISON2
1CLLE-LTC (CNRS, UTM, EPHE), Universite´ de Toulouse, Toulouse, France
2Department of Psychology, Centre for Critical Incident Research, University of Liverpool,iverpool University, Liverpool, UK.
opyright # 2008 John Wiley & Sons, Ltd.
unlikely. For example, ‘the offender is probably a male’. They can also include precise
numerical expressions such as, ‘There is a 60% chances that the offender will be white’. If a
claim is not qualified by a verbal or a numerical uncertainty phrase, then the profiler is
indicating to the enquiry team that the claim is certain. For example in the absence of any
uncertainty qualifier, the statement, ‘the offender will have previous convictions’ might be
understood as a statement of complete certainty.
To verify that profilers duly reported the uncertainty surrounding their claims, Collins
and Alison (2002) content analysed 26 offender profiles constructed by a range of profilers,
the majority of which were produced during the years 1996–2000. They identified 107
different verbal probability qualifiers, which they argued could be divided into two broad
categories: a ‘possible’ low-probability category (i.e. characterising claims as having a low
probability of being true) and a high-probability category (i.e. characterising claims as
having a high probability of being true). However, 46% of the claims examined were not
characterised by any uncertainty qualifier at all, suggesting that they may be perceived as
statements of certainty (a situation which is extremely unlikely in any field of expert
advice).
However in a contemporary study, Almond, Alison, and Porter (2007) examined
47 behavioural investigative advice reports produced by the National Policing
Improvements Agency in the year 2005 and discovered that 18% of all the claims were
unqualified. This more recent figure is comparatively lower than that observed by Collins
and Alison (2002). Almond et al. (2007) coded the remaining claims as either verbal or
numerical qualifying expressions, with the verbal probability expressions divided into the
same two high/low categories previously identified by Collins and Alison (2002). Results
revealed that 59% of these qualifiers could be coded as ‘probable’ and only 5% as
‘possible’. The statistical terms were also categorised as either high or low probability
qualifiers. Similar results emerged from this analysis: 16% of these qualifiers represented a
probability higher than 50%, whilst only 2% represented a probability of less than 50%. So,
overall, this line of research has shown that most claims presented within contemporary
BIA reports were qualified by verbal probability expressions and were presented as highly
probable statements of certainty.
The fact that probability expressions are now used in most profiling claims is reassuring,
since investigators can more readily assess the significance of the inferences drawn from
claims when they are qualified by varying levels of certainty. However, exactly how
qualifiers affect the interpretation of the claims is currently unknown. There are some
well-documented examples of disastrous errors caused by differential understanding and
usage of verbal probabilities. For example, Karelitz and Budescu (2004) discussed the
costly consequences of the different interpretation of the term fair chance by the U.S Joint
Chief of Staff and the Central Intelligence Agency which led to the Bay of Pigs invasion in
1961. In a similar vein, if a Senior Investigating Officer (SIO) were to misinterpret the
strength of an offender profiling claim, this could have serious consequences for the
investigation. A misunderstanding about the likelihood of an offender having
pre-convictions (and thus appearing in a Police National Computer record) could if
unchecked, direct an enquiry down an unproductive route. For that reason, it is important
that research considers how verbal and numerical probabilities affect uncertainty
interpretations. This will give profilers and BIA’s a better awareness of pertinent issues
when constructing their reports and will help to minimise any misinterpretations between
themselves and the SIO.
Copyright # 2008 John Wiley & Sons, Ltd. Appl. Cognit. Psychol. (2008)
DOI: 10.1002/acp
values of the probability scale for any given probability expression. Membership functions
Interpretation of profiling claimsthus describe how verbal probability expressions map on the different values of the
numerical probability scale for different individuals. They specify the meaning of a
particular verbal probability expression by defining (i) its range of possible numerical
interpretations, (ii) the symmetry of the mapping and (iii) the probability value which has
the highest membership value. For example, non-null membership values for the word
likely spread from p¼ .08 to p¼ 1, indicating that likely could be interpreted to refer to
almost any numerical probability value. On average, the maximum membership value (the
membership function’s peak value) for likely is assigned at a probability of .85 (Budescu,
Karelitz, & Wallsten, 2003). This suggests that likely will be interpreted by most people as
conveying an 85% probability. Whereas membership functions are generally quite stable
for a given individual, they can vary greatly from an individual to another (Budescu &
Wallsten, 1995). This line of research suggests that considerable variation in individuals’There is a large body of decision-making literature that examines the interpretation of
probability words and expressions. Several studies have revealed that, paradoxically,
although people prefer to communicate uncertainties with verbal probability expressions
they prefer to receive it with numerical probability expressions (Brun & Teigen, 1988; Erev
& Cohen, 1990). Speakers’ preference for verbal expressions is generally said to occur
because words, as opposed to numbers, are thought to be better understood by those who
receive the information. In contrast, decision-makers who receive uncertain information
tend to think they will make more accurate inferences based on numerical information,
although this belief is not necessarily correct (Erev & Cohen, 1990).
In the remainder of this introduction, the paper considers the research on verbal
probabilities and uses this as a basis for predicting how verbal probabilities may affect the
interpretation of profiling claims.
INTER-INDIVIDUAL VARIABILITY IN NUMERICAL INTERPRETATIONS
OF PROBABILITY WORDS
When communicating uncertainty, the vocabulary people use to express states of
uncertainty is rich and varied. In an effort to investigate the numerical meaning of verbal
probabilities, Reagan, Mosteller, and Youtz (1989) found that over 282 different verbal
probability expressions had been used in 37 studies of verbal uncertainty. As well as being
numerous, verbal probabilities are also very vague (Brun & Teigen, 1988; Reagan et al.,
1989; Teigen & Brun, 1995). Unsurprisingly, the most robust finding in studies of verbal
uncertainty communication is the extremely high variability in individuals’ interpretation
of numerical probabilities conveyed within a verbal probability expression. Although
people perceive the meaning of verbal probabilities consistently (individuals derive the
same interpretation across multiple situations), there is a wide variation across different
individuals in the interpretation of verbal probability expressions (Teigen & Brun, 1995).
Wallsten, Budescu, Rapoport, Zwick, and Forsyth (1986) argued that individuals’
interpretations of verbal probability expressions are best understood in terms of
membership functions. Memberships range from 0 for a numerical value that cannot be
conveyed by the verbal probability to 1 for a numerical value that is typically conveyed bynumerical interpretations might also be observed within the context of profiling claims.
Copyright # 2008 John Wiley & Sons, Ltd. Appl. Cognit. Psychol. (2008)
DOI: 10.1002/acp
consequences, the perceived dangerousness of an offender characteristic may, for example,
also influence the numeric interpretation. This study consequently hypothesised that a
G. Villejoubert et al.dangerous offender characteristic statement would be assigned a higher numeric
probability function than a harmless characteristic, even where qualified by the samePERCEIVED AS MORE LIKELY
As discussed previously the perceived severity of the outcome qualified is an additional
contextual feature. Individuals tend to interpret a given probability word as more probable
when it qualifies a severe outcome in a medical scenario (Bonnefon & Villejoubert, 2006;
Weber & Hilton, 1990). Assuming the physician is polite and will try to ‘sugar-coat’
severe—hence threatening—news, individuals infer that the true numerical probability of
the severe outcome is much higher than that usually communicated by the physician’s
probability qualifier (Bonnefon & Villejoubert, 2006). This ‘severity effect’ may also play
a role in interpreting profiling claims because a severe outcome has potentially harmfulMprob
CopyORE DANGEROUS BEHAVIOURAL CHARACTERISTICS WILL BEThe word likely will receive different numerical interpretations depending on context. In
interpreting the level of uncertainty communicated, individuals tend to combine the range
of values which they feel are expressed by the word likely with contextual information
about the base-rate of the outcome being predicted (in this case, the probability of snow in
December in the two locations specified). Thus, when asked to give numerical
interpretations for such outcomes, individuals typically assign higher numerical values
to ‘likely’ when it qualifies a high base-rate outcome rather than a low base-rate outcome
(Wallsten, Fillenbaum, & Cox, 1986; Weber & Hilton, 1990). In the example above,
individuals would assign a higher numerical value to statement 2. Consequently, one might
expect that a high base-rate offender characteristic would be assigned a higher numerical
probability than a low base-rate characteristic, despite them being qualified by the same
probability word (Hypothesis 2).Therefore, we hypothesised that individuals’ membership functions of verbal probability
expressions qualifying profiling claims would be heterogeneous (Hypothesis 1).
PERCEIVED BEHAVIOURAL CHARACTERISTIC BASE-RATES WILL
AFFECT NUMERICAL INTERPRETATIONS
In addition to high variability across different individuals’ interpretations, several external
factors, such as the outcome’s base-rate as well as its seriousness have also been shown to
have substantial effects on the numerical interpretation of verbal probabilities. Thus, the
peak, spread and shape of probability words’ membership functions also depends on the
context within which any given claim arises (Budescu & Wallsten, 1995). Thus, a likely
outcome can be thought to have different numerical probabilities of occurrence depending
on its perceived base-rate probability. For example, compare the following two statements:
(1) It is likely that it will snow in Liverpool, England, next December.ability word (Hypothesis 3).
right # 2008 John Wiley & Sons, Ltd. Appl. Cognit. Psychol. (2008)
DOI: 10.1002/acp
ON CHARACTERISTIC FRAMING
Probability words used to qualify an outcome can be assigned a variety of different
numerical interpretations, depending on (i) individuals’ personal membership functions for
these words, (ii) on the outcome’s base-rate or its severity. Furthermore, recent research
suggests that the way an outcome is framed will also affect how likely it will be perceived.
Verbal probability qualifiers generate different perspectives on the perceived probability
of an occurrence or non-occurrence (Moxey & Sanford, 2000; Teigen & Brun, 1995). Some
probability words and expressions such as probable or a small chance are said to have a
‘positive directionality’. This means that they focus our attention on a possible occurrence
of the outcome characterised. Alternatively, when an outcome is qualified by a negative
probability term such as doubtful or not quite certain, they focus our attention on
non-occurrence.
A direct consequence of ‘perspective effect’ is that when an outcome is framed
positively, individuals prefer using positive probability words to qualify uncertainty. For
example, when a medical examination reveals positive reactions to some of the tests,
individuals prefer to say, that it is possible the patient has the disease, thus representing a
‘positive verbal probability’. Conversely, they will prefer to use a negative word and say,
for example, that it is uncertain whether or not the patient has the disease when told that not
all the tests showed positive reactions. A mirror pattern occurs when individuals are told of
the quantities of tests showing negative results (Teigen & Brun, 2003).
Generally, systematic relationships between the directionality of a given probability
expression and the shape and location of its membership function are observed. Thus,
positive phrases are typically interpreted as conveying probabilities above .5, whereas
negative phrases are typically interpreted as denoting probabilities below .5. The
membership functions of positive probability expressions are, therefore, positively skewed
with peak values above .50. In contrast membership functions of negative probability
expressions are negatively skewed with peak values below .50 (Budescu et al., 2003).
Typically, profiling claims most often refer to behavioural characteristics that may or
may not be true of the suspected offender. Thus, the perceived level of uncertainty
associated with a given behavioural characteristic may depend on whether the
characteristic is presented in a positive frame (i.e. presence of an offender characteristic)
or in a negative frame (i.e. absence of an offender characteristic). The current paper
hypothesises that the perceived level of uncertainty associated with a given probability
word such as likely would be higher when this word qualifies a positively framed
characteristic (‘the offender has a history of sexually inappropriate behaviour’) compared
to when it qualified negatively (‘the offender does not have a history of sexually
inappropriate behaviour’) (Hypothesis 4a). Similarly, probability words chosen to express
a given numerical probability such as 80% probability may convey a higher level of
uncertainty when qualifying a positively framed characteristic rather than a negatively
framed characteristic (Hypothesis 4b).
PERCEIVED LIKELIHOOD WILL DEPEND ON PROFILING CLAIM
FRAMING
A given profiling claim may focus on the chances that a target attribute is true or will occur
vs. the chances that the alternative target attribute is not true or will not occur. For example,
Copyright # 2008 John Wiley & Sons, Ltd. Appl. Cognit. Psychol. (2008)
DOI: 10.1002/acp
LOW-PROBABILITY CHARACTERISTICS WILL BE
EXPERIMENT
G. Villejoubert et al.Profilers use probability phrases to indicate the level of uncertainty of any given claim.
Previous research has shown that profilers use various verbal and numerical probabilities
expressions to convey such uncertainties (Collins & Alison, 2002). However, there is
currently no research examining how these terms can affect the interpretation of offender
profiling claims. Based on existing literature on the interpretation of verbal probabilities,
we formulated a number of hypotheses concerning the factors that will affect the
interpretation of the uncertainty qualifying profiling claims. These hypotheses were tested
using an online questionnaire.
Method
Participants
Seventy participants (20 men, 45 women, 5 did not specify their gender, mean age¼UNDERSTOOD AS CONFIRMATIONS
Although high numerical probabilities are typically expressed with positive probability
words such as very probable and low probabilities, this is not always the case with negative
words such as very improbable. An event with an 80% probability may be described as not
completely certain, whilst an event with a 20% probability may be described as possible. In
fact, low probabilities outcomes (10%-probability, 25% probability outcomes) are more
often interpreted as asserting an outcome rather than denying it (Teigen & Brun, 1995).
Moreover, Evans (1998) demonstrated the existence of a ‘matching bias’, defined as the
tendency to only consider information whose lexical content matches that of the
information presented in the propositional rule to be tested. We consequently hypothesised
that where a behavioural characteristic is mentioned in a profiling claim; this would still be
judged as more likely to represent the characteristic of the offender even when the claim
that qualifies the characteristic is assigned a low numerical probability (Hypothesis 6).than when it is described as offering a 10% chance of failure, even though the two
descriptions are formally equivalent (Levin & Gaeth, 1988; Russo & Schomaker, 1989).
Thus the numerical interpretation of a verbal uncertainty qualifier should be higher when
inferred from a positively framed claim such as ‘It is likely that X’ rather than from a
negatively framed claim such as ‘It is unlikely that X’ (Hypothesis 5a). Conversely, we
expected that the probability word chosen to express the numerical probability of a claim
would convey higher levels of uncertainty when it was inferred from a positively framed
claim (e.g. there is a 70% probability that X is true) rather than from a negatively framed
one (e.g. there is a 30% probability that X is not true) (Hypothesis 5b).a profiler could say that, ‘it is likely that the offender has a short temper’ (positive frame) or,
‘it is unlikely that the offender does not have a short temper’ (negative frame). Thus
profiling claim framing entails both the framing of the behavioural characteristic and its
associated probability. Research on the role of description framing has shown that when an28.5 years, SD¼ 11.3 years) were recruited through postings on the University of
Copyright # 2008 John Wiley & Sons, Ltd. Appl. Cognit. Psychol. (2008)
DOI: 10.1002/acp
were White British. Half were employed and 44% were still studying. Forty-one per cent
had achieved a Post-Graduate Degree, 27% a Degree and 21% had achieved A-levels.
Design
All participants completed the same online questionnaire containing 48 questions
presented in two sections. The first section presented a series of questions aimed at testing
the effect of the different contextual factors we had identified on the numerical
interpretations that could be assigned to probability words or on the choice of verbal
probability words. The second section was aimed at assessing the variability in individuals’
numerical interpretations of verbal probabilities using membership functions for a series of
expressions qualifying different offender characteristics.
Interpretation of profiling claimsMaterials and procedure
The first page of the questionnaire introduced the study as part of a larger project
examining the content of offender profiles and behavioural investigative advice and the
ways in which such advice is interpreted and used. Participants were reminded that their
answers would remain confidential and anonymous. Before beginning the questionnaire,
participants were asked to indicate that they consented to participate in the study and that
they were aware that they could withdraw from the study at anytime.
The hypotheses were tested by displaying a series of statements presented as originating
from a number of offender profiles that had been compiled to assist the police in their
apprehension of an unknown suspect. Participants were asked to read each statement
carefully and answer the associated question.
In order to examine the variation that exists across individuals’ interpretation of verbal
probability words (Hypothesis 1), we elicited membership functions for ten different words
using the Multiple Stimuli Method (Budescu et al., 2003). So, after having read an offender
profiling claim such as ‘It is very probable that the offender will be male’, participants
were presented with 11 10-point Likert scales, corresponding to 11 levels of numerical
probability (0%, 10%, up to 100% probability). For each of the probabilities presented
(0%, 10%, etc.), participants were asked to indicate the extent to which the profiler could
have had this probability in mind when making his claim. They reported their answer by
selecting a number between 1 (absolutely not) and 10 (absolutely). Table 1 summarises the
10 probability phrases examined alongside the offender characteristic they qualified. The
Table 1. Offender profiling claims used to test for Hypothesis 1
Claim
1. It suggests that the offender will have previous sexual convictions
2. It is very likely that the offender will have a manual unskilled occupation
3. It is uncertain whether the offender will be employed
4. It is likely that the offender will be aged 50 years or older
5. It is somewhat doubtful that the offender will live in the local area
6. It is possible that the offender will not live in the local area
7. It is very unlikely that the offender will be married
8. It is probable that the offender will be single
9. It is quite unlikely that the offender will be aged less than 25 years old
10. It is improbable that the offender will have previous convictionsCopyright # 2008 John Wiley & Sons, Ltd. Appl. Cognit. Psychol. (2008)
DOI: 10.1002/acp
base-rate characteristic. A similar manipulation was used to assess the effect of perceived
dangerousness on the interpretation of the uncertainty associated with offender profiling
claims (Hypothesis 3). Table 2 summarises the claims used for this section. For each of
these claims participants were asked, ‘What do you think the chances are that the offender
[characteristic]? ___%’.
To evaluate the effect of characteristic framing on the level of probability assigned to a
given probability word (Hypothesis 4a), we presented the following two pairs of claims
where the same probability phrase qualified either a positively framed behavioural
characteristic or a negatively framed characteristic (the words in brackets were presented in
the claim using a negative frame):
(a) It is likely that the offender will (will not) have a history of sexually inappropriate
behaviour.
(b) It is probable that the offender will (will not) be a loner.Th
profilIn
cha
fram
cha
W
prob
How
neg
thei
(c)
(d)
Copyple identified by Almond et al. (2007).
e effect of base-rates on the interpretation of the uncertainty associated with offender
ing claims (Hypothesis 2) was examined using three pairs of claims. The sameselec
samtion of these terms was based on the most regularly used terms in the contemporaryBase-rate
High . . .of White British origin . . .of Christian faith . . .have previous convictions
Low . . .of Chinese origin . . .of Pagan faith . . .have previous convictions
for fraud
Dangerousness
High . . .a sexual predator . . .a psychopath . . .kill again
Low . . .a homosexual . . .a postman . . .return to the crime sceneDime
manipnsion
ulated
Probability phrase
. . .will probably be. . . . . .is possibly. . . . . .is likely to. . .Table 2. Claims presented to participants as a function of base-rate and dangerousnessorder to collect comparable judgements, participants were asked to evaluate the
nces that the offender did present the target characteristic when it was positively
ed, but they were asked to evaluate the chances that the offender did not present this
racteristic when it was negatively framed.
e used a similar procedure to evaluate the effect of characteristic framing on the
ability word used to convey a given level of numerical probability (Hypothesis 4b).
ever, we used the same numerical probability to qualify either a positively or a
atively framed characteristic and we asked participants to complete a sentence using
r own preferred probability word. The two claims used were:
There is an 80% probability that the offender will (will not) have a pre-conviction for
violence.
There is a 60% probability that the offender will (will not) live within one mile of the
crime scene.
right # 2008 John Wiley & Sons, Ltd. Appl. Cognit. Psychol. (2008)
DOI: 10.1002/acp
asked to choose a word to convey the probability that the offender did present the target
characteristic for positively framed characteristics and a word to convey the probability
that the offender did not present it for negatively framed ones.
The effect of claim framing on the numerical interpretation of verbal probabilities
(Hypothesis 5a) was tested by presenting the following two claims qualified by opposite
probability words and opposite characteristic framings:
Interpretation of profiling claimsTable 3. Summary suspect information presented to test for Hypothesis 6
Suspect no. Age Marital status Occupation
1 28 Single Unemployed
2 21 Co-habiting Soldier
3 41 Divorced Construction worker
4 44 Married Lorry driverasked
the oCopto rank order the suspects from the most likely to the least likely to have committed
ffence.status and the occupation of four suspects (see Table 3). In each case, participants were(i) There is a 10% probability (a 75% probability) that the offender is a construction
worker.
(j) There is a 25% probability (a 70% probability) that the offender is aged between 18
and 25 years.
(k) There is a 20% probability (a 80% probability) that the offender will be single.
Each claim was presented with the same summary information about the age, the maritalwhicyability (Hypothesis 6). To this end, we presented the following three different claims,
h were either qualified with a low or a high probability:offen
probacteristic mentioned in a profiling claim would be judged more likely to be the
der even when the claim that qualified the characteristic yielded a low numerical(g) There is a 70% probability (a 30% probability) that the offender will (will not) have a
short temper and be quite aggressive.
(h) There is an 80% probability (a 20% probability) that the offender will (will not) have
children.
Once again, in both cases, participants were asked to verbally express the chances that
the offender would present the characteristic mentioned using their own preferred choice of
words.
Finally, we wanted to examine whether suspects who presented a behavioural
charchara
enting two identical claims qualified by complementary probability values and
cteristics. The claims used were:word
prescure with women). In a similar vein, the role of claim framing on the choice of verbal
s to qualify a given numerical probability (Hypothesis 5b) was examined bythe c
inseharacteristic mentioned in the claim (e.g. the chances that the offender would be(e) It is probable (rather improbable) that the offender will (will not) collect pornography.
(f) It is likely (rather unlikely) that the offender will (will not) be insecure with women.
In both cases, participants were asked to evaluate the chances that the offender presentedright # 2008 John Wiley & Sons, Ltd. Appl. Cognit. Psychol. (2008)
DOI: 10.1002/acp
likely.
G. Villejoubert et al.The role of perceived base-rate and dangerousness of offender characteristics
Beyond inter-individual variability in the interpretation of verbal probability phrases, we
also expected to demonstrate how contextual features, such as the perceived base-rate of a
given offender profiling claim (Hypothesis 2) and the perceived dangerousness of the claim
(Hypothesis 3) influence interpretation. Both hypotheses were tested using a 3 (probability
word) 2 (high vs. low base-rates/Dangerousness) within-subject Analysis of Variance
(ANOVA). Figure 3 presents the mean probability judgements observed for each
probability word ( probably, possibly and likely) associated with (i) high and low base-rate
levels, and (ii) high and low levels of dangerousness. The probability word used to qualify
the claim had a significant effect on the numerical probability assigned to the claim;
Mprobably¼ 57.96, SE¼ 2.12; Mpossibly¼ 39.26, SE¼ 2.03 and Mlikely¼ 61.86, SE¼ 1.79;
F(2, 136)¼ 80.81, MSE¼ 249.18, h2p ¼ .54, p< .001 for claims with differing base-rates;
Mprobably¼ 60.54, SE¼ 1.98; Mpossibly¼ 37.99, SE¼ 2.11 and Mlikely¼ 66.59, SE¼ 1.70;
F(2, 138)¼ 118.03, MSE¼ 269.55, h2p ¼ .63, p< .001 for claims with differing
dangerousness.
As anticipated in Hypothesis 2, numerical probability judgements were also influenced
by the claims’ perceived base-rates. High base-rate claims systematically led to higher
mean probability judgements; Mhigh¼ 57.08, SE¼ 1.56; Mlow¼ 48.98, SE¼ 2.18; F(1,
68)¼ 19.25, MSE¼ 352.83, h2p ¼ .22, p< .001. Similarly, and in line with Hypothesis 3,
claims related to more dangerous behaviour characteristics were perceived as more likelyRESULTS
Inter-individual variability
We expected to observe considerable variability between individuals in their interpretation
of verbal qualifying profiling claims (Hypothesis 1). Figure 1 presents the membership
functions for each probability phrase used. The error bars (denoting the means standard
deviations) further indicate the extent of the variability in interpretations dependent on the
verbal probability term used. Table 4 presents the membership functions, mean peak,
minimum and maximum values, together with standard deviations and coefficients of
variation. There was little agreement amongst individuals concerning the numerical pro-
bability typically conveyed within a verbal probability phrase. This latter point is further
illustrated by Figure 2, which represents histograms representing the distributions of the
peak values assigned to any given word. These graphs reveal that phrases such as
it suggests are especially ambiguous. The phrase is interpreted by some individuals as
signifying a probability lower than 20%, whilst others interpret this as conveying a
probability greater than 60% similar variations exist for probable, unlikely or quite
unlikely, which are assigned peak values that encompass the whole scale of probabilities.
Greater consensus was found for the phrases somewhat doubtful or improbable, which were
interpreted as denoting probabilities lower than 20% by the majority of individuals.
Similarly, uncertain or possible were generally understood as typically conveying a 50% or
less probability. The phrases likely and very likely were generally understood to conveythan those related to more benign characteristics; Mdangerous¼ 59.06, Mbenign¼ 51.01,
Copyright # 2008 John Wiley & Sons, Ltd. Appl. Cognit. Psychol. (2008)
DOI: 10.1002/acp
characteristics in offender profiling claims based on the judgements of all 69 subjects tested. Points
represent the mean membership for a given probability. Vertical lines depict standard deviations from
the means
Copyright # 2008 John Wiley & Sons, Ltd. Appl. Cognit. Psychol. (2008)
DOI: 10.1002/acp
Interpretation of profiling claims
qualifying behavioural characteristics in offender profiling claims
Table 4. Membership functions statistics for the probability words used to test for Hypothesis 1
Probability word N
Peak Min Max Skew
M SD Cv (%) M SD M SD M SD
Improbable (10) 62 33.1 31.5 95 10.5 16.5 78.5 22.5 0.26 0.52
Somewhat doubtful (5) 62 37.3 29.9 80 9.7 15.5 76.8 22.8 0.15 0.46
Uncertain (3) 64 41.9 26.4 63 14.8 15.6 83.8 20.5 0.03 0.35
Possible (6) 62 49.5 25.6 52 12.3 13.6 85.3 18.2 0.01 0.41
Suggests (1) 63 50.0 37.2 74 14.8 18.6 78.1 26.3 0.03 0.63
Very unlikely (7) 62 60.8 28.2 46 20.3 20.4 92.3 12.5 0.12 0.46
Quite unlikely (9) 62 63.9 24.5 38 23.5 20.3 91.5 14.5 0.16 0.42
Probable (8) 62 68.4 22.2 32 21.8 18.3 94.4 10.2 0.22 0.41
Likely (4) 58 68.8 19.6 28 25.5 19.9 95.3 9.0 0.20 0.40
Very likely (2) 64 73.3 26.5 36 29.1 22.3 95.3 13.8 0.29 0.49
Notes: The words are sorted by their mean peak value. The numbers in brackets refer to the associated claim used
as reported in Table 1.
Copyright # 2008 John Wiley & Sons, Ltd. Appl. Cognit. Psychol. (2008)
DOI: 10.1002/acp
G. Villejoubert et al.
Interpretation of profiling claimsness levels. Error bars depict confidence intervals around the meansF(1, 69)¼ 19.25, MSE¼ 245.24, h2p ¼ .29, p< .001. The interactions effects were not
significant in either analyses; ps> .15.
The role of characteristic framing
We hypothesised that the perceived level of certainty associated with likely or probable
would differ depending on whether these words qualified a positively framed characteristic
(e.g. the offender will have a history of sexually inappropriate behaviour) compared to
when it qualified a negatively framed one (e.g. the offender will not have. . .). The words
likely and probable elicited similar numerical probability judgements; F(1, 68)¼ 0.45,
MSE¼ 230.24, h2p ¼ .01, p¼ .51. In line with Hypothesis 4a, however, the valence of the
behavioural characteristic had a significant effect on mean probability judgements. As
expected, positively framed characteristics led to higher judgements than negatively
framed ones; Mpositive¼ 62.34, Mnegative¼ 54.36, F(1, 68)¼ 17.21, MSE¼ 255.63,
h2p ¼ .20, p< .001.
Similarly, we expected that probability words chosen to communicate a numerical
probability associated with positively framed characteristic would convey greater certainty
than those with a negatively framed characteristic (Hypothesis 4b). A total of 98 different
probability expressions were elicited by the questions associated with claims (c), (d) and
(g), (h) which were used to test for Hypotheses 4b and 5b, respectively (see the ‘Method’
section for a description of these claims). We presented the probability phrases elicited by
our participants to a new set of 20 independent coders and asked them to provide the most
typical probability value representative of the phrases listed. We then used the median
probability value given for each word by the independent coders as proxy for the degree of
Copyright # 2008 John Wiley & Sons, Ltd. Appl. Cognit. Psychol. (2008)
DOI: 10.1002/acp
Previous results revealed that the same probability word or probability value will convey
different levels of certainty depending on whether it is associated with a positively framed
characteristic or a negatively framed one. We also hypothesised that logically equivalent
claims would be seen to convey different levels of certainty, depending on the valence of
the frame used. In order to test Hypothesis 5a, we compared the perceived level of certaintycertainty communicated by participants. The resulting scores were not normally distributed
so we analysed these data with Wilcoxon-signed ranks tests. The tests results and summary
statistics for Hypothesis 4b are presented in the first half of Table 5. As expected, the level
of certainty communicated by words associated to positively framed characteristics was
generally higher than that communicated by words associated with negatively framed
characteristics. This trend, however, was only statistically significant for claim (c) ‘There is
an 80% probability that the offender will (will not) have a pre-conviction for violence’.
Table 5. Wilcoxon-signed ranks test results for Hypotheses 4b and 5b
Claims
Ranks
Positive Negative Ties
N M
P
N M
P
N z
Behavioural characteristic framing (Hypothesis 4b)
(c) 32 18.97 59.00 40 14.75 607.00 23 4.33*
(d) 15 19.03 285.50 16 13.16 210.50 28 0.74
Claim framing (Hypothesis 5b)
(g) 44 25.41 1118.00 3 3.33 10.00 11 5.87*
(h) 49 28.70 1406.50 4 6.13 24.50 8 6.13*
p< .001.based on positive claims (e.g. It is probable that the offender will. . .) to that associated with
negative claims (e.g. It is rather improbable that the offender will not. . .). A printing error
in the questionnaire prevented the use of statements (f) as the characteristic was framed
positively in both claims. Hypothesis 5a was, however, confirmed by comparing
judgements based on the positive framing of claim (e) to those based on its negative frame
equivalent. Thus, the average probability that the offender collected pornography was
judged higher by participants informed that it was probable that the offender would collect
pornography (positively framed claim) compared to participants informed that it was
rather improbable that the offender would not collect pornography (negatively framed
claim); Mpositive¼ 62.77, SD¼ 19.53, Mnegative¼ 47.99, SD¼ 29.40, t(68)¼ 4.15,
d¼ 0.60, p< .001.
Similarly, we hypothesised that probability words chosen to communicate a numerical
probability associated with positively framed claim would convey more certainty than
those chosen to communicate the same probability associated with a negatively framed
claim (Hypothesis 5b). We used the same procedure used to test for Hypothesis 4b to
analyse the degree of certainty communicated by probabilities phrases elicited. The tests
results and summary statistics for Hypothesis 5b are presented in the second half of Table 5.
Copyright # 2008 John Wiley & Sons, Ltd. Appl. Cognit. Psychol. (2008)
DOI: 10.1002/acp
higher level of certainty when based on positive rather than negative claims.
The impact of low-probability statistics on suspect identification
Our last hypothesis concerned the impact of matching behavioural characteristics on
perceived likelihood of guilt. We expected that suspects who presented a behavioural
characteristic mentioned in a profiling claim would be judged more likely to be the
offender even when the claim qualified the characteristic with a low numerical probability
(Hypothesis 6). To test this hypothesis we computed the proportion of positive
identifications (i.e. identifications of a suspect as either most likely or second most likely to
have committed the offence) depending on whether or not a suspect description contained a
characteristic matching the offender’s characteristic as well as the stated probability that
the offender would present this characteristic. A total of 1649 identifications were available
(4 suspects 3 offender characteristics 2 levels of probability for the offender
characteristic 70 subjects—missing data).
Overall, suspects who presented a matching characteristic were positively identified
77.3% (N¼ 415) of the time, whereas suspects who did not were positively identified
Interpretation of profiling claimsFigure 4. Percentage of positive identifications as a function of suspect–offender matching and42.7% (N¼ 1234) of the time. The rate of positive identifications was thus significantly
higher when suspects’ descriptions matched a characteristic of the offender; x2(1,
N¼ 1649)¼ 165.56; p< .001. As expected, the stated probability of the offender
presenting the named characteristic did not have an effect on positive identifications.
Suspects were equally likely to be positively identified when the offender’s characteristic
was associated with a low probability or with a high probability; x2(1, N¼ 1649)¼ 0.005;
p¼ .94. There was, however, an interaction between the probability of the offender
presenting a characteristic and whether or not a suspect’s description matched this
characteristic, as shown in Figure 4. Suspects were always more likely to be positivelyoffender’s characteristic probability. Error bars depict confidence intervals around the percentages
Copyright # 2008 John Wiley & Sons, Ltd. Appl. Cognit. Psychol. (2008)
DOI: 10.1002/acp
base-rate claims or claims relating to a benign characteristic.
G. Villejoubert et al.The results also confirmed that the framing of behavioural characteristics presented in
profile claims also affected the perceived probability of occurrence. The same probability
word was interpreted as denoting a lower level of uncertainty (i.e. a higher probability of
occurring) when referring to the presence of a characteristic in an offender rather than its
absence. This was also true for numerical probabilities: a given numerical level ofidentified when they presented a matching characteristic with the offender but the effect of
suspect–offender description matching was significantly more pronounced when the
offender was said to be highly likely to present the matching characteristic. This result
demonstrates that the matching bias is even stronger for high probability matching
statements.
DISCUSSION
The aim of this research was to examine how probability expressions affect the inter-
pretation of the information provided in offender profiles. Based on an online questionnaire
we were able to demonstrate that uncertain claims about offenders are interpreted
differently by different people. Our results showed that several uncertainty expressions led
to considerable variation. Although participants generally agreed that phrases such as
improbable or somewhat doubtful communicated very low probabilities, phrases such as
uncertain or possible consistently were interpreted to communication probabilities at
around chance level. Expressions such as likely and very likely clearly communicated
higher probabilities above 50% chance, whereas other expressions, such as it suggests were
found very ambiguous and were interpreted as denoting very low probabilities or very high
probabilities. This interpretative diversity may induce an illusion of valid communication,
which could result in misunderstandings between a Behavioural Investigative Advisor’s
intended meaning and the Senior Investigating Officer’s (SIOs) interpretation (Budescu &
Wallsten, 1985). Such opportunities for miscommunication have been shown to be higher
for verbal opposed to numerical risk communication both experimentally (Budescu,
Weinberg, & Wallsten, 1988) and in field (medical) settings (Bryant & Norman, 1980).
However, relying on numerical probability estimates may not always be possible,
especially in an arena, where there is a relative paucity of hard, scientific facts. Therefore,
when uncertainty can only be characterised verbally, BIA’s may wish to focus on the least
ambiguous terms.
Our results also demonstrated how base-rates and perceived dangerousness influenced
the way in which uncertain claims are interpreted. Thus, verbal probability expressions
associated with a high base-rate claim led to higher subjective judgements of the
probability of the claim being true compared to low base-rate claims. Similarly, extending
upon the well-documented severity effect in medical contexts (Bonnefon & Villejoubert,
2006; Weber & Hilton, 1990), our research showed that more dangerous offender
characteristics were perceived as more likely to occur. One strategy to counteract such
effects might be to put greater emphasis on the uncertainty of a claim when it relates to a
high base-rate or highly dangerous characteristic. For example, if a BIA concludes that it is
probable that an offender may be a sexual predator (a dangerous characteristic), they could
report that it is possible in order to prevent SIOs to overestimate the probability of such auncertainty was communicated with probability expressions denoting less uncertainty
Copyright # 2008 John Wiley & Sons, Ltd. Appl. Cognit. Psychol. (2008)
DOI: 10.1002/acp
behavioural characteristic. Another framing effect was observed between logically
equivalent claims either presented in a positive frame or in a negative frame. Thus, a claim
presented in a positive frame (e.g. ‘it is probable that the offender would collect
pornography’) led to higher subjective probabilities for the claim being true than when the
same claim was presented in a negative frame (e.g. ‘it is rather improbable that the offender
would not collect pornography’). Once again, the same effect of claim framing was found
when the original claim was qualified by a numerical probability: probability words
qualifying a claim conveyed higher level of certainty when they were based of positive
claims than when they were based on negative ones. From a general standpoint, these
results both confirm and expand upon previous results by Teigen and Brun (2003), in which
the interpretation of verbal probabilities depended on claim framing in general contexts
and had not systematically distinguished the role of characteristic vs. claim framing on the
uncertainty conveyed by a statement qualified by a probability phrase.
With regards to issues directly relevant to offender profiling, these results suggest that
BIAs may need to consider what is the most appropriate way to frame behavioural
characteristics (as either present or absent) or claims (as either true or not true) when
reporting uncertain claims. Research has shown that where individuals are encouraged to
engage in deeper processing of such information, they are less likely to draw biased
interpretations of risk information in general (Kahneman, 2003; Natter & Berry, 2005). In
particular, framing effects are reduced when recipients are encouraged to think about the
claim in both a positive frame and its negative equivalent (Maule, 1989; Maule &
Villejoubert, 2007). These results suggest that SIO’s interpretations of BIA’s profile claims
will be less influenced by framing effects if such claims are presented both with positive
and negative frames, thereby reducing interpretive error. Likewise, SIOs may consider
reframing claims they read in the opposite frame in order to avoid being erroneously
influenced by the way in which the claim is presented.
Finally, our results have shown that when prioritising suspects based on profile claims,
suspects are more often classified as being either the most likely or the second most likely
to have committed an offence if they present a characteristic that matches the offender
characteristic in the profile claim. Such positive identifications were systematic when
the claim was qualified by a high probability, although were still highly prevalent when the
claim was qualified by a low probability. Individuals tend to rely on a similarity heuristic to
make probability judgements (Kahneman & Frederick, 2002). It is therefore possible that
individuals rely on such heuristics when they judge that the suspect who most resembles
the offender was the most likely to have committed an offence. This is not necessarily an
issue where the probability that the offender presents the characteristic is also high.
However, where the characteristic in the behavioural claim is reported as having a low
probability of occurrence, this could lead to erroneous prioritisation, where suspects
presenting a matching characteristic are given too much priority.
Needless to say, a weakness of the study was the reliance on a non-police sample.
However, previous studies in both lay samples and experts have revealed similar issues
(Maule & Villejoubert, 2007). Thus, whilst important to test the transferability of such
findings operationally, our results are probably indicative of general findings. Moreover,
although advice concerning probable offender characteristics is rarely used in court,
especially in the UK (Alison, Bennell, Mokros, & Ormerod, 2002), similar fact evidence in
behavioural analysis has been used in the courts and is, therefore, actively listened to and
read by members of the jury (Alison, West, & Goodwill, 2004). As such, our results yield
Copyright # 2008 John Wiley & Sons, Ltd. Appl. Cognit. Psychol. (2008)
DOI: 10.1002/acp
G. Villejoubert et al.more accurate inferences based on numerical information, although this intuition is not
necessarily correct (Erev & Cohen, 1990). Therefore, SIO’s preferences for numerical
information as well as the accuracy of their inference made on the basis of verbal and
numerical information could be tested in future experiments.
ACKNOWLEDGEMENTS
This research is part of a larger project funded by the Leverhulme Trust with support by the
National Policing Improvements Agency. The authors are grateful to Lee Rainbow and the
Behavioural Investigative Advisors at the National Policing Improvements Agency for
comments on this study.
REFERENCES
Alison, L., & Crego, J. (2008). Policing critical incidents: Leadership and critical incident
management. Devon: Willan Publishers.
Alison, L., Mclean, C., & Almond, L. (2007). Profiling suspects. In T. Newburn, T. Williamson, & A.
Wright (Eds.), Handbook of criminal investigation. Devon, Cullumpton: Willan Publishers.
Alison, L., West, A., & Goodwill, A. (2004). The academic and the practitioner: Pragmatists’ views
of offender profiling. Psychology, Public Policy and Law, 10, 71–101.
Alison, L., Bennell, C., Mokros, A., & Ormerod, D. (2002). The personality paradox in offender
profiling: A theoretical review of the processes involved in deriving background characteristics
from crime scene actions. Psychology, Public Policy and Law, 8, 115–135.
Almond, L., Alison, L., & Porter, L. (2007). An evaluation and comparison of claims made in
behavioural investigative advice reports compiled by the national policing improvements agencyThe present research offered a comprehensive overview of ways in which uncertain
profiling claims may be misinterpreted. As we mentioned in the introduction, research that
is directly relevant to improve the quality of behavioural investigative advice is scarce. A
major contribution of the present research was to extend results from previous research
examining risk communication in medical and business settings to the domain of
investigative advice. Raising awareness of these issues should assist in improving the
quality of the provision of such advice and the interpretation and utilisation in operationalinteresting prospects for a greater appreciation of how such advice is interpreted beyond
the remit of investigative experts. Indeed, other areas of advice also bear examination,
including the interpretation of covert surveillance and other forms of law enforcement and
military/security intelligence. Alison and Crego (2008), amongst many others have
indicated how high stake decisions might be made based on uncertain information and how,
in many cases, such uncertain information may come from external sources (e.g. from
other agencies, expert advisor reports, source management and handlers etc.). An increased
awareness of interpretative frameworks that may occur across this range of contexts, as
well as research-informed advice on how such reports could be best constructed could
prove invaluable.
CONCLUSIONin the United Kingdom. Journal of Investigative Psychology and Offender Profiling (in press).
Copyright # 2008 John Wiley & Sons, Ltd. Appl. Cognit. Psychol. (2008)
DOI: 10.1002/acp
the severity bias in the interpretation of probability phrases. Psychological Science, 17, 747–751.
Brun, W., & Teigen, K. (1988). Verbal probabilities: Ambigious, context-dependent, or both?
Organizational Behavior and Human Decision Processes, 41, 390–404.
Bryant, G., & Norman, G. (1980). Expressions of probability: Words and numbers. New England
Journal of Medicine, 302, 411–427.
Budescu, D., & Wallsten, T. (1985). Consistency in interpretation of probabilistic phrases. Organ-
izational Behavior and Human Decision Processes, 36, 391–485.
Budescu, D. V., & Wallsten, T. S. (1995). Processing linguistic probabilities: General principles and
empirical evidence. In J. Busemeyer, R. Hastie, & D. L. Medin (Eds.), Decision making from a
cognitive perspective. New York: Academic Press.
Budescu, D., Karelitz, T., & Wallsten, T. (2003). Predicting the directionality of probability words
from their membership functions. Journal of Behavioral Decision Making, 16, 159–180.
Budescu, D., Weinberg, S., & Wallsten, T. (1988). Decisions based on numerically and verbally
expressed uncertainties. Journal of Experimental Psychology: Human Perception and Perform-
ance, 14, 281–294.
Collins, S., & Alison, L. (2002). How certain are offender profilers about the claims they make?
Internal report for the National Crime and Operations Faculty, Hook, Bramshill, UK.
Erev, I., & Cohen, B. L. (1990). Verbal versus numerical probabilities: Efficiency, biases, and the
preference paradox. Organizational Behavior and Human Decision Processes, 45, 1–18.
Evans, J. St. B. T. (1998). Matching bias in conditional reasoning: Do we understand it after 25 years?
Thinking and Reasoning, 4, 45–82.
Kahneman, D., & Frederick, S. (2002). Representativeness revisited: Attribute substitution in
intuitive judgment. In T. Gilovich, D. Griffin, & D. Kahneman (Eds.), Heuristics and biases:
The psychology of intuitive judgment. Cambridge, UK: Cambridge University Press.
Kahneman, D. (2003). A perspective on judgment and choice: Mapping bounded rationality.
American Psychologist, 58, 697–720.
Karelitz, T., & Budescu, D. (2004). You say ‘‘probable’’ and I say ‘‘likely’’: Improving interpersonal
communication with verbal probability phrases. Journal of Experimental Psychology: Applied, 10,
25–41.
Levin, I. P., & Gaeth, G. J. (1988). How consumers are affected by the framing of attribute
information before and after consuming the product. Journal of Consumer Research, 15, 374–378.
Maule, A. J. (1989). Positive and negative decision frames: A verbal protocol analysis of the Asian
Disease problem of Tversky and Kahneman. In H. Montgomery, & O. Svenson (Eds.), Process and
structure in human decision making. Chichester, UK: Wiley.
Maule, A. J., & Villejoubert, G. (2007). What lies beneath: Reframing framing effects. Thinking and
Reasoning, 13, 25–44.
Moxey, L. M., & Sanford, A. J. (2000). Communicating quantities: A review of psycholinguistic
evidence of how expressions determine perspectives. Applied Cognitive Psychology, 14, 237–255.
Natter, H. N., & Berry, D. C. (2005). Effects of active information processing on the understanding of
risk information. Applied Cognitive Psychology, 19, 123–135.
Reagan, R., Mosteller, F., & Youtz, C. (1989). Quantitative meanings of verbal probability
expressions. Journal of Applied Psychology, 74, 433–442.
Russo, J., & Schomaker, P. (1989). Decision traps. New York: Schuster & Simon.
Teigen, K., & Brun, W. (1995). Yes, but it is uncertain: Direction and communicative intention of
verbal probabilistic terms. Acta Psychologica, 88, 233–258.
Teigen, K., & Brun, W. (2003). Verbal probabilities: A question of frame? Journal of Behavioural
Decision Making, 16, 53–72.
Wallsten, T., Budescu, D., Rapoport, A., Zwick, R., & Forsyth, B. (1986). Measuring the vague
meanings of probability terms. Journal of Experimental Psychology: General, 115, 348–365.
Wallsten, T., Fillenbaum, S., & Cox, J. (1986). Base rate effects on the interpretation of probability
and frequency expressions. Journal of Memory and Language, 25, 571–587.
Weber, E., & Hilton, D. (1990). Contextual effects in the interpretation of probability words:
Perceived base rate and severity of events. Journal of Experimental Psychology: Human Percep-
tion and Performance, 16, 781–789.Copyright # 2008 John Wiley & Sons, Ltd. Appl. Cognit. Psychol. (2008)
DOI: 10.1002/acp
Sign up today - FREE
Mendeley saves you time finding and organizing research. Learn more
- All your research in one place
- Add and import papers easily
- Access it anywhere, anytime


