The habit of abstract pursuits makes learned men much infe-
rior to the average in the power of visualization, and much more
exclusively occupied with words in their “thinking.”
Bertrand Russell (1919b)
1. Introduction
For the last several decades, the fields of cognition and per-
ception have diverged. Researchers in these two areas know
ever less about each other’s work, and their discoveries have
had diminishing influence on each other. In many universi-
ties, researchers in these two areas are in different programs,
and sometimes in different departments, buildings, and uni-
versity divisions. One might conclude from this lack of con-
tact that perception and cognition reflect independent or
modular systems in the brain. Perceptual systems pick up in-
formation from the environment and pass it on to separate
systems that support the various cognitive functions, such as
language, memory, and thought. I will argue that this view is
fundamentally wrong. Instead, cognition is inherently per-
ceptual, sharing systems with perception at both the cogni-
tive and the neural levels. I will further suggest that the di-
vergence between cognition and perception reflects the
widespread assumption that cognitive representations are in-
herently nonperceptual, or what I will call amodal.
1.1. Grounding cognition in perception
In contrast to modern views, it is relatively straightforward
to imagine how cognition could be inherently perceptual.
As Figure 1 illustrates, this view begins by assuming that
perceptual states arise in sensory-motor systems. As dis-
cussed in more detail later (sect. 2.1), a perceptual state can
contain two components: an unconscious neural represen-
tation of physical input, and an optional conscious experi-
ence. Once a perceptual state arises, a subset of it is ex-
tracted via selective attention and stored permanently in
BEHAVIORAL AND BRAIN SCIENCES (1999) 22, 577–660
Printed in the United States of America
© 1999 Cambridge University Press 0140-525X/99 $12.50 577
Perceptual symbol systems
Lawrence W. Barsalou
Department of Psychology, Emory University, Atlanta, GA 30322
barsalou@emory.edu userwww.service.emory.edu/~barsalou/
Abstract: Prior to the twentieth century, theories of knowledge were inherently perceptual. Since then, developments in logic, statis-
tics, and programming languages have inspired amodal theories that rest on principles fundamentally different from those underlying
perception. In addition, perceptual approaches have become widely viewed as untenable because they are assumed to implement record-
ing systems, not conceptual systems. A perceptual theory of knowledge is developed here in the context of current cognitive science and
neuroscience. During perceptual experience, association areas in the brain capture bottom-up patterns of activation in sensory-motor
areas. Later, in a top-down manner, association areas partially reactivate sensory-motor areas to implement perceptual symbols. The stor-
age and reactivation of perceptual symbols operates at the level of perceptual components – not at the level of holistic perceptual expe-
riences. Through the use of selective attention, schematic representations of perceptual components are extracted from experience and
stored in memory (e.g., individual memories of green, purr, hot). As memories of the same component become organized around a com-
mon frame, they implement a simulator that produces limitless simulations of the component (e.g., simulations of purr). Not only do
such simulators develop for aspects of sensory experience, they also develop for aspects of proprioception (e.g., lift, run) and introspec-
tion (e.g., compare, memory, happy, hungry). Once established, these simulators implement a basic conceptual system that represents
types, supports categorization, and produces categorical inferences. These simulators further support productivity, propositions, and ab-
stract concepts, thereby implementing a fully functional conceptual system. Productivity results from integrating simulators combinato-
rially and recursively to produce complex simulations. Propositions result from binding simulators to perceived individuals to represent
type-token relations. Abstract concepts are grounded in complex simulations of combined physical and introspective events. Thus, a per-
ceptual theory of knowledge can implement a fully functional conceptual system while avoiding problems associated with amodal sym-
bol systems. Implications for cognition, neuroscience, evolution, development, and artificial intelligence are explored.
Keywords: analogue processing; categories; concepts; frames; imagery; images; knowledge; perception; representation; sensory-motor
representations; simulation; symbol grounding; symbol systems
Lawrence Barsalou is Professor
of Psychology at Emory University,
having previously held positions at
the Georgia Institute of Technology
and the University of Chicago. He is
the author of over 50 scientific pub-
lications in cognitive psychology and
cognitive science. His research ad-
dresses the nature of human knowl-
edge and its roles in language, memory, and thought,
focusing specifically on the perceptual bases of knowl-
edge, the structural properties of concepts, the dynamic
representation of concepts, the construction of cate-
gories to achieve goals, and the situated character of
conceptualizations. He is a Fellow of the American Psy-
chological Society and of the American Psychological
Association, he served as Associate Editor for the Jour-
nal of Experimental Psychology: Learning, Memory,
and Cognition, and he is currently on the governing
board of the Cognitive Science Society.

long-term memory. On later retrievals, this perceptual
memory can function symbolically, standing for referents in
the world, and entering into symbol manipulation. As col-
lections of perceptual symbols develop, they constitute the
representations that underlie cognition.
Perceptual symbols are modal and analogical. They are
modal because they are represented in the same systems as
the perceptual states that produced them. The neural sys-
tems that represent color in perception, for example, also
represent the colors of objects in perceptual symbols, at
least to a significant extent. On this view, a common repre-
sentational system underlies perception and cognition, not
independent systems. Because perceptual symbols are
modal, they are also analogical. The structure of a percep-
tual symbol corresponds, at least somewhat, to the percep-
tual state that produced it.1
Given how reasonable this perceptually based view of
cognition might seem, why has it not enjoyed widespread
acceptance? Why is it not in serious contention as a theory
of representation? Actually, this view dominated theories
of mind for most of recorded history. For more than
2,000 years, theorists viewed higher cognition as inherently
perceptual. Since Aristotle (4th century BC/1961) and Epi-
curus (4th century BC/1994), theorists saw the representa-
tions that underlie cognition as imagistic. British empiri-
cists such as Locke (1690/1959), Berkeley (1710/1982), and
Hume (1739/1978) certainly viewed cognition in this man-
ner. Images likewise played a central role in the theories of
later nativists such as Kant (1787/1965) and Reid (1764/
1970; 1785/1969). Even recent philosophers such as Rus-
sell (1919b) and Price (1953) have incorporated images
centrally into their theories. Until the early twentieth cen-
tury, nearly all theorists assumed that knowledge had a
strong perceptual character.
After being widely accepted for two millennia, this view
withered with mentalism in the early twentieth century. At
that time, behaviorists and ordinary language philosophers
successfully banished mental states from consideration in
much of the scientific community, arguing that they were
unscientific and led to confused views of human nature
(e.g., Ryle 1949; Watson 1913; Wittgenstein 1953). Because
perceptual theories of mind had dominated mentalism to
that point, attacks on mentalism often included a critique
of images. The goal of these attacks was not to exclude im-
ages from mentalism, however, but to eliminate mentalism
altogether. As a result, image-based theories of cognition
disappeared with theories of cognition.
1.2. Amodal symbol systems
Following the cognitive revolution in the mid-twentieth
century, theorists developed radically new approaches to
representation. In contrast to pre-twentieth century think-
ing, modern cognitive scientists began working with repre-
sentational schemes that were inherently nonperceptual.
To a large extent, this shift reflected major developments
outside cognitive science in logic, statistics, and computer
science. Formalisms such as predicate calculus, probability
theory, and programming languages became widely known
and inspired technical developments everywhere. In cog-
nitive science, they inspired many new representational
languages, most of which are still in widespread use today
(e.g., feature lists, frames, schemata, semantic nets, proce-
dural semantics, production systems, connectionism).
These new representational schemes differed from ear-
lier ones in their relation to perception. Whereas earlier
schemes assumed that cognitive representations utilize
perceptual representations (Fig. 1), the newer schemes as-
sumed that cognitive and perceptual representations con-
stitute separate systems that work according to different
principles. Figure 2 illustrates this assumption. As in the
framework for perceptual symbol systems in Figure 1, per-
ceptual states arise in sensory-motor systems. However, the
next step differs critically. Rather than extracting a subset
of a perceptual state and storing it for later use as a symbol,
an amodal symbol system transduces a subset of a percep-
tual state into a completely new representation language
that is inherently nonperceptual.
As amodal symbols become transduced from perceptual
states, they enter into larger representational structures,
such as feature lists, frames, schemata, semantic networks,
and production systems. These structures in turn constitute
a fully functional symbolic system with a combinatorial syn-
tax and semantics, which supports all of the higher cogni-
tive functions, including memory, knowledge, language,
and thought. For general treatments of this approach, see
Dennett (1969), Newell and Simon (1972), Fodor (1975),
Pylyshyn (1984), and Haugeland (1985). For reviews of spe-
cific theories in psychology, see E. Smith and Medin (1981),
Rumelhart and Norman (1988), and Barsalou and Hale
(1993).
It is essential to see that the symbols in these systems are
amodal and arbitrary. They are amodal because their inter-
nal structures bear no correspondence to the perceptual
states that produced them. The amodal symbols that rep-
resent the colors of objects in their absence reside in a dif-
ferent neural system from the representations of these col-
ors during perception itself. In addition, these two systems
use different representational schemes and operate ac-
cording to different principles.
Because the symbols in these symbol systems are
amodal, they are linked arbitrarily to the perceptual states
that produce them. Similarly to how words typically have
arbitrary relations to entities in the world, amodal symbols
have arbitrary relations to perceptual states. Just as the
word “chair” has no systematic similarity to physical chairs,
the amodal symbol for chair has no systematic similarity to
Barsalou: Perceptual symbol systems
578 BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4
Figure 1. The basic assumption underlying perceptual symbol
systems: Subsets of perceptual states in sensory-motor systems are
extracted and stored in long-term memory to function as symbols.
As a result, the internal structure of these symbols is modal, and
they are analogically related to the perceptual states that produced
them.

perceived chairs. As a consequence, similarities between
amodal symbols are not related systematically to similarities
between their perceptual states, which is again analogous to
how similarities between words are not related systemati-
cally to similarities between their referents. Just as the
words “blue” and “green” are not necessarily more similar
than the words “blue” and “red,” the amodal symbols for
blue and green are not necessarily more similar than the
amodal symbols for blue and red.2
Amodal symbols bear an important relation to words and
language. Theorists typically use linguistic forms to repre-
sent amodal symbols. In feature lists, words represent fea-
tures, as in:
CHAIR (1)
seat
back
legs
Similarly in schemata, frames, and predicate calculus ex-
pressions, words represent relations, arguments, and val-
ues, as in:
EAT (2)
Agent 5 horse
Object 5 hay
Although theorists generally assume that words do not lit-
erally constitute the content of these representations, it is
assumed that close amodal counterparts of words do. Al-
though the word “horse” does not represent the value of
Agent for EAT in (2), an amodal symbol that closely paral-
lels this word does. Thus, symbolic thought is assumed to
be analogous in many important ways to language. Just as
language processing involves the sequential processing of
words in a sentence, so conceptual processing is assumed to
involve the sequential processing of amodal symbols in list-
like or sentence-like structures (e.g., Fodor & Pylyshyn
1988).
It is important to see that this emphasis on amodal and
arbitrary symbols also exists in some, but not all, connec-
tionist schemes for representing knowledge (e.g., McClel-
land et al. 1986; Rumelhart et al. 1986). Consider a feed-
forward network with back propagation. The input units in
the first layer constitute a simple perceptual system that
codes the perceived features of presented entities. In con-
trast, the internal layer of hidden units is often interpreted
as a simple conceptual system, with a pattern of activation
providing the conceptual representation of an input pat-
tern. Most importantly, the relation between a conceptual
representation and its perceptual input is arbitrary for tech-
nical reasons. Prior to learning, the starting weights on the
connections between the input units and the hidden units
are set to small random values (if the values were all 0, the
system could not learn). As a result, the conceptual repre-
sentations that develop through learning are related arbi-
trarily to the perceptual states that activate them. With dif-
ferent starting weights, arbitrarily different conceptual
states correspond to the same perceptual states. Even
though connectionist schemes for representation differ in
important ways from more traditional schemes, they often
share the critical assumption that cognitive representations
are amodal and arbitrary.
Connectionist representational schemes need not neces-
sarily work this way. If the same associative network repre-
sents information in both perception and cognition, it
grounds knowledge in perception and is not amodal (e.g.,
Pulvermüller 1999). As described later (sects. 2.2.1, 2.5),
shared associative networks provide a natural way to view
the representation of perceptual symbols.
1.2.1. Strengths. Amodal symbol systems have many pow-
erful and important properties that any fully functional con-
ceptual system must exhibit. These include the ability to
represent types and tokens, to produce categorical infer-
ences, to combine symbols productively, to represent
propositions, and to represent abstract concepts. Amodal
symbol systems have played the critical role of making these
properties central to theories of human cognition, making
it clear that any viable theory must account for them.
1.2.2. Problems. It has been less widely acknowledged that
amodal symbol systems face many unresolved problems.
First, there is little direct empirical evidence that amodal
symbols exist. Using picture and word processing tasks,
some researchers have explicitly tested the hypothesis that
conceptual symbols are amodal (e.g., Snodgrass 1984;
Theios & Amhrein 1989). However, a comprehensive re-
view of this work concluded that conceptual symbols have
a perceptual character (Glaser 1992; also see Seifert 1997).
More recently, researchers have suggested that amodal vec-
tors derived from linguistic context underlie semantic pro-
cessing (Burgess & Lund 1997; Landauer & Dumais 1997).
However, Glenberg et al. (1998b) provide strong evidence
against these views, suggesting instead that affordances de-
rived from sensory-motor simulations are essential to se-
mantic processing.
Findings from neuroscience also challenge amodal sym-
bols. Much research has established that categorical knowl-
edge is grounded in sensory-motor regions of the brain (for
reviews see Damasio 1989; Gainotti et al. 1995; Pulver-
müller 1999; also see sect. 2.3). Damage to a particular sen-
sory-motor region disrupts the conceptual processing of
categories that use this region to perceive physical exem-
plars. For example, damage to the visual system disrupts
the conceptual processing of categories whose exemplars
are primarily processed visually, such as birds. These find-
ings strongly suggest that categorical knowledge is not
amodal.3
Barsalou: Perceptual symbol systems
BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4 579
Figure 2. The basic assumption underlying amodal symbol sys-
tems: Perceptual states are transduced into a completely new rep-
resentational system that describes these states amodally. As a re-
sult, the internal structure of these symbols is unrelated to the
perceptual states that produced them, with conventional associa-
tions establishing reference instead.

In general, the primary evidence for amodal symbols is
indirect. Because amodal symbols can implement concep-
tual systems, they receive indirect support through their in-
strumental roles in these accounts. Notably, however,
amodal symbols have not fared well in implementing all
computational functions. In particular, they have encoun-
tered difficulty in representing spatio-temporal knowledge,
because the computational systems that result are cumber-
some, brittle, and intractable (e.g., Clark 1997; Glasgow
1993; McDermott 1987; Winograd & Flores 1987). Al-
though amodal symbol systems do implement some com-
putational functions elegantly and naturally, their inade-
quacies in implementing others are not encouraging.
Another shortcoming of amodal symbol systems is their
failure to provide a satisfactory account of the transduction
process that maps perceptual states into amodal symbols
(Fig. 2). The lack of an account for such a critical process
should give one pause in adopting this general framework.
If we cannot explain how these symbols arise in the cogni-
tive system, why should we be confident that they exist?
Perhaps even more serious is the complete lack of cognitive
and neural evidence that such a transduction process actu-
ally exists in the brain.
A related shortcoming is the symbol grounding problem
(Harnad 1987; 1990; Newton 1996; Searle 1980), which is
the converse of the transduction problem. Just as we have
no account of how perceptual states become mapped to
amodal symbols during transduction, neither do we have an
account of how amodal symbols become mapped back to
perceptual states and entities in the world. Although
amodal theories often stress the importance of symbol in-
terpretation, they fail to provide compelling accounts of the
interpretive scheme that guides reference. Without such an
account, we should again have misgivings about the viabil-
ity of this approach.4
A related problem concerns how an amodal system im-
plements comprehension in the absence of physical ref-
erents. Imagine that amodal symbols are manipulated to
envision a future event. If nothing in the perceived
environment grounds these symbols, how does the system
understand its reasoning? Because the processing of
amodal symbols is usually assumed to be entirely syntactic
(based on form and not meaning), how could such a system
have any sense of what its computations are about? It is of-
ten argued that amodal symbols acquire meaning from as-
sociated symbols, but without ultimately grounding termi-
nal symbols, the problem remains unsolved. Certainly
people have the experience of comprehension in such situ-
ations.
One solution is to postulate mediating perceptual repre-
sentations (e.g., Harnad 1987; Höffding 1891; Neisser
1967). According to this account, every amodal symbol is as-
sociated with corresponding perceptual states in long-term
memory. For example, the amodal symbol for dog is associ-
ated with perceptual memories of dogs. During transduc-
tion, the perception of a dog activates these perceptual
memories, which activate the amodal symbol for dog. Dur-
ing symbol grounding, the activation of the amodal symbol
in turn activates associated perceptual memories, which
ground comprehension. Problematically, though, percep-
tual memories are doing all of the work, and the amodal
symbols are redundant. Why couldn’t the system simply use
its perceptual representations of dogs alone to represent
dog, both during categorization and reasoning?
The obvious response from the amodal perspective is
that amodal symbols perform additional work that these
perceptual representations cannot perform. As we shall see,
however, perceptual representations can play the critical
symbolic functions that amodal symbols play in traditional
systems, so that amodal symbols become redundant. If we
have no direct evidence for amodal symbols, as noted ear-
lier, then why postulate them?
Finally, amodal symbol systems are too powerful. They
can explain any finding post hoc (Anderson 1978), but of-
ten without providing much illumination. Besides being
unfalsifiable, these systems often fail to make strong a pri-
ori predictions about cognitive phenomena, especially
those of a perceptual nature. For example, amodal theories
do not naturally predict distance and orientation effects in
scanning and rotation (Finke 1989; Kosslyn 1980), although
they can explain them post hoc. Such accounts are not par-
ticularly impressive, though, because they are uncon-
strained and offer little insight into the phenomena.
1.2.3. Theory evaluation. Much has been made about the
ability of amodal theories to explain any imagery phenom-
enon (e.g., Anderson 1978). However, this ability must be
put into perspective. If perceptual theories predict these ef-
fects a priori, whereas amodal theories explain them post
hoc, why should this be viewed as a tie? From the perspec-
tive of inferential statistics, Bayesian reasoning, and philos-
ophy of science, post hoc accounts should be viewed with
great caution. If a priori prediction is favored over post hoc
prediction in these other areas, why should it not be favored
here? Clearly, greater credence must be given to a theory
whose falsifiable, a priori predictions are supported than to
a theory that does not predict these findings a priori, and
that accounts for them post hoc only because of its unfalsi-
fiable explanatory power.
Furthermore, the assessment of scientific theories de-
pends on many other factors besides the ability to fit data.
As philosophers of science often note, theories must also be
evaluated on falsifiability, parsimony, the ability to produce
provocative hypotheses that push a science forward, the ex-
istence of direct evidence for their constructs, freedom
from conceptual problems in their apparatus, and integra-
bility with theory in neighboring fields. As we have seen,
amodal theories suffer problems in all these regards. They
are unfalsifiable, they are not parsimonious, they lack direct
support, they suffer conceptual problems such as transduc-
tion and symbol grounding, and it is not clear how to inte-
grate them with theory in neighboring fields, such as per-
ception and neuroscience. For all of these reasons, we
should view amodal theories with caution and skepticism,
and we should be open to alternatives.
1.3. The current status of perceptual symbol systems
The reemergence of cognition in the mid-twentieth cen-
tury did not bring a reemergence of perceptually based cog-
nition. As we have seen, representational schemes moved
in a nonperceptual direction. Furthermore, theorists were
initially hostile to imbuing modern cognitive theories with
any perceptual character whatsoever. When Shepard and
Metzler (1971) offered initial evidence for image-like rep-
resentations in working memory (not long-term memory!),
they encountered considerable resistance (e.g., Anderson
1978; Pylyshyn 1973; 1981). [See also Pylyshyn: “Computa-
Barsalou: Perceptual symbol systems
580 BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4

tional Models and Empirical Constraints” BBS 1(1) 1978;
“Computation and Cognition” BBS 3(1) 1980; “Is Vision
Continuous with Cognition?” BBS 22(3) 1999.] When
Kosslyn (1980) presented his theory of imagery, he argued
adamantly that permanent representations in long-term
memory are amodal, with perceptual images existing only
temporarily in working memory (see also Kosslyn 1976).
The reasons for this resistance are not entirely clear. One
factor could be lingering paranoia arising from the attacks
of behaviorists and ordinary language philosophers. An-
other factor could be more recent criticisms of imagery in
philosophy, some of which will be addressed later (e.g.,
Dennett 1969; Fodor 1975; Geach 1957). Perhaps the most
serious factor has been uncharitable characterizations of
perceptual cognition that fail to appreciate its potential.
Critics often base their attacks on weak formulations of the
perceptual approach and underestimate earlier theorists.
As a result, perceptual theories of knowledge are widely
misunderstood.
Consider some of the more common misunderstandings:
perceptual theories of knowledge are generally believed to
contain holistic representations instead of componential
representations that exhibit productivity. These theories
are widely believed to contain only conscious mental im-
ages, not unconscious representations. The representations
in these theories are often assumed to arise only in the sen-
sory modalities, not in other modalities of experience, such
as proprioception and introspection. These theories are
typically viewed as containing only static representations,
not dynamic ones. These theories are generally construed
as failing to support the propositions that underlie descrip-
tion and interpretation. And these theories are often as-
sumed to include only empirical collections of sense data,
not genetically constrained mechanisms.
Careful readings of earlier thinkers, however, indicate
that perceptual theories of knowledge often go consider-
ably beyond this simplistic stereotype. Many philosophers,
for example, have assumed that perceptual representations
are componential and produce representations produc-
tively (e.g., Locke, Russell, Price). Many have assumed that
unconscious representations, then referred to as “disposi-
tions” and “schemata,” produce conscious images (e.g.,
Locke, Kant, Reid, Price). Many have assumed that images
can reflect nonsensory experience, most importantly intro-
spection or “reflection” (e.g., Locke, Hume, Kant, Reid).
Many have assumed that images can support the type-
token mappings that underlie propositions (e.g., Locke,
Reid, Russell, Price). Many have assumed that native mech-
anisms interpret and organize images (e.g., Kant, Reid). All
have assumed that images can be dynamic, not just static,
representing events as well as snapshots of time.
As these examples suggest, perceptual theories of knowl-
edge should be judged on the basis of their strongest mem-
bers, not their weakest. My intent here is to develop a pow-
erful theory of perceptual symbols in the contexts of
cognitive science and neuroscience. As we shall see, this
type of theory can exhibit the strengths of amodal symbol
systems while avoiding their problems.
More and more researchers are developing perceptual
theories of cognition. In linguistics, cognitive linguists have
made the perceptual character of knowledge a central as-
sumption of their theories (e.g., Fauconnier 1985; 1997;
Jackendoff 1987; Johnson 1987; Lakoff 1987; 1988; Lakoff
& Johnson 1980; Lakoff & Turner 1989; Langacker 1986;
1987; 1991; 1997; Sweetser 1990; Talmy 1983; 1988; Turner
1996). In psychology, these researchers include Paivio
(1971; 1986), Miller and Johnson-Laird (1976), Hutten-
locher (1973; 1976), Shannon (1987), J. Mandler (1992),
Tomasello (1992), L. Smith (L. Smith & Heise 1992; L.
Smith et al. 1992; Jones & L. Smith 1993), Gibbs (1994),
Glenberg (1997), Goldstone (Goldstone 1994; Goldstone
& Barsalou 1998), Wu (1995), Solomon (1997), Mac-
Whinney (1998), and myself (Barsalou 1993; Barsalou &
Prinz 1997; Barsalou et al. 1993; in press). In philosophy,
these researchers include Barwise and Etchemendy (1990;
1991), Nersessian (1992), Peacocke (1992), Thagard
(1992), Davies and Stone (1995), Heal (1996), Newton
(1996), Clark (1997), and Prinz (1997; Prinz & Barsalou, in
press a). In artificial intelligence, Glasgow (1993) has shown
that perceptual representations can increase computational
power substantially, and other researchers are grounding
machine symbols in sensory-motor events (e.g., Bailey et al.
1997; Cohen et al. 1997; Rosenstein & Cohen 1998). Many
additional researchers have considered the role of percep-
tual representations in imagery (e.g., Farah 1995; Finke
1989; Kosslyn 1994; Shepard & Cooper 1982; Tye 1991),
but the focus here is on perceptual representations in long-
term knowledge.
1.4. Recording systems versus conceptual systems
It is widely believed that perceptually based theories of
knowledge do not have sufficient expressive power to im-
plement a fully functional conceptual system. As described
earlier (sect. 1.2.1), a fully functional conceptual system
represents both types and tokens, it produces categorical
inferences, it combines symbols productively to produce
limitless conceptual structures, it produces propositions by
binding types to tokens, and it represents abstract concepts.
The primary purpose of this target article is to demonstrate
that perceptual symbol systems can implement these func-
tions naturally and powerfully.
The distinction between a recording system and a con-
ceptual system is central to this task (Dretske 1995; Hauge-
land 1991). Perceptually based theories of knowledge are
typically construed as recording systems. A recording sys-
tem captures physical information by creating attenuated
(not exact) copies of it, as exemplified by photographs,
videotapes, and audiotapes. Notably, a recording system
does not interpret what each part of a recording contains –
it simply creates an attenuated copy. For example, a photo
of a picnic simply records the light present at each point in
the scene without interpreting the types of entities present.
In contrast, a conceptual system interprets the entities in
a recording. In perceiving a picnic, the human conceptual
system might construe perceived individuals as instances of
tree, table, watermelon, eat, above, and so forth. To accom-
plish this, the conceptual system binds specific tokens in
perception (i.e., individuals) to knowledge for general types
of things in memory (i.e., concepts). Clearly, a system that
only records perceptual experience cannot construe indi-
viduals in this manner – it only records them in the holistic
context of an undifferentiated event.
A conceptual system has other properties as well. First,
it is inferential, allowing the cognitive system to go beyond
perceptual input. Theorists have argued for years that the
primary purpose of concepts is to provide categorical infer-
ences about perceived individuals. Again, this is not some-
Barsalou: Perceptual symbol systems
BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4 581

thing that recording systems accomplish. How does a photo
of a dog go beyond what it records to provide inferences
about the individual present? Second, a conceptual system
is productive in the sense of being able to construct com-
plex concepts from simpler ones. This, too, is not something
possible with recording systems. How could a photo of
some snow combine with a photo of a ball to form the con-
cept of snowball? Third, a conceptual system supports the
formulation of propositions, where a proposition results
from binding a concept (type) to an individual (token) in a
manner that is true or false. Again, this is something that
lies beyond the power of recording systems. How does a
photo of a dog implement a binding between a concept and
an individual?
As long as perceptually based theories of knowledge are
viewed as recording systems, they will never be plausible,
much less competitive. To become plausible and competi-
tive, a perceptually based theory of knowledge must exhibit
the properties of a conceptual system. The primary purpose
of this target article is to demonstrate that this is possible.
Of course, it is important to provide empirical support for
such a theory as well. Various sources of empirical evidence
will be offered throughout the paper, especially in section
4, and further reports of empirical support are forthcoming
(Barsalou et al., in press; Solomon & Barsalou 1999a;
1999b; Wu & Barsalou 1999). However, the primary sup-
port here will be of a theoretical nature. Because so few
theorists currently believe that a perceptually based theory
of knowledge could possibly have the requisite theoretical
properties, it is essential to demonstrate that it can. Once
this has been established, an empirical case can follow.
1.5. Overview
The remainder of this paper presents a theory of percep-
tual symbols. Section 2 presents six core properties that im-
plement a basic conceptual system: perceptual symbols are
neural representations in sensory-motor areas of the brain
(sect. 2.1); they represent schematic components of per-
ceptual experience, not entire holistic experiences (sect.
2.2); they are multimodal, arising across the sensory modal-
ities, proprioception, and introspection (sect. 2.3). Related
perceptual symbols become integrated into a simulator that
produces limitless simulations of a perceptual component
(e.g., red, lift, hungry, sect. 2.4). Frames organize the per-
ceptual symbols within a simulator (sect. 2.5), and words as-
sociated with simulators provide linguistic control over the
construction of simulations (sect. 2.6).
Section 3 presents four further properties, derived from
the six core properties, that implement a fully functional
conceptual system: simulators can be combined combina-
torially and recursively to implement productivity (sect.
3.1); they can become bound to perceived individuals to im-
plement propositions (sect. 3.2). Because perceptual sym-
bols reside in sensory-motor systems, they implement vari-
able embodiment, not functionalism (sect. 3.3). Using
complex simulations of combined physical and introspec-
tive events, perceptual symbol systems represent abstract
concepts (sect. 3.4).
Section 4 sketches implications of this approach. Viewing
knowledge as grounded in sensory-motor areas changes
how we think about basic cognitive processes, including
categorization, concepts, attention, working memory, long-
term memory, language, problem solving, decision making,
skill, reasoning, and formal symbol manipulation (sect. 4.1).
This approach also has implications for evolution and de-
velopment (sect. 4.2), neuroscience (sect. 4.3), and artificial
intelligence (sect. 4.4).
2. Core properties
The properties of this theory will not be characterized for-
mally, nor will they be grounded in specific neural mecha-
nisms. Instead, this formulation of the theory should be
viewed as a high-level functional account of how the brain
could implement a conceptual system using sensory-motor
mechanisms. Once the possibility of such an account has
been established, later work can develop computational im-
plementations and ground them more precisely in neural
systems.
Because this target article focuses on the high level ar-
chitecture of perceptual symbol systems, it leaves many de-
tails unspecified. The theory does not specify the features
of perception, or why attention focuses on some features
but not others. The theory does not address how the cogni-
tive system divides the world into categories, or how ab-
straction processes establish categorical knowledge. The
theory does not explain how the fit between one represen-
tation and another is computed, or how constraints control
the combination of concepts. Notably, these issues remain
largely unresolved in all theories of knowledge – not just
perceptual symbol systems – thereby constituting some of
the field’s significant challenges. To provide these missing
aspects of the theory would exceed the scope of this article,
both in length and ambition. Instead, the goal is to formu-
late the high-level architecture of perceptual symbol sys-
tems, which may well provide leverage in resolving these
other issues. From here on, footnotes indicate critical as-
pects of the theory that remain to be developed.
Finally, this target article proposes a theory of knowl-
edge, not a theory of perception. Although the theory relies
heavily on perception, it remains largely agnostic about the
nature of perceptual mechanisms. Instead, the critical
claim is that whatever mechanisms happen to underlie per-
ception, an important subset will underlie knowledge as
well.
2.1. Neural representations in sensory-motor systems
Perceptual symbols are not like physical pictures; nor are
they mental images or any other form of conscious subjec-
tive experience. As natural and traditional as it is to think of
perceptual symbols in these ways, this is not the form they
take here. Instead, they are records of the neural states that
underlie perception. During perception, systems of neu-
rons in sensory-motor regions of the brain capture infor-
mation about perceived events in the environment and in
the body. At this level of perceptual analysis, the informa-
tion represented is relatively qualitative and functional
(e.g., the presence or absence of edges, vertices, colors, spa-
tial relations, movements, pain, heat). The neuroscience lit-
erature on sensory-motor systems is replete with accounts
of this neural architecture (e.g., Bear et al. 1996; Gazzaniga
et al. 1998; Zeki 1993). There is little doubt that the brain
uses active configurations of neurons to represent the prop-
erties of perceived entities and events.
This basic premise of modern perceptual theory under-
Barsalou: Perceptual symbol systems
582 BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4

lies the present theory of perceptual symbol systems: a per-
ceptual symbol is a record of the neural activation that
arises during perception. Essentially the same assumption
also underlies much current work in imagery: common
neural systems underlie imagery and perception (e.g.,
Crammond 1997; Deschaumes-Molinaro et al. 1992; Farah
1995; Jeannerod 1994; 1995; Kosslyn 1994; Zatorre et al.
1996). The proposal here is stronger, however, further as-
suming that the neural systems common to imagery and
perception underlie conceptual knowledge as well.
This claim by no means implies that identical systems un-
derlie perception, imagery, and knowledge. Obviously, they
must differ in important ways. For example, Damasio
(1989) suggests that convergence zones integrate informa-
tion in sensory-motor maps to represent knowledge. More
generally, associative areas throughout the brain appear to
play this integrative role (Squire et al. 1993). Although
mechanisms outside sensory-motor systems enter into con-
ceptual knowledge, perceptual symbols always remain
grounded in these systems. Complete transductions never
occur whereby amodal representations that lie in associa-
tive areas totally replace modal representations. Thus,
Damasio (1989) states that convergence zones “are unin-
formed as to the content of the representations they assist
in attempting to reconstruct. The role of convergence zones
is to enact formulas for the reconstitution of fragment-
based momentary representations of entities or events in
sensory and motor cortices” (p. 46).5
2.1.1. Conscious versus unconscious processing. Al-
though neural representations define perceptual symbols,
they may produce conscious counterparts on some occa-
sions. On other occasions, however, perceptual symbols
function unconsciously, as during preconscious processing
and automatized skills. Most importantly, the basic defini-
tion of perceptual symbols resides at the neural level: un-
conscious neural representations – not conscious mental
images – constitute the core content of perceptual sym-
bols.6
Both the cognitive and neuroscience literatures support
this distinction between unconscious neural representa-
tions and optional conscious counterparts. In the cognitive
literature, research on preconscious processing indicates
that conscious states may not accompany unconscious pro-
cessing, and that if they do, they follow it (e.g., Marcel
1983a; 1983b; Velmans 1991). Similarly, research on skill
acquisition has found that conscious awareness falls away as
automaticity develops during skill acquisition, leaving un-
conscious mechanisms largely in control (e.g., Schneider &
Shiffrin 1977; Shiffrin 1988; Shiffrin & Schneider 1977).
Researchers have similarly found that conscious experience
often fails to reflect the unconscious mechanisms control-
ling behavior (e.g., Nisbett & Wilson 1977). In the neuro-
science literature, research on blindsight indicates that un-
conscious processing can occur in the absence of conscious
visual images (e.g., Cowey & Stoerig 1991; Weiskrantz
1986; see also Campion, Lotto & Smith: “Is Blindsight an
Effect of Scattered Light, Spared Cortex, and Near-
Threshold Vision?” BBS 2(3) 1983). Similarly, conscious
states typically follow unconscious states when processing
sensations and initiating actions, rather than preceding
them (Dennett & Kinsbourne 1992; Libet 1982; 1985).
Furthermore, different neural mechanisms appear respon-
sible for producing conscious and unconscious processing
(e.g., Farah & Feinberg 1997; Gazzaniga 1988; Schacter et
al. 1988).
Some individuals experience little or no imagery. By dis-
tinguishing unconscious perceptual processing from con-
scious perceptual experience, we can view such individuals
as people whose unconscious perceptual processing under-
lies cognition with little conscious awareness. If human
knowledge is inherently perceptual, there is no a priori rea-
son it must be represented consciously.
2.2. Schematic perceptual symbols
A perceptual symbol is not the record of the entire brain
state that underlies a perception. Instead, it is only a very
small subset that represents a coherent aspect of the state.
This is an assumption of many older theories (e.g., Locke
1690/1959), as well as many current ones (e.g., Langacker
1986; J. Mandler 1992; Talmy 1983). Rather than contain-
ing an entire holistic representation of a perceptual brain
state, a perceptual symbol contains only a schematic aspect.
The schematic nature of perceptual symbols falls out nat-
urally from two attentional assumptions that are nearly ax-
iomatic in cognitive psychology: Selective attention (1) iso-
lates information in perception, and (2) stores the isolated
information in long-term memory. First, consider the role
of selective attention in isolating features. During a per-
ceptual experience, the cognitive system can focus atten-
tion on a meaningful, coherent aspect of perception. On
perceiving an array of objects, attention can focus on the
shape of one object, filtering out its color, texture, and po-
sition, as well as the surrounding objects. From decades of
work on attention, we know that people have a sophisti-
cated and flexible ability to focus attention on features (e.g.,
Norman 1976; Shiffrin 1988; Treisman 1969), as well as on
the relations between features (e.g., Treisman 1993). Al-
though nonselected information may not be filtered out
completely, there is no doubt that it is filtered to a signifi-
cant extent (e.g., Garner 1974; 1978; Melara & Marks
1990).7
Once an aspect of perception has been selected, it has a
very high likelihood of being stored in long-term memory.
On selecting the shape of an object, attention stores infor-
mation about it. From decades of work on episodic mem-
ory, it is clear that where selective attention goes, long-term
storage follows, at least to a substantial extent (e.g., Barsa-
lou 1995; F. Craik & Lockhart 1972; Morris et al. 1977; D.
Nelson et al. 1979). Research on the acquisition of auto-
maticity likewise shows that selective attention controls
storage (Compton 1995; Lassaline & Logan 1993; Logan &
Etherton 1994; Logan et al. 1996). Although some nonse-
lected information may be stored, there is no doubt that it
is stored to a much lesser extent than selected information.
Because selective attention focuses constantly on aspects of
experience in this manner, large numbers of schematic rep-
resentations become stored in memory. As we shall see
later, these representations can serve basic symbolic func-
tions. Section 3.1 demonstrates that these representations
combine productively to implement compositionality, and
section 3.2 demonstrates that they acquire semantic inter-
pretations through the construction of propositions. The
use of “perceptual symbols” to this point anticipates these
later developments of the theory.
Finally, this symbol formation process should be viewed
in terms of the neural representations described in section
Barsalou: Perceptual symbol systems
BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4 583

2.1. If a configuration of active neurons underlies a per-
ceptual state, selective attention operates on this neural
representation, isolating a subset of active neurons. If se-
lective attention focuses on an object’s shape, the neurons
representing this shape are selected, and a record of their
activation is stored. Such storage could reflect the Hebbian
strengthening of connections between active neurons (e.g.,
Pulvermüller 1999), or the indirect integration of active
neurons via an adjacent associative area (e.g., Damasio
1989). Conscious experience may accompany the symbol
formation process and may be necessary for this process to
occur initially, falling away only as a symbol’s processing be-
comes automatized with practice. Most fundamentally,
however, the symbol formation process selects and stores a
subset of the active neurons in a perceptual state.
2.2.1. Perceptual symbols are dynamic, not discrete. Once
a perceptual symbol is stored, it does not function rigidly as
a discrete symbol. Because a perceptual symbol is an asso-
ciative pattern of neurons, its subsequent activation has dy-
namical properties. Rather than being reinstated exactly on
later occasions, its activations may vary widely. The subse-
quent storage of additional perceptual symbols in the same
association area may alter connections in the original pattern,
causing subsequent activations to differ. Different contexts
may distort activations of the original pattern, as connections
from contextual features bias activation toward some fea-
tures in the pattern more than others. In these respects, a
perceptual symbol is an attractor in a connectionist network.
As the network changes over time, the attractor changes. As
the context varies, activation of the attractor covaries. Thus,
a perceptual symbol is neither rigid nor discrete.
2.2.2. Perceptual symbols are componential, not holistic.
Theorists often view perceptual representations as con-
scious holistic images. This leads to various misunder-
standings about perceptual theories of knowledge. One is
that it becomes difficult to see how a perceptual represen-
tation could be componential. How can one construct a
schematic image of a shape without orientation combined
holistically? If one imagines a triangle consciously, is orien-
tation not intrinsically required in a holistic image?
It may be true that conscious images must contain cer-
tain conjunctions of dimensions. Indeed, it may be difficult
or impossible to construct a conscious image that breaks
apart certain dimensions, such as shape and orientation. If
a perceptual symbol is defined as an unconscious neural
representation, however, this is not a problem. The neurons
for a particular shape could be active, while no neurons for
a particular orientation are. During the unconscious pro-
cessing of perceptual symbols, the perceptual symbol for a
particular shape could represent the shape componentially,
while perceptual symbols for other dimensions, such as ori-
entation, remain inactive. The neuroanatomy of vision sup-
ports this proposal, given the presence of distinct channels
in the visual system that process different dimensions, such
as shape, orientation, color, movement, and so forth.
When conscious images are constructed for a perceptual
symbol, the activation of other dimensions may often be re-
quired. For example, consciously imagining a triangle may
require that it have a particular orientation. [See Edelman:
“Representation if Representation of Similarities” BBS
21(4) 1998.] However, these conscious representations
need not be holistic in the sense of being irreducible to
schematic components. For example, Kosslyn and his col-
leagues have shown that when people construct conscious
images, they construct them sequentially, component by
component, not holistically in a single step (Kosslyn et al.
1988; Roth & Kosslyn 1988; see also Tye 1993).
2.2.3. Perceptual symbols need not represent specific in-
dividuals. Contrary to what some thinkers have argued,
perceptual symbols need not represent specific individuals
(e.g., Berkeley 1710/1982; Hume 1739/1978). Because of
the schematicity assumption and its implications for human
memory, we should be surprised if the cognitive system
ever contains a complete representation of an individual.
Furthermore, because of the extensive forgetting and re-
construction that characterize human memory, we should
again be surprised if the cognitive system ever remembers
an individual with perfect accuracy, during either conscious
or unconscious processing. Typically, partial information is
retrieved, and some information may be inaccurate.
As we shall see later, the designation of a perceptual sym-
bol determines whether it represents a specific individual
or a kind – the resemblance of a symbol to its referent is not
critical (sect. 3.2.8). Suffice it to say for now that the same
perceptual symbol can represent a variety of referents, de-
pending on how causal and contextual factors link it to ref-
erents in different contexts (e.g., Dretske 1995; Fodor
1975; Goodman 1976; Schwartz 1981). Across different
pragmatic contexts, a schematic drawing of a generic sky-
scraper could stand for the Empire State Building, for sky-
scrapers in general, or for clothing made in New York City.
A drawing of the Empire State Building could likewise
stand for any of these referents. Just as different physical
replicas can stand for each of these referents in different
contexts, perceptual representations of them can do so as
well (Price 1953). Thus, the ability of a perceptual symbol
to stand for a particular individual need not imply that it
must represent an individual.
2.2.4. Perceptual symbols can be indeterminate. Theo-
rists sometimes argue that because perceptual representa-
tions are picture-like, they are determinate. It follows that
if human conceptualizations are indeterminate, perceptual
representations cannot represent them (e.g., Dennett
1969; but see Block 1983). For example, it has been argued
that people’s conceptualizations of a tiger are indeterminate
in its number of stripes; hence they must not be represent-
ing it perceptually. To my knowledge, it has not been veri-
fied empirically that people’s conceptualizations of tigers
are in fact indeterminate. If this is true, though, a percep-
tual representation of a tiger’s stripes could be indetermi-
nate in several ways (Schwartz 1981; Tye 1993). For exam-
ple, the stripes could be blurred in an image, so that they
are difficult to count. Or, if a perceptual symbol for stripes
had been extracted schematically from the perception of a
tiger, it might not contain all of the stripes but only a patch.
In later representing the tiger, this free-floating symbol
might be retrieved to represent the fact that the tiger was
striped, but, because it was only a patch, it would not imply
a particular number of stripes in the tiger. If this symbol
were used to construct stripes on the surface of a simulated
tiger, the tiger would then have a determinate number of
stripes, but the number might differ from the original tiger,
assuming for any number of reasons that the rendering of
the tiger’s surface did not proceed veridically.
Barsalou: Perceptual symbol systems
584 BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4

The two solutions considered thus far assume conscious
perceptual representations of a tiger. Unconscious neural
representations provide another solution. It is well known
that high-level neurons in perceptual systems can code in-
formation qualitatively. For example, a neuron can code the
presence of a line without coding its specific length, posi-
tion, or orientation. Similarly, a neuron can code the spatial
frequency of a region independently of its size or location.
Imagine that certain neurons in the visual system respond
to stripes independently of their number (i.e., detectors for
spatial frequency). In perceiving a tiger, if such detectors
fire and become stored in a perceptual representation, they
code a tiger’s number of stripes indeterminately, because
they simply respond to striped patterning and do not cap-
ture any particular number of stripes.
Qualitatively oriented neurons provide a perceptual rep-
resentation system with the potential to represent a wide
variety of concepts indeterminately. Consider the repre-
sentation of triangle. Imagine that certain neurons repre-
sent the presence of lines independently of their length, po-
sition, and orientation. Further imagine that other neurons
represent vertices between pairs of lines independently of
the angle between them. Three qualitative detectors for
lines, coupled spatially with three qualitative detectors for
vertices that join them, could represent a generic triangle.
Because all of these detectors are qualitative, the lengths of
the lines and the angles between them do not matter; they
represent all instances of triangle simultaneously. In this
manner, qualitatively specified neurons support perceptual
representations that are not only indeterminate but also
generic.8
2.3. Multimodal perceptual symbols
The symbol formation process just described in section 2.2
can operate on any aspect of perceived experience. Not
only does it operate on vision, it operates on the other four
sensory modalities (audition, haptics, olfaction, and gusta-
tion), as well as on proprioception and introspection. In any
modality, selective attention focuses on aspects of per-
ceived experience and stores records of them in long-term
memory, which later function as symbols. As a result, a wide
variety of symbols is stored. From audition, people acquire
perceptual symbols for speech and the various sounds
heard in everyday experience. From touch, people acquire
perceptual symbols for textures and temperatures. From
proprioception, people acquire perceptual symbols for
hand movements and body positions.
Presumably, each type of symbol becomes established in
its respective brain area. Visual symbols become estab-
lished in visual areas, auditory symbols in auditory areas,
proprioceptive symbols in motor areas, and so forth. The
neuroscience literature on category localization supports
this assumption. When a sensory-motor area is damaged,
categories that rely on it during the processing of perceived
instances exhibit deficits in conceptual processing (e.g.,
Damasio & Damasio 1994; Gainotti et al. 1995; Pulver-
müller 1999; Warrington & Shallice 1984). For example,
damage to visual areas disrupts the conceptual processing
of categories specified by visual features (e.g., birds). Anal-
ogously, damage to motor and somatosensory areas disrupts
the conceptual processing of categories specified by motor
and somatosensory features (e.g., tools). Recent neuro-
imaging studies on people with intact brains provide con-
verging evidence (e.g., A. Martin et al. 1995; 1996; Pulver-
müller 1999; Rösler et al. 1995). When normal subjects
perform conceptual tasks with animals, visual areas are
highly active; when they perform conceptual tasks with
tools, motor and somatosensory areas are highly active.
Analogous findings have also been found for the conceptual
processing of color and space (e.g., DeRenzi & Spinnler
1967; Levine et al. 1985; Rösler et al. 1995).
As these findings illustrate, perceptual symbols are mul-
timodal, originating in all modes of perceived experience,
and they are distributed widely throughout the modality-
specific areas of the brain. It should now be clear that “per-
ceptual” is not being used in its standard sense here. Rather
than referring only to the sensory modalities, as it usually
does, it refers much more widely to any aspect of perceived
experience, including proprioception and introspection.
2.3.1. Introspection. Relative to sensory-motor processing
in the brain, introspective processing is poorly understood.
Functionally, three types of introspective experience appear
especially important: representational states, cognitive oper-
ations, and emotional states. Representational states include
the representation of an entity or event in its absence, as well
as construing a perceived entity as belonging to a category.
Cognitive operations include rehearsal, elaboration, search,
retrieval, comparison, and transformation. Emotional states
include emotions, moods, and affects. In each case, selective
attention can focus on an aspect of an introspective state and
stores it in memory for later use as a symbol. For example,
selective attention could focus on the ability to represent
something in its absence, filtering out the particular entity or
event represented and storing a schematic representation of
a representational state. Similarly, selective attention could
focus on the process of comparison, filtering out the partic-
ular entities compared and storing a schematic representa-
tion of the comparison process. During an emotional event,
selective attention could focus on emotional feelings, filter-
ing out the specific circumstances leading to the emotion,
and storing a schematic representation of the experience’s
“hot” components.
Much remains to be learned about the neural bases of in-
trospection, although much is known about the neural
bases of emotion (e.g., Damasio 1994; LeDoux 1996). To
the extent that introspection requires attention and work-
ing memory, the neural systems that underlie them may be
central (e.g., Jonides & E. Smith 1997; Posner 1995; Rush-
worth & Owen 1998). Like sensory-motor systems, intro-
spection may have roots in evolution and genetics. Just as
genetically constrained dimensions underlie vision (e.g.,
color, shape, depth), genetically constrained dimensions
may also underlie introspection. Across individuals and cul-
tures, these dimensions may attract selective attention, re-
sulting in the extraction of similar perceptual symbols for
introspection across individuals and cultures. Research on
mental verbs in psychology and linguistics suggests what
some of these dimensions might be (e.g., Cacciari & Levo-
rato 1994; Levin 1995; Schwanenflugel et al. 1994). For ex-
ample, Schwanenflugel et al. report that the dimensions of
perceptual/conceptual, certain/uncertain, and creative/
noncreative organize mental verbs such as see, reason,
know, guess, and compare. The fact that the same dimen-
sions arise cross-culturally suggests that different cultures
conceptualize introspection similarly (e.g., D’Andrade 1987;
Schwanenflugel et al., in press).
Barsalou: Perceptual symbol systems
BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4 585

2.4.3. Concepts, conceptualizations, and categories. Ac-
cording to this theory, the primary goal of human learning
is to establish simulators. During childhood, the cognitive
system expends much of its resources developing simula-
tors for important types of entities and events. Once indi-
viduals can simulate a kind of thing to a culturally accept-
able degree, they have an adequate understanding of it.
What is deemed a culturally competent grasp of a category
may vary, but in general it can be viewed as the ability to
simulate the range of multimodal experiences common to
the majority of a culture’s members (cf. Romney et al.
1986). Thus, people have a culturally acceptable simulator
for chair if they can construct multimodal simulations of the
chairs typically encountered in their culture, as well as the
activities associated with them.
In this theory, a concept is equivalent to a simulator. It is
the knowledge and accompanying processes that allow an
individual to represent some kind of entity or event ade-
quately. A given simulator can produce limitless simulations
of a kind, with each simulation providing a different con-
ceptualization of it. Whereas a concept represents a kind
generally, a conceptualization provides one specific way of
thinking about it. For example, the simulator for chair can
simulate many different chairs under many different cir-
cumstances, each comprising a different conceptualization
of the category. For further discussion of this distinction be-
tween permanent knowledge of a kind in long-term mem-
ory and temporary representations of it in working memory,
see Barsalou (1987; 1989; 1993; also see sect. 2.4.5).
Simulators do not arise in a vacuum but develop to track
meaningful units in the world. As a result, knowledge can
accumulate for each unit over time and support optimal in-
teractions with it (e.g., Barsalou et al. 1993; 1998; Millikan
1998). Meaningful units include important individuals
(e.g., family members, friends, personal possessions) and
categories (e.g., natural kinds, artifacts, events), where a
category is a set of individuals in the environment or intro-
spection. Once a simulator becomes established in memory
for a category, it helps identify members of the category on
subsequent occasions, and it provides categorical infer-
ences about them, as described next.11
2.4.4. Categorization, categorical inferences, and afford-
ances. Tracking a category successfully requires that its
members be categorized correctly when they appear. View-
ing concepts as simulators suggests a different way of think-
ing about categorization. Whereas many theories assume
that relatively static, amodal structures determine category
membership (e.g., definitions, prototypes, exemplars, the-
ories), simulators suggest a more dynamic, embodied ap-
proach: if the simulator for a category can produce a satis-
factory simulation of a perceived entity, the entity belongs
in the category. If the simulator cannot produce a satisfac-
tory simulation, the entity is not a category member.12
Besides being dynamic, grounding categorization in per-
ceptual symbols has another important feature: the knowl-
edge that determines categorization is represented in
roughly the same manner as the perceived entities that
must be categorized. For example, the perceptual simula-
tions used to categorize chairs approximate the actual per-
ceptions of chairs. In contrast, amodal theories assume that
amodal features in concepts are compared to perceived en-
tities to perform categorization. Whereas amodal theories
have to explain how two very different types of representa-
tion are compared, perceptual symbol systems simply as-
sume that two similar representations are compared. As a
natural side effect of perceiving a category’s members, per-
ceptual knowledge accrues that can be compared directly
to perceived entities during categorization.
On this view, categorization depends on both familiar
and novel simulations. Each successful categorization
stores a simulation of the entity categorized. If the same en-
tity or a highly similar one is encountered later, it is assigned
to the category because the perception of it matches an ex-
isting simulation in memory. Alternatively, if a novel entity
is encountered that fails to match an existing simulation,
constructing a novel simulation that matches the entity can
establish membership. Explanation-based learning as-
sumes a similar distinction between expertise and creativ-
ity in categorization (DeJong & Mooney 1986; T. Mitchell
et al. 1986), as do theories of skill acquisition (Anderson
1993; Logan 1988; Newell 1990), although these ap-
proaches typically adopt amodal representations.
As an example, imagine that the simulator for triangle
constructs three lines and connects their ends uniquely.
Following experiences with previous triangles, simulations
that match these instances become stored in the simulator.
On encountering these triangles later, or highly similar
ones, prestored simulations support rapid categorization,
thereby implementing expertise. However, a very different
triangle, never seen before, can also be categorized if the
triangle simulator can construct a simulation of it (cf. Miller
& Johnson-Laird 1976).
Categorization is not an end in itself but provides access
to categorical inferences. Once an entity is categorized,
knowledge associated with the category provides predic-
tions about the entity’s structure, history, and behavior, and
also suggests ways of interacting with it (e.g., Barsalou 1991;
Ross 1996; see also sect. 3.2.2). In this theory, categorical
inferences arise through simulation. Because a simulator
contains a tremendous amount of multimodal knowledge
about a category, it can simulate information that goes be-
yond that perceived in a categorized entity. On perceiving
a computer from the front, the simulator for computer can
simulate all sorts of things not perceived, such as the com-
puter’s rear panel and internal components, what the com-
puter will do when turned on, what tasks it can perform,
how the keys will feel when pressed, and so forth. Rather
than having to learn about the entity from scratch, a per-
ceiver can run simulations that anticipate the entity’s struc-
ture and behavior and that suggest ways of interacting with
it successfully.
Simulators also produce categorical inferences in the ab-
sence of category members. As described later, simulations
provide people with a powerful ability to reason about en-
tities and events in their absence (sects. 2.6, 3.1, 3.2, 4.1,
4.2). Simulations of future entities, such as a rental home,
allow people to identify preparations that must be made in
advance. Simulations of future events, such as asking a fa-
vor, allow people to identify optimal strategies for achiev-
ing success. To the extent that future category members are
similar to previous category members, simulations of pre-
vious members provide reasonable inferences about future
members.13
Deriving categorical inferences successfully requires
that simulations preserve at least some of the affordances
present in actual sensory-motor experiences with category
members (cf. Gibson 1979; also see S. Edelman 1998). To
Barsalou: Perceptual symbol systems
BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4 587

the extent that simulations capture affordances from per-
ception and action, successful reasoning about physical
situations can proceed in their absence (Glenberg 1997;
Glenberg et al. 1998b; Newton 1996). Agents can draw in-
ferences that go beyond perceived entities, and they can
plan intelligently for the future. While sitting in a restau-
rant and wanting to hide from someone entering, one could
simulate that a newspaper on the table affords covering
one’s face completely but that a matchbook does not. As a
result of these simulations, the newspaper is selected to
achieve this goal rather than the matchbook. Because the
simulations captured the physical affordances correctly, the
selected strategy works.
2.4.5. Concept stability. Equating concepts with simula-
tors provides a solution to the problem of concept stability.
Previous work demonstrates that conceptualizations of a
category vary widely between and within individuals (Bar-
salou 1987; 1989; 1993). If different people conceptualize
bird differently on a given occasion, and if the same indi-
vidual conceptualizes bird differently across occasions, how
can stability be achieved for this concept?
One solution is to assume that a common simulator for
bird underlies these different conceptualizations, both be-
tween and within individuals. First, consider how a simula-
tor produces stability within an individual. If a person’s dif-
ferent simulations of a category arise from the same
simulator, then they can all be viewed as instantiating the
same concept. Because the same simulator produced all of
these simulations, it unifies them. Between individuals, the
key issue concerns whether different people acquire simi-
lar simulators. A number of factors suggest that they should,
including a common cognitive system, common experience
with the physical world, and socio-cultural institutions that
induce conventions (e.g., Newton 1996; Tomasello et al.
1993). Although two individuals may represent the same
category differently on a given occasion, each may have the
ability to simulate the other’s conceptualization. In an un-
published study, subjects almost always viewed other sub-
jects’ conceptualizations of a category as correct, even
though their individual conceptualizations varied widely.
Each subject produced a unique conceptualization but ac-
cepted those of other subjects because they could be simu-
lated. Furthermore, common contextual constraints during
communication often drive two people’s simulations of a
category into similar forms. In another unpublished study,
conceptualizations of a category became much more stable
both between and within subjects when constructed in a
common context. Subjects shared similar simulators that
produced similar conceptualizations when constrained ad-
equately.
2.4.6. Cognitive penetration. The notion of a simulator is
difficult to reconcile with the view that cognition does not
penetrate perception (Fodor 1983). According to the im-
penetrability hypothesis, the amodal symbol system under-
lying higher cognition has little or no impact on processing
in sensory-motor systems because these systems are mod-
ular and therefore impenetrable. In contrast, the construct
of a simulator assumes that sensory-motor systems are
deeply penetrable. Because perceptual symbols reside in
sensory-motor systems, running a simulator involves a par-
tial running of these systems in a top-down manner.
In an insightful BBS review of top-down effects in vision,
Pylyshyn (1999) concludes that cognition only produces
top-down effects indirectly through attention and decision
making – it does not affect the content of vision directly.
Contrary to this conclusion, however, much evidence in-
dicates that cognition does affect the content of sensory-
motor systems directly. The neuroscience literature on
mental imagery demonstrates clearly that cognition estab-
lishes content in sensory-motor systems in the absence of
physical input. In visual imagery, the primary visual cortex,
V1, is often active, along with many other early visual areas
(e.g., Kosslyn et al. 1995). In motor imagery, the primary
motor cortex, M1, is often active, along with many other
early motor areas (e.g., Crammond 1997; Deschaumes-
Molinaro et al. 1992; Jeannerod 1994; 1995). Indeed, mo-
tor imagery not only activates early motor areas, it also stim-
ulates spinal neurons, produces limb movements, and
modulates both respiration and heart rate. When sharp-
shooters imagine shooting a gun, their entire body behaves
similarly to actually doing so. In auditory imagery, activation
has not yet been observed in the primary auditory cortex,
but activation has been observed in other early auditory ar-
eas (e.g., Zatorre et al. 1996). These findings clearly demon-
strate that cognition establishes content in sensory-motor
systems in the absence of physical input.
A potential response is that mental imagery arises solely
within sensory-motor areas – it is not initiated by cognitive
areas. In this vein, Pylyshyn (1999) suggests that perceptual
modules contain local memory areas that affect the content
of perception in a top-down manner. This move under-
mines the impenetrability thesis, however, at least in its
strong form. As a quick perusal of textbooks on cognition
and perception reveals, memory is widely viewed as a basic
cognitive process – not as a perceptual process. Many re-
searchers would probably agree that once memory is im-
ported into a sensory-motor system, cognition has been im-
ported. Furthermore, to distinguish perceptual memory
from cognitive memory, as Pylyshyn does, makes sense only
if one assumes that cognition utilizes amodal symbols. Once
one adopts the perspective of perceptual symbol systems,
there is only perceptual memory, and it constitutes the rep-
resentational core of cognition. In this spirit, Damasio
(1989) argues eloquently that there is no sharp discontinu-
ity between perceptual and cognitive memory. Instead,
there is simply a gradient from posterior to anterior associ-
ation areas in the complexity and specificity of the memo-
ries that they activate in sensory-motor areas. On Damasio’s
view, memory areas both inside and outside a sensory-
motor system control its feature map to implement cogni-
tive representations. In this spirit, the remainder of this tar-
get article assumes that top-down cognitive processing in-
cludes all memory effects on perceptual content, including
memory effects that originate in local association areas.
Nevertheless, Pylyshyn (1999) makes compelling argu-
ments about the resiliency of bottom-up information in
face-to-face competition with contradicting top-down in-
formation. For example, when staring at the Müller-Lyer il-
lusion, one cannot perceive the horizontal lines as equiva-
lent in length, even though one knows cognitively that they
are. Rather than indicating impenetrability, however, this
important observation may simply indicate that bottom-up
information dominates top-down information when they
conflict (except in the degenerate case of psychosis and
other hallucinatory states, when top-down information does
dominate bottom-up information). Indeed, Marslen-Wil-
Barsalou: Perceptual symbol systems
588 BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4

son and Tyler (1980), although taking a nonmodular inter-
active approach, offer exactly this account of bottom-up
dominance in speech recognition. Although top-down pro-
cessing can penetrate speech processing, it is overridden
when bottom-up information conflicts. If semantic and syn-
tactic knowledge predict that “The cowboy climbed into the
———” ends with “saddle,” but the final word is actually
“jacuzzi,” then “jacuzzi” overrides “saddle.”
On this view, sensory-motor systems are penetrable but
not always. When bottom-up information conflicts with
top-down information, the former usually dominates.
When bottom-up information is absent, however, top-down
information penetrates, as in mental imagery. Perhaps most
critically, when bottom-up and top-down information are
compatible, top-down processing again penetrates, but in
subtle manners that complement bottom-up processing.
The next section (sect. 2.4.7) reviews several important
phenomena in which bottom-up and top-down processing
simultaneously activate sensory-motor representations as
they cooperate to perceive physical entities (i.e., implicit
memory, filling-in, anticipation, interpretation). Recent
work on simultaneous imagery and perception shows
clearly that these two processes work well together when
compatible (e.g., Craver-Lemley & Reeves 1997; Gilden et
al. 1995).
Perhaps the critical issue in this debate concerns the de-
finition of cognition. On Pylyshyn’s view, cognition con-
cerns semantic beliefs about the external world (i.e., the be-
lief that the horizontal lines are the same length in the
Müller-Lyer illusion). However, this is a far narrower view
of cognition than most cognitive psychologists take, as evi-
denced by journals and texts in the field. Judging from these
sources, cognitive psychologists believe that a much
broader array of processes and representations – including
memory – constitutes cognition.
Ultimately, as Pylyshyn suggests, identifying the mecha-
nisms that underlie intelligence should be our primary goal,
from the most preliminary sensory processes to the most
abstract thought processes. Where we actually draw the line
between perception and cognition may not be all that im-
portant, useful, or meaningful. In this spirit, perceptual
symbol systems attempt to characterize the mechanisms
that underlie the human conceptual system. As we have
seen, the primary thesis is that sensory-motor systems rep-
resent not only perceived entities but also conceptualiza-
tions of them in their absence. From this perspective, cog-
nition penetrates perception when sensory input is absent,
or when top-down inferences are compatible with sensory
input.
2.4.7. A family of representational processes. Evolution
often capitalizes on existing mechanisms to perform new
functions (Gould 1991). Representational mechanisms in
sensory-motor regions of the brain may be such mecha-
nisms. Thus far, these representational mechanisms have
played three roles in perceptual symbol systems: (1) In per-
ception, they represent physical objects. (2) In imagery,
they represent objects in their absence. (3) In conception,
they also represent objects in their absence. On this view,
conception differs from imagery primarily in the con-
sciousness and specificity of sensory-motor representa-
tions, with these representations being more conscious and
detailed in imagery than in conception (Solomon & Barsa-
lou 1999a; Wu & Barsalou 1999). Several other cognitive
processes also appear to use the same representational
mechanisms, including implicit memory, filling-in, antici-
pation, and interpretation. Whereas perception, imagery,
and conception perform either bottom-up or top-down
processing exclusively, these other four processes fuse com-
plementary mixtures of bottom-up and top-down process-
ing to construct perceptions.
In implicit memory (i.e., repetition priming), a percep-
tual memory speeds the perception of a familiar entity (e.g.,
Roediger & McDermott 1993; Schacter 1995). On seeing a
particular chair, for example, a memory is established that
speeds perception of the same chair later. Much research
demonstrates the strong perceptual character of these
memories, with slight deviations in perceptual features
eliminating facilitative effects. Furthermore, imagining an
entity produces much the same facilitation as perceiving it,
suggesting a common representational basis. Perhaps most
critically, implicit memory has been localized in sensory-
motor areas of the brain, with decreasing brain activity re-
quired to perceive a familiar entity (e.g., Buckner et al.
1995). Thus, the representations that underlie implicit
memory reside in the same systems that process entities
perceptually. When a familiar entity is perceived, implicit
memories become fused with bottom-up information to
represent it efficiently.
In filling-in, a perceptual memory completes gaps in bot-
tom-up information. Some filling-in phenomena reflect
perceptual inferences that are largely independent of mem-
ory (for a review, see Pessoa et al. 1998). In the perception
of illusory contours, for example, low-level sensory mecha-
nisms infer edges on the basis of perceived vertices (e.g.,
Kanizsa 1979). However, other filling-in phenomena rely
heavily on memory. In the phoneme restoration effect,
knowledge of a word creates the conscious perceptual ex-
perience of hearing a phoneme where noise exists physi-
cally (e.g., Samuel 1981; 1987; Warren 1970). More signif-
icantly, phoneme restoration adapts low-level feature
detectors much as if physical phonemes had adapted them
(Samuel 1997). Thus, when a word is recognized, its mem-
ory representation fills in missing phonemes, not only in ex-
perience, but also in sensory processing. Such findings
strongly suggest that cognitive and perceptual representa-
tions reside in a common system, and that they become
fused to produce perceptual representations. Knowledge-
based filling-in also occurs in vision. For example, knowl-
edge about bodily movements causes apparent motion to
deviate from the perceptual principle of minimal distance
(Shiffrar & Freyd 1990; 1993). Rather than filling in an arm
as taking the shortest path through a torso, perceivers fill it
in as taking the longer path around the torso, consistent
with bodily experience.
In perceptual anticipation, the cognitive system uses past
experience to simulate a perceived entity’s future activity.
For example, if an object traveling along a trajectory disap-
pears, perceivers anticipate where it would be if it were still
on the trajectory, recognizing it faster at this point than at
the point it disappeared, or at any other point in the display
(Freyd 1987). Recent findings indicate that knowledge af-
fects the simulation of these trajectories. When subjects be-
lieve that an ambiguous object is a rocket, they simulate a
different trajectory compared to when they believe it is a
steeple (Reed & Vinson 1996). Even infants produce per-
ceptual anticipations in various occlusion tasks (e.g., Bail-
largeon 1995; Hespos & Rochat 1997). These results fur-
Barsalou: Perceptual symbol systems
BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4 589

3.2. Propositions
Another important lesson that we have learned from
amodal symbol systems is that a viable theory of knowledge
must implement propositions that describe and interpret
situations (e.g., Anderson & Bower 1973; Goodman 1976;
Kintsch 1974; Norman et al. 1975; Pylyshyn 1973; 1978;
1981; 1984). A given situation is capable of being construed
in an infinite number of ways by an infinite number of
propositions. Imagine being in a grocery store. There are
limitless ways to describe what is present, including (in
amodal form):
CONTAINS (grocery store, apples) (3)
ABOVE (ceiling, floor)
As these examples illustrate, different construals of the sit-
uation result from selecting different aspects of the situa-
tion and representing them in propositions. Because an in-
finite number of aspects can be propositionalized, selecting
the propositions to represent a situation is an act of cre-
ativity (Barsalou & Prinz 1997). Different construals can
also result from construing the same aspects of a situation
in different ways, as in:
ABOVE (ceiling, floor) (4)
BELOW ( floor, ceiling)
Bringing different concepts to bear on the same aspects of
a situation extends the creative construction of propositions
further.
Construals of situations can be arbitrarily complex, re-
sulting from the ability to embed propositions hierarchi-
cally, as in:
CAUSE (HUNGRY (shopper), BUY (shopper, groceries)) (5)
The productive properties of amodal symbols are central to
constructing complex propositions.
Not all construals of a situation are true. When a con-
strual fails to describe a situation accurately, it constitutes a
false proposition, as in:
CONTAINS (grocery store, mountains) (6)
Similarly, true and false propositions can be negative, as in
the true proposition:
NOT (CONTAINS (grocery store, mountains)) (7)
Thus, propositions can construe situations falsely, and they
can indicate negative states.
Finally, propositions represent the gist of comprehen-
sion. Comprehenders forget the surface forms of sentences
rapidly but remember the conceptual gist for a long time
(e.g., Sachs 1967; 1974). Soon after hearing “Marshall gave
Rick a watch,” listeners would probably be unable to spec-
ify whether they had heard this sentence as opposed to
“Rick received a watch from Marshall.” However, listeners
would correctly remember that it was Marshall who gave
Rick the watch and not vice versa, because they had stored
the proposition:
GIVE (Agent 5 marshall, Recipient 5 rick, Object 5 watch) (8)
Thus, propositions capture conceptualizations that can be
paraphrased in many ways.
Basically, propositions involve bringing knowledge to
bear on perception, establishing type-token relations be-
tween concepts in knowledge and individuals in the per-
ceived world. This requires a conceptual system that can
combine types (concepts) productively to form hierarchical
structures, and that can then map these structures onto in-
dividuals in the world. It is fair to say that this ability is not
usually recognized as possible in perceptual theories of
knowledge, again because they are widely construed as
recording systems. Indeed, this belief is so widespread that
the term “propositional” is reserved solely for nonpercep-
tual theories of knowledge. As we shall see, however, if one
adopts the core properties of perceptual symbol systems,
the important properties of propositions follow naturally.
Because perceptual symbol systems have the same poten-
tial to implement propositions, they too are propositional
systems.19
3.2.1. Type-token mappings. To see how perceptual sym-
bol systems implement type-token mappings, consider Fig-
ure 5A. The large panel with a thick solid border stands for
a perceived scene that contains several individual entities.
On the far left, the schematic drawing of a jet in a thin solid
border stands for the simulator that underlies the concept
jet. The other schematic drawing of a jet in a thick dashed
border represents a specific simulation that provides a good
fit of the perceived jet in the scene. Again, such drawings
are theoretical notations that should not be viewed as literal
images. The line from the simulator to the simulation
stands for producing the simulation from the simulator. The
line from the simulation to the perceived individual stands
for fusing the simulation with the individual in perception.
Barsalou: Perceptual symbol systems
BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4 595
Figure 5. (A) Examples of how perceptual symbol systems rep-
resent true propositions by fusing perceived entities with simu-
lations constructed from simulators. (B) Example of a complex
hierarchical proposition. (C) Example of an alternative interpre-
tation of the same aspects of the scene in panel B. Boxes with thin
solid lines represent simulators; boxes with thick dashed lines rep-
resent simulations; boxes with thick solid lines represent per-
ceived situations.

The activation of the simulator for jet in Figure 5A results
from attending to the leftmost individual in the scene. As
visual information is picked up from the individual, it pro-
jects in parallel onto simulators in memory. A simulator be-
comes increasingly active if (1) its frame contains an exist-
ing simulation of the individual, or if (2) it can produce a
novel simulation that provides a good fit (sect. 2.4.4). The
simulation that best fits the individual eventually controls
processing.20 Because the simulation and the perception
are represented in a common perceptual system, the final
representation is a fusion of the two. Rather than being a
pure bottom-up representation, perception of the individ-
ual includes top-down information from the simulation
(sect. 2.4.6). The constructs of meshing, blending, percep-
tual hypotheses, and perceptual anticipation are closely re-
lated (Fauconnier & Turner 1998; Glenberg 1997; Kosslyn
1994; Neisser 1976).
Binding a simulator successfully with a perceived indi-
vidual via a simulation constitutes a type-token mapping.
The simulator is a type that construes a perceived token as
having the properties associated with the type. Most im-
portantly, this type-token mapping implicitly constitutes a
proposition, namely, the one that underlies “It is true that
the perceived individual is a jet.” In this manner, percep-
tual symbol systems establish simple propositions.
3.2.2. Categorical inferences. Binding a simulator to a per-
ceived individual allows perceivers to draw wide variety of
categorical inferences (sects. 2.4.4, 2.4.7). If the jet in Fig-
ure 5A should become occluded, or if the perceivers should
turn away, a simulation of its trajectory could predict where
it will reappear later. Simulating the jet might further sug-
gest many aspects of it that are not visible. For example, the
simulation might suggest that the jet contains pilots, pas-
sengers, luggage, and fuel. Similarly, it might suggest that
the jet is likely to fly horizontally, not vertically, and is even-
tually likely to decrease altitude, put down its wheels, and
land at an airport. From bringing the multimodal simulator
for jet to bear on the perceived individual, a wealth of top-
down inference becomes available. Finally, the binding
process updates the simulator for jet. As selective attention
extracts perceptual symbols from the current individual and
uses them to construct a simulation, they become inte-
grated into the underlying frame that helped produce it (as
illustrated in Fig. 3).
3.2.3. False and negative propositions. So far, we have
only considered true propositions, namely, successful bind-
ings between simulators and perceived individuals. Not all
attempted bindings succeed, however, and when they do
not, they constitute false propositions. Similarly, negative
propositions occur when the explicitly noted absence of
something in a simulation corresponds to an analogous ab-
sence in a perceived situation. Because false and negative
propositions receive detailed treatment in section 3.4 on
abstract concepts, further discussion is deferred until then.
3.2.4. Multiple construals. Because a given situation can be
construed in infinite ways, propositional construal is cre-
ative. One way of producing infinite propositions is by se-
lecting different aspects of the situation to construe. Figure
5A illustrates how perceptual symbol systems implement
this ability. If perceivers were to focus attention on the up-
permost individual in the scene, rather than on the leftmost
individual, they would construe the scene differently.
Rather than bringing the simulator for jet to bear, they
would bring the simulator for balloon to bear. The success-
ful mapping that results represents a different proposition,
namely, the one that underlies “It is true that the perceived
individual is a balloon.” Because infinitely many aspects of
the scene can be selected and bound to simulators, an infi-
nite number of propositions describing the scene are pos-
sible.
3.2.5. Productively produced hierarchical propositions.
Perceptual symbol systems readily implement complex hi-
erarchical propositions. In Figure 5B, a hierarchical simu-
lation of a balloon above a cloud is constructed productively
from the simulators for balloon, cloud, and above (sect. 3.1).
This hierarchical simulation in turn is fused successfully
with individuals in the perceived scene and the regions they
occupy. The result is the representation of the hierarchical
proposition that underlies, “It is true that there is a balloon
above a cloud.”
3.2.6. Alternative interpretations. Perceptual symbol sys-
tems also readily represent alternative interpretations of
the same individuals in a scene. As Figure 5C illustrates, the
simulator for below can be mapped into the same aspects
of the perceived situation as the simulator for above (Fig.
5B). Because the same spatial configuration of regions sat-
isfies both, either can represent a true proposition about
the scene, differing only in where attention is focused (i.e.,
the upper or lower region). In this manner, and in many
others as well, perceptual symbols support different inter-
pretations of the same information. To the extent that dif-
ferent simulations can be fit successfully to the same per-
ceived information, different interpretations result.
3.2.7. Gist. A perceptual simulation represents a gist that
can be paraphrased in multiple ways. Imagine that someone
hears the sentence, “The balloon is above the cloud.” To rep-
resent the sentence’s meaning, the listener might construct
a simulation that focuses attention on the upper region of
above, as in Figure 5B. When later trying to remember the
sentence, however, the listener might construct a simulation
that has lost information about where attention resided. As
a result, it is impossible to specify whether the earlier sen-
tence had been “The balloon is above the cloud” or “The
cloud is below the balloon.” As information becomes lost
from the memory of a simulation, paraphrases become in-
creasingly likely. Furthermore, because different simulators
can often be mapped into the remaining information, addi-
tional paraphrases become possible.
In summary, perceptual symbol systems readily imple-
ment all the fundamental properties of propositions re-
viewed earlier for amodal symbols systems. Perceptual
symbol systems produce type-token mappings that provide
a wealth of inferences about construed individuals. They
produce alternative construals of the same scene, either by
selecting different aspects of the scene to simulate, or by
simulating the same aspects in different ways. They repre-
sent complex hierarchical propositions by constructing sim-
ulations productively and then mapping these complex sim-
ulations into scenes. They represent the gist of sentences
with simulations that can be paraphrased later with differ-
ent utterances. Finally, as described later, they represent
false and negative propositions through failed and absent
Barsalou: Perceptual symbol systems
596 BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4

though the shape might be ambiguous in isolation, when
linked historically to a particular simulator over the course
of a simulation, it is not.
Conversely, another problem of image construal con-
cerns how radically different images can be construed as
images of the same thing. Consider Schwartz’s (1981) ex-
ample of a circle. Normally, a circle in full view looks round.
If its lower half is occluded, however, it looks like a semi-
circle. If partially occluded by visual clutter, it looks like a
circular arrangement of line segments. If rotated 45°
around its vertical axis, it looks like an oval. If rotated 90°,
it looks like a straight vertical line. How do we know that
these five different images all represent the same entity?
Again, the construct of a simulator provides a solution. If
the simulator for circle constructs an instance and then
transforms it into different surface images, the transforma-
tional history of this particular circle links all these images
so that they are construed as the same individual. As de-
scribed for concept stability (sect. 2.4.5), a simulator unifies
all the simulations it produces into a single concept. The ex-
ample here is a similar but more specific case where a sim-
ulator unifies all the perspectives and conditions under
which a given individual can be simulated.
3.3. Variable embodiment
Variable embodiment is the idea that a symbol’s meaning
reflects the physical system in which it is represented (Clark
1997; Damasio 1994; Glenberg 1997; Johnson 1987; Lakoff
1987; Lakoff & Johnson 1980; Newton 1996). As different
intelligent systems vary physically, their symbol systems are
likely to vary as well. Unlike productivity and propositional
construal, variable embodiment does not exist in amodal
symbol systems. It exists only in perceptual symbol systems,
arising from their first two core properties: neural repre-
sentations in perceptual systems (sect. 2.1), and schematic
symbol formation (sect. 2.2). Before considering variable
embodiment in perceptual symbol systems, it is useful to
consider its absence in amodal symbol systems.
According to the functionalist perspective that has dom-
inated modern cognitive science, the symbol system un-
derlying human intelligence can be disembodied. Once we
characterize the computational properties of this system
successfully, we can implement it in other physical systems
such as computers. Thus, functionalism implies that the
computational system underlying human intelligence can
be understood independently of the human body. It further
implies that the same basic symbol system operates in all
normal humans, independent of their biological idiosyn-
crasies. Just as computer software can be characterized in-
dependently of the particular hardware that implements it,
the human symbol system can be characterized indepen-
dently of the biological system that embodies it. For ac-
counts of this view, see Putnam (1960), Fodor (1975), and
Pylyshyn (1984); for critiques, see Churchland (1986) and
G. Edelman (1992).
The discrete referential nature of amodal symbols lies at
the heart of modern functionalism. To see this, consider the
transformations of the word “CUP” in Figure 6 and their
lack of implications for reference. As Figure 6 illustrates,
the word “CUP” can increase in size, it can rotate 45° coun-
terclockwise, and the . on the P can become separated
from the *. In each case, the transformation implies nothing
new about its referent. If “CUP” originally referred to the
referent on the left of Figure 6, its reference does not
change across these transformations. Making “CUP” larger
does not mean that its referent now appears larger. Rotat-
ing “CUP” 45° counterclockwise does not imply that its ref-
erent tipped. Separating the . on the P horizontally from
the * does not imply that the handle of the cup has now bro-
ken off. Because words bear no structural relations to their
referents, structural changes in a word have no implications
for analogous changes in reference. As long as the conven-
tional link between a word and its referent remains intact,
the word refers to the referent discretely in exactly the same
way across transformations.
Because amodal symbols refer in essentially the same
way as words do, they also refer discretely. Changes in their
form have no implications for their meaning. It is this dis-
crete property of amodal symbols that makes functionalism
possible. Regardless of how an amodal symbol is realized
physically, it serves the same computational function. Phys-
ical variability in the form of the symbol is irrelevant, as long
as it maintains the same conventional links to the world, and
the same syntactic relations to other amodal symbols.
Perceptual symbols differ fundamentally. Unlike an
amodal symbol, variations in the form of a perceptual sym-
bol can have semantic implications (cf. Goodman 1976). As
Figure 6 illustrates, the schematic perceptual symbol for a
cup can increase in size, it can rotate 45° counterclockwise,
and the handle can become separated from the cup. In each
case, the transformation implies a change in the referent
(Fig. 6). Increasing the size of the perceptual symbol im-
plies that the referent appears larger, perhaps because the
perceiver has moved closer to it. Rotating the perceptual
symbol 45° counterclockwise implies that the cup has
tipped analogously. Separating the handle from the cup in
the perceptual symbol implies that the handle has become
detached from the referent. Because perceptual symbols
bear structural relations to their referents, structural
changes in a symbol imply structural changes in its referent,
at least under many conditions.23
The analogically referring nature of perceptual symbols
makes their embodiment critical. As the content of a sym-
bol varies, its reference may vary as well. If different intel-
ligent systems have different perceptual systems, the con-
ceptual systems that develop through the schematic symbol
formation process may also differ. Because their symbols
contain different perceptual content, they may refer to dif-
ferent structure in the world and may not be functionally
equivalent.
3.3.1. Adaptive roles of variable embodiment. Variable
embodiment allows individuals to adapt the perceptual
symbols in their conceptual systems to specific environ-
ments. Imagine that different individuals consume some-
what different varieties of the same plants because they live
in different locales. Through perceiving their respective
foods, different individuals develop somewhat different
perceptual symbols to represent them. As a result, some-
what different conceptual systems develop through the
schematic symbol formation process, each tuned optimally
to its typical referents.24
Variable embodiment performs a second useful function,
ensuring that different individuals match perceptual sym-
bols optimally to perception during categorization (sect.
2.4.4). Consider the perception of color. Different individ-
uals from the same culture differ in the detailed psy-
Barsalou: Perceptual symbol systems
598 BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4

chophysical structure of their color categories (Shevell &
He 1995; V. Smith et al. 1976). Even individuals with nor-
mal color vision discriminate the same colors in somewhat
different ways, because of subtle differences in their per-
ceptual systems. As a result, when the schematic symbol
formation process establishes the ability to simulate colors,
somewhat different simulators arise in different individu-
als. Because each simulator is grounded in a subtly differ-
ent implementation of the same perceptual system, it rep-
resents the same symbols in subtly different manners. Most
important, each individual’s simulator for color optimally
matches their symbols for colors to their perceptions of col-
ors. As different individuals search for bananas at the gro-
cery store, they can simulate the particular representation
of yellow that each is likely to perceive on encountering
physical instances.
Because humans vary on all phenotypic traits to some ex-
tent, they are likely to vary on all the perceptual discrimi-
nations that could be extracted to form perceptual symbols,
not just color. Similar variability should arise in the percep-
tion of shape, texture, space, pitch, taste, smell, movement,
introspection, and so forth. If so, then variable embodiment
allows the human conceptual system to adapt itself natu-
rally to variability in perceptual systems. In contrast, such
adaptability is not in the spirit of functionalism. Because
functionalism rests on amodal symbols that bear no struc-
tural relation to their referents, it neither anticipates nor ac-
commodates individual variability in perception.
3.3.2. Variable embodiment in conceptual variability and
stability. Earlier we saw that conceptual variability arises
out of simulators (sect. 2.4.5). When the simulator for a cat-
egory produces different simulations, conceptual variabil-
ity arises both between and within individuals. Variable em-
bodiment provides a second source of variability. Different
individuals represent the same concept differently because
their perceptual symbols become tuned to somewhat dif-
ferent physical environments and develop in somewhat dif-
ferent perceptual systems.
We also saw earlier that simulators provide conceptual
stability (sects. 2.4.5, 3.2.9). When different simulations can
be traced back to the same simulator, they become unified
as instances of the same concept. Embodiment similarly
provides stability. Although perceptual systems produce
variable embodiment across different individuals, they also
produce shared embodiment at a more general level. Be-
cause most humans have roughly the same mechanisms for
perceiving color, they have roughly the same conceptual
representations of it. Although perceptual systems induce
idiosyncrasies in perception and conception, they also in-
duce basic commonalities (Newton 1996).
3.4. Abstract concepts
Representing abstract concepts poses a classic challenge for
perceptual theories of knowledge. Although representing
concrete objects has sometimes seemed feasible, repre-
Barsalou: Perceptual symbol systems
BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4 599
Figure 6. An example of how transforming a word or an amodal symbol fails to produce an analogous transformation in reference,
whereas transforming a perceptual simulation does.

senting events, mental states, social institutions, and other
abstract concepts has often seemed problematic.
3.4.1. Representing abstract concepts metaphorically.
Cognitive linguists have suggested that metaphor provides
a perceptually based solution to the representation of ab-
stract concepts (e.g., Gibbs 1994; Johnson 1987; Lakoff
1987; Lakoff & Johnson 1980; Lakoff & Turner 1989;
Turner 1996). For example, Lakoff and Johnson (1980) sug-
gest that the concrete domain of liquid exploding from a
container represents the abstract concept of anger. Al-
though metaphor most certainly plays a major role in elab-
orating and construing abstract concepts, it is not sufficient
for representing them (Barsalou et al. 1993; Murphy 1996).
Instead, a direct, nonmetaphorical representation of an ab-
stract domain is essential for two reasons: first, it constitutes
the most basic understanding of the domain. Knowing only
that anger is like liquid exploding from a container hardly
constitutes an adequate concept. If this is all that people
know, they are far from having an adequate understanding
of anger. Second, a direct representation of an abstract do-
main is necessary to guide the mapping of a concrete do-
main into it. A concrete domain cannot be mapped system-
atically into an abstract domain that has no content.
As research on emotion shows (e.g., G. Mandler 1975;
Shaver et al. 1987; Stein & Levine 1990), people have di-
rect knowledge about anger that arises from three sources
of experience. First, anger involves the appraisal of an ini-
tiating event, specifically, the perception that an agent’s goal
has been blocked. Second, anger involves the experience of
intense affective states. Third, anger often involves behav-
ioral responses, such as expressing disapproval, seeking re-
venge, and removing an obstacle. As people experience
each of these three aspects of anger, they develop knowl-
edge directly through the schematic symbol formation
process (sect. 2.2).
Although people may understand anger metaphorically
at times, such understanding elaborates and extends the di-
rect concept. Furthermore, metaphorical language may of-
ten indicate polysemy rather than metaphorical conceptu-
alization (Barsalou et al. 1993). For example, when
someone says, “John exploded in anger,” the word “ex-
plode” may function polysemously. “Explode” may have
one sense associated with heated liquid exploding in con-
tainers, and another associated with the rapid onset of an-
gry behavior. Rather than activating conceptual knowledge
for liquid exploding from a container, “explode” may simply
activate a perceptual simulation of angry behavior directly.
As in the direct interpretation of indirect speech acts that
bypass literal interpretation (Gibbs 1983; 1994), familiar
metaphors may produce direct interpretations that bypass
metaphorical mappings. Just as “Can you pass the salt?” by-
passes its literal meaning to arrive directly at a pragmatic re-
quest, so can “explode” bypass its concrete sense to arrive
directly at its introspective sense. Although novel meta-
phors may typically require a metaphorical mapping, fa-
miliar metaphors may bypass this process through poly-
semy.
3.4.2. Representing abstract concepts directly with per-
ceptual symbols. Ideally, it should be shown that percep-
tual symbol systems can represent all abstract concepts di-
rectly. Such an analysis is not feasible, however, given the
large number of abstract concepts. An alternative strategy
is to select quintessential abstract concepts and show that
perceptual accounts are possible. The next two subsections
provide perceptual accounts of two such concepts, truth
and disjunction, as well as related concepts in their seman-
tic fields. If challenging abstract concepts like these can be
represented perceptually, we have good reason to believe
that other abstract concepts are also tractable.
In developing perceptual accounts of these abstract con-
cepts and others, a general trend has emerged. Across these
concepts, three mechanisms appear central to their repre-
sentation. First, an abstract concept is framed against the
background of a simulated event sequence. Rather than be-
ing represented out of context in a single time slice, an ab-
stract concept is represented in the context of a larger body
of temporally extended knowledge (Barwise & Perry 1983;
Fillmore 1985; Langacker 1986; Newton 1996; Yeh &
Barsalou 1999a; 1999b). As we saw earlier in the sections
on simulators (sect. 2.4) and framing (sect. 2.5.3), it is pos-
sible to simulate event sequences perceptually, and it is pos-
sible for a simulator to frame more specific concepts. Thus,
perceptual symbol systems can implement the framing that
underlies abstract concepts.
Second, selective attention highlights the core content of
an abstract concept against its event background (Lan-
gacker 1986; Talmy 1983). An abstract concept is not the
entire event simulation that frames it but is a focal part of
it. As we saw earlier in the section on schematic symbol for-
mation (sect. 2.2), it is possible to focus attention on a part
of a perceptual simulation analytically. In this way, percep-
tual symbol systems capture the focusing that underlies ab-
stract concepts.
Third, perceptual symbols for introspective states are
central to the representation of abstract concepts. If one
limits perceptual symbols to those that are extracted from
perception of the external world, the representation of ab-
stract concepts is impossible. As we saw earlier in the sec-
tion on multimodal symbols, the same symbol formation
process that operates on the physical world can also oper-
ate on introspective and proprioceptive events. As a result,
the introspective symbols essential to many abstract con-
cepts can be represented in a perceptual symbol system. Al-
though many different introspective events enter into the
representation of abstract concepts, propositional construal
appears particularly important (sect. 3.2). As we shall see,
the act of using a mental state to construe the physical world
is often central.
Together, these three mechanisms – framing, selectivity,
and introspective symbols – provide powerful tools for rep-
resenting abstract concepts in perceptual symbol systems.
As we shall see shortly, they make it possible to formulate
accounts of truth, disjunction, and related concepts. This
success suggests a conjecture about the representation of
abstract concepts: framing, selectivity, and introspective
symbols allow a perceptual symbol system to represent any
abstract concept. This conjecture in turn suggests a general
strategy for discovering these representations. First, iden-
tify an event sequence that frames the abstract concept.
Second, characterize the multimodal symbols that repre-
sent not only the physical events in the sequence but also
the introspective and proprioceptive events. Third, identify
the focal elements of the simulation that constitute the core
representation of the abstract concept against the event
background. Finally, repeat the above process for any other
event sequences that may be relevant to representing the
Barsalou: Perceptual symbol systems
600 BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4

concept (abstract concepts often refer to multiple events,
such as marriage referring to a ceremony, interpersonal re-
lations, domestic activities, etc.). If the conjecture is cor-
rect, this strategy should always produce a satisfactory ac-
count of an abstract concept.
Of course, the success of this strategy does not entail that
people actually represent abstract concepts this way. Such
conclusions await empirical assessment and support. How-
ever, success is important, because it demonstrates that
perceptual symbol systems have the expressive power to
represent abstract concepts. In a pilot study with Karen
Olseth Solomon, we have obtained preliminary support for
this account. Subjects produced more features about event
sequences and introspection for abstract concepts, such as
truth and magic, than for concrete concepts, such as car and
watermelon. Recent neuroscience findings are also consis-
tent with this proposal. As reviewed by Pulvermüller
(1999), abstract concepts tend to activate frontal brain re-
gions. Of the many functions proposed for frontal regions,
two include the coordination of multimodal information
and sequential processing over time. As just described, ab-
stract concepts exhibit both properties. On the one hand,
they represent complex configurations of multimodal in-
formation that must be coordinated. On the other, they rep-
resent event sequences extended in time. Thus, frontal ac-
tivation is consistent with this proposal.
3.4.3. Representing truth, falsity, negation, and anger.
This analysis does not attempt to account for all senses of
truth, nor for its formal senses. Only a core sense of peo-
ple’s intuitive concept of truth is addressed to illustrate this
approach to representing abstract concepts. In the pilot
study with Solomon, subjects described this sense fre-
quently. Figure 7A depicts the simulated event sequence
that underlies it. In the first subevent, an agent constructs
a perceptual simulation of a balloon above a cloud using the
productive mechanisms described earlier. The agent might
have constructed this simulation on hearing a speaker say,
“There’s a balloon above a cloud outside,” or because the
agent had seen a balloon above a cloud earlier, or for some
other reason. Regardless, at this point in the event se-
quence, the agent has constructed a simulation. In the sec-
ond subevent, the agent perceives a physical situation (i.e.,
the scene inside the thick solid border), and attempts to
map the perceptual simulation into it. The agent may at-
tempt this mapping because a speaker purported that the
statement producing the simulation was about this particu-
lar situation, because the agent remembered that the sim-
ulation resulted from perceiving the situation earlier, or for
some other reason. The agent then assesses whether the
simulation provides an accurate representation of the situ-
ation, as described earlier for propositional construal. In
this case, it does. Analogous to the simulation, the situation
contains a balloon and a cloud, and the balloon is above the
cloud. On establishing this successful mapping, the agent
might say, “It’s true that a balloon is above a cloud,” with
“true” being grounded in the mapping.
This account of truth illustrates the three critical mech-
anisms for representing abstract concepts. First, a simu-
lated event sequence frames the concept. Second, the con-
cept is not the entire simulation but a focal part of it,
specifically, the outcome that the simulation provides an ac-
curate construal of the situation. Third, perceptual symbols
for introspective events are central to the concept, includ-
ing those for a perceptual simulation, the process of com-
paring the perceptual simulation to the perceived situation,
and the outcome of establishing a successful mapping be-
tween them. After performing this complex event sequence
on many occasions, a simulator develops for truth, that is,
people learn to simulate the experience of successfully
mapping an internal simulation into a perceived scene.25
The concept of falsity is closely related to the concept of
truth. The account here addresses only one sense of peo-
ple’s intuitive concept for falsity, although it too is polyse-
mous and has formal interpretations. As Figures 7A and 7B
illustrate, very similar event sequences underlie these two
concepts. In both, a simulation is constructed initially, fol-
lowed by the perception of a situation. The two sequences
differ only in whether the simulation can or cannot be
mapped successfully into the situation. Whereas the map-
ping succeeds for truth, it fails for falsity. Thus, a speaker,
after failing to map the simulation into the situation in Fig-
ure 7B, might say, “It is false that there is a balloon above a
cloud,” with “false” being grounded in the failed mapping.
The slant mark through the line between the simulated bal-
loon and its absence in perception is a theoretical device,
not a cognitive entity, that stands for a failed attempt at fus-
ing them.26
The concept of negation is also closely related to the con-
cept of truth. Again, the account here addresses only one
sense of people’s intuitive concept. As Figure 7C illustrates,
negation results from explicitly noting the absence of a
binding between a simulator and a simulation. In this
particular case, the simulator for balloon is not bound to
anything in the upper region of the above simulation. Ex-
plicitly noting the absence of a binding becomes part of the
Barsalou: Perceptual symbol systems
BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4 601
Figure 7. (A) Accounting for one sense of truth using perceptual
symbols. (B) Accounting for one sense of falsity using perceptual
symbols. (C) Accounting for one sense of negation using percep-
tual symbols. Boxes with thin solid lines represent simulators;
boxes with thick dashed lines represent simulations; boxes with
thick solid lines represent perceived situations.

schematic simulation in the same perceptual systems that
perceive the individual, with the simulation adding inferred
features. Because the simulation and the perception are
represented in shared perceptual systems, they become
fused together. In contrast, standard views assume that
amodal features in a nonperceptual system become active
to represent a perceived individual, with the cognitive and
perceptual representations remaining separate.
The concepts that underlie categorization also take on
new forms from the perspective of perceptual symbol sys-
tems. Rather than being a static amodal structure, a concept
is the ability to simulate a kind of thing perceptually. Be-
cause simulators partially reproduce perceptual experi-
ence, spatiotemporal relations and transformations become
central to concepts. Whereas such knowledge is cumber-
some and brittle in amodal symbol systems (e.g., Clark
1997; Glasgow 1993; McDermott 1987; Winograd & Flores
1987), it “rides for free” in perceptual symbol systems
(Goldstone & Barsalou 1998). Conceptual combination
similarly takes on a new character in perceptual symbol sys-
tems. Rather than being the set-theoretic combination of
amodal features, it becomes the hierarchical construction
of complex perceptual simulations.
Perceptual symbol systems also offer a different per-
spective on the proposal that intuitive theories provide
background knowledge about concepts (Murphy & Medin
1985). As we saw earlier (sect. 2.5.3), concepts are often
framed in the context of perceptually simulated events.
This type of framing may underlie many of the effects at-
tributed to intuitive theories, and it may account for them
in a more implicit and less formal manner. Rather than as-
suming that people use theories to frame concepts, it may
be more realistic to assume that people use simulated
events from daily experience. For example, an intuitive the-
ory about biology is often assumed to frame concepts of nat-
ural kinds. Alternatively, simulations of biological events
may frame these concepts and represent implicitly the the-
oretical principles that underlie them. Specifically, simula-
tions of birth, growth, and mating may frame concepts like
animal and represent theoretical principles such as moth-
ers give birth to babies, who grow up to produce their own
offspring. Simulations of various transformations may fur-
ther support inferences that distinguish natural kinds from
artifacts, such as the ability of leg to bend for human but not
for table. Similarly, simulations of rolling out pizzas versus
the minting of quarters may underlie different inferences
about variability in diameter (Rips 1989). Event simulations
may also represent functions, such as the function of cups
being represented in simulations of drinking from them.
Whenever an intuitive theory appears necessary to explain
a conceptual inference, a simulated event sequence may
provide the requisite knowledge.
4.1.3. Attention. Attention is central to schematic symbol
formation. By focusing on an aspect of perceptual experi-
ence and transferring it to long-term memory, attention
overcomes the limits of a recording system. Attention is a
key analytic mechanism in parsing experience into the
schematic components that ultimately form concepts. Al-
though attention has played important roles in previous
theories of concepts (e.g., Nosofsky 1984; Trabasso &
Bower 1968), its role here is to create the schematic per-
ceptual symbols that compose simulators.
Once a simulator becomes established, it in turn controls
attention (cf. Logan 1995). As a simulator produces a sim-
ulation, it controls attention across the simulation. Thus, at-
tention becomes a semantic feature, as when it focuses on
the upper versus the lower region of the same spatial rep-
resentation to distinguish above from below. The control of
attention can also contrast a focal concept against a back-
ground simulation, as we saw in the sections on framing
(sect. 2.5.3) and abstract concepts (sect. 3.4). As these ex-
amples illustrate, attention takes on important new roles in
the context of perceptual symbol systems.
Perceptual symbol systems also provide natural accounts of
traditional attentional phenomena. For example, automatic
processing is the running of a highly compiled simulation,
whereas strategic processing is the construction of a novel
simulation using productive mechanisms. Priming is essen-
tially perceptual anticipation (Glenberg 1997; Neisser 1976),
namely, the top-down activation of a simulation that matches
a perceptual input and facilitates its processing. In con-
trast, inhibition is the top-down suppression of a simulator.
4.1.4. Working memory. Current accounts of working mem-
ory construe it as a complex set of limited capacity mecha-
nisms, including an executive processor and several modal-
ity specific buffers, such as an articulatory loop and a visual
short-term memory (Baddeley 1986). From the perspective
of perceptual symbol systems, working memory is the sys-
tem that runs perceptual simulations. The articulatory loop
simulates language just heard or about to be spoken. The vi-
sual short-term buffer simulates visual experience just seen
or currently imagined. The motor short-term buffer simu-
lates movements just performed or about to be performed.
The gustatory short-term buffer simulates tastes just experi-
enced or anticipated. The executive processor simulates the
execution of procedures just executed or about to be per-
formed. Not only do these working memory systems oper-
ate during perception, movement, and problem solving,
they can also be used to simulate these activities offline.
Standard theories of cognition assume that working mem-
ory contains perceptual representations, whereas long-
term memory contains amodal representations (e.g.,
Baddeley 1986; Kosslyn 1980). From the perspective of per-
ceptual symbol systems, both systems are inherently per-
ceptual, sharing neural systems with perception. Whereas
long-term memory contains simulators, working memory
implements specific simulations.
4.1.5. Long-term memory. Encoding information into long-
term memory is closely related to categorization, because
both involve propositional construal. When an individual is
encountered, its categorization into a type-token relation
encodes a proposition (sect. 3.2.1). To the extent that this
proposition receives processing, it becomes established in
long-term memory. The many varieties of elaboration and
organization in the encoding literature can all be viewed in
this manner. Indeed, various findings suggest that elabora-
tion and organization are inherently perceptual (e.g.,
Brandimonte et al. 1997; Intraub et al. 1998; M. Martin &
Jones 1998). From the perspective of perceptual symbol
systems, encoding produces a fusion of bottom-up sensa-
tion and top-down simulation (sect. 2.4.7). Under concep-
tually driven orienting tasks, a fusion contains increased in-
formation from simulation; under data-driven orienting
tasks, a fusion contains increased information from sensa-
tion (cf. Jacoby 1983). Furthermore, rich sensory informa-
Barsalou: Perceptual symbol systems
604 BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4

tion suggests that the memory resulted from perception,
not imagination (Johnson & Raye 1981).
Once a type-token fusion becomes stored in long-term
memory, it can be retrieved on later occasions. Because the
fusion is a perceptual representation, its retrieval is essen-
tially an attempt to simulate the original entity or event.
Thus, memory retrieval is another form of perceptual sim-
ulation (Glenberg 1997; cf. Conway 1990; Kolers & Roedi-
ger 1984), with fluent simulations producing attributions of
remembrance (Jacoby et al. 1989). Such simulations can
become active unconsciously in implicit memory, or they
can become active consciously in explicit memory. Recon-
structive memory reflects the unbidden contribution of a
simulator into the retrieval process. As a memory is re-
trieved, it produces a simulation of the earlier event. As the
simulation becomes active, it may differ somewhat from the
original perception, perhaps because of less bottom-up
constraint. To the extent that the remembered event’s fea-
tures have become inaccessible, the simulator converges on
its default simulation. The result is the wide variety of re-
constructive effects reported in the literature.
4.1.6. Language processing. Language comprehension
can be viewed as the construction of a perceptual simula-
tion to represent the meaning of an utterance or text. As
suggested earlier, the productive properties of language
guide this process, with a combination of words providing
instructions for building a perceptual simulation (Lan-
gacker 1986; 1987; 1991; 1997; MacWhinney 1998). A typ-
ical sentence usually contains reference to specific individ-
uals, as well as predications of them. For example, “the cup
on Anna’s desk is blue” refers to a particular cup on a par-
ticular desk, and predicates that it is blue. To construct a
simulation of this sentence, the comprehender simulates
the individual desk and cup and then specializes the color
of the cup. Later sentences update the simulation by chang-
ing the individuals present and/or transforming them.
Thus, “it contains pens and pencils” adds new individuals to
the simulation, inserting them inside the cup. The afford-
ances of a simulation may often produce inferences during
comprehension. For example, spatial properties of the
pens, pencils, and cup determine that the pens and pencils
sit vertically in the cup, leaning slightly against its lip. If the
pens and pencils had instead been placed in a drawer, their
orientation would have been horizontal. In an amodal rep-
resentation, such inferences would not be made, or they
would require cumbersome logical formulae.
As individuals and their properties become established in
a simulation, they become fused with the simulators that con-
strue them. Thus, the pens, pencils, and cup become fused
with simulators for pen, pencil, and cup, and the blue color
of the cup becomes fused with blue. As a result, the simula-
tors produce inferences about the individuals as needed. For
example, if the text stated, “Anna checked whether one of the
pens worked, and it didn’t,” the simulator for pen might sim-
ulate inferences such as Anna pressed a button on top of a
pen and when the pen didn’t work, an uncomfortable feeling
resulted from the dry pen scraping across the paper. Simi-
larly, if the text stated, “Anna would have preferred a cup in
a lighter shade of blue,” the simulator for blue might simu-
late a lighter shade that simulates the cup negatively but that
produces a more positive affective response in the simulation
of Anna’s mental state. As comprehension proceeds, repre-
sentations of individuals develop, as in the perception of a
physical scene. Simultaneously, simulators become fused
with these imagined individuals to represent propositions
about them, much like the type-token mappings that develop
during the categorization of physical entities.29
As these examples illustrate, perceptual simulation offers
a natural account of how people construct the meanings of
texts, or what other researchers have called situation mod-
els and mental models (e.g., Johnson-Laird 1983; Just &
Carpenter 1987; van Dijk & Kintsch 1983). A variety of
findings can be interpreted as evidence that perceptual
simulation underlies these models (e.g., Black et al. 1979;
Bransford & Johnson 1973; Gernsbacher et al. 1990; Gibbs
1994; Glenberg et al. 1987; Intraub & Hoffman 1992; Mor-
row et al. 1987; Potter & Faulconer 1975; Potter et al. 1986;
Potter et al. 1977; Von Eckardt & Potter 1985; Rinck et al.
1997; Wilson et al. 1993).
4.1.7. Problem solving, decision making, and skill. From
the perspective of perceptual symbol systems, problem solv-
ing is the process of constructing a perceptual simulation
that leads from an initial state to a goal state. Problem solvers
can work forward from the initial state or backward from the
goal state, but in either case they attempt to simulate a plan
that achieves the goal. In novice problem solving, the diffi-
culty is finding satisfactory components of the simulation
and ordering them properly. If novices start from the initial
state, they may not know the next event to simulate that will
lead to the goal. A component may be added to the simula-
tion that has been successful for achieving this type of goal
in the past, or a component might be added because its sim-
ulated affordances suggest that it may work. For example,
one might want to remove caulk between a wall and the
counter in a kitchen. If one has never done this before, var-
ious plans can be simulated to see which works best (e.g., us-
ing a scraper versus a chemical solvent).
Decision making can be viewed as specializing a simu-
lated plan in different ways to see which specialization pro-
duces the best outcome (cf. the simulation heuristic of Kah-
neman & A. Tversky 1982). In evaluating plans to remove
caulk from a joint, a decision must be made about how to
specialize the region of the handheld instrument. As possi-
ble specializations are retrieved, each is simulated to assess
which works best. A wide variety of decisions can be viewed
this way, including decisions about purchases, occupations,
social events, and so forth. For each, an agent simulates a
plan to achieve a goal and then tries out disjunctive spe-
cializations of a region to see which yields the most promis-
ing outcome. In essence, making a decision involves con-
structing an ad hoc category for a plan region and selecting
one of its members (sect. 3.4.4).
Skill results from compiling simulations for most of the
plans in a domain through extensive experience (cf. Ander-
son 1993; Logan 1988; Newell 1990). Rather than search-
ing for plan components and making decisions about how
to specialize plan regions, experts retrieve compiled simu-
lations that achieve goals with minimal transformation.
During plan execution, simulated plans run slightly ahead
of perception. As in propositional construal, simulation and
perception become closely entrained. Expertise is achieved
when an agent can almost always simulate what is about to
occur, rarely stopping to revise the simulation.
4.1.8. Reasoning and formal symbol manipulation. To see
how perceptual symbol systems could underlie logical rea-
Barsalou: Perceptual symbol systems
BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4 605

soning, consider modus ponens. In modus ponens, if the
premise XrY is true, and if the premise X is also true, then
the conclusion Y follows. Shortly, a formal account of
modus ponens in perceptual symbol systems will be pre-
sented, but first an implicit psychological account is devel-
oped. Imagine that an agent is told, If a computer is a Mac-
intosh, then it has a mouse. On hearing this, the agent
constructs a perceptual simulation of a Macintosh that in-
cludes a mouse, thereby representing the premise, XrY,
informally (cf. Johnson-Laird 1983). On a later occasion,
when a particular Macintosh is encountered (i.e., premise
X), it activates the simulation of a Macintosh constructed
earlier, which includes a mouse. As the simulation is fused
with the individual, a mouse is simulated, even if a physical
mouse is not perceived. Through this psychological ana-
logue to modus ponens, the inference that the individual
has a mouse is drawn (i.e., Y).
Similar use of simulation could underlie syllogisms such
as Every B is C, A is B, therefore A is C. Imagine that, over
time, an agent experiences a mouse with every Macintosh.
As a result, mice become strongly established in the simu-
lator for Macintosh, so that all simulations of a Macintosh
include one (i.e., Every B is C). Later, when a new entity is
categorized as a Macintosh (i.e., A is B), the simulator for
Macintosh produces a simulation that includes a mouse,
thereby drawing the syllogistic inference (i.e., A is C).
To the extent that C does not always covary with B, the
certainty of C decreases, giving the inference a statistical
character (Oaksford & Chater 1994). If an agent has expe-
rienced Macintoshes without mice, the inference that a
perceived individual has one is less certain, because simu-
lations can be constructed without them as well as with
them. To the extent that simulations with mice are easier to
construct than simulations without mice, however, the in-
ference is compelling. As a simulation becomes more flu-
ent, its perceived likelihood increases (Jacoby et al. 1989).
Widespread content effects in reasoning are consistent
with perceptual simulation. As many researchers have re-
ported, reasoning improves when the abstract variables in
arguments are replaced with familiar situations. For exam-
ple, people draw the invalid inference of affirming the con-
sequent in arguments stated with abstract variables (i.e., re-
ceiving the premises XrY and Y, and then concluding X).
However, in a familiar domain such as computers, if X is a
Macintosh and Y is a mouse, people do not affirm the con-
sequent, because they can think of non-Macintosh com-
puters that have mice. From the perspective of perceptual
symbol systems, this improvement reflects the ability to
construct relevant simulations (cf. Johnson-Laird 1983). To
the extent that the critical events have been experienced,
information becomes stored that produces the simulations
necessary to drawing only valid inferences.
Perceptual simulation offers a similar account of causal
reasoning. Cheng and Novick (1992) propose that the
strength of a causal inference reflects the difference be-
tween the probability of an event leading to an outcome and
the probability of the event not leading to the outcome,
with increasingly large differences producing increasingly
strong inferences. A perceptual symbol system can com-
pute these differences using perceptual simulations. To the
extent that it is easier to simulate an event leading to an out-
come than the event not leading to the outcome, the event
is construed as a likely cause of the outcome. Indeed, peo-
ple appear to construct such simulations to assess causality
(Ahn & Bailenson 1996; Ahn et al. 1995). Covariation is also
critical, though, because the more the event and outcome
covary, the greater the fluency of the simulation, and the
stronger the causal attribution (cf. Jacoby et al. 1989).
Much additional research in social cognition illustrates that
the ease of simulating a scenario underlies the acceptabil-
ity of a causal explanation (e.g., K. Markman et al. 1993;
Pennington & Hastie 1992; Wells & Gavinski 1989).
Finally, it is possible to explain formal symbol manipula-
tion in logic and mathematics through the simulation of ar-
bitrary symbols. From perceptual experience with external
symbols and operations, the ability to construct analogous
simulations internally develops. For example, after watch-
ing an instructor work through modus ponens, a student
may develop the ability to simulate the formal procedure
internally. The student learns to simulate the two premises
followed by the inferred conclusion, analogous to how they
would be manipulated externally (i.e., simulate “XrY” and
“X” as given, then simulate “Y” as true). Furthermore, the
student develops the ability to simulate, replacing variables
with constants. Thus, if students receive “Macintoshr
mouse, Macintosh” in a problem, they know that memory
should be searched for a simulated rule that this pattern of
constants can specialize. On retrieving the simulation for
modus ponens, the students see that the perceived form of
the constants matches the simulated form of the inference
rule, indicating that the simulation can be applied. Running
the simulation requires first specializing the variables in the
simulated premises with the constants from the problem.
The simulation then produces “Y” as an inference and spe-
cializes it with its value from the first premise, thereby sim-
ulating the correct conclusion, “mouse.”
As this example illustrates, the same basic processes that
simulate natural entities and events can also simulate for-
mal entities and events. It is worth adding, though, that
people often construct nonformal simulations to solve for-
mal problems. For example, mathematicians, logicians, and
scientists often construct visual simulations to discover and
understand formalisms (e.g., Barwise & Etchemendy 1990,
1991; Hadamard 1949; Thagard 1992). Nonacademics sim-
ilarly use nonformal simulations to process formalisms (e.g.,
Bassok 1997; Huttenlocher et al. 1994). Whereas proofs en-
sure the properties of a formalism, perceptual simulations
often lead to its discovery in the first place.
4.2. Implications for evolution and development
Amodal symbol systems require a major leap in evolution.
Assuming that nonhuman animals do not have amodal sym-
bol systems, humans must have acquired a radically new
form of neural hardware to support a radically new form of
representation. Of course, this is possible. However, if a
more conservative evolutionary path also explains the hu-
man conceptual system, parsimony favors it, all other fac-
tors being equal.
Not only do perceptual symbol systems offer a more par-
simonious account of how intelligence evolved, they also es-
tablish continuity with nonhuman animals. On this view,
many animals have perceptual symbol systems that allow
them to simulate entities and events in their environment.
Such simulations could produce useful inferences about
what is likely to occur at a given place and time, and about
what actions will be effective. Because many animals have
attention, working memory, and long-term memory, they
Barsalou: Perceptual symbol systems
606 BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4

could readily extract elements of perception analytically, in-
tegrate them in long-term memory to form simulators, and
construct specific simulations in working memory. If so,
then the human conceptual ability is continuous with the
nonhuman conceptual ability, not discontinuous.
Where human intelligence may diverge is in the use of
language to support shared simulations (e.g., Donald 1991;
1993; Tomasello et al. 1993). Evolution may have built upon
perceptual symbol systems in nonhuman primates by
adding mechanisms in humans for uttering and decoding
rapid speech, and for linking speech with conceptual simu-
lations. Rather than evolving a radically new system of rep-
resentation, evolution may have developed a linguistic sys-
tem that extended the power of existing perceptual symbol
systems. Through language, humans became able to con-
trol simulations in the minds of others, including simula-
tions of mental states. As a result, humans became able to
coordinate physical and mental events in the service of
common goals. Whereas nonhumans primarily construct
simulations individually in response to immediate physical
and internal environments, humans construct simulations
jointly in response to language about nonpresent situations,
thereby overcoming the present moment.
Human development may follow a similar path, with on-
togeny loosely recapitulating phylogeny (K. Nelson 1996).
Similar to nonhumans, infants may develop simulators and
map them into their immediate world. During early devel-
opment, infants focus attention selectively on aspects of ex-
perience, integrate them in memory, and construct simula-
tors to represent entities and events (cf. Cohen 1991; Jones
& L. Smith 1993; J. Mandler 1992; L. Smith et al. 1992).
Long before infants use language, they develop the ability
to simulate many aspects of experience. By the time infants
are ready for language, they have a tremendous amount of
knowledge in place to support its acquisition. As they en-
counter new words, they attach them to the relevant simu-
lators. New words may sometimes trigger the construction
of a new simulator, or a new aspect of an existing one (cf. E.
Markman 1989; Tomasello 1992). Much of the time, how-
ever, new words may map onto existing simulators and their
parts. As linguistic skill develops, children learn to con-
struct simulations productively from other people’s utter-
ances, and to construct utterances that convey their inter-
nal simulations to others.
Analogous to perceptual symbol systems being continu-
ous across evolution, perceptual symbol systems are con-
tinuous across development, with the addition of linguistic
control added to achieve social coordination and cultural
transmission. Certainly, perceptual symbol systems must
change somewhat over both evolution and development.
Indeed, the principle of variable embodiment implies such
differences (sect. 3.3). To the extent that different animals
have different perceptual and bodily systems, they should
have different conceptual systems. Similarly, as infants’ per-
ceptual and bodily systems develop, their conceptual sys-
tems should change accordingly. Nevertheless, the same
basic form of conceptual representation remains constant
across both evolution and development, and a radically new
form is not necessary.
4.3. Implications for neuroscience
Much more is known about how brains implement per-
ception than about how they implement cognition. If
perception underlies cognition, then what we know about
perception can be used to understand cognition. Neural ac-
counts of color, form, location, and movement in percep-
tion should provide insights into the neural mechanisms
that represent this information conceptually. Much re-
search has established that mental imagery produces
neural activity in sensory-motor systems, suggesting that
common neural mechanisms underlie imagery and per-
ception (e.g., Crammond 1997; Deschaumes-Molinaro et
al. 1992; Farah 1995; Jeannerod 1994; 1995; Kosslyn 1994;
Zatorre et al. 1996). If perceptual processing also under-
lies cognition, then common sensory-motor mechanisms
should be active for all three processes. As described ear-
lier (sect. 2.3), increasing neuroscientific evidence sup-
ports this hypothesis, as does increasing behavioral evi-
dence (e.g., Barsalou et al., in press; Solomon & Barsalou
1999a; 1999b; Wu & Barsalou 1999).
Because perception, imagery, and cognition are not iden-
tical behaviorally, their neuroanatomical bases should not
be identical. Theorists have noted neuroanatomical differ-
ences between perception and imagery (e.g., Farah 1988;
Kosslyn 1994), and differences also certainly exist between
perception and cognition. The argument is not that per-
ception and cognition are identical. It is only that they share
representational mechanisms to a considerable extent.
4.4. Implications for artificial intelligence
Modern digital computers are amodal symbol systems.
They capture external input using one set of sensing mech-
anisms (e.g., keyboards, mice) and map it to a different set
of representational mechanisms (e.g., binary strings in
memory devices). As a result, arbitrary binary strings come
to stand for the states of input devices (e.g., 1011 stands for
a press of the period key).
Nevertheless, a perceptual symbol system could be im-
plemented in a current computer. To see this, imagine that
a computer has a set of peripheral devices that connect it to
the world. At a given point in time, the “perceptual state”
of the machine is the current state of its peripheral devices.
If the machine can focus selectively on a small subset of a
perceptual state, associate the subset’s components in
memory to form a perceptual symbol, integrate this mem-
ory with related memories, and later reproduce a superim-
position of them in the peripheral device to perform con-
ceptual processing, it is a perceptual symbol system. The
machine stores and simulates perceptual symbols in its per-
ceptual systems to perform basic computational operations
– it does not process transduced symbols in a separate cen-
tral processor.
In such a system, a perceptual symbol can later become
active during the processing of new input and be fused with
the input to form a type-token proposition (in a peripheral
device). If different perceptual symbols become fused with
the same perceptual input, different construals result. The
system could also implement productivity by combining the
activation of peripheral states in a top-down manner. Most
simply, two symbols could be activated simultaneously and
superimposed to form a more complex state never encoun-
tered.
Variable embodiment has implications for such imple-
mentations. Because peripheral devices in computers dif-
fer considerably from human sensory-motor systems, im-
plementations of perceptual symbol systems in computers
Barsalou: Perceptual symbol systems
BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4 607

should have a distinctly nonhuman character. Contrary to
the claims of functionalism, computers should not be capa-
ble of implementing a human conceptual system, because
they do not have the requisite sensory-motor systems for
representing human concepts. Although it is intriguing to
consider how a perceptual symbol system might be imple-
mented technologically, it is probably naive to believe that
such a system would correspond closely to human intelli-
gence. Such correspondence awaits the development of ar-
tifacts that are much more biological in nature.
5. Conclusion
Once forgotten, old ideas seem new. Perhaps the percep-
tual approach to cognition has not been altogether forgot-
ten, but it has appeared so strange and improbable in mod-
ern cognitive science that it has been relegated to the
periphery. Rather than resurrecting older perceptual theo-
ries and comparing them to amodal theories, reinventing
perceptual theories in the contexts of cognitive science and
neuroscience may be more productive. Allowing these con-
texts to inspire a perceptual theory of cognition may lead to
a competitive and perhaps superior theory.
Clearly, every aspect of the theory developed here must
be refined, including the process of schematic symbol for-
mation, the construct of a simulator, the productive use of
language to construct simulations, the fusing of simulations
with perceived individuals to produce propositions, the
ability to represent abstract concepts, and so forth. Ideally
these theoretical analyses should be grounded in neural
mechanisms, and ideally they should be formalized com-
putationally. Furthermore, a strong empirical case needs to
be established for myriad aspects of the theory. Again, the
goal here has only been to demonstrate that it is possible to
ground a fully functional conceptual system in sensory-
motor mechanisms, thereby giving this classic theory new
life in the modern context.
ACKNOWLEDGMENTS
This work was supported by National Science Foundation Grants
SBR-9421326 and SBR 9796200. I am indebted to Karen Olseth
Solomon, Ling-Ling Wu, Wenchi Yeh, and Barbara Luka for their
contributions to the larger project in which this paper originates,
and to Jesse Prinz for extensive guidance in reading the philo-
sophical literature. For helpful comments on earlier drafts, I am
grateful to Katy Boerner, Eric Dietrich, Shimon Edelman, Arthur
Glenberg, Robert Goldstone, Annette Herskovits, Alan Malter,
Jean Mandler, Robert McCauley, Douglas Medin, Gregory Mur-
phy, Lance Rips, and Steven Sloman, none of whom necessarily
endorse the views expressed. I am also grateful to Howard Nus-
baum for discussion on cognitive penetration, and to Gay Snod-
grass for permission to use drawings from the Snodgrass and Van-
derwart (1980) norms. Finally, I am grateful to Stevan Harnad for
supporting this article, for extensive editorial advice of great value,
and for coining the term “simulator” as a replacement for “simu-
lation competence” in previous articles.
NOTES
1. This is not a claim about correspondence between percep-
tual symbols and the physical world. Although the structure of
perceptual symbols may correspond to the physical world in some
cases, it may not in others. For example, philosophers have often
argued that a correspondence exists for primary qualities, such as
shape, but not for secondary qualities, such as color (e.g., K.
Lehrer 1989). Neuroscientists have similarly noted topographic
correspondences between neuroanatomical structure and physi-
cal structure (e.g., Tootel et al. 1982; Van Essen 1985).
2. Throughout this paper, double quotes signify words, and
italics signify conceptual representations, both modal and amodal.
Thus, “chair” signifies the word chair, whereas chair signifies the
corresponding concept.
3. Some researchers argue that visual agnosia and optic apha-
sia support a distinction between perception and conception. Be-
cause these disorders are characterized by normal perceptual abil-
ities but damaged conceptual and semantic abilities, they suggest
that perceptual and conceptual abilities reside in distinct neural
systems. Detailed study of these disorders, however, suggests cau-
tion in drawing such conclusions (e.g., Hillis & Caramazza 1995).
When careful behavioral assessments are performed, correlated
deficits between perception and conception may actually be pres-
ent. Also, bottom-up control of sensory-motor areas may remain
after top-down control is lost (sect. 2.4.6). Rather than providing
evidence for two different representational systems, these disor-
ders may provide evidence for two different ways of activating a
common representational system.
4. Various causal accounts of symbol grounding have been pro-
posed in the philosophical literature, but these typically apply only
to a small fragment of concepts and fail to provide a comprehen-
sive account of how symbols in general are grounded. Further-
more, empirical evidence for these theories is typically lacking.
5. Specifying the features computed in the sensory-motor cor-
tices constitutes an undeveloped aspect of the theory. To a con-
siderable extent, this problem belongs to a theory of perception
(but see Schyns et al. 1998). Nevertheless, whatever features turn
out to be important for perception should also be at least some-
what important for cognition. Specifying how associative areas
store patterns of features in sensory-motor areas constitutes an-
other undeveloped aspect of the theory, although the connection-
ist literature suggests many possibilities.
6. When conscious states do occur, they do not necessarily ex-
hibit one-to-one mappings with the neural states that produced
them (Pessoa et al. 1998).
7. Specifying how the cognitive system knows where to focus
attention during the symbol formation process constitutes an un-
developed component of the theory. Barsalou (1993) and Logan
(1995) suggest several possibilities, but this is a central issue that
remains far from resolved in any theory.
8. Jesse Prinz suggested this account of triangle. Note that the
qualitative neurons in this account constitute a modal representa-
tion, not an amodal one. As defined earlier (sect. 1.1), modal refers
to the fact that the same neurons represent triangles perceptually
and conceptually. Thus, the qualitative neurons that represent tri-
angle are not arbitrarily linked to the neural states that arise while
perceiving triangles. Instead, the neurons that represent triangle
conceptually are a subset of those that are active when triangles
are processed perceptually.
9. Recent research suggests that object-oriented reference
frames may not be essential to categorization (e.g., S. Edelman
1998; Tarr & Pinker 1989; but see Biederman & Gerhardstein
1993). Regardless, the proposal here is that object-oriented refer-
ence frames organize knowledge about a type of entity. Although
such reference frames may not always become active during fa-
miliar categorizations, they almost certainly exist in categorical
knowledge, as suggested by people’s robust ability to construct
three-dimensional images and perform transformations on them
(see Finke, 1989, for a review).
10. Section 2.5 provides preliminary accounts of the frame for-
mation and simulation processes. Nevertheless, many crucial as-
pects of these processes remain undeveloped, including (1) the in-
tegration of memories into frames, (2) the retrieval of information
from frames to construct simulations, (3) the integration and
transformation of information in simulations, and (4) the devel-
opment of abstractions.
11. Why cognitive systems divide the world into some cate-
gories but not others remains largely unresolved, as in other theo-
Barsalou: Perceptual symbol systems
608 BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4

On the virtues of going all the way
Shimon Edelmana and Elise M. Breenb
aSchool of Cognitive and Computing Sciences, University of Sussex at
Brighton, Falmer BN1 9QH, England; bLaboratory for Cognitive Brain
Mapping, Brain Research Institute, Institute of Physical and Chemical
Research (RIKEN), Wako, Saitama 351-01, Japan.
shimone@cogs.susx.ac.uk www.cogs.susx.ac.uk/users/shimone
Abstract: Representational systems need to use symbols as internal stand-
ins for distal quantities and events. Barsalou’s ideas go a long way towards
making the symbol system theory of representation more appealing, by
delegating one critical part of the representational burden – dealing with
the constituents of compound structures – to image-like entities. The tar-
get article, however, leaves the other critical component of any symbol sys-
tem theory – the compositional ability to bind the constituents together –
underspecified. We point out that the binding problem can be alleviated
if a perceptual symbol system is made to rely on image-like entities not
only for grounding the constituent symbols, but also for composing these
into structures.
Supposing the symbol system postulated by Barsalou is perceptual
through and through – what then? The target article outlines an
intriguing and exciting theory of cognition in which (1) well-spec-
ified, event- or object-linked percepts assume the role tradition-
ally allotted to abstract and arbitrary symbols, and (2) perceptual
simulation is substituted for processes traditionally believed to re-
quire symbol manipulation, such as deductive reasoning. We take
a more extreme stance on the role of perception (in particular, vi-
sion) in shaping cognition, and propose, in addition to Barsalou’s
postulates, that (3) spatial frames, endowed with a perceptual
structure not unlike that of the retinotopic space, pervade all sen-
sory modalities and are used to support compositionality.
In the target article too, the concept of a frame is invoked as a
main explanatory tool in the discussion of compositionality. The
reader is even encouraged to think of a frame as a structure with
slots where pointers to things and events can be inserted. This,
however, turns out to be merely a convenient way to visualize an
entity borrowed from artificial intelligence: a formal expression
in several variables, each of which needs to be bound to things
or events. An analogy between this use of frames and the second
labor of Heracles suggests itself: opting for perceptual symbols
without offering a perceptual solution to the binding problem
is like chopping off the Hydra’s heads without staunching the
stumps.
The good news is that there is a perceptually grounded alter-
native to abstract frames: spatial (e.g., retinotopic) frames. The
origins of this idea can be traced to a number of sources. In vision,
it is reminiscent of O’Regan’s (1992) call to consider the visual
world (which necessarily possesses an apparently two-dimensional
spatial structure) as a kind of external memory. In language, a
model of sentence processing based on spatial data structures
(two-dimensional activation maps) has been proposed a few years
ago (Miikkulainen 1993). In a review of the latter work, one of us
pointed out that the recourse to a spatial substrate in the process-
ing of temporal structure may lead to a welcome unification of the-
ories of visual and linguistic representation (Edelman 1994).
From the computational standpoint, such unification could be
based on two related principles. The first of these is grounding the
symbols (Harnad 1990) in the external reality; this can be done by
imparting to the symbols some structure that would both help to
disambiguate their referents and help manipulate the symbols to
simulate the manipulation of the referent objects. This principle
is already incorporated into Barsalou’s theory (cf. his Fig. 6). The
second principle, which is seen to be a generalization of the first
one, is grounding the structures built of symbols.
In the case of vision, structures (that is, scene descriptors) can
be naturally grounded in their distal counterparts (scenes) simply
by representing the scene internally in a spatial data structure (as
envisaged by O’Regan). This can be done by “spreading” the per-
ceptual symbols throughout the visual field, so that in the repre-
sentation (as in the world it reflects) each thing is placed literally
where it belongs. To keep down the hardware costs, the system
may use channel coding (Snippe & Koenderink 1992; i.e., repre-
sent the event “object A at location L” by a superposition of a few
events of the form “object A at location Li”).
In the case of language, structures do not seem to have anything
like a natural grounding in any kind of spatial structure (not count-
ing the parse trees that linguists of a certain persuasion like to
draw on two-dimensional surfaces). We conjecture, however, that
such a grounding is conceivable, and can be used both for repre-
senting and manipulating semantic spaces, and for holding syn-
tactic structures in memory (which needs then to be merely a
replica, or perhaps a shared part, of the visual scene memory). To
support this conjecture, one may look for a “grammar” of spatial
relations that would mirror all the requisite theoretical constructs
invented by linguists for their purposes (the “syntactic” approach
to vision, popular for a brief time in the 1980s, may have remained
barren because it aimed to explain vision in terms of language, and
not vice versa). Alternatively, it may be preferable to aim at
demonstrating performance based on our idea (e.g., by imple-
menting a version of Barsalou’s system in which spatial buffers
would play the role of the frames), rather than to argue futilely
about theories of competence.
In neurobiology, perhaps the best piece of evidence for a per-
ceptual symbol system of the sort proposed by Barsalou is pro-
vided by phantom limb phenomena (Ramachandran & Hirstein
1998). These can set in very rapidly (Borsook et al. 1998), are
known to occur in the congenitally limb deficient or in early child-
hood amputees (Melzack et al. 1997), and may even be induced
in normal subjects (Ramachandran & Hirstein 1998). In a beauti-
ful experiment, Ramachandran and Rogers-Ramachandran (1996)
superimposed a mirror image of the intact arms of amputees onto
the space occupied by a phantom arm, and found that movements
of the mirrored intact hand produced corresponding kinesthetic
sensations in the phantom hand, even in a subject who had not ex-
perienced feelings of movement in his phantom hand for some ten
years prior to testing. Likewise, touching of the intact mirrored
hand produced corresponding, well-localized touch sensations in
the phantom hand. These findings support the idea of the world
– somatosensory, as well as visual – serving as an external mem-
ory (O’Regan 1992), and suggest a stronger relationship between
visuospatial and tactile/proprioceptive representations of “body
space” for normal subjects. It is interesting that these representa-
tions may be linked, in turn, to the mental lexicon: electromyo-
gram (EMG) responses to words with semantic content relating
to pain were found to be significantly different in the stumps of
amputees with chronic phantom limb pain, compared to the EMG
in the intact contralateral limb in the same subjects (Larbig et al.
1996).
Finally, the phantom phenomenon is not limited to the percept
of “body space” but may also be demonstrated in other modalities,
notably, in the auditory system (Muhlnickel et al. 1998). All this
suggests that perceptual symbols, along with a spatial frame pro-
vided by the experience of the external world, may (1) solve the
symbol grounding problem and (2) circumvent the binding prob-
lem – two apparently not immortal heads of the Hydra that besets
symbolic theories of knowledge representation. [See also Edel-
man: “Representation is Representation of Similarities.” BBS
21(4) 1998.]
Commentary/Barsalou: Perceptual symbol systems
614 BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4

Creativity, simulation, and conceptualization
Gilles Fauconnier
Department of Cognitive Science, University of California, San Diego,
La Jolla, CA 92093. gfauconnier@ ucsd.edu
Abstract: Understanding the role of simulation in conceptualization has
become a priority for cognitive science. Barsalou makes a valuable con-
tribution in that direction. The present commentary points to theoretical
issues that need to be refined and elaborated in order to account for key
aspects of meaning construction, such as negation, counterfactuals, quan-
tification or analogy. Backstage cognition, with its elaborate bindings,
blendings, and mappings, is more complex than Barsalou’s discussion
might suggest. Language does not directly carry meaning, but rather serves,
along with countless other situational elements, as a powerful instrument
for prompting its construction.
When a child plays with sugar cubes and matchboxes, pretending
that they are cars and buses, driving them all over the kitchen
table, and emitting loud “brrr” noises, there is simulation going on
in the everyday sense of the word. And there is also simulation go-
ing on in the neurobiological and conceptual sense evoked by
Barsalou in his target article. If the child’s activity makes sense and
provides enjoyment, it is because the brain is activating and run-
ning some of the dynamic conceptual schemas linked to cars and
buses. The projection of such a dynamic frame overlooks and over-
rides most of the “essential” features commonly associated with
vehicles – size, appearance, roads, internal mechanics, and so on.
And language is applied effortlessly to the situation: “Look,
Mummy, the car hit the bus. The engine’s broken.” The activity is
in no way exotic. Very young children master it, and use it to de-
velop their conceptual and linguistic systems. The children are in
no way deluded. They can maintain simultaneous representations
of objects as sugar cubes and as cars. They can mix frames and say
things like “Hey, don’t put the car in your coffee!” to the adult pick-
ing up the cube.
The human capacity for extracting, projecting, and combining
dynamic schemas is anything but trivial. It lies at the heart of con-
ceptual creativity. Yet, scientific accounts of meaning seldom do it
justice. Focusing only on necessary and sufficient conditions, or
on prototypes, or on statistical inference from reality, will quickly
rule out our sugar cube adventure. And by doing so, it will rule out
most of the power of language and thought. Words like car go far
beyond picking out categories of objects. They prompt us to
construct mappings and run dynamic schemas that can be wildly
different depending on circumstances. Reflect if you will on the
contribution of the conceptual world of automobiles to the un-
derstanding of expressions like put a tiger in your tank, Maytag is
the Cadillac of washing machines, If cars were men, you’d like
your daughter to marry this one [a Volvo ad], If cars had wings,
we wouldn’t need bridges.
In stressing the crucial role of simulation in conceptualization,
and in proposing the general scheme of perceptual symbols to ap-
proach the issue in a neurobiologically plausible way, Barsalou
does the cognitive science community a great service, offering to
liberate it from the chains of amodal computation. In attempting
to cover an extremely wide range of issues, there are many com-
plications that Barsalou does not address. Here are a few of them.
Note first that even the simple example above of children at play
remains a problem for a literal application of Barsalou’s percep-
tual symbols theory. The mapping of his Figure 3 will fail for the
sugar cube and so will many other aspects of perceptual/concep-
tual mappings commonly applicable to driving. The key here is se-
lective projection. Creative conceptual projection operates on the
basis of partial mapping selecting relevant aspects of a complex
conceptual domain (Fauconnier & Turner 1998; Tomasello 1998).
This suggests that the requirement of a common simulator (sect.
2.4.5) is too strong. More generally, the presupposition in Barsa-
lou’s footnote 2, that for a word there is a corresponding concept,
is dubious. Wittgensteinian musings and findings in cognitive se-
mantics (Lakoff 1987) suggest a connection of words to multiple
schemas/simulators with many kinds of connections between
them, not a direct word to concept correspondence.
A second challenge is the theoretical understanding of emer-
gent properties. Barsalou is quite right to point out that we can
conceive of running chairs and croquet flamingos through simu-
lation rather than by feature composition, and that this reflects af-
fordances captured through the formation process. But what is
this formation process? We do not get it for free from the per-
ceptual symbol hypothesis. Efforts are made in Fauconnier and
Turner 1998, Coulson 1997, and Langacker 1991 to characterize
some high level operations and optimality principles that could
yield such emergent structure. The problem applied to concep-
tual blends generally, including counterfactuals (Tetlock et al.
1996; Turner & Fauconnier 1998), fictive motion (Talmy 1996),
and grammatical constructions (Mandelblit 1997) is a difficult
one. Interesting theories of neural computation that achieve some
of this are explored by the Neural Theory of Language Project
(Narayan 1997; Shastri & Granner 1996).
A third issue is how to unleash the power of quantifiers, con-
nectives, negation, and variables in a perceptual symbol system. I
think Barsalou is on the right track when he thinks of elementary
negation in terms of failed mappings. But the proposal needs to
be fleshed out. One virtue of a scheme like Barsalou’s is that it nat-
urally makes some of the conception part of perception because
perceptual symbols and dynamic frames are activated along with
more basic perception. We don’t see an object with features x, y,
z in 3D position p and then compute that it’s a balloon above a
cloud. Rather we see it directly, conceptually and perceptually all
at once, as a balloon above a cloud. Negation, conceived of as
failed mapping, is different, because there is an infinite number
of possible failed mappings for any situation, but we don’t want the
cognitive system to try to register all these possible failures. In-
stead, elementary negation is typically used to contrast one situa-
tion with another that is very close to it. “The balloon is under the
cloud” is a more likely negation of “The balloon is above the cloud”
than “A typewriter is on a desk,” even though the latter would lead
to massive mapping failure in that situation. So the notion of failed
mapping needs to be elaborated and constrained. Furthermore,
the simulations must be able to run internally in the absence of ac-
tual events, and so the failed mappings should also be between al-
ternative simulations. Quantifiers, generics, and counterfactuals
pose obvious similar problems for the extension of Barsalou’s
scheme. One rewarding way to deal with such problems is to study
the more complex mappings of multiple mental spaces that orga-
nize backstage cognition (Fauconnier 1997; Fauconnier &
Sweetser 1996) and the cognitive semantics of quantifiers (Lan-
gacker 1991).
Many other points should be addressed here, but cannot be for
lack of space. Let me only mention that metaphorical and analog-
ical mappings (Gibbs 1994; Hofstadter 1995; Indurkhya 1992;
Lakoff & Johnson 1999) play a greater and more complex role than
Barsalou suggests, and also that in counterfactual and other con-
ceptual blends we routinely find simulations that have no possible
counterparts in the real world (Fauconnier & Turner 1998; Turner
& Fauconnier 1998).
An important general point for cognitive scientists is that lan-
guage does not directly carry meaning. Rather, it serves as a pow-
erful means of prompting dynamic on-line constructions of mean-
ing that go far beyond anything explicitly provided by the lexical
and grammatical forms (Fauconnier 1997). Fortunately for hu-
mans and unfortunately for scientists, the lightning speed opera-
tion of our phenomenal backstage cognition in a very rich mental,
physical, and social world is largely unconscious and not accessi-
ble to direct observation. Still, that is what needs to be studied in
order to really account for the famed productivity and creativity of
language and thought (Tomasello 1998).
Commentary/Barsalou: Perceptual symbol systems
BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4 615

Spatial symbol systems and spatial
cognition: A computer science perspective
on perception-based symbol processing
Christian Freksa, Thomas Barkowsky,
and Alexander Klippel
Department for Informatics and Cognitive Science Program, University
of Hamburg, D-22527 Hamburg, Germany.
{freksa; barkowsky; klippel}@informatik.uni-hamburg.de
Abstract: People often solve spatially presented cognitive problems more
easily than their nonspatial counterparts. We explain this phenomenon by
characterizing space as an inter-modality that provides common structure
to different specific perceptual modalities. The usefulness of spatial struc-
ture for knowledge processing on different levels of granularity and for in-
teraction between internal and external processes is described. Map rep-
resentations are discussed as examples in which the usefulness of spatially
organized symbols is particularly evident. External representations and
processes can enhance internal representations and processes effectively
when the same structures and principles can be implicitly assumed.
The role of spatial relations for cognition. Neural representations
resulting from perception are often organized in sensoritopic rep-
resentations, that is, according to spatial structure manifested in
the perceived configuration. As entities are perceived in spatial re-
lation to one another, a representation that preserves spatial rela-
tions is obtained without costly transformations. In linking the ex-
ternal and the internal worlds, perception processes make use of
spatial organization principles.
A special feature of structure-preserving representations is that
the same type of process can operate on both the represented and
the representing structure (Palmer 1978). [See also Palmer:
“Color, Consciousness, and the Isomorphism Constraint” BBS
22(6) 1999.] Thus, events in the external world can be reproduced
internally.
According to Barsalou (sect. 2.4), simulations with perceptual
symbols should underlie the same restrictions as their corre-
sponding perception processes, as they share the properties for
arrangement and order of occurrence with “real” perceptions.
From this perspective, space – similar to time – plays a twofold
role for cognitive processes: spatial location may be represented
as a perceptual symbol (as a result of perception) and space pro-
vides organizing structure for perceptual symbol systems.
As spatial structure is relevant to perception across modality
boundaries and as it behaves like a modality with respect to struc-
tural constraints, space can be viewed as an inter-modality com-
bining the advantages of specificity and generality of amodal and
modal representations, respectively. This inter-modality property
of space may be essential for multi-modal sensory integration and
for cross-modal interaction.
Specificity versus generality of representations. “General
purpose computers” have been praised for their capability in im-
plementing arbitrary concepts and processes. This capability has
been achieved by breaking up the structures of specific domains
and reducing everything to very general primitive principles that
apply to all domains. The generality of today’s computer systems
is achieved at the expense of an alienation of these systems from
a real environment.
The computer is a “brain” that is only superficially connected to
the external world (cf. Barsalou sect. 4.4). In reasoning about the
world and in simulating processes in the world, computers carry
out operations that do not structurally correspond to operations in
the world. To make computers function properly according to the
rules of a given domain, the specific domain structure must be
mimicked and described in an abstract way in the general-purpose
structure. As the domain-specific structure is not given intrinsi-
cally but is simulated through expensive computational processes,
certain operations are achieved at much higher computational
cost than in a more specialized structure.
Conversely, highly specific modal structures as manifested in
sensors can do little but react to the specific stimuli they are ex-
posed to. To be effective, they have to be strongly adapted to their
environment. In particular, they are not able to perform cross-
modal tasks, unless they have a specific interface to establish this
connection.
Thus, spatial structure appears to provide a good combination
of the advantages of abstract general and concrete specific levels
of representation. In spatial structures, a certain degree of ab-
straction is possible. At the same time, important relations rele-
vant to our perception, conceptualization, reasoning, and action
are maintained.
Spatial structures provide principal advantages for processing
information about spatial environments (Glasgow et al. 1995).
This can be attributed to the fact that – unlike other information
structures – spatial relationships are meaningful on many differ-
ent granularity levels.
Distinctions that are made on high-resolution levels disappear
on lower resolution levels. Preserving spatial structure, different
meaningful conceptualization levels can be achieved simply by
“looking” more closely or less closely (Freksa 1997).
This property of spatial structure can also be utilized on more
abstract levels: conceptual spaces in which small local changes can
be ignored in favor of the similarities on a higher level form con-
ceptual neighborhoods (Freksa 1991) with interesting computa-
tional properties (Freksa & Barkowsky 1996).
Maps as external representations for cognitive processes.
Maps and map-like representations convey knowledge by depict-
ing spatial entities in a planar external medium. Relevant geo-
graphic entities are identified and depicted by more or less ab-
stract symbols that form spatially neighboring pictorial entities on
the map’s surface (Barkowsky & Freksa 1997).
The usefulness of maps depends strongly on the spatial struc-
ture of the representational medium and on human spatio-per-
ceptual recognition and interpretation abilities. The symbols in
the map are perceived in relation to one another. The pictorial
map space is organized in spatial analogy to the world it repre-
sents. Hence, the map becomes an interface between spatio-per-
ceptual and symbolic concepts that are integrated in a single rep-
resentational structure.
Maps as external knowledge representations preserve spatial
relations adapted from the perception of the represented spatial
environment. Hence they support the conception and use of spa-
tial information in a natural way. The spatial structure of maps ex-
tends the cognitive facilities of the map user to the external rep-
resentation medium (Scaife & Rogers 1996).
The cognitive information processing principles that are ap-
plicable to both internal and external spatial representations can
be investigated by studying processes on external spatial repre-
sentation media. This seems especially valid regarding spatially or-
ganized knowledge for which the types of relations relevant to
both internal and external conceptions of environments can be
studied (Berendt et al. 1998). Moreover, as maps combine both
spatio-analogical and abstract symbolic information, the relevance
of spatial organization principles for non-spatial information can
be investigated.
Extending brain power: External memory and process model.
The idea of using the same principles for internal and external
knowledge organization is computationally appealing. Common
structures support the use of internal processes for using exter-
nally represented knowledge. Consequently, external knowledge
can serve effectively as an extension of internal memory struc-
tures.
Furthermore, internal and external processes that strongly cor-
respond to one another can greatly enhance the communication
between the internal and the external worlds. A main reason for
this is that only a little information needs to be exchanged to refer
to a known corresponding process, as the common structures can
act implicitly. In anthropomorphic terms we could say that it is
possible to empathize with familiar processes but not with unfa-
miliar ones.
Commentary/Barsalou: Perceptual symbol systems
616 BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4

specific individuals . . . we should be surprised if the cognitive sys-
tem ever contains a complete representation of an individual. . . .
we should again be surprised if the cognitive system ever remem-
bers an individual with perfect accuracy” (sect. 2.2.3). This again
shows the contrast with the logical approach, in which there can
be no question of the completeness or accuracy of an individual
constant term in denoting an individual; relative completeness is
out of the question because an individual constant term is atomic,
and accuracy is given by the logician’s fiat.
At his most careful, Barsalou speaks of “perceived individuals”
or “individuals in the perceived world,” and this is where his ver-
sion of proposition is based. “Binding a simulator with a perceived
individual . . . constitutes a type-token mapping. . . . This type-
token mapping implicitly constitutes a proposition” (sect. 3.2.1).
Thus, in the very act of perceiving something as belonging to some
pre-established mental category, I can entertain a proposition, be-
cause my attention at the moment of perception is fixed on this
one particular thing. The argument of the predicate is given by my
attention at the moment. The numerical singularity of the argu-
ment comes from the bottom up, because I am attending to just
one thing; the predication comes from a match between some per-
ceived property of the thing and top down information about a
pre-existing mental category. Thus, a Barsalovian proposition is
formed.
The numerical singularity of a perceived object is a product of
the observer’s attention at the time of perception. An observer
may focus on a pile of rice or on a single grain, or on a pair of boots
attended to as a pair, or on two separate individual boots. If I at-
tend to two objects, simultaneously but individually (say, a man
and his shaving mirror), my perception of them as two individual
objects at the time provides slots for two arguments, and I can find
a pre-established 2-place mental relation (say, use) to classify the
perceived event. (The limit on how many individuals one can at-
tend to at once may be similar to the limits of subitization in young
children – up to about four items.)
A classic formal semantic model (e.g., Cann 1993) might con-
tain a set of individual entities, each satisfying the predicate ant;
the denotation of the predicate ant is the set. Each individual ant
is the denotation of some individual constant (e.g., a1, a2, a3, . . .)
in the logical language. Thus, a1 and a2 are distinguished by the
fiat according to which the logician constructs his model. The
proposition ant (a1) is a different proposition from ant (a2); and
no further distinction – for example by a predicate applying to one
ant but not to another – is necessary in the logical system. The rep-
resentation of the proposition in the logical language tells us which
ant the proposition is about.
By contrast, in terms of perception, I can attend to certain prop-
erties of an object and judge from these properties that it is an ant,
but the perceived properties cannot tell me which ant it is. I know
that the world contains more than one ant, because I have some-
times seen many together, but all I know is that I have seen some
ant. Barsalou’s account of the storing of a basic proposition in long-
term memory describes it as involving a “type-token fusion” (sect.
4.1.5). This fusion is not further described, but it must in fact re-
sult in the loss of identity of the token, the “whichness” of the orig-
inally perceived object.
Barsalou is usually careful to prefix “perceived” onto “individ-
ual.” We can see how an individual can be perceived, but can an
individual be cognized, and if so, how? If individuals lose their
“whichness” during the process of storing a type-token fusion in
long-term memory, how can the perceptual symbols (the simula-
tors) in my mind for an ant and for my mother differ in a way that
echoes the classical difference between a set and an individual (or
between a property and an individual concept)?
A cognized individual (as opposed to a perceived individual) can
be constructed by the process involving the three mechanisms
which Barsalou claims are central to the formation of abstract con-
cepts, namely, framing, selectivity, and introspective symbols. I
have many experiences of my mother and form a simulator allow-
ing me to recognize her and anyone exactly like her. But I am never
presented with evidence that there is anyone exactly like her, de-
spite the thousands of opportunities for such a person to appear.
Whenever I perceive my mother, there is never anyone else pre-
sent with exactly her properties. The same can be said of the sun or
the moon, which I also represent as cognized individuals.
This account accords well with Barsalou’s account of abstrac-
tion: “First, an abstract concept is framed against the background
of a simulated event sequence” (sect. 3.4.2). In the case of my
mother, the event sequence is drawn from all my experiences of
her. “Second, selective attention highlights the core content of an
abstract concept against its event background” (sect. 3.4.2). The
selected core content of the abstract notion of a cognized individ-
ual is that this particular simulator is sui generis, that one has never
encountered two perceived individuals together fitting this simu-
lator. “Third, perceptual symbols for introspective states are cen-
tral to the representation of abstract concepts” (sect. 3.4.2). An in-
trospective state involved in the abstract notion of an individual is
comparison of percepts.
Consider, briefly, classic cases of the identity relation as ex-
pressed by Clark Kent is Superman or The Morning Star is the
Evening Star. Each such sentence seems to express a proposition
equating two different cognized individuals. Before Lois Lane re-
alized that Clark Kent is Superman, she had two distinct cognized
individual simulators. She always and only saw Clark Kent wear-
ing glasses and a baggy suit in the newspaper office; and she al-
ways and only saw Superman flying through the air in his red and
blue cape and catsuit. On the day when she saw Clark Kent be-
come Superman, these two individual concepts merged into a sin-
gle, more complex cognized individual. If, at some later date, she
actually explained to someone, in English, “You see, Clark Kent IS
Superman,” she was not thereby reflecting two separate individu-
als in her own cognition, but collaboratively assuming that her
hearer would not yet have merged the two concepts.
There is no space to explore the role that language plays in es-
tablishing abstract cognized individuals. The grammaticalization
of deictic terms into definite determiners, such as English the, and
the fossilization of definite descriptions into proper names, like
Baker and The Rockies, have given us devices for expressing the
abstract notion “individual.” The existence of stable cognized in-
dividuals in our minds, as opposed to merely transient perceived
individuals, in terms of which Barsalou mainly writes, must be
central to the ability of humans to “construct simulations jointly in
response to language about nonpresent situations, thereby over-
coming the present moment” (sect. 4.2, para. 3).
Identity, individuals, and the reidentification of particulars are
classic problems in metaphysics and the philosophy of language.
Barsalou has articulated a view of cognitive symbols which, if de-
veloped, can shed valuable light on these classic problems.
Creativity of metaphor in perceptual
symbol systems
Bipin Indurkhya
Department of Computer, Information, and Communication Sciences, Tokyo
University of Agriculture and Technology, Tokyo 184-8588, Japan.
bipin@cc.tuat.ac.jp
Abstract: A metaphor can often create novel features in an object or a sit-
uation. This phenomenon has been particularly hard to account for using
amodal symbol systems: although highlighting and downplaying can ex-
plain the shift of focus, it cannot explain how entirely new features can
come about. We suggest here that the dynamism of perceptual symbol sys-
tems, particularly the notion of simulator, provides an elegant account of
the creativity of metaphor. The elegance lies in the idea that the creation
of new features by a metaphor proceeds in the same way as the creation
of regular features in a perceptual symbol.
Barsalou has outlined an ambitious program to reaffirm the role of
perception in cognition. The proposal is indeed quite sketchy, as
Commentary/Barsalou: Perceptual symbol systems
BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4 621

the author himself explicitly acknowledges, and many details will
have to be worked out before a skeptic can be convinced of its vi-
ability. However, I find it intriguing, and applaud the author’s at-
tempt to rescue perception from the back seat and put it back in
the driver’s seat where it belongs. Indeed, the perceptual ground-
ing of all cognition and the close bi-directional coupling between
perception and cognition have largely been ignored by cognitive
scientists and philosophers alike for most of this century. Chalmer
et al. (1929) and Goodman (1976; 1978) are among the few notable
exceptions to this general trend. For instance, Goodman echoed an
essentially Kantian theme when he wrote: “Although conception
without perception is merely empty, perception without concep-
tion is blind (totally inoperative)” (Goodman 1978, p. 6, emphasis
Goodman’s). Though this slogan is certainly witty, a comprehensive
framework uniting conception and perception has not yet been at-
tempted, as far as I am aware, until this target article by Barsalou.
I would like to focus my comments on a specific phenomenon
having to do with the creativity of metaphor, where a metaphor
creates a novel feature of an object or a situation. Recognizing the
role of perception has a particularly illuminating effect in explain-
ing this phenomenon, and we show how it can be addressed in the
framework of perceptual symbol systems outlined by Barsalou.
Philosophers have long noted that metaphors can create new
perspectives or new features on a certain object, situation, or ex-
perience; and in this creation lies the true cognitive force of
metaphor (Black 1962a; 1979; Hausman 1984; 1989). This kind of
creativity in problem solving has been studied, among others, by
Gordon (1961; 1965), Koestler (1964) and Schön (1963; 1979).
Recent psychological research has also demonstrated this aspect
of creativity in understanding metaphorical juxtaposition in poetry
(Gineste et al. 1997; Nueckles & Janetzko 1997; Tourangeau &
Rips 1991).
Cognitive psychology and cognitive science research have
largely addressed metaphor within amodal symbol systems, where
metaphors are seen to arise from mappings between the symbols
of two domains. The best these approaches can do is to use the no-
tion of salience, together with highlighting and downplaying, to
argue that creativity of metaphor consists in highlighting low-
salience features of an object or situation. However, highlighting
and downplaying does not explain how completely new features
can be created.
An example can perhaps illustrate this point. In Stephen
Spender’s well-known poem “Seascape,” the poet compares the
ocean to a harp. In reading the poem, one’s attention is invariably
drawn to the way the rays of the sun reflect on the ripples of a calm
ocean. This is not so much a matter of highlighting certain aspects
of the amodal representation of the ocean, but creating a new rep-
resentation for it. If one insists on explaining this as highlighting,
then the amodal representation of the ocean quickly grows to an
enormous proportion, as it must include every possible feature
that can ever be associated with the ocean by any metaphor or any
other cognitive mechanism. (See also Indurkhya 1992.)
The notion of a simulator, however, allows for a rather elegant
explanation of how new features can be created. As the simulators
of both the harp and the ocean are activated, they try to build a
simulation together. In this process, certain features of each co-
operate (we might say that they resonate), while others cancel
each other out. In this example, one might build an association be-
tween the images of the light reflecting on the strings of a harp as
they lie waiting to be strummed and the sunlight playing on the
ripples that seem to stand still in a calm ocean. The fact that the
simulators contain the perceptual information about their referents
is very crucial here, as the resonance occurs between the percep-
tual components, which cannot be reduced to a mapping between
the symbolic levels. (After the metaphor has been assimilated, one
can try to explain the metaphor verbally and symbolically. How-
ever, in many such metaphors, the verbal explanations are usually
long and tortuous, and often hopelessly inadequate.)
The resonance of features might be unique in the sense that this
particular confluence of features might never have existed before,
and could not have been activated by either of the concepts alone.
Thus, a new feature emerges, which can then be explicitly incor-
porated in some perceptual symbol schema (see also Indurkhya
1998). What is elegant about this account is that the emergence of
new features by a metaphor is akin to the way features are first cre-
ated, when a perceptual symbol schema is formed directly by se-
lective attention (cf. sect. 2.2).
This account is akin to Fauconnier and Turner’s conceptual
blending approach (Turner & Fauconnier 1995; see also Faucon-
nier’s accompanying commentary in this issue), where they show
how concepts from many spaces blend together to produce
metaphorical meanings. Barsalou’s simulators extend Fauconnier
and Turner’s theory in a significant way by not only incorporating
the perceptual dimension, but also giving it a paramount role to
play; so that the account of creativity of metaphor outlined above
can better be dubbed “perceptual blending.”
Indeed, imagery and perceptual episodic memory have been
known to play a key role in understanding certain metaphors
(Marschark et al. 1983; Paivio 1979). This claim has been strength-
ened by recent neurolinguistic research (Bottini 1994; Burgess &
Chiarello 1996), and some researchers argue that metaphors are
essentially grounded in perception (Dent-Read & Szokolszky
1993). What Barsalou has tried to show us here is that metaphors
are not special in this way, because all concepts are essentially
grounded in perception.
The uncanny power of words
Paul J. M. Jorion
Théorie et Praxis, Maison des Sciences de l’Homme, 75270 Cedex 06, Paris.
paul jorion@email.msn.com aris.ss.uci.edu/~jorion
Abstract: In their quality as acoustic or visual percepts, words are linked
to the emotional values of the state-of-affairs they evoke. This allows them
to engender meanings capable of operating nearly entirely detached from
percepts. Such a laying flat of meanings permits deliberation to take place
within the window of consciousness. In such a theatre of the imagination,
linguistically triggered, resides the originality of the human psyche.
I’m going for a walk in a wilderness park in Southern California.
At the entrance of the park I am handed a leaflet about possible
sightings of mountain lions. There are some instructions about
how to act when faced with a puma. It sounds pretty exciting. I am
on my own. I start walking, wondering how likely it is that I will
be faced with the animal. Soon enough my attention drifts away,
I’m day-dreaming while I walk, more or less absorbed in the “in-
ner monologue” of the stream of consciousness. At some point I
hear “within my head” – as part of inner speech: “I’m seeing one
. . . I’m seeing one.” This makes me focus on the scene. Indeed
(the “indeed” with its element of pause is essential in the process),
I suddenly realize that about fifty yards away from me on the trail,
there’s a puma, heading away from me into the brush.
What does this little piece of introspection reveal? There has
been a visual perception of the animal no doubt, but it accedes to
consciousness in a very indirect manner: the actual awareness is
triggered by a sentence of inner speech, that is, the visual percept
generates a piece of linguistic behaviour that then focuses my vi-
sual attention. I must have seen the animal unconsciously, which
I then only properly see at the conscious level once the percept
has been translated into words within inner speech.
A number of supposed coincidences work undoubtedly in the
same manner. I am at the station, and I think to myself, “This good
Jonathan, what a pity I haven’t seen him for so many years.” And
lo and behold! Would you believe this? Here’s Jonathan! What an
extraordinary coincidence! Well no: I must have perceived him
unconsciously (the mystery of peripheral vision?) half a second or
so before. Enough time to have the inner speech elaborate on how
true a friend he is.
Commentary/Barsalou: Perceptual symbol systems
622 BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4

tion, and sometimes do interesting abstract thinking, such as
math, for which a strong tie to perception is either irrelevant or a
nuisance.
A view from cognitive linguistics
Ronald W. Langacker
Department of Linguistics, University of California, San Diego, La Jolla, CA
92093-0108. rlangacker@ucsd.edu
Abstract: Barsalou’s contribution converges with basic ideas and empiri-
cal findings of cognitive linguistics. They posit the same general architec-
ture. The perceptual grounding of conceptual structure is a central tenet
of cognitive linguistics. Our capacity to construe the same situation in al-
ternate ways is fundamental to cognitive semantics, and numerous paral-
lels are discernible between conceptual construal and visual perception.
Grammar is meaningful, consisting of schematized patterns for the pair-
ing of semantic and phonological structures. The meanings of grammati-
cal elements reside primarily in the construal they impose on conceptual
content. This view of linguistic structure appears to be compatible with
Barsalou’s proposals.
Barsalou’s stimulating contribution converges in myriad respects
with basic ideas and empirical findings of cognitive linguistics.
Perceptual symbol systems are a promising vehicle for the imple-
mentation of cognitive linguistic descriptions, which in turn reveal
a rich supply of phenomena that both support the general con-
ception and offer significant challenges for its future develop-
ment.
Perceptual symbol systems and cognitive linguistic approaches
posit the same basic architecture, involving abstraction from ex-
perience, the flexible activation of abstracted symbols in top-
down processing, and their recursive combination producing an
open-ended array of complex conceptualizations. A central claim
of “cognitive grammar” (Langacker 1987; 1990; 1991) is that all
linguistic units arise from actual events of language use through
the general processes of schematization and categorization.
Grounded in bodily experience, elaborate conceptual systems are
constructed through imaginative devices such as metaphorical
projection (Lakoff & Johnson 1980; Lakoff & Turner 1989; Turner
1987), the creation of mental spaces (Fauconnier 1985; 1997),
conceptual blending (Fauconnier & Sweetser 1996; Fauconnier &
Turner 1998), and productive simulations. The same devices, as
well as the conceptual structures already assembled, are dynami-
cally employed in the construction of linguistic meanings. Rather
than being mechanically derived from the meanings of its parts,
an expression’s meaning is actively constructed, largely in top-
down fashion, from the fragmentary clues provided by its consti-
tutive elements. Its coherence derives from their reconciliation in
a complex simulation that draws on all available resources (con-
ceptual, linguistic, and contextual).
The perceptual grounding of conceptual structure is a basic
tenet of cognitive linguistics. “Image schemas” abstracted from
sensory and kinesthetic experience have been claimed by Johnson
(1987) and by Lakoff (1987) to be the building-blocks of concep-
tual structure, the abstract commonality preserved or imposed in
metaphorical mappings (Lakoff 1990), and the structures most
critically employed in reasoning. I myself have speculated that all
conceptualization is ultimately derivable from certain irreducible
realms of experience – including time, space, as well as sensory,
emotive, and kinesthetic domains – however many levels of orga-
nization may intervene. I have further suggested that sensory and
motor images are essential components of linguistic meanings
(Langacker 1987). Talmy (1996) has coined the word “ception” to
indicate the extensive parallels he has noted between concep-
tion and perception. The same parallels have led me (Langacker
1995) to use the term “viewing” for both (without however taking
any position on the extent to which cognition might be visually
grounded).
Fundamental to a conceptualist semantics is the recognition of
our multifaceted ability to conceive and portray the same situation
in alternate ways, resulting in subtly different linguistic meanings.
This capacity for “construal” has numerous components that are
posited through linguistic analysis and strongly supported by their
descriptive utility. With striking consistency, these components of
construal bear evident similarities to basic aspects of visual per-
ception (Langacker 1993). For instance, linguistic expressions can
characterize a situation at any desired level of precision and detail
(“granularity”), as seen in lexical hierarchies such as thing . crea-
ture . reptile . snake . rattlesnake . sidewinder. This progres-
sion from highly schematic to increasingly more specific expres-
sions seems quite analogous to the visual experience of seeing
something with progressively greater acuity while walking up to it
from a distance.
In visual perception, we can distinguish the maximal field of
view, the general region of viewing attention (e.g., the stage, in
watching a play), and the specific focus of attention within that re-
gion (e.g., a particular actor). An analogous set of constructs is re-
quired for the semantic description of linguistic expressions. The
term knee, for example, evokes the conception of the body for its
“maximal scope,” selects the leg as its “immediate scope” (general
locus of attention), within which it “profiles” the major joint (the
conceptual referent, or focus of attention). In expressions that
profile relationships, one participant – called the “trajector” – is
accorded a kind of prominence analogous to that of the “figure”
within a visual scene. For instance, above and below profile refer-
entially identical relationships along the vertical axis; their se-
mantic contrast resides in whether trajector status is conferred on
the higher or the lower participant. Thus X is above Y is used to
specify the location of X, while Y is below X specifies the location
of Y. I take this contrast as being akin to the perceptual phenom-
enon of figure/ground reversal.
This is merely a sample of the parallels one can draw between
visual perception and the constructs needed for conceptual se-
mantic description. These constructs are not required just for
expressions pertaining to visual, spatial, or even physical situations
– they are fully general, figuring in the characterization of expres-
sions relating to any realm of thought and experience, both con-
crete and abstract. Moreover, they also prove crucial for grammar.
An expression’s grammatical class, for example, is determined by
the nature of its profile in particular (not its overall conceptual
content), and trajector status is the basis for subjecthood (Lan-
gacker 1998). Once the importance of construal is fully recog-
nized, the conceptual import of grammatical structure is dis-
cernible. In fact, the pivotal claim of cognitive grammar is that all
valid grammatical constructs are meaningful, their semantic contri-
butions residing primarily in the construal they impose on con-
ceptual content. Lexicon and grammar are seen as forming a
continuum fully describable as assemblies of “symbolic structures”
(i.e., pairings between phonological and semantic structures). I be-
lieve this conception of linguistic structure to be broadly compat-
ible with Barsalou’s notion of perceptual symbol systems.
Can handicapped subjects use perceptual
symbol systems?
F. Lowenthal
Cognitive Sciences, University of Mons, B-7000 Mons, Belgium.
francis.lowenthal@umh.ac.be www.umh.ac.be/~scoglab/index.html
Abstract: It is very tempting to try to reconcile perception and cognition
and perceptual symbol systems may be a good way to achieve this; but is
there actually a perception-cognition continuum? We offer several argu-
ments for and against the existence of such a continuum and in favor of
the choice of perceptual symbol systems. One of these arguments is purely
theoretical, some are based on PET-scan observations and others are
based on research with handicapped subjects who have communication
Commentary/Barsalou: Perceptual symbol systems
BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4 625

bodied approach to cognition and it subtly changes the way cog-
nitive scientists approach mental content, thereby changing what
tools theories of mental content or intentionality can use.
A core distinction in cognitive science is the one between per-
ceptual and conceptual representation. Barsalou points out that
this distinction is problematic, because there is no successful
demonstration of how these types of representations are con-
nected. He suggests that this difficulty should be taken as a sign
that perceptual representations are all there is. There are no
amodal representations.
In order to make this account plausible, it is critical to demon-
strate how key aspects of conceptual representation like abstrac-
tion can be cashed out in a perceptual symbol system (PSS). These
functions of conceptual representation are accomplished in a PSS
by assuming that perceptual representations have role-argument
structure in the form of frames.
It is not surprising that Barsalou suggests that frames form the
core of PSS. He has long regarded frame representations as being
well suited to be the medium underlying conceptual representa-
tion (Barsalou 1992). Further, many cognitive scientists adopted
representational systems with some kind of role-argument struc-
ture in order to facilitate the abstraction process (see e.g., Fodor
1981a; Markman 1999; Shank 1982). And finally, structured rep-
resentations have been suggested as the basis of models of central
perceptual processes like object recognition (Biederman 1987;
Marr 1982). Thus, there is some reason to believe that perceptual
representations are structured, and structured as Barsalou claims
they are.
Naturally, the assumption that frames are critical to perceptual
symbol systems leads to the question of where the capacity to rep-
resent information with frames comes from. There are two possi-
bilities. One is that the frames develop over the life of a cognitive
system from simple unstructured representations (e.g., vectors or
independent features). A second possibility is that the capacity to
build frames is an inherent capability of a PSS. These possibilities
are just a version of the standard “learned versus innate” debate
in cognitive science.
To our knowledge, all attempts to construct complex structured
representational schemes starting with only unstructured repre-
sentations have foundered. For example, attempts to account for
the development of complex representational capacities using as-
sociative connectionist models were not successful (Fodor &
Pylyshyn 1988; Marcus 1998). It is possible to create complex rep-
resentational structures in a connectionist model, but such struc-
ture has to be built in ahead of time using other techniques that
give the connectionist system a classical structuring capability
(e.g., Shastri & Ajjanagadde 1993). A very good example of this is
Smolensky’s interesting work on connectionism. His tensor prod-
uct approach does have classical constituent structure already
built in (see the series of papers: Smolensky 1990; Fodor &
McLaughlin 1990; Smolensky 1995; and McLaughlin 1997).
In light of this difficulty, Barsalou assumes that the capacity to
form structured representations is an inherent component of a
PSS. That is, he assumes stimuli that contact a cognitive agent are
converted into frame representations early on, and that the ca-
pacity to do this is innate in the system or organism. Although
structured representations have been incorporated into models of
perceptual processes, they are not a necessary component of mod-
els of perception (e.g., Ullman 1996).
The assumption that representations are structured is extraor-
dinarily important for the PSS account. Once these frames are
constructed, many of the techniques of concept formation and ab-
straction used by proponents of structured amodal representa-
tions can be incorporated into PSS including the ability to make
similarity comparisons and to reason by analogy (Gentner 1983;
Gentner & Markman 1997). Thus, once Barsalou assumes that
representations in a PSS are frames, the ability of this system to
account for higher level cognitive abilities is virtually assured.
Where the perceptual symbol system approach differs from
previous approaches to structured representations is in assuming
that the components of representational frames are tied to per-
ception rather than being derived from a central multimodal rep-
resentation language (or perhaps from language ability itself ).
However, it is here that Barsalou’s approach has its greatest
promissory note. It is a bold step to posit that the structure repre-
sentations that form the basis of conceptual abilities are closely
tied to perception. It is now critical to demonstrate how a true per-
ceptual system could give rise to representations of this type.
Development, consciousness, and the
perception/mental representation distinction
Lorraine McCune
Department of Educational Psychology, Rutgers University, New Brunswick,
NJ 08903. mccune@rci.rutgers.edu
Abstract: Perceptual symbol systems provide a welcome alternative to
amodal encapsulated means of cognitive processing. However, the rela-
tions between perceived reality and internal mentation require a more dif-
ferentiated approach, reflecting both developmental differences between
infant and adult experience and qualitative differences between con-
sciously perceived and mentally represented contents. Neurological evi-
dence suggests a developmental trajectory from initial perceptual states in
infancy to a more differentiated consciousness from two years of age on.
Children’s processing of and verbal expressions regarding motion events
provides an example of the changing capacity for mental experience.
Barsalou defines a “symbol” as a “record,” a neurologically em-
bodied result of the neural state underlying a given perceptual ex-
perience. Perceptual symbols so defined are described as the
medium for all cognitive processing, both conscious and uncon-
scious. The author is persuasive in his argument that an embod-
ied modal system, directly relating biology to experience is a par-
simonious and effective basis for mental processes. However,
theoretical considerations regarding the term “symbolic” and de-
velopmental issues require further elucidation.
In the realm of consciousness, perception has traditionally been
considered distinct from mental, “imaginal” or “symbolic” repre-
sentation (Sartre 1984; Werner & Kaplan 1963). Perception is the
ongoing experience of present reality, while representation, ac-
complished through the mediation of symbols, allows considera-
tion of past, future, and counterfactual phenomena. Both types of
conscious experience could have their basis in the perceptual sym-
bol systems as defined, but the distinction between them needs to
be addressed within the theory.
It has become common to roughly equate infant mentation with
that of adults as Barsalou does, “From birth, and perhaps before,
the infant simulates its expected experience and assesses whether
these simulations map successfully into what actually occurs”
(sect. 3.4.3, para. 6). The author acknowledges the importance of
developmental transitions, but only up to a point, noting that “as
infants’ perceptual and bodily systems develop, their conceptual
systems should change accordingly. Nevertheless the same basic
form of conceptual representation remains constant across both
evolution and development, and a radically new form is not nec-
essary” (sect. 4.2, para. 5). I would like to suggest that the infant’s
initial capacity from birth and before is limited to direct percep-
tual experiences of which neural records may be kept, based on
the infant’s attentional focus, as Barsalou proposes. But during the
first two years the child develops the initial capacity for mentally
representing reality, a transition which can be documented be-
haviorally as well as on the basis of changing neurological struc-
tures. Without such mental representation capacity the child is
clearly incapable of entertaining propositions as described in sec-
tion 3.2. The capacity for sensory and motor understanding of
such relationships as “above” and “below” is not equivalent to
propositional analysis.
Observable behavioral changes are demonstrated in activities
Commentary/Barsalou: Perceptual symbol systems
BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4 627

own movements, in the visual experience (or memory) of another’s
movements, or auditorily in music. When one recognizes the iden-
tity in these different rhythms, is this a mapping among simula-
tions, a mapping from simulations to perceptions, or an amodal
detection of temporal consistency? More complex: How is a non-
perceptual analogy recognized if a frame needs to be perceptually
specialized?
Barsalou’s analysis of truth, though impressive, seems more a
description of how to decide if something is true while already
having a notion of truth to decide about. A matching between a vi-
sual simulation and a visual experience could represent not only
“truth,” but also “similar,” “comparable,” and “looks like.” Given
that “truth” does not seem to be a perceptual construct, more
seems necessary to justify the claim that people’s core intuitive
sense of “truth” is represented by matching simulation to percep-
tion. Similar problems seem likely with other perceptually un-
specifiable terms such as “can,” “might,” “electricity,” “ignorant,”
and even “thing.” It will, then, be interesting to see how far per-
ception-based cognition can go in elucidating how we understand
the world and ourselves.
Introspection and the secret agent
Natika Newton1
Nassau County Community College, New York, NY 11530.
nnewton@suffolk.lib.ny.us
Abstract: The notion of introspection is unparsimonious and unnecessary
to explain the experiential grounding of our mentalistic concepts. Instead,
we can look at subtle proprioceptive experiences, such as the experience
of agency in planning motor acts, which may be explained in part by the
phenomenon of collateral discharge or efference copy. Proprioceptive sen-
sations experienced during perceptual and motor activity may account for
everything that has traditionally been attributed to a special mental activ-
ity called “introspection.”
Barsalou’s target article presents a convincing case for perceptual
rather than amodal symbol systems. Considered in light of a pro-
posal like this one, the traditional amodal view appears, in hind-
sight, strikingly misguided and implausible. My only serious ob-
jection concerns Barsalou’s remarks on introspection (sects. 2.3
and 2.4) which he sees as “an aspect of perceptual experience” like
proprioception. He points out that introspective processing is rel-
atively poorly understood.
It is questionable whether there is an aspect of perceptual ex-
perience that corresponds to “introspection.” Barsalou does not
define the term, and talks about it somewhat in the style of Locke
who believed that we can attend to either outer or inner experi-
ence, and when doing the latter, will perceive mental operations
– for example, comparing, representing, and so on. Why believe
that there are specific brain mechanisms for processing percep-
tual experiences of other brain mechanisms? There would seem
to be more parsimonious ways to explain what was traditionally
called “introspection” without taking that risky step toward infi-
nite regress, especially without evidence of any corresponding
brain structures.
Barsalou does talk about selective attention and our ability to
abstract, and suggests that in introspection, selective attention can
focus on our “ability to represent something in its absence.” This
procedure seems to me to be a case of theorizing within a partic-
ular conceptual framework involving “the mind,” “representa-
tion,” and so on, but not requiring any ability to experience the
“ability to represent.” The concept of representing, if Barsalou is
right, must be perceptually grounded like any other concept; but
it can be grounded in the perceptual experience of external rela-
tions between representer and represented, not necessarily in the
activity of representing all by itself. The ability to represent things
may well be logically entailed by our representational experience
together with the theories we hold about the brain, but that en-
tailment is not the same thing as the representing ability being an
experiential aspect of the process of representation, in the way
that, for example, visual features such as colors or outlines can be.
Instead of talking about introspection, I suggest that we look for
subtle proprioceptive experiences that accompany activities like
representing, attending to, and so forth, and that may be unno-
ticed but taken for granted as essential aspects of the experiences
of those activities together with their sensory objects. For exam-
ple, when one sees a visible object, one is aware not only of the
visible features of the object, but of the proprioceptive sensations
of focusing the eyes and orienting the body toward the object.
These ordinary perceptual experiences may be all that is neces-
sary to ground the concept of “seeing.”
One aspect of experience that has traditionally seemed to re-
quire a special form of access like introspection is the experience
of agency that accompanies all voluntary intentional activity, even
subtle perceptual activity such as ocular control. But even this ex-
perience is proprioceptive. It appears to be produced by corollary
discharge, or efference copy: information about motor commands
to the muscles that is sent to the parietal lobe and to the cerebel-
lum from the motor and premotor cortex, for purposes of com-
parison with the expected movement. This information about mo-
tor commands is attended to by the subject normally only when the
expected feedback does not match the commands, or is lacking al-
together as when the relevant limb is paralyzed. In ocular motor
control, the experience is almost continuous, since the eyes must
be constantly adjusted in response to shifts in the visual field (Houk
et al. 1996). The subjective feeling of agency is strikingly absent
when motor activity is initiated artificially in experimental condi-
tions; in such cases the movement is “automatically referred to a
source (an intention) distinct from the self” (Jeannerod 1997, p.
189). There is evidence that information from efference copy is
held in working memory, and hence would be accessible to con-
sciousness like any other perceptual experience (Houk et al. 1996).
Barsalou has supplied us with all the basic mechanisms needed
to ground our entire conceptual system in sensorimotor experi-
ence. All that is needed is to tease out a few more of the tricks used
by our bodies to inform us of what we are doing and what is hap-
pening to us so that we can be, or at least feel that we are, full in-
tentional agents. Efference copy is one trick that bears more study.
Unless systems like this prove to be inadequate to explain self-
awareness, concepts like introspection should probably be kept in
the archives.
NOTE
1. Author’s mailing address is 15 Cedar Lane, Setauket, NY 11733.
Can metacognition be explained in terms
of perceptual symbol systems?
Ruediger Oehlmann
University of Essex, The Data Archive, Psychology Unit, Wivenhoe Park,
Colchester, CO4 3SQ, England. oehlmann@essex.ac.uk
dawww.essex.ac.uk/~roehl
Abstract: Barsalou’s theory of perceptual symbol systems is considered
from a metacognitive perspective. Two examples are discussed in terms of
the proposed perceptual symbol theory. First, recent results in research
on feeling-of-knowing judgement are used to argue for a representation of
familiarity with input cues. This representation should support implicit
memory. Second, the ability of maintaining a theory of other people’s be-
liefs (theory of mind) is considered and it is suggested that a purely simu-
lation-based view is insufficient to explain the available evidence. Both ex-
amples characterize areas where Barsalou’s theory would benefit from
additional detail.
Barsalou presents a series of very careful and challenging argu-
ments for a perceptual theory of knowledge. This commentary will
Commentary/Barsalou: Perceptual symbol systems
BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4 629

patterns will grow increasingly distinct functionally and end up as
members of different neural circuits.
What kind of neural representation will this sort of process gen-
erate? Consider a layer of cortex whose neurons respond to dif-
ferent feature analyzers at the sensory interface (e.g., shape, color,
texture). A first encounter with a given object would set up a fairly
diffuse pattern of cortical activity and would lead to a modest
strengthening of the connections among the active elements.
Because every object is encountered in some context, some of the
active elements would represent features of the object, whereas
others would represent contextual features. Given repeated en-
counters with the same object in varying contexts, the neurons
representing the object’s features will grow into a highly intercon-
nected network while those representing contextual features will
tend to fall away. Hebb coined the terms “recruitment” and “frac-
tionation” to describe these two processes whereby neurons be-
come incorporated into or excluded from a network.
The same processes of recruitment and fractionation occurring
at a stage further removed from the sensory interface can grow
neural representations that stand for classes of objects. Given re-
peated encounters with a set of objects that share certain features,
birds for example, the neural units responding to the most invari-
ant features (e.g., feathers and beaks) will grow into a highly in-
terconnected functional unit, whereas the more variable features
(e.g., color, size) will be excluded from the set of core elements.
There is in principle no limit to the number of different levels of
abstraction at which this process might operate. Given sufficient
cortical “depth” and sufficiently varied experience, we might ex-
pect an individual to develop neural representations that stand not
only for classes of objects, but for classes of classes, and so on ad
infinitum. An assembly of neurons that forms in this fashion will
exhibit many of the properties Barsalou attributes to perceptual
symbols. It will be schematic, in that it represents only a subset of
the features that any actual object manifests at any given time. It
subserves categorization, in that the same assembly responds to
varying instances of some class of objects that have features in
common. It is inherently perceptual, dynamic, and can participate
in reflective thought. In addition, it allows for recognition on the
basis of partial input, typicality effects, prototype formation, and
imagery. And note that we get all of this, in a sense, for free. The
mechanisms of recruitment and fractionation that build the re-
quired perceptual symbols at the same time isolate the invariant
features of some class of objects without the need for any outside
intelligence to guide the process along.
By contrast, Barsalou seems to place much of the burden for
perceptual symbol formation on an unexplained process of selec-
tive attention. According to his theory, attention selects some par-
tial aspect of a perceptual experience and stores this aspect in
long-term memory. Moreover, he offers no account of why in any
given perceptual experience attention might focus on some fea-
tures and not on others. This we find troubling, for in the absence
of any principled constraint on which subsets of experience come
to serve as dynamic internal representations, Barsalou’s percep-
tual symbols seem only marginally less arbitrary than the symbols
postulated by the amodal and connectionist theories he rightly
criticizes.
By distancing himself from connectionist approaches in gen-
eral, Barsalou not only undermines his capacity to deal with the
particular problem of categories and abstractions, but also his ca-
pacity to specify any sort of mechanism for the perceptual ap-
proach. He perhaps did not realize that some forms of connec-
tionism, in particular the Hebbian or “active symbol” version, are
well suited to support and give structure to his position.
Truth and intra-personal concept stability
Mark Siebel
Research Group in Communication and Understanding, University of Leipzig,
04109 Leipzig, Germany. siebel@rz.uni-leipzig.de
Abstract: I criticize three claims concerning simulators: (1) That a simu-
lator provides the best-fitting simulation of the perceptual impression one
has of an object does not guarantee, pace Barsalou, that the object belongs
to the simulator’s category. (2) The people described by Barsalou do not
acquire a concept of truth because they are not sensitive about the poten-
tial inadequacy of their sense impressions. (3) Simulator update prevents
Barsalou’s way of individuating concepts (i.e., identifying them with simu-
lators) from solving the problem of intra-personal concept stability be-
cause to update a simulator is to change its content, and concepts with dif-
ferent contents are distinct.
Barsalou’s theory involves three claims about simulators that I
wish to criticize. They concern (1) true propositions, (2) the de-
velopment of a concept of truth, and (3) intra-personal concept
stability.
1. According to Barsalou (sects. 2.4.4 and 3.2.3), one entertains
a true proposition to the effect that a perceived individual belongs
to a certain category if one’s simulator for that category provides
the best-fitting simulation of the perceptual impression one has of
the individual. However, that account rests on a confusion be-
tween what seems to be true (to a person) and what is true.
Consider someone who sees a horse that looks like a donkey to
him; the horse causes a neural representation that resembles the
impressions he normally has when he sees a donkey. Conse-
quently, the simulator that provides the best-fitting simulation of
the person’s impression is his simulator for donkeys, not for horses.
Thus, according to Barsalou, the person does not only classify the
horse as being a donkey, his conceptual system also provides a true
proposition. Because the simulator for donkeys produces a satis-
factory simulation of the horse, the horse should belong to the
simulator’s category. But, obviously, a horse does not become a
donkey because it seems to be a donkey. Otherwise it would be
nearly impossible to make a mistake in classifying an object. The
resemblance between a perceptual representation of an object
and representations created by a simulator for a certain category
does not guarantee the object’s category membership. Whether
the object belongs to the category depends on the veridicality of
those representations.
2. Barsalou’s account of the way we acquire a concept of truth
is infected by a similar problem. In Barsalou’s view, people acquire
a concept of truth by “learn[ing] to simulate the experience of suc-
cessfully mapping an internal simulation into a perceived scene”
(sect. 3.4.3, para. 2). They develop such a concept by, for example,
comparing perceptual simulations they form by hearing the ut-
terance of a sentence and perceptual impressions they have of a
corresponding scene. If the simulation resembles the impression
to a certain extent, they learn to reply to the utterance with “That’s
true.”
Roughly, truth is correspondence to the facts: a representation
is true if and only if it represents a realized state of affairs. What
Barsalou’s people learn, however, is not construing truth as corre-
spondence to the facts, but as correspondence to certain repre-
sentations, namely, sense impressions of perceived scenes. Their
concept of “truth” is instantiated whenever the simulation caused
by hearing an utterance corresponds to the impression caused by
looking at a certain scene – regardless of whether the impression
itself is veridical. Barsalou’s people are not sensitive about the po-
tential inadequacy of their sense impressions. In a word, they do
not develop a concept of truth because it is too easy to elicit assent
from them: they say “That’s true” whenever the way in which they
conceptualize the meaning of the utterance resembles the way in
which they perceive the crucial scene. They do not know what it
is for an utterance to be true because they do not take into con-
sideration that their perceptual impressions can lead them astray.
3. As Barsalou (sect. 2.4.5) himself recognizes, an adequate
Commentary/Barsalou: Perceptual symbol systems
632 BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4

theory of concepts must allow for a certain extent of intrapersonal
concept stability. It must allow, for example, that the concept one
uses in classifying an object on Thursday is the same as the con-
cept one used on Monday to classify another object. Barsalou
(sects. 2.4.3–2.4.5) tries to achieve stability by identifying con-
cepts with simulators: even if different conceptualizations (i.e.,
simulations) are responsible for classifying different objects as be-
ing members of the same category, the same concept is used in
these processes as long as the same simulator provides these best-
fitting simulations.
That way of individuating concepts, however, remains too fine-
grained to save intra-personal concept stability, for Barsalou
(sects. 2.4.4 and 3.2.2) says that every successful simulation up-
dates the corresponding simulator by integrating perceptual sym-
bols of the categorized object into it. In other words, the catego-
rization of an object supplies the simulator with new perceptual
information about the members of its category so that it is able to
create further simulations afterwards. Thus, updating a simulator
(i.e., a concept) means to change its content; and since concepts
with different contents are distinct, the update leads to another
concept.
Concepts are to be individuated by their extension (their refer-
ents) and their content (the information they contain about their
referents). Bertrand’s concept of donkeys is already different from
Susan’s concept of horses because it represents another category.
And if Bertrand’s concept of horses does not contain the same in-
formation as Susan’s concept of horses, they are likewise distinct,
although they have the same extension. To use a Fregean term,
even concepts for the same category differ if their “modes of pre-
sentation” are distinct, that is, they represent that category in dif-
ferent ways.
Hence, an update of a simulator entails that it is no longer the
same concept. Although the category of objects it represents re-
mains the same, we have a different concept afterwards, because
there is something added to its content, namely, perceptual infor-
mation about the object categorized before. Hence it is hard to see
how Barsalou’s account could save concept stability within indi-
viduals. If the categorization of an object is followed by an update
of the corresponding simulator, then the next assignment of an ob-
ject to the same category does not arise from the same simulator
because its content changed. Because different contents imply
different concepts, updating prevents a person from using the
same concept twice.
In order to save concept stability within individuals, one must
not identify concepts with simulators as a whole, but only with a
certain core of them. Much as shared concepts between individu-
als only require similar simulators, the identity of a concept within
an individual merely requires that the underlying simulator not
have changed too much.
A perceptual theory of knowledge:
Specifying some details
Aaro Toomela
Department of Psychology, University of Tartu, Tartu, Estonia EE 50410.
toomela@psych.ut.ee
Abstract: We attempt to resolve some details of Barsalou’s theory. (1) The
mechanism that guides selection of perceptual information may be the ef-
ferent control of activity. (2) Information about a world that is not acces-
sible to the senses can be constructed in the process of semiotic media-
tion. (3) Introspection may not be a kind of perception; rather, semiotically
mediated information processing might be necessary for the emergence
of introspection.
Barsalou describes a perceptual theory of knowledge for which
the need emerges from the historical divergence of the fields of
cognition and perception in psychology. Russian psychology, how-
ever, especially Vygotsky-Luria’s school of psychology, never sep-
arated thinking from perception (Luria 1969; 1973; 1979; Vygot-
sky 1934/1996). Barsalou also suggests that there is no theory of
knowledge to explain when emergent features arise and why (note
18). Yet the primary objective of the field of semiotics is to un-
derstand semiosis and the knowledge-making activity (Danesi
1998). The question of how novel information is generated is thor-
oughly analyzed in the Tartu-Moscow school of semiotics (e.g.,
Lotman 1983; 1984; 1989). Thus, there are some ideas in Vygot-
sky-Luria’s psychology and in semiotics that may be useful for fill-
ing gaps in Barsalou’s theory.
Selection of information may be guided by the efferent control
of activity. According to Barsalou, selective attention has a central
role in the construction of perceptual symbols (e.g., sect. 2.2), but
he does not explain how the cognitive system knows where to fo-
cus attention during the symbol formation process (note 7). He as-
sumes that a conceptual system may be based on perceptual sym-
bols. However, if we do not know how attention selects perceptual
information then there is a possibility that the cognitive system
that underlies selective attention is amodal. (Of course that sys-
tem may also be modal, but the point is that Barsalou cannot ex-
clude amodal systems from his theory without demonstrating that
such a selection mechanism – which “knows” where to focus at-
tention – is modal.)
Luria’s theory offers a modal solution to the problem of atten-
tion in the form of a special system in cognitive functioning that is
responsible for programming, regulating, and monitoring activity.
In addition to afferent modalities, there is an efferent (basically
motor) modality. The same system supports attention (Luria 1969;
1973). It is possible that isolation of novel information in percep-
tion for subsequent storage in long-term memory is realized by
that efferent system. There is some supporting evidence, for ex-
ample, the disappearance of stabilized retinal images (Alpern
1972; Cornsweet 1970), showing that activity (eye-movement) is
necessary for visual sensation. There is also evidence that visual
agnosias are accompanied with disorders in eye-movements
(Luria 1969; 1973). This finding indicates that activity is also nec-
essary for the functioning of central perceptual mechanisms.
The relationship between consciousness and introspection is
unclear. Barsalou suggests that conscious experience may be nec-
essary for the symbol formation process to occur initially (sect.
2.2). By “consciousness,” Barsalou means awareness of experience
or qualia (personal communication, December 1, 1998). At the
same time, he assumes that the term “perceptual” refers both to
the sensory modalities and to introspection (sect. 2.3.1). Accord-
ing to him, introspection represents experiences (representational
states, cognitive operations, and emotional states; sect. 2.3.1). It is
not clear how it is possible to be aware of experiences for building
perceptual-introspective knowledge about the same experiences.
We return to this problem below.
Information about a world that is not accessible to the senses
cannot be based on perceptual symbols alone. Barsalou assumes
that every kind of knowledge can be represented with perceptual
symbols. The perceptual theory of knowledge is not very convinc-
ing regarding abstract concepts even though the solution pro-
posed there is not entirely implausible. However, there exists a
kind of knowledge that must be amodal in essence: the knowledge
about a world which is qualitatively out of reach of our senses. Hu-
mans do not possess perceptual mechanisms for perceiving elec-
tromagnetic fields, electrons, and other submolecular particles.
None of the modalities of the human perception can react to a
neutrino. How such knowledge is constructed is not explained in
Barsalou’s theory (note 18 partly recognizes this problem).
One solution to the question of how novel information is gen-
erated is proposed by Lotman (e.g., 1983; 1984; 1989). According
to him novel information emerges only if at least two different
mechanisms of information processing interact, the novelty
emerging when the results of one type of information processing
is “translated” into the other. Humans possess such mechanisms,
sensory-perceptual and amodal-linguistic. Novel information is
Commentary/Barsalou: Perceptual symbol systems
BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4 633

created in the process of semiotic mediation, in the “dialogue” be-
tween the perceptual and linguistic mechanisms. In the process of
semiotic mediation, perceptual events acquire entirely new mean-
ings (cf. Lotman 1983; Toomela 1996). The same basic mechanism
may be responsible for the emergence of novel knowledge about
a world that is accessible to the senses. In this case novelty
emerges when information about the same object or phenomenon
is processed in a dialogue between different sensory modalities.
Introspection may be a kind of semiotically mediated knowl-
edge. It is not possible to go into detail here, but it is worth con-
sidering the possibility that introspection and consciousness rep-
resent a qualitatively novel knowledge that is not accessible to our
senses. It might not be accidental that “introspective processing is
poorly understood” (sect. 2.3.1). To the best of my knowledge,
there is no direct evidence that humans possess an introspective
system analogous to sensory mechanisms. Rather, introspection
might be a result of semiotic mediation (cf. Luria 1973; 1979; Vy-
gotsky 1925/1982; 1934/1996).
The idea that humans possess different mechanisms for pro-
cessing the same information fits with some of Barsalou’s conclu-
sions. First, the right hemisphere represents individuals in per-
ception and comprehension and the left hemisphere represents
simulators that construe them (note 29). It is also possible, how-
ever, that right hemisphere processes information more in terms
of perceptual characteristics and the left hemisphere more with
“linguistic simulators” (cf. Lotman 1983). Second, in sect. 3.4.2,
Barsalou refers to the finding that abstract concepts tend to acti-
vate frontal brain regions. Prefrontal brain regions are also known
to be related to verbal regulation of behavior and verbal thinking
(Luria 1969; 1973). The construction of abstract concepts may re-
quire semiotically mediated thinking supported by frontal struc-
tures. Finally, Barsalou suggests that the dissociation of visual ag-
nosia and optic aphasia may provide evidence for two different
ways of activating a common representational system (note 3).
That interpretation also accords with the idea that the same in-
formation might be processed in different ways in the same cog-
nitive system.
Most of my remarks have concerned information in footnotes
where undeveloped aspects of the theory are recognized. There
are many theoretically important ideas in the body of the target
article that not only fit with other perceptual theories but also ex-
tend our understanding of the role of perception in cognition.
Some interesting ways to improve the perceptual theory of knowl-
edge might be found in Vygotsky-Luria’s school of psychology and
in Lotman’s school of semiotics.
ACKNOWLEDGMENT
This work was supported by The Estonian Science Foundation.
External symbols are a better bet
than perceptual symbols
A. J. Wells
Department of Social Psychology, The London School of Economics and
Political Science, London WC2A 2AE, England. a.j.wells@lse.ac.uk
Abstract: Barsalou’s theory rightly emphasizes the perceptual basis of
cognition. However, the perceptual symbols that he proposes seem ill
suited to carry the representational burden entailed by the architecture in
which they function, given that Barsalou accepts the requirement for pro-
ductivity. A more radical proposal is needed in which symbols are largely
external to the cognizer and linked to internal states via perception.
I strongly endorse Barsalou’s emphasis on the perceptual nature
of cognition and I share his view that theories based on amodal
symbol systems suffer from serious difficulties. Transduction and
symbol grounding (cf. sect. 1.2.2), in particular, are fundamental
problems. Barsalou proposes that at least some of the problems
can be solved by hypothesizing a class of internal representations,
that is, perceptual symbols, that stand in a different relation to the
proximal stimuli that produced them than do amodal symbols.
The proposal looks radical at first sight but on further examination
seems rather modest. It is precisely the modesty of the proposal
that gets it into difficulties and I wish to recommend a more rad-
ical proposal.
Much traditional theorizing about the linkages between the ex-
ternal world and internal thought processes acknowledges the fol-
lowing broad classes of entities. There are distal events and ob-
jects, proximal stimuli, perceptual states and cognitive states.
Barsalou seems to acknowledge these classes. Roughly, it is as-
sumed that distal entities cause proximal stimuli, which cause per-
ceptual states, which cause cognitive states. The latter are often
assumed to consist (at least in part) of structured symbolic ex-
pressions. Cognitive computation over input symbol structures
leads to output symbol structures that guide behavior. Behavior
can have causal consequences for distal entities thus closing the
causal loop.
Barsalou’s proposal is modest because it is concerned, essen-
tially, with just one link in the chain, the link between perceptual
states and symbol structures. Significantly, his proposal does not
require a revised account of either the perceptual states that pro-
duce symbol structures or the cognitive computations that trans-
form them. His core claim is that changing the nature of the sym-
bol structures is sufficient to repair the deficiencies in amodal
symbol systems theorizing. Thus he proposes simply that amodal
symbol structures should be replaced with perceptual symbol
structures which have two primary characteristics not shared with
amodal structures. Perceptual symbol structures are hypothesized
to be modality specific (or multi-modal) and analogical. The force
of the latter point is that “the structure of a perceptual symbol cor-
responds, at least somewhat, to the perceptual state that produced
it” (sect. 1.1). Thus the suggestion is that the distinction between
perceptual symbols and amodal symbols is akin to the distinction
between analogue and digital modes of representation. The pri-
mary requirements have a number of further consequences dis-
cussed in section 2.2. Perceptual symbols are dynamic not discrete,
contextually variable, nonspecific and potentially indeterminate.
They have, in other words, a degree of plasticity that is not char-
acteristic of amodal symbols. They are, nevertheless, required to
support the full range of computations needed for a fully fledged
conceptual system. Most important in this regard, Barsalou argues
that perceptual symbol structures must be capable of unbounded
generativity and recursive elaboration (sect. 3.1).
Sympathetic though I am to the idea that perception and cog-
nition should be better integrated, I think that Barsalou’s proposal
is almost guaranteed not to work. It is essential for unbounded
generativity and recursion that primitive symbol tokens be amodal
and reliably identifiable independent of context. Barsalou pro-
poses to give up these essential characteristics without a clear pic-
ture of how the much sloppier perceptual symbols are supposed
to perform the same functions. He is right to emphasize the con-
straints on symbols arising from the neural foundations of the cog-
nitive system, and right to note that the evidence for amodal sym-
bols is not strong, but changing the nature of the symbols does not
seem to be the solution to the problem. If a theory of cognitive
processing requires unbounded generativity and recursion, then
introducing sloppy symbols is a weakness. A better bet, if you be-
lieve that neural systems cannot support amodal symbols and the
requisite compositional and recursive processes, is to question the
nature of the requirement for productivity. It is also unclear how
perceptual symbols solve the transduction problem. It is not clear,
for example, that abstraction from perceptual states via selective
attention is superior to any other current proposal in this regard.
Can one then characterize the requirement for productivity in
a way that relieves the burden on internal representation and al-
lows perceptual processes to play a larger part in cognitive pro-
cessing generally? I believe that it can be done but it requires a
more radical picture of the nature of cognitive architecture than
Commentary/Barsalou: Perceptual symbol systems
634 BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4

the one Barsalou paints. The traditional view of productivity relies
primarily on internal resources for both storage and processing.
Hence the requirement for internal structures that can store ar-
bitrarily complex, iterated, nested symbolic expressions. It is quite
plausible, however, given the existence of external symbol storage
media such as books, that productivity might be understood as an
interaction between external symbol structures and internal pro-
cesses. In the most extreme case there need be no internal sym-
bol storage. An argument in favor of models of cognitive architec-
ture with external symbolic resources has been advanced by Wells
(1998) based on a re-analysis of Turing’s theory of computation. If
one argues that cognitive architecture extends into the environ-
ment of the cognizer, cognition does indeed become integrally
linked to perception. Moreover, the representational burden on
internal states can be greatly reduced. Donald (1991; see also BBS
multiple book review of Donald’s Origins of the Modern Mind
BBS 16(4) 1993) also emphasizes the significance of external sym-
bol storage for theories of cognitive architecture.
Barsalou’s proposal demonstrates the need for a theory of this
kind. He, among others, has pointed out the substantial difficul-
ties faced by theories based on amodal symbol systems. His
achievement in the present target article is to have set out a com-
prehensive outline of what a theory based on perceptual symbols
would be like. He has, so to speak, covered all the angles and in-
dicated the central problems that any theory using perceptual
symbols would have to solve. In doing this, he has, I think, shown
that such a theory is unlikely to succeed.
ACKNOWLEDGMENT
I am grateful to Bradley Franks for discussion and advice.
Perceiving abstract concepts
Katja Wiemer-Hastings and Arthur C. Graesser
Department of Psychology, The University of Memphis, Memphis, TN 38152-
6400. kwiemer@latte.memphis.edu graesser@cc.memphis.edu
www.psyc.memphis.edu/faculty/graesser/graesser.htm
Abstract: The meanings of abstract concepts depend on context. Percep-
tual symbol systems (PSS) provide a powerful framework for representing
such context. Whereas a few expected difficulties for simulations are con-
sistent with empirical findings, the theory does not clearly predict simula-
tions of specific abstract concepts in a testable way and does not appear to
distinguish abstract noun concepts (like truth) from their stem concepts
(such as true).
Do perceptual symbol systems (PSS) solve the “challenge of any
perceptual representation system,” that is, can they account for
the representation of abstract concepts? Abstract concepts have
no perceivable referents. It is therefore challenging to specify
their representation. It is well documented that comparatively few
attributes are generated for abstract concepts in free generation
tasks (Graesser & Clark 1985; Markman & Gentner 1993; McNa-
mara & Sternberg 1983). Hampton (1981) has shown that many
abstract concepts are not structured as prototypes.
With such difficulties in mind, it may seem somewhat surpris-
ing that a perceptually based theory should provide a compelling
framework to represent abstract concepts. However, we argue
that the nature of abstract concepts requires such an approach.
Previous theories have emphasized that such concepts depend on
context (e.g., Quine 1960). Grounding abstract concepts in con-
text, however, only works if context provides useful information
for their meaning representation. That is, context itself needs to
have a representation that is grounded. And of course, the ulti-
mate grounding mechanism of humans is perception.
Concept representations characterize a concept and distinguish
it from other concepts. Barsalou demonstrates that PSS can in
principle represent abstract concepts, but can it also handle their
differentiation? This issue is important. Abstract concepts are of-
ten very similar to each other, therefore requiring very fine dis-
tinction. Examples for similar abstract concepts are argument, dis-
agreement, and conflict. The similarity of such word sets should
result in very similar simulations because the concepts occur in
similar contexts. At the same time, however, we know that there
are subtle differences in meaning, and such differences must be
captured by representations. Do PPS capture such differences?
Suppose separate groups were asked to list attributes for either
argument or for disagreement. Conceivably, both groups would
come up with similar lists, suggesting identical representations.
However, if asked to generate attributes for both concepts, they
might attempt to list distinctive features via a careful comparison
of simulations. This illustrates a problem: PSS have the power to
represent abstract concepts, but the theory does not provide prin-
cipled predictions for simulations of particular abstract concepts.
Instead, ad hoc simulations account for a concept depending on a
context or on contrast concepts.
Contexts of abstract concepts may vary considerably. Hampton
(1981) pointed out that abstract concepts are “almost unlimited in
the range of possible new instances that could be discovered or in-
vented” (p. 153). This is especially true for a subset of abstract
nouns that have a content, such as belief, truth, and idea. Content
concepts reflect various degrees of abstraction, as illustrated for
the concept of truth:
1. I knew he was not telling me the truth. (Truth is a concrete
referent in a situation; it is a “fact.”)
2. I know that he always tells the truth. (Truth is a feature that is
shared by a set of statements.)
3. Scientists want to know the truth. (Truth is the general, absolute
knowledge of how things are.)
Barsalou’s suggested simulation for truth is appropriate for (1) and
(2), in that it involves the comparison of facts and statements. For
(3), however, it seems that the simulation needs to express a higher
level of abstractness, because this truth (the set of facts) is as yet
unknown. It is unclear how PSS would handle cases like (3).
A second interesting aspect of content concepts is that their con-
tent can vary infinitely. This may be associated with the observa-
tion that abstract concepts are difficult to define and overall take
longer to process (for an overview see Schwanenflugel 1991). Con-
sider the following expressions of ideas: He might be in the other
room, perhaps you need to dial a 1 first, let’s have pizza, and so on.
How can an abstracted representation of idea be constructed from
such examples? What do these ideas have in common?
They probably share contextual aspects, such as an initial prob-
lem: I cannot find him, I cannot get through, or what should we
eat? Then, somebody expresses an idea. Consequently, different
things may happen: An opposing reaction (We had pizza yester-
day), a new behavior to solve the problem (he was indeed in the
other room, or dialing 1 does not help either), or an alternative idea
(I’ve just looked there. Perhaps he is outside?). Due to the varia-
tion, it may be difficult to identify which aspects are critically re-
lated to the word meaning and should therefore be used to frame
the simulation. However, if concepts are presented in a specific
context, the context can frame the simulation and provide clear
referents. Compare the following contexts:
1. “It is too bad that he quit his job. He always had such good
ideas!”
2. “I have lost my key somewhere.”
“Did you search your jacket pockets?”
“No – great idea!”
Example (1) requires some abstraction of idea in the context of a
job, whereas in (2), the idea is a concrete suggestion to do some-
thing. A simulation should be easier to construct in the second
case because the situation sets up an event sequence. This does
not present a problem, however, since it is consistent with the ob-
servation that abstract language is processed faster if presented in
a meaningful context (Schwanenflugel & Shoben 1983).
Commentary/Barsalou: Perceptual symbol systems
BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4 635

Finally, it is unclear how a perceptual symbols system distin-
guishes between a stem concept (“true”) and a complex abstract
noun (“truth”). It is true that the moon orbits the earth. Also, oc-
casionally it is true that we are tired. We know that these state-
ments are true by comparison with the facts. It seems that the sim-
ulations for true and truth would be the same. However, it could
be argued that truth and something being true are very different
concepts.
Perceptual symbols in language
comprehension: Can an empirical
case be made?
Rolf A. Zwaan, Robert A. Stanfield, and Carol J. Madden
Department of Psychology, Florida State University, Tallahassee, FL 32306-
1270. freud.psy.fsu.edu:80/~zwaan/
Abstract: Perceptual symbol systems form a theoretically plausible alter-
native to amodal symbol systems. At this point it is unclear whether there
is any truly diagnostic empirical evidence to decide between these systems.
We outline some possible avenues of research in the domain of language
comprehension that might yield such evidence. Language comprehension
will be an important arena for tests of the two types of symbol systems.
Barsalou provides convincing theoretical arguments for why a per-
ceptual symbol system should be preferred over amodal systems:
greater parsimony, the ability to predict rather than postdict ef-
fects, resolution of the symbol-grounding problem, consistency
with the notion of embodiment, and avoidance of an evolutionary
quantum leap presupposition. He also discusses examples of em-
pirical evidence. It is not clear to us at this point how diagnostic
these are with respect to a comparison between perceptual and
amodal symbol systems. For example, Barsalou (sect. 2.3) inter-
prets neuroimaging evidence showing that visual areas of the brain
are active during conceptual tasks involving animals and motor ar-
eas during conceptual tasks involving tools as support for percep-
tual symbol systems. Proponents of amodal systems might grant
that representations are stored in the corresponding perceptual
areas of the brain, but still maintain that these representations are
amodal. The problem, as Barsalou (sect. 1.2.2) notes is that
amodal systems might be able to explain (away) many findings on
a post-hoc basis.
Because we find Barsalou’s view congenial and inspiring, we at-
tempt to identify some existing and possible lines of research that
might have diagnostic value with respect to the comparison be-
tween perceptual and amodal symbol systems and as such could
bolster the case for perceptual symbol systems empirically.
Amodal propositional systems are arguably more dominant in the
area of language comprehension than in any other area of cogni-
tion. Therefore, it is a crucial arena for evaluating perceptual and
amodal symbol systems.
We will first clarify our understanding of language comprehen-
sion in a perceptual-symbol framework, extrapolating from Barsa-
lou’s comments. Language comprehension is tantamount to the
construction of a mental model of the described state of affairs: a
situation model (e.g., Zwaan & Radvansky 1998). Barsalou (sect.
2.4.2) argues that situation models are, in terms of a perceptual-
symbol framework, equivalent to surface level simulations only.1
We agree with this characterization. Situation models represent
specific situations bound in time and space. As such, they are to-
kens, whereas related knowledge structures, such as scripts and
frames, are types. We understand simulators to be types and sim-
ulations to be tokens. Thus, frames and scripts are assemblies of
simulators (which differ in that the simulators become activated
in a specified temporal order in scripts but not in frames), whereas
situation models are simulations generated in part by frames and
scripts. These simulations run more smoothly – and perhaps seem
more perception-like – when their simulators can be assembled
and sequenced quickly. This happens when comprehenders have
a great deal of background knowledge, for example in the form of
frames and scripts. Thus, skilled text comprehension is analogous
(but not identical!) to the perception of unfolding events. Such a
perspective might be better equipped to explain how readers be-
come immersed in narrative worlds (Zwaan, in press) than would
amodal systems.
How could research on situation models yield evidence that is
diagnostic with respect to a comparison between perceptual and
amodal symbol systems? We sketch five existing and potential
lines of research.
1. Perceptual effects during comprehension. Morrow and Clark
(1989) demonstrated that readers’ interpretation of motion words,
such as “approach” depends on perceptual characteristics of the
situation. Furthermore, comprehenders provide faster responses
to words denoting events and objects present in the situation com-
pared to discontinued events and nonpresent objects (e.g., Mac-
donald & Just 1989; Zwaan 1996). There is no straightforward way
to account for these findings using propositional analyses.
2. Rapid integration of perceptual and language-based infor-
mation. Klatzky et al. (1989) found that the comprehension of ver-
bally described actions is facilitated when subjects are allowed to
first form their hand into a shape appropriate for the action (for
example, fingers pinched together when reading about throwing
a dart).
3. Interference between secondary perceptual/imagery task
and situation-model construction. Fincher-Kiefer (1998) recently
showed that comprehenders who had to keep a high-imagery sen-
tence unrelated to the text in working memory had greater diffi-
culty constructing a situation model than comprehenders who had
to keep a low-imagery sentence in working memory.
4. Effects of embodiment on comprehension. We know of no re-
search in this domain. Yet, this type of evidence might yield strong
evidence against amodal systems. For example, it might be that a
basketball player represents the situation described by “He picked
up the basketball” differently (one-handed) from a smaller person
(two-handed). One would also assume that women who have
given birth construct different mental representations of a story
about childbirth than other women or than men. Amodal systems
would have no a priori way to account for this.
5. Physiological effects of comprehension. Clinical psycholo-
gists have amassed a great deal of relevant evidence, which is not
discussed by Barsalou. For example, Lang and his colleagues have
conceptualized and demonstrated how emotional imagery can
have powerful physiological effects (Lang 1979).
Presumably, the evidence yielded by these lines of research
would not be equivalent in diagnostic value. For example, the
rapid integration of language-based and perceptual information
and the physiological effects of comprehension are not necessar-
ily predicted by amodal symbol systems, but might be accommo-
dated by them post hoc, maybe with some additional assumptions.
The occurrence of perceptual effects during comprehension
would be more difficult to accommodate by amodal systems. Even
more diagnostic would be demonstrations of interference of un-
related perceptual representations on the construction of situa-
tion models. Finally, the most diagnostic evidence, in our view,
would be demonstrations of variable embodiment on compre-
hension processes. Perhaps none of these lines of research will
yield convincing evidence for perceptual symbols independently,
but taken together they might. At the very least, they would force
proponents of amodal symbol systems to postulate an increasingly
unwieldy and implausible range of special assumptions to postdict
the findings.
Barsalou has provided a viable alternative against reigning
amodal symbol systems. He has successfully integrated ideas and
concepts from linguistics, philosophy, and neuroscience into a
cognitive-psychological framework. The perceptual-symbol frame-
work shines a searchlight on research topics like the five listed
above, which have received scattered attention in the past. As
such, it could provide a unified account of the extant and future
Commentary/Barsalou: Perceptual symbol systems
636 BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4

findings and thus influence the agenda for cognitive science in
years to come.
NOTE
1. This might not be entirely accurate for mental models in general, be-
cause some are more generic and therefore more like simulators; see John-
son-Laird’s (1983, pp. 422–29) typology.
Author’s Response
Perceptions of perceptual symbols
Lawrence W. Barsalou
Department of Psychology, Emory University, Atlanta, GA 30322.
barsalou@emory.edu userwww.service.emory.edu/~barsalou/
Abstract: Various defenses of amodal symbol systems are ad-
dressed, including amodal symbols in sensory-motor areas, the
causal theory of concepts, supramodal concepts, latent semantic
analysis, and abstracted amodal symbols. Various aspects of per-
ceptual symbol systems are clarified and developed, including
perception, features, simulators, category structure, frames, anal-
ogy, introspection, situated action, and development. Particular
attention is given to abstract concepts, language, and computa-
tional mechanisms.
I am grateful for the time and energy that the commenta-
tors put into reading and responding to the target article.
They raised many significant issues and made many excel-
lent suggestions. Even the strongest criticisms were useful
in clarifying misunderstandings and in addressing matters
of importance. I have organized the commentaries into two
general groups: (1) those that defended amodal symbol sys-
tems and (2) those that addressed the properties of per-
ceptual symbol systems. Because the second group was so
large, I have divided it into four smaller sections. In the
first, I address a wide variety of issues surrounding percep-
tual symbol systems that include perception, features, sim-
ulators, frames, introspection, and so forth. The final three
sections address the topics raised most often: abstract con-
cepts, language, and computational mechanisms.
R1. Defense of amodal symbol systems
No commentator responded systematically to the criticisms
of amodal symbol systems listed in sections 1.2.2 and 1.2.3
of the target article. Little direct evidence exists for amodal
symbols; it is difficult to reconcile amodal symbols with ev-
idence from neuroscience; the transduction and grounding
of amodal symbols remain unspecified; representing space
and time computationally with amodal systems has not
been successful; amodal symbol systems are neither parsi-
monious nor falsifiable. Expecting a single commentator to
address all of these concerns in the space provided would
certainly be unfair. I was struck, however, by how little at-
tempt overall there was to address them. Having once been
deeply invested in amodal symbol systems myself (e.g.,
Barsalou 1992), I understand how deeply these convictions
run. Nevertheless, defending amodal symbol systems
against such criticisms seems increasingly important to
maintaining their viability.
A handful of commentators defended amodal systems in
various other ways. Although most of these defenses were
concrete and direct, others were vague and indirect. In ad-
dressing these defenses, I proceed from most to least con-
crete.
R1.1. Amodal symbols reside in sensory-motor areas of
the brain. The defense that struck me as most compelling
is the suggestion that amodal symbols reside in sensory-mo-
tor regions of the brain (Aydede; Zwaan et al.). A related
proposal is that perceptual representations are not consti-
tutive of concepts but only become epiphenomenally active
during the processing of amodal symbols (Adams &
Campbell). In the target article, I suggested that neuro-
science evidence implicates sensory-motor regions of the
brain in knowledge representation (sect. 2.1., 2.2, and 2.3).
When a lesion exists in a particular sensory-motor region,
categories that depend on it for the perceptual processing
of their exemplars can exhibit knowledge deficits. Because
the perception of birds, for example, depends heavily on
visual object processing, damage to the visual system can
produce a deficit in category knowledge. Neuroimaging
studies of humans with intact brains similarly show that
sensory-motor areas become active during the processing
of relevant categories. Thus, visual regions become active
when accessing knowledge of birds.
Several commentators noted correctly that these find-
ings are consistent with the assumptions that (a) amodal
symbols represent concepts and that (b) these symbols re-
side in sensory-motor regions (Adams & Campbell;
Aydede; Zwaan et al.). If these assumptions are correct,
then damage to a sensory-motor region could produce a
deficit in category knowledge. Similarly, sensory-motor re-
gions should become active when people with intact brains
process categories.
This is an important hypothesis that requires careful em-
pirical assessment. Damasio’s (1989) convergence zones
provide one way to frame the issue. According to his view,
local associative areas in sensory-motor regions capture pat-
terns of perceptual representation. Later, associative areas
reactivate these sensory-motor representations to simulate
experience and thereby support cognitive processing. As
the quotation from Damasio (1989) in section 2.1 illus-
trates, he believes that reactivating sensory-motor repre-
sentations is necessary for representing knowledge – acti-
vation in a nearby associative area never stands in for them.
In principle, however, activation in local associative areas
could stand in for sensory-motor representations during
symbolic activity, thereby implementing something along
the lines of amodal symbols, with perceptual representa-
tions ultimately being epiphenomenal.
Behavioral findings argue against this proposal. Studies
cited throughout the target article show that perceptual
variables predict subjects’ performance on conceptual
tasks. For example, Barsalou et al. (1999) report that oc-
clusion affects feature listing, that size affects property ver-
ification, and that detailed perceptual form predicts property
priming. The view that activation in associative areas rep-
resents concepts does not predict these effects or explain
them readily. Instead, these effects are more consistent
with the view that subjects are reactivating sensory-motor
representations. Studies on language comprehension simi-
larly exhibit such effects (sects. 4.1.6, R4.5, and Zwaan
et al.).
Response/Barsalou: Perceptual symbol systems
BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4 637

takes me to mean that no silicon-based system could ac-
quire human knowledge. He then shows how computers that
implement LSA mimic humans on a wide variety of knowl-
edge-based tasks. I never claimed, however, that amodal
systems cannot mimic humans behaviorally! Indeed, sec-
tion 1.2 noted that amodal symbols have so much power
that they can probably mimic all human behavior. My claim
in section 4.4 was different, namely, if knowledge is
grounded in sensory-motor mechanisms, then humans and
computers will represent knowledge differently, because
their sensory-motor systems differ so radically. I acknowl-
edged the possibility that future technological develop-
ments might produce computers with sensory-motor sys-
tems closer to ours, in which case their knowledge could be
represented similarly. No one is currently under the illu-
sion, however, that the input-output systems of today’s
computers are anything like human sensory-motor systems.
Thus, if knowledge is implemented in perceptual systems,
it would have to take very different forms.
In making his case for LSA, Landauer fails to address
the problems I raise for amodal symbol systems in section
1.2: (1) What is the direct evidence that co-occurrence fre-
quency between words controls performance? A system
based on co-occurrence frequencies can mimic human be-
havior, thereby providing indirect evidence that this con-
struct is useful, but is there evidence implicating this basic
unit of analysis directly? (2) What is the neural story? Do
lesion and neuroimaging data provide support? (3) How are
perception and conception linked? Everyone would agree
that a conceptual system helps us interpret perceived
events in the world, but how do word co-occurrences map
onto perceived individuals in the world? When we perceive
an object, how is it categorized using LSA? When we con-
ceptualize an object, how do we find its referent in the
world? (4) How does LSA represent knowledge about
space and time? It is difficult to see how a system that only
tracks word co-occurrence can represent these basic as-
pects of perception. Other concerns about LSA arise, as
well. How does LSA accomplish the conceptual functions
addressed in the target article, such as distinguishing types
and tokens, implementing productivity, and representing
propositions? Using only word correlations, it does not
seem likely that it can implement these basic human abili-
ties. Finally, are co-occurrences tracked only between
words, or between corresponding amodal symbols, as well?
If the latter is the case, what is the nature of this amodal sys-
tem, and how is it related to language?
It is important to note that correlation – not causation –
underlies LSA’s accounts of human data. Because word co-
occurrence exists in the world and is not manipulated ex-
perimentally, infinitely many potential variables are con-
founded with it. To appreciate this problem, recall how LSA
works. On a given task, the likelihood of a particular re-
sponse (typically a word) is predicted by how well it corre-
lates with the overall configuration of stimulus elements
(typically other words). Thus, responses that are highly cor-
related with stimulus elements are more likely to be pro-
duced than less correlated responses. The problem is that
many, many other potential variables might well be corre-
lated with word co-occurrence, most notably, the percep-
tual similarity of the things to which the words refer. It is an
interesting finding that computer analyses of word correla-
tions produce a system that mimics human behaviors. It
does not follow, though, that these correlations are causally
important. Of course the human brain could work like LSA,
but to my knowledge, no evidence exists to demonstrate
this, or to rule out that variables correlated with word co-
occurrence are the critical causal factors.
A number of empirical findings suggest that knowledge
is not grounded in word co-occurrence. First, aphasics typ-
ically lose only language; they do not lose knowledge
(Lowenthal). If knowledge were simply represented in
word co-occurrence, they should lose knowledge, as well.
Second, Glenberg reports that perceptual affordances en-
ter into language comprehension, and that word co-occur-
rence cannot explain these results (also see sects. 4.1.6 and
R4.5). Landauer believes that this is because words for
certain idioms, metaphors, and modern phrases have not
been included in the language corpus that underlies LSA,
but again the problem of correlation versus causation arises.
Landauer wants to believe that these missing entries have
causal significance, when actually their status is merely cor-
relational. Because Glenberg’s affordances are also correla-
tional, more powerful laboratory techniques are needed to
tease these factors apart. To the extent that sensory-motor
affordances are implicated in conceptual processing, how-
ever, LSA has no way of handling them. In this spirit,
Solomon and Barsalou (1999a; 1999b) controlled the
strength of lexical associations in the property verification
task and found effects of perceptual variables. In a task in-
volving only linguistic stimuli, nonlinguistic factors signifi-
cantly affected performance when linguistic factors were
held constant.
One way to extend LSA would be to apply its basic as-
sociative mechanism to perceptual elements. Just as co-
occurrences of words are tracked, so are co-occurrences in
perception. Depending on the specific details of the for-
mulation, such an approach could be virtually identical to
the target article’s proposal about frames. As described in
section 2.5., a frame accumulates components of percep-
tion isolated by selective attention, with the associative
strength between components reflecting how often they
are processed together. Thus, the co-occurrence of percep-
tual components lies at the heart of these perceptually
grounded frames. Notably, however, it is proposed that
these structures are organized by space and time, not sim-
ply by associative strength. To later simulate perceptual ex-
perience, the mere storage of associations will not be suffi-
cient – extensive perceptual structure will be necessary, as
well. If so, a simple extension of LSA from words to per-
ception probably will not work.
Furthermore, we have known since Chomsky (1957) that
the human brain is not simply a generalization mechanism.
Besides generalizing around known exemplars, the brain
produces infinite structures from finite elements produc-
tively. Not only is this true of language, it is also true of con-
ception (Fodor & Pylyshyn 1988). People can conceive of
concepts that they have never encountered and that are
not generalizations of past concepts. Neither a word-based
nor a perception-based version of LSA can implement
productivity. There is no doubt that the human brain is an
associative device to a considerable extent, yet it is also a
productive device. Although LSA may explain the associa-
tive aspects of cognition, it appears to have no potential for
explaining the productive aspects.
Finally, in a superb review of picture-word processing,
Glaser (1992) concluded that a mixture of two systems ac-
counts for the general findings in these paradigms. Under
Response/Barsalou: Perceptual symbol systems
BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4 639

to establish symbols that serve higher goals of the system.
The open-endedness of the symbols that people use should
not be viewed as a weakness of perceptual symbol systems.
Instead, it is something that any theory needs to explain,
and something that perceptual symbol systems explain
through the flexible use of selection attention (sect. 2.2). It
would be much more problematic if the theory were sim-
ply limited to symbols that reflect the computations of low-
level sensory-motor processing.
Ultimately, Schwartz et al. have another agenda. They
argue that schematic category representations result from
recruitment and fractionation, not from selective attention.
It is implicit in their view that holistic perceptual states are
recorded during learning, with differences across them
canceling out to produce schematic representations of per-
ceptual symbols. Similarly, Gabora argues for the impor-
tance of these mechanisms. Although I endorse these
mechanisms, I strongly disagree with the view that selective
attention plays no role in learning. As discussed in sections
2.2 and 4.1.3, selective attention has been strongly impli-
cated in learning across decades of research in many areas.
Furthermore, it is naive to believe that holistic recordings
of perception occur, as Hochberg argues eloquently (also
see Hochberg 1998). Being highly flexible does not make
selective attention irrelevant to conceptual processing. In-
stead, it is absolutely essential for creating a conceptual sys-
tem (sects. 1.4 and 4.1.3).
Hochberg and Toomela each propose that active infor-
mation-seeking and goal pursuit guide selective attention
and hence the acquisition of perceptual symbols. Glen-
berg’s arguments about the importance of situated action
are also consistent with this view. I agree strongly that top-
down goal-achievement often guides selective attention,
such that perceptual information relevant to these pursuits
becomes acquired as perceptual symbols. However, bot-
tom-up mechanisms may produce perceptual symbols, as
well. For example, attention-grabbing onsets and changes
in perception may attract processing and produce a per-
ceptual symbol of the relevant content. Similarly, the per-
ceptual layout of a physical stimulus may guide attention
and the subsequent extraction of perceptual symbols. In
this spirit, Brewer notes that physical proximity better pre-
dicts recall order for the objects in a room than does their
conceptual relatedness. Thus, bottom-up as well as top-
down factors are probably important to selecting the con-
tent of perceptual symbols. Furthermore, appealing to the
importance of top-down factors pushes the difficult issues
back a level. Attention is clearly under the control of goals
and action to a considerable extent, yet how do we charac-
terize the mechanisms underlying goals and actions that
guide attentional selection?
R2.3. Abstraction, concepts, and simulators. I am the
first to admit that specifying the mechanisms underlying
simulators constitutes perhaps the most central challenge
of the theory, and I noted this throughout the target article.
Later I will have more to say about implementing simula-
tors in specific mechanisms (sect. R5.3). Here I focus on
some misunderstandings about them.
Without citing any particular statement or section of the
target article, Ohlsson claims that I equated selection with
abstraction. According to him, I believe that selectively
storing a wheel while perceiving a car amounts to creating
an abstraction of cars in general. Nowhere does the target
article state that selection is abstraction. On the contrary,
this is a rather substantial and surprising distortion of my
view. What I did equate roughly with selection was schema-
tization, namely, the information surrounding focal content
is filtered out, leaving a schematic representation of the
component (sect. 2.2). However, I never claimed that a
schematization is an abstraction, or that it constitutes a con-
cept. Rather, I argued that the integration of many
schematic memories into a simulator is what constitutes a
concept (sect. 2.4). Actually, I never said much about ab-
straction per se but focused instead on the development of
types (sects. 2.4.3. and 3.2.1). If I were to characterize ab-
straction in terms of perceptual symbol systems, I would
define it as the development of a simulator that reenacts the
wide variety of forms a kind of thing takes in experience. In
other words, a simulator contains broad knowledge about a
kind of thing that goes beyond any particular instance.
Adams & Campbell claim that I characterize simulators
in terms of the functions they perform, such as categoriza-
tion and productivity. Most basically, however, I defined
simulators as bodies of knowledge that generate top-down
reenactments or simulations of what a category is like (sect.
2.4). Clearly, this is a functional specification, but it is not
the one that Adams & Campbell attribute to me. Adams &
Campbell further complain that I fail to say anything about
the mechanisms that underlie simulators (so do Dennett
& Viger and Schwartz et al.). In section 2.4, however, I
defined a simulator as a frame plus the mechanisms that op-
erate on it to produce specific simulations. In section 2.5, I
provided quite a bit of detail on the frames that underlie
simulators. I acknowledged that this account is far from
complete, and I agree that developing a full-blown compu-
tational theory is extremely important. For the record,
though, I certainly did say something about the mecha-
nisms that underlie simulators, and I continue to believe
that these ideas provide useful guidance in developing spe-
cific accounts (sect. R5.1).
Landau believes that simulations are not capable of cap-
turing the underlying content or abstractions of concepts.
If I understand correctly, she is making the same argument
Adams & Campbell made earlier that perceptual knowl-
edge is epiphenomenal and not constitutive of concepts.
In a sense, this is also similar to Ohlsson’s argument that
storing part of a category instance does not represent the
respective category. Again, however, the claim is that an en-
tire simulator – not a specific perceptual representation –
comes to stand for a category. As just discussed, a simulator
captures a wide variety of knowledge about the category,
making it general, not specific. Furthermore, the abstrac-
tion process described by Gabora and by Schwartz et al.
enters into the frames that underlie simulators, thereby
making them generic (sects. R1.5 and R2.4). Finally, a sim-
ulator stands in a causal relation to its physical category in
the world (sect. R1.2). On perceiving an instance of cate-
gory, a simulator is causally activated, bringing broad cate-
gory knowledge to bear on the instance (sects. 2.4.3 and
3.2.1). For all these reasons, simulators function as con-
cepts.
Finally, Siebel takes issue with my claim that bringing
the same simulator to bear across different simulations of a
category provides stability among them (sect. 2.4.5). Thus,
I claimed that the different simulations a person constructs
of birds are unified because the same simulator for bird
produced all of them. As Siebel correctly notes, each time
Response/Barsalou: Perceptual symbol systems
642 BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4

a simulator becomes active, the perceptual symbols
processed becomes stored or strengthened, thereby chang-
ing the simulator’s content. Thus, the same simulator – in
terms of its content – cannot be brought to bear on differ-
ent simulations to provide stability. “Same,” however, does
not refer to content but to the body of knowledge that
stands in causal relation historically to a physical category.
Stability results from the fact that one particular body of
knowledge typically becomes active when a particular cat-
egory is conceptualized, even though that body of knowl-
edge changes in content over time. Because all conceptual-
izations can be linked to this particular body of knowledge
historically, they gain stability.
R2.4. Conceptual essences and family resemblances.
Lurking behind the concern that simulators cannot repre-
sent concepts may well be the belief that concepts have
essences. When Ohlsson, Landau, and Adams & Camp-
bell question whether perceptual representations capture
underlying constitutive content, I take them to be worrying
about this. Since Wittgenstein (1953), however, there have
been deep reservations about whether concepts have nec-
essary and sufficient features. For this reason, modern the-
ories often portray essences as people’s naive belief that
necessary and sufficient features define concepts, even
when they actually do not (e.g., Gleman & Diesendruck, in
press). Should a concept turn out to have defining features,
though, a simulator could readily capture them. If a feature
occurs across all the instances of a category, and if selective
attention always extracts it, the feature should become well
established in the frame that underlies the simulator (sect.
2.5, Fig. 3; sect. R1.5; Gabora; Schwartz et al.). As a re-
sult, the feature is almost certain to become active later in
simulations of the category constructed. Simulators have
the ability to extract features that are common across the
instances of a category, should they exist.
Research on the content of categories indicates that few
features if any are typically common to all members of a cat-
egory (e.g., Malt 1994; Malt et al., in press; Rosch & Mervis
1975). However, a small subset of features is likely to be
true of many category members, namely, features that are
characteristic of the category but not defining. Should a
family resemblance structure of this sort exist in a category,
perceptual symbol systems are quite capable of extracting
it. Again, following the discussion of frames (sect. 2.5, Fig.
3) and abstraction (sects. R1.5 and R2.3), the more often a
feature is extracted during the perceptual processing of a
category, the better established it becomes in the category’s
frame. To the extent that certain features constitute a sta-
tistical regularity, the frame for the category will capture
this structure and manifest it across simulations con-
structed later. The more a feature is processed perceptually,
the more it should occur in category simulations.
Finally, Ohlsson seems to believe that a perceptual view
of concepts could only work if common perceptual features
underlie all instances of a concept. Nowhere did I claim
this, and there is no reason it must be true. As just de-
scribed, the frame that underlies a simulator captures the
statistical distribution of features for a category, regardless
of whether it possesses common features. For a category
lacking common features, its simulator will produce simu-
lations that do not share features with all other simulations
but are related instead by a family resemblance structure.
Fauconnier raises the related point that a word may not
be associated with a single simulator, taking me to mean
that a simulator only produces simulations that share com-
mon features. Again, however, there is no a priori reason
that a simulator cannot produce disjunctive simulations.
Because different perceptions may be associated with the
same word, different perceptual symbols may be stored dis-
junctively in the frame of the associated simulator and may
later produce disjunctive simulations. As Fauconnier sug-
gests, the content of a simulator is accessed selectively to
project only the information relevant in a particular context,
with considerably different information capable of being
projected selectively from occasion to occasion (sect. 2.4.3).
R2.5. Frames and productivity. Barsalou and Hale (1993)
propose that all modern representation schemes evolved
from one of two origins: propositional logic or predicate cal-
culus. Consider some of the differences between these two
formalisms. Most basically, propositional logic contains bi-
nary variables that can be combined with simple connec-
tives, such as and, or, and implies, and with simple opera-
tors, such as not. One problem with propositional logic is its
inability to represent fundamentally important aspects of
human thought and knowledge, such as conceptual rela-
tions, recursion, and bindings between arguments and val-
ues. Predicate calculus remedied these problems through
additional expressive power.
Barsalou and Hale show systematically how many mod-
ern representation schemes evolved from propositional
logic. By allowing binary variables to take continuous forms,
fuzzy logic developed. By replacing truth preservation with
simple statistical relations between variables, the construct
of a feature list developed, leading to prototype and exem-
plar models. By adding complicated activation and learning
algorithms to prototype models, connectionism followed.
Notably, neural nets embody the spirit of propositional
logic because they implement simple continuous units un-
der a binary interpretation, linked by simple connectives
that represent co-occurrence. In contrast, consider repre-
sentation schemes that evolved from predicate calculus.
Classic frame and schema theories maintain conceptual re-
lations, binding, and recursion, while relaxing the require-
ment of truth preservation. Perceptual symbol systems sim-
ilarly implement these functions (sect. 2.5.1), while relaxing
the requirement that representations be amodal or lan-
guage-like.
It is almost universally accepted now that representation
schemes lacking conceptual relations, binding, and recur-
sion are inadequate. Ever since Fodor and Pylyshyn’s
(1988) classic statement, connectionists and dynamic sys-
tems theorists have been trying to find ways to implement
these functions in descendants of propositional logic. In-
deed, the commentaries by Edelman & Breen, Mark-
man & Dietrich, and Wells all essentially acknowledge
this point. The disagreement lies in how to implement these
functions.
Early connectionist formulations implemented classic
predicate calculus functions by superimposing vectors for
symbolic elements in predicate calculus expressions (e.g.,
Pollack 1990; Smolensky 1990; van Gelder 1990). The psy-
chological validity of these particular approaches, however,
has never been compelling, striking many as arbitrary tech-
nical attempts to introduce predicate calculus functions
into connectionist nets. More plausible cognitive accounts
that rely heavily on temporal asynchrony have been sug-
Response/Barsalou: Perceptual symbol systems
BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4 643

gested by Shastri and Ajjanagadde (1993) and Hummel and
Holyoak (1997). My primary concern with this approach is
that it is amodal and therefore suffers from the problems
noted in section 1.2.
Edelman & Breen suggest that spatial relations provide
considerable potential for implementing conceptual rela-
tions, binding, and recursion. Furthermore, they note that
these functions achieve productivity through the combina-
torial and recursive construction of simulations. As sections
2.5 and 3.1 indicate, I agree. Perceptual symbol systems
provide more than a means of representing individual con-
cepts; they provide powerful means of representing rela-
tions between them. My one addition to Edelman &
Breen’s proposal would be to stress the importance of tem-
poral relations – not just spatial relations – as the basis of
frames and productivity. The discussion of ad hoc cate-
gories in section 3.4.4 illustrates this point. One can think
of a simulation as having both spatial and temporal dimen-
sions. For example, a simulation of standing on a chair to
change a light bulb contains entities and events distributed
over both space and time. As suggested in section 3.4.4, ad
hoc categories result from disjunctively specializing space-
time regions of such simulations. Thus, the entity that is
stood on to change a light bulb constitutes a space-time re-
gion, which can be specialized with simulations of different
objects. Similarly, the agent standing on the chair can be
specialized differently, as can the burned-out light bulb, the
new light bulb, the fixture, and so forth. Each region that
can be specialized constitutes a potential attribute in the
frame for this type of event. Note that such regions are not
just defined spatially, they are also defined temporally.
Thus, the regions for the burned out bulb, the new bulb,
and the chair all occupy different regions of time in the sim-
ulation, not just different regions of space. Because there
are essentially an infinite number of space-time regions in
a simulation, there are potentially an infinite number of
frame attributes (Barsalou 1992; 1993).
Markman & Dietrich observe that frame structure
arises in perceptual symbol systems through nativist mech-
anisms. With appropriate acknowledgment of epigenesis
(Elman et al. 1996) and eschewing genetic determinism, I
agree. Frame structure arises from the basic mechanisms of
perception. Indeed, I have often speculated informally that
predicate calculus also originated in perception. Markman
& Dietrich further speculate that language is not the origin
of frame structure. Conversely, one might speculate that
the frame-like structure of language originated in percep-
tion as well. Because perception has relations, binding, and
recursion, language evolved to express this structure
through verbs, sentential roles, and embedded clause struc-
ture, respectively. Finally, Markman & Dietrich suggest
that frame structure is not learned, contrary to how certain
learning theorists might construe its origins in neural nets.
I agree that the basic potential for frames lies deep in the
perceptual architecture of the brain. No doubt genetic reg-
ulation plays an important role in producing the attentional,
spatial, and temporal mechanisms that ultimately make
frames possible. Yet, I hasten to add that specific frames are
most certainly learned, and that they may well obey con-
nectionist learning assumptions during their formation.
Again, though, these connectionist structures are not
amodal but are instead complex spatio-temporal organiza-
tions of sensory-motor representations.
Finally, Wells agrees that frame structure is important.
Without providing any justification, though, he claims that
only amodal symbol systems can implement productivity,
not perceptual symbol systems. Perhaps perceptual symbol
systems do not exhibit exactly the same form of productiv-
ity as amodal symbol systems, but to say that they do not
implement any at all begs the question. As section 3.1 illus-
trated in considerable detail, the hierarchical composition
of simulations implements the combinatoric and recursive
properties of productivity quite clearly. Wells then pro-
ceeds to argue that productivity does not reside in the brain
but resides instead in interactions with the environment.
This reminds me of the classic behaviorist move to put
memory in the environment as learning history, and I pre-
dict that Wells’s move will be as successful.
I sympathize with Wells’s view that classic representa-
tional schemes have serious flaws. As Hurford notes, how-
ever, it is perhaps unwise to throw out the symbolic baby
with the amodal bath water. There are many good reasons
for believing that the brain is essentially a representational
device (Dietrich & Markman, in press; Prinz & Barsalou, in
press b). Some theorists have gone so far as arguing that
evolution selected representational abilities in humans to
increase their fitness (Donald 1991; 1993). Perceptual sym-
bol systems attempt to maintain what is important about
representation while similarly attempting to maintain what
is important about connectionism and embodied cognition
(sect. R6). The brain is a statistical, embodied, and repre-
sentational device. It is our ability to represent situations
offline, and to represent situations contrary to perception,
that make us such amazing creatures.
R2.6. Analogy, metaphor, and complex simulations. As
noted by Markman & Dietrich, the ability of perceptual
symbol systems to represent frames makes it possible to ex-
plain phenomena that require structured representations,
including analogy, similarity, and metaphor. Indurkhya
further notes that, by virtue of being perceptual, frames
provide powerful means of explaining how truly novel fea-
tures emerge during these phenomena. In a wonderful ex-
ample, he illustrates how blending a perceptual simulation
of sunlight with a perceptual simulation of the ocean’s sur-
face the emergent feature of harp strings vibrating back and
forth. Indurkhya argues compellingly that perceptual sym-
bol systems explain the emergence of such features much
more naturally than do amodal symbol systems. In this
spirit, he suggests that conceptual blending be called “per-
ceptual blending” to highlight the importance of perceptual
representations, and he cites empirical evidence showing
that perception enters significantly into metaphor compre-
hension.
Fauconnier notes that I fail to acknowledge the com-
plexity of simulations in language and thought. He provides
a compelling example of a child who uses sugar cubes and
matchbooks to simulate cars and buses driving down the
street. Clearly, such examples involve complicated simula-
tions that reside on multiple levels, and that map into one
another in complex ways. Indeed, this particular example
pales in complexity next to Fauconnier and Turner’s (1998)
examples in which people juxtapose perception of the cur-
rent situation with past, future, and counterfactual situa-
tions. In these examples, people must represent several sit-
uations simultaneously for something they say to make
sense. To my mind, the fact that the human conceptual sys-
tem can represent such complicated states-of-affairs illus-
Response/Barsalou: Perceptual symbol systems
644 BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4

trates how truly remarkable it is. Brewer’s observation that
simulations provide a good account of how scientists repre-
sent models further illustrates this ability.
The evolution of the human frontal lobes provides one
way to think about people’s ability to construct multiple
simulations simultaneously and to map between them. In-
creasingly large frontal lobes may have provided greater in-
hibitory power to suppress perception and to represent
nonpresent situations (cf. Carlson et al. 1998; Donald 1991;
1993; Glenberg et al. 1998). Taking this idea a little further
provides leverage in explaining humans’ ability to represent
multiple nonpresent situations simultaneously. Not only
can humans simulate absent situations, we can simulate
several absent situations simultaneously and map between
them. The possession of a powerful inhibitory and control
system in the frontal lobes may well make this possible.
R2.7. Introspection, emotion, and metacognition. Two
commentators found my inclusion of introspective processes
problematic and unparsimonious (Newton, Toomela). Ul-
timately, they worry about the mechanisms that perceive in-
trospective events. In place of introspection, Newton sug-
gests that we seek subtle proprioceptive events that ground
intuitive understandings of introspection. For example, ex-
periences of collateral discharge while executing move-
ments might ground the intuitive concept of agency,
whereas experiences of eye movements might ground the
intuitive concept of seeing. Similarly, Toomela suggests that
introspective phenomena arise through interactions of
more basic systems, such as perception and language.
Although I am sympathetic to these proposals and could
imagine them being correct to some extent, I am not at all
convinced that it is either necessary or wise to explain all in-
trospection in these ways. Most significantly, people clearly
experience all sorts of internal events, including hunger, fa-
tigue, and emotion. Finding ways to ground them in more
externally-oriented events seems difficult, and denying
their internal experience seems highly counterintuitive.
Another problem for Newton and Toomela is explain-
ing the central role that the representation of mental states
plays in modern developmental and comparative psychol-
ogy. Research on theory of mind shows that people repre-
sent what they and others are representing (or are not rep-
resenting) (e.g., Hala & Carpendale 1997). A primary
argument to emerge from this literature is that represent-
ing mental states is a central ability that distinguishes hu-
mans from other species (Tomasello & Call 1997). If we do
away with introspection, how do we represent minds in a
theory of mind? Also, how do we explain the wide variety of
metacognitive phenomena that people exhibit (Char-
land)?
Regarding the problem of what mechanisms perceive
representing, I disagree with Newton and Toomela that
this leads to an infinite regress whereby additional mecha-
nisms to perceive representations must be added to the ba-
sic mechanisms that perceive the world. Rather, I suspect
that the brain uses one set of mechanisms to perceive both
the world and representations. Recall the basic premise of
perceptual symbol systems: Top-down simulations of sen-
sory-motor systems represent knowledge, optionally pro-
ducing conscious states in the process (sect. 2.1). Perhaps
the mechanisms that produce conscious experience of sen-
sory-motor systems when driven by bottom-up sensory pro-
cessing also produce consciousness of the same systems
when driven by top-down activation from associative areas
(sect. 2.4.7). No new mechanisms are necessary. The one
critical requirement is knowing the source of the informa-
tion that is driving conscious experience. In psychosis, this
awareness is lacking, but most of the time, a variety of cues
is typically sufficient, such as whether our eyes are closed
(Glenberg et al. 1998), and the vividness of the experience
(Johnson & Raye 1981).
I agree strongly with Charland that emotion is central
to perceptual symbol systems, and that emotion’s role in this
framework must be developed further. I also agree that
there may well be specific brain circuits for processing par-
ticular emotions, which later support conceptualizations of
these emotions (e.g., circuits for fear and anxiety; Davis
1998). Where I disagree with Charland is in his apparent
claim that emotion is a self-contained symbol system. I fur-
ther disagree with his readings of Damasio (1994) and
Lazarus (1991) that perceptual images and appraisal mech-
anisms belong to an autonomous symbol system for emo-
tion. This is certainly not my reading of their work. On the
contrary, I strongly suspect that these cognitive aspects of
emotion are provided directly by cognitive – not emotion –
mechanisms. Rather than containing its own cognitive sys-
tem, I suspect that emotion mechanisms complement a
separate cognitive system. I hasten to add, however, that
these two systems are so tightly coupled that dissociating
them creates a distortion of how each one functions alone
(Damasio 1994). Also, reenacting states in emotion systems
– not amodal systems in a general knowledge store – con-
stitutes the basis of representing emotions symbolically in
conceptual processing.
Finally, Oehlmann presents two metacognitive phenom-
ena and asks how perceptual symbol systems explain them.
First, how do people know that they know the solution to a
problem without having to simulate the entire solution? Se-
lective attention and schematization provide one account
(sect. 2.2). On simulating a solution, selective attention
stores the initial and final states in working memory, drop-
ping the intermediate states. By then switching back and
forth between the initial and final states, an abbreviated ver-
sion of the simulation becomes associated with the entire
simulation. On later perceiving the initial conditions for
these simulations, both become active, with the abbreviated
one finishing much sooner. If the final state in the abbrevi-
ated simulation is sufficient to produce a response, waiting
for the complete simulation to finish is unnecessary. As this
example illustrates, selective attention and schematicity pro-
vide powerful editing functions on simulations, just as in
productivity, making it possible to rise above complete sim-
ulations of experience (sect. 3.1). Again, a perceptual sym-
bol system is not a recording system.
Oehlmann’s second problem is how perceptual symbol
systems explain performance on the false belief task. Al-
though children can be shown to recognize that another
person has a different belief than they do, they neverthe-
less forget this at times and attribute their own belief to the
other person. Oehlmann claims that if beliefs were simply
simulated, this pattern of results should not occur. If chil-
dren can simulate another person’s belief correctly on one
occasion, why do they not simulate it correctly on all occa-
sions? An obvious explanation is that children at this age are
limited in their ability to simulate other people’s mental
states. Under optimal conditions, they have enough cogni-
tive resources to run different simulations for themselves
Response/Barsalou: Perceptual symbol systems
BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4 645

differentiate the senses of an abstract concept. Using the
concept of idea, Wiemer-Hastings & Graesser illustrate
how situations may well provide leverage on this task. I sus-
pect that the same strategy is also likely to provide leverage
on the different senses of truth that they raise, such as the
truth of a single utterance, the truth of all a person’s utter-
ances, and scientists trying to discover the truth. As just de-
scribed for falsity, differences in how simulations are as-
sessed against perceived situations may distinguish the first
two senses. In the first sense, a single simulation for one
particular utterance must be compared to a single per-
ceived situation. In the second sense, a simulation for each
claim a person makes must be compared to each respective
situation. To account for the scientific sense of truth, a com-
pletely new set of situations from the scientific enterprise is
required, including the formulation of hypotheses, the
conducting of research, the assessment of the hypotheses
against data, and so forth. Only within this particular set of
experiences does the scientific sense of truth make sense.
Again, specifying the meaning of an abstract concept re-
quires searching for the critical content in the relevant
background situations.
R3.6. How do you represent X? Various commentators
wondered how perceptual symbols could represent all sorts
of other abstract concepts. These commentaries fall into
two groups. First, there was the commentator who followed
my strategy in section 3.4.2 for representing abstract con-
cepts and discovered that it does indeed provide leverage
on representing them. Second, there were the commenta-
tors who complained that perceptual symbol systems fail to
represent particular abstract concepts, yet who apparently
did not try this strategy. At least these commentators do not
report that this strategy failed on being tried.
First, consider the commentator who tried the strategy.
Hurford notes that the target article failed to provide an
account of the concept individual. On examining relevant
situations that contain familiar individuals, Hurford in-
duced an account of this concept: A unique individual ex-
ists whenever we fail to perceive anything exactly like it si-
multaneously in a given situation. Thus, we conclude that
our mother is a unique individual because we never per-
ceive anyone exactly like her simultaneously. For the same
reason, we conclude that the sun and moon are unique in-
dividuals. As Hurford further notes, mistakes about the
multiple identities of the same individual follow naturally
from this account. Because we perceive Clark Kent and Su-
perman as each being unique, we never realize that they are
the same individual (the same is true of the Morning Star
and the Evening Star). Only after viewing the transition of
one into the other do we come to believe that they are the
same individual. Like my account of truth, Hurford’s ac-
count of individual may need further development. Re-
gardless, it illustrates the potential of the strategy in section
3.4.2 for discovering and representing the content of ab-
stract concepts.
Finally, consider the commentators who doubted that
perceptual symbol systems could represent particular ab-
stract concepts, apparently without trying the strategy in
section 3.4.2. Adams & Campbell expressed skepticism
about chiliagons and myriagons; Brewer expressed skepti-
cism about entropy, democracy, evolution, and because;
Mitchell & Clement expressed skepticism about can,
might, electricity, ignorant, and thing; Ohlsson expressed
skepticism about health care system, middle class, recent
election, and internet. Again, if there were no shortage of
space, I would be happy to illustrate how the proposed
strategy in section 3.4.2 might represent these concepts.
Again, the content of an abstract concept must be specified,
regardless of whether it is to be represented perceptually
or amodally, and the proposed strategy is a useful means of
identifying it. Furthermore, I remain convinced that simu-
lations of this content are sufficient to represent it, and that
amodal redescriptions are unnecessary.
R4. Language
Another frequently raised topic was the relation between
language and perceptual symbol systems.
R4.1. Linguistic forms per se do not carry meaning. I con-
tinue to be amazed by how often sophisticated researchers
believe that language per se carries meaning. Most com-
monly, this belief has to do with internal speech. As some-
one listens to imagined speech in introspection, the speech
is presumed to carry meaning as the person “thinks in lan-
guage.” As Fauconnier notes, however, language does not
carry meaning directly. Instead, language is a powerful
means of controlling the online construction of conceptu-
alizations (sect. 2.7). One can hear a foreign language thou-
sands of times, but unless the words become linked to their
referents in experience, they remain meaningless. The
words do not inherently carry meaning.
In arguing that we think in language when processing ab-
stract concepts, Jorion strikes me as being under the illu-
sion that language carries meaning (a similar criticism of
Landauer was presented earlier in sect. R1.4). In a series
of examples, Jorion argues that concepts are represented
in words, not in percepts. The fact that some people are
highly conscious of internal speech, though, does not entail
that words are the causal forces in their thinking. As most
cognitive scientists agree, the bulk of the action in cognition
is unconscious, and a lot more is happening than conscious
inner speech. If language is so central to thought, how can
aphasics think, and how can Lowenthal’s subjects learn?
How can people be thinking of something and not know the
word for it? Where do all the inferences in comprehension
come from that are not expressed directly in linguistic
forms?
R4.2. Language in knowledge acquisition. I hasten to add
that language most certainly plays critical roles in the ac-
quisition and use of knowledge. For example, much re-
search on conceptual development has shown that words
provide important signals for developing new concepts
(e.g., Gelman et al. 1998; Gelman & Tardif 1988; Markman
1989). On hearing a new word, children often infer that it
refers to a new set of entities that share important com-
monalities. Hurford similarly notes how linguistic ele-
ments may signal the presence of individuals in domains of
discourse, perhaps assisting in the acquisition of individual.
Toomela similarly suggests that language is central to
forming abstract concepts. Finally, Langacker’s examples
of construal illustrate how the word associated with a per-
ceived entity determines its conceptualization, with differ-
ent words producing different conceptualizations (sects.
2.4.7 and 3.2.6).
Lowenthal provides an example to the contrary, show-
Response/Barsalou: Perceptual symbol systems
BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4 649

ing that people can acquire new knowledge without lan-
guage. Even when lesions in language areas of the brain dis-
rupt linguistic processing, people still learn new concepts
and skills. As I have argued throughout this reply, such
demonstrations strongly question the grounding of knowl-
edge in language. Again, it by no means follows that lan-
guage is normally irrelevant.
R4.3. Language in human evolution. In section 4.2., I sug-
gested that human intelligence may have diverged from
nonhuman primates with the evolution of language, citing
Donald (1991; 1993). Gabora, however, cites a passage
from Donald in which he states that some sort of concep-
tual foundation, which may have been mimetic culture,
must have preceded speech. In other words, the divergence
must have occurred after the evolution of some more basic
representational ability, not after the evolution of language.
I stand corrected. Gabora has indeed described Donald’s
correct position. Ultimately, though, my point was that hu-
mans evolved the means to control joint simulations about
nonpresent situations. By developing the ability to repre-
sent nonpresent situations jointly in the past and future, hu-
mans increased their fitness substantially. Using the body to
act out nonpresent situations (mimesis) may well have been
the first stage of this evolution, with language building on it
later. Again, my point was that developing the ability to sim-
ulate nonpresent situations constituted a major develop-
ment in human evolution.
R4.4. Cognitive linguistics. As noted by Fauconnier and
Langacker, cognitive linguistics and perceptual symbol
systems share many common assumptions. Most impor-
tantly, they assume that conceptualizations are grounded in
the experiential systems of the brain and body. What I tried
to show in the target article is that both approaches belong
to a much larger tradition that stretches back over two mil-
lennia (sects. 1.1 and 1.3). Embodied approaches to cogni-
tion are hardly novel and only appear so in the context of
the twentieth century. What is new and exciting is the rein-
vention of this approach in the contexts of modern cogni-
tive science and neuroscience. Contrary to Landau and
Ohlsson’s claim that perceptual symbol systems offer noth-
ing new, it and other current formulations depart from ear-
lier formulations through the incorporation of modern find-
ings, insights, and tools.
Quite a few people have suggested to me informally that
cognitive linguistics and perceptual symbol systems are
complementary, lying at different levels of analysis. Al-
though cognitive linguists develop accounts of semantics
and grammar in terms of experiential constructs, these ac-
counts often fail to map clearly into well-established cogni-
tive and neural mechanisms. One goal of perceptual sym-
bol systems was to provide an account of how a conceptual
system could be grounded in these mechanisms. Showing
that concepts can be grounded in perception and move-
ment not only supports the assumptions of cognitive lin-
guists, it provides a layer of theoretical machinery that may
be useful to their accounts. Conversely, cognitive linguists’
analyses of myriad linguistic structures strongly suggest var-
ious conceptual structures that we are well advised to look
for in cognition and neuroscience. As usual, the interplay
between levels is not only stimulating but probably neces-
sary for success in achieving the ultimate goals of these
fields.
R4.5. Perceptual simulation in comprehension. Because
amodal theories of knowledge have dominated accounts of
language comprehension, evidence that perceptual simula-
tion underlies comprehension instead is significant. If the
meanings of texts are grounded in simulations, one of the
strongest arguments for the amodal view is in danger.
Zwaan et al. review a variety of findings which suggest that
simulations do indeed underlie comprehension. Zwaan et al.
also suggest a variety of future projects that could explore
this issue further. In section 4.1.6 of the target article, a va-
riety of additional studies are cited that support embodied
comprehension, and Barsalou (in press) provides an evolu-
tionary account of why comprehension might have become
grounded in perceptual simulation. Clearly, much additional
research is needed to address this hypothesis thoroughly,
but existing evidence suggests that it may well be true.
R5. Computational accounts
As I mentioned throughout the target article, developing
computational accounts of perceptual symbol systems is
important (e.g., sects. 2 and 5). A number of commentators
agreed, and had a variety of things to say about this issue.
R5.1. Functional specifications versus mechanisms.
Several commentators suggested that the target article pro-
vided only functional specifications of perceptual symbol
systems and failed to say anything at all about the underly-
ing mechanisms (Adams & Campbell; Dennett & Viger;
Schwartz et al.). As I acknowledged in the target article, I
did not provide a full-blown computational account of per-
ceptual symbol systems. Nevertheless, I certainly did have
something to say about mechanisms. Throughout the target
article, I discussed the viability of Damasio’s (1989) con-
vergence zones, arguing that they provide an ideal mecha-
nism for implementing perceptual symbols (sects. 2.1, 2.2,
2.3, 2.4.6, 2.4.7, and 4.3). In a convergence zone, one layer
of units represents information in a sensory-motor domain,
and a second layer in an association area captures patterns
of activation in the first layer. By activating a pattern of units
in the associative area, a sensory-motor representation is
reenacted. The mechanics of such systems are basic lore in
connectionism, and in citing convergence zones frequently
throughout the target article, I was pointing to an obvious
and well-known mechanism for implementing perceptual
symbols.
During discussions of symbol formation (sects. 2.2 and
2.3), I suggested a second line of mechanisms that are cen-
tral to perceptual symbol systems. Specifically, I suggested
that the well-known mechanisms of attention and memory
extract and store components of experience for later use as
symbols. Although I did not provide detailed accounts of
how these mechanisms work, I did indeed incorporate
them centrally. Furthermore, it would be possible to draw
on detailed accounts in the literature to develop my pro-
posed account of how perceptual symbols are extracted
from experience.
The discussions of frames similarly proposed mecha-
nisms that underlie the formation of simulators. In section
2.5 and Figure 3, I sketched an account of the associative
mechanisms that underlie frames. In section 2.5.1, I de-
fined the components of frames and described how they
could be implemented in perceptual symbol systems.
Response/Barsalou: Perceptual symbol systems
650 BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4

ing attention. However, mechanisms for representing
space and time are perhaps more basic. In the absence of
spatio-temporal reference systems, it is impossible to com-
pute an attentional trajectory. Guiding attention to achieve
a goal presupposes the ability to guide attention through
space and time. Thus, for still another reason, developing
space-time mechanisms is essential to implementing per-
ceptual symbol systems. Once these mechanisms are in
place, it will be possible to formulate mechanisms that
guide attention selectively over space-time regions to
process them.
Perhaps I am naive about computational modeling, but
the four aspects of perceptual symbol systems whose im-
plementation I just outlined strike me as particularly chal-
lenging. To my knowledge, nothing “off the shelf” is avail-
able for fully implementing these functions, nor are they
likely to be implemented in a few weeks’ work. So please
forgive me for not yet having implemented the theory in the
target article! Perhaps some readers will agree that sketch-
ing the theory’s basic mechanisms and illustrating their po-
tential for achieving a fully functional conceptual system
constitutes a useful contribution.
R6. Conclusion
The way I see it, three basic approaches to knowledge exist
in modern cognitive science and neuroscience: (1) classic
representational approaches based on amodal symbols, (2)
statistical and dynamical approaches such as connectionism
and neural nets, and (3) embodied approaches such as clas-
sic empiricism, cognitive linguistics, and situated action. If
I were to say that there is value in all three of these ap-
proaches, I might be viewed as simply being diplomatic. On
the contrary, I believe that each of these approaches has dis-
covered something fundamentally important about human
knowledge, which the other two have not. Classic amodal
approaches have discovered the importance of structured
representations, productivity, and propositions. Statistical
approaches have discovered the importance of generaliza-
tion, partial matches, adaptation, frequency effects, and
pattern completion. Embodied approaches have discov-
ered the importance of grounding knowledge in sensory-
motor mechanisms and the significance of environments
and action for cognition. What I have tried to do in formu-
lating perceptual symbol systems is to integrate the positive
contributions of all three approaches. Regardless of
whether my particular formulation succeeds, I predict that
whatever approach ultimately does succeed will similarly
attempt to integrate representation, statistical processing,
and embodiment.
ACKNOWLEDGMENT
This work was supported by National Science Foundation Grants
SBR-9421326 and SBR-9796200.
References
Letters “a” and “r” appearing before authors’ initials refer to target article
and response, respectively.
Adams, F. & Aizawa, K. (1994) Fodorian semantics. In: Mental representation, ed.
S. Stich & T. Warfield. Blackwell. [FA]
Ahn, W. & Bailenson, J. (1996) Causal attribution as a search for underlying
mechanisms: An explanation of the conjunction fallacy and the discounting
principle. Cognitive Psychology 31:82–123. [aLWB]
Ahn, W., Kalish, C. W., Medin, D. L. & Gelman, S. A. (1995) The role of
covariation versus mechanism information in causal attribution. Cognition
54:299–352. [aLWB]
Aizawa, K. (1997) Explaining systematicity. Mind and Language 12:115–36. [FA]
Alpern, M. (1972) Eye movements. In: Visual psychophysics, ed. D. Jameson &
L. M. Hurvich. Springer-Verlag. [AT]
Anderson, J. R. (1978) Arguments concerning representations for mental imagery.
Psychological Review 85:249–77. [aLWB]
(1993) Rules of the mind. Erlbaum. [aLWB]
Anderson, J. R. & Bower, G. H. (1973) Human associative memory. Winston.
[aLWB]
Aristotle (1961) De anima, Books II and III, trans. D. W. Hamlyn. Oxford
University Press. (Original work published in 4th century B.C.). [aLWB]
Baddeley, A. D. (1986) Working memory. Clarendon. [aLWB]
Bailey, D., Feldman, J., Narayanan, S. & Lakoff, G. (1997) Embodied lexical
development. Proceedings of the Nineteenth Annual Meeting of the Cognitive
Science Society. Erlbaum. [aLWB]
Baillargeon, R. (1995) A model of physical reasoning in infancy. In: Advances in
infancy research, vol. 9. Ablex. [aLWB]
Barkowsky, T. & Freksa, C. (1997) Cognitive requirements on making and
interpreting maps. In: Spatial information theory: A theoretical basis for GIS,
ed. S. Hirtle & A. Frank. Springer. [CF]
Barsalou, L. W. (1983) Ad hoc categories. Memory and Cognition 11:211–27.
[aLWB]
(1985) Ideals, central tendency, and frequency of instantiation as determinants
of graded structure in categories. Journal of Experimental Psychology:
Learning, Memory, and Cognition 11:629–54. [aLWB]
(1987) The instability of graded structure in concepts. In: Concepts and
conceptual development: Ecological and intellectual factors in categorization,
ed. U. Neisser. Cambridge University Press. [aLWB]
(1989) Intra-concept similarity and its implications for inter-concept similarity.
In: Similarity and analogical reasoning, ed. S. Vosniadou & A. Ortony.
Cambridge University Press. [aLWB]
(1991) Deriving categories to achieve goals. In: The psychology of learning and
motivation: Advances in research and theory, vol. 27. Academic Press.
[aLWB]
(1992) Frames, concepts, and conceptual fields. In: Frames, fields, and
contrasts: New essays in lexical and semantic organization, ed. A. Lehrer &
E. F. Kittay. Erlbaum. [arLWB, ABM]
(1993) Flexibility, structure, and linguistic vagary in concepts: Manifestations of
a compositional system of perceptual symbols. In: Theories of memories, ed.
A. C. Collins, S. E. Gathercole & M. A. Conway. Erlbaum. [arLWB, WFB]
(1995) Storage side effects: Studying processing to understand learning. In:
Goal-driven learning, ed. A. Ram & D. Leake. MIT Press. [aLWB]
(in press) Language comprehension: Archival memory or preparation for
situated action? Discourse Processes. [rLWB]
Barsalou, L. W. & Hale, C. R. (1993) Components of conceptual representation:
From feature lists to recursive frames. In: Categories and concepts:
Theoretical views and inductive data analysis, ed. I. Van Mechelen, J.
Hampton, R. Michalski & P. Theuns. Academic Press. [arLWB]
Barsalou, L. W., Huttenlocher, J. & Lamberts, K. (1998) Basing categorization on
individuals and events. Cognitive Psychology 36:203–72. [aLWB]
Barsalou, L. W. & Prinz, J. J. (1997) Mundane creativity in perceptual symbol
systems. In: Conceptual structures and processes: Emergence, discovery, and
change, ed. T. B. Ward, S. M. Smith & J. Vaid. American Psychological
Association. [aLWB]
Barsalou, L. W., Solomon, K. O. & Wu, L. L. (in press) Perceptual simulation in
conceptual tasks. In: Cultural, typological, and psychological perspectives in
cognitive linguistics: The proceedings of the 4th Conference of the
International Cognitive Linguistics Association, vol. 3, ed. M. K. Hiraga, C.
Sinha & S. Wilcox. John Benjamins. [arLWB]
Barsalou, L. W., Yeh, W., Luka, B. J., Olseth, K. L., Mix, K. S. & Wu, L.-L. (1993)
Concepts and meaning. In: Chicago Linguistics Society 29: Papers from the
parasessions on conceptual representations, vol. 2, ed. K. Beals, G. Cooke, D.
Kathman, K. E. McCullough, S. Kita & D. Testen. Chicago Linguistics
Society. [aLWB]
Barwise, J. & Etchemendy, J. (1990) Information, infons, and inference. In:
Situation theory and its application, ed. R. Cooper, K. Mukai & J. Perry.
University of Chicago Press. [aLWB]
(1991) Visual information and valid reasoning. In: Visualization in mathematics,
ed. W. Zimmerman & S. Cunningham. Mathematical Association of America.
[aLWB]
Barwise, J. & Perry, J. (1983) Situations and attitudes. MIT Press. [aLWB]
Bassok, M. (1997) Object-based reasoning. In: The psychology of learning and
motivation, vol. 37, ed. D. L. Medin. Academic Press. [aLWB]
Bates, E., Thal, D., Trauner, D., Fenson, J., Aram, D., Eisele, J. & Nass, R. (1997)
Response/Barsalou: Perceptual symbol systems
652 BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4

From first words to grammar in children with focal brain injury.
Developmental Neuropsychology 13(3): 275–343. [LM]
Bear, M. F., Connors, B. W. & Paradiso, M. A. (1996) Neuroscience: Exploring the
brain. Williams & Wilkins. [aLWB]
Berendt, B., Barkowsky, T., Freksa, C. & Kelter, S. (1998) Spatial representation
with aspect maps. In: Spatial cognition - an interdisciplinary approach to
representing and processing spatial knowledge, ed. C. Freksa, C. Habel &
K. F. Wender. Springer. [CF]
Berkeley, G. (1710/1982) A treatise concerning the principles of human knowledge.
Hackett. (Original work published in 1710). [aLWB]
Biederman, I. (1987) Recognition-by-components: A theory of human image
understanding. Psychological Review 94:115–47. [aLWB, ABM]
Biederman, I. & Gerhardstein, P. C. (1993) Recognizing depth-rotated objects:
Evidence and conditions for three-dimensional viewpoint invariance. Journal
of Experimental Psychology: Human Perception and Performance 19:1162–
82. [aLWB]
Black, J. B., Turner, T. J. & Bower, G. H. (1979) Point of view in narrative
comprehension, memory, and production. Journal of Verbal Learning and
Verbal Behavior 18:187–98. [aLWB]
Black, M. (1962a) Metaphor. In: Models and metaphors, ed. M. Black. Cornell
University Press. [BI]
(1962b) Models and archetypes. In: Models and metaphors, ed. M. Black.
Cornell University Press. [WFB]
(1979) More about metaphor. In: Metaphor and thought, ed. A. Ortony.
Cambridge University Press. [BI]
Block, N. (1983) Mental pictures and cognitive science. Philosophical Review
93:499–542. [aLWB]
Bloom, P. & Markson, L. (1998) Intention and analogy in children’s naming of
pictorial representations. Psychological Science 9:200–204. [aLWB]
Borsook, D., Becerra, L., Fishman, S., Edwards, A., Jennings, C. L., Stojanovic,
M., Papinicolas, L., Ramachandran, V. S., Gonzalez, R. G. & Breiter, H.
(1998) Acute plasticity in the human somatosensory cortex following
amputation. Neuroreport 9:1013–17. [SE]
Bottini, G., Corcoran, R., Sterzi, R., Paulesu, E., Schenone, P., Scarpa, P.,
Frackowiak, R. S. J. & Frith, C. D. (1994) The role of the right hemisphere in
the interpretation of figurative aspects of language: A positron emission
tomography activation study. Brain 117:1241–53. [BI]
Braitenberg, V. (1978) Cortical architectonics: General and areal. In: Architectonics
of the cerebral cortex, ed. M. A. Brazier & H. Petsche. Raven Press. [DAS]
Brandimonte, M. A., Schooler, J. W. & Gabbino, P. (1997) Attenuating verbal
overshadowing through color retrieval cues. Journal of Experimental
Psychology: Learning, Memory, and Cognition 23:915–31. [aLWB]
Bransford, J. D. & Johnson, M. K. (1973) Considerations of some problems of
comprehension. In: Visual information processing, ed. W. G. Chase. Academic
Press. [aLWB]
Brewer, W. F. (1975) Memory for ideas: Synonym substitution. Memory and
Cognition 3:458–64. [WFB]
(in press) Scientific theories and naive theories as forms of mental
representation: Psychologism revived. Science and Education. [WFB]
Brewer, W. F. & Pani, J. R. (1983) The structure of human memory. In: The
psychology of learning and motivation: Advances in research and theory, vol.
17, ed. G. H. Bower. Academic Press. [rLWB]
(1996) Reports of mental imagery in retrieval from long-term memory.
Consciousness and Cognition 5:265–87. [WFB]
Brewer, W. F. & Treyens, J. C. (1981) Role of schemata in memory for places.
Cognitive Psychology 13:207–30. [WFB]
Brooks, L. R. (1978) Nonanalytic concept formation and memory for instances. In:
Cognition and categorization, ed. E. Rosch & B. B. Lloyd. Erlbaum.
[aLWB]
Bruner, J. S. (1966) On cognitive growth. In: Studies in cognitive growth, ed. J. S.
Bruner, R. R. Olver & P. M. Greenfield. Wiley. [FL]
Buckner, R. L., Petersen, S. E., Ojemann, J. G., Miezin, F. M., Squire, L. R. &
Raichle, M. E. (1995) Functional anatomical studies of explicit and implicit
memory retrieval tasks. Journal of Neuroscience 15:12–29. [aLWB]
Burbaud, P. (1988) Traitement numérique et latéralisation corticale (IRMf). First
meeting of the Société d’anatomie fonctionnelle cérébrale, Paris, 1988
(personal communication). [FL]
Burbaud, P., Degreze, P., Lafon, P., Franconi, J.-M., Bouligand, B., Bioulac, B.,
Caille, J.-M. & Allard, M. (1995) Lateralization of prefrontal activation during
internal mental calculation: A functional magnetic resonance imaging study.
APStracts 2:0263N. Journal of Neurophysiology 74(5):2194–2200. [FL]
Burgess, C. & Chiarello, C. (1996) Neurocognitive mechanisms underlying
metaphor comprehension and other figurative language. Metaphor and
Symbolic Activity 11(1):67–84. [BI]
Burgess, C. & Lund, K. (1997) Modelling parsing constraints with high-
dimensional context space. Language and Cognitive Processes 12:177–210.
[arLWB]
Cacciari, C. & Levorato, M. C. (1994) “See,” “look,” and “glimpse”: What makes
the difference? Paper presented at the Meeting of the Psychonomic Society,
St. Louis, MO. [aLWB]
Cann, R. (1993) Formal semantics. Cambridge University Press. [JRH]
Carey, S. (1985) Conceptual change in childhood. MIT Press. [aLWB]
Carlson, S. M., Moses, L. J. & Hix, H. R. (1998) The role of inhibitory processes in
young children’s difficulties with deception and false belief. Child
Development 69:672–91. [rLWB]
Carnap, R. (1958) Introduction to symbolic logic and its applications. Dover.
[JRH]
Carroll, L. (1960) The annotated Alice: Alice’s adventures in wonderland and
Through the looking glass. Bramhall House. [aLWB]
Chalmers, D. J., French, R. M. & Hofstader, D. R. (1992) High-level perception,
representation, and analogy: A critique of artificial intelligence methodology.
Journal of Experimental and Theoretical Artificial Intelligence 4:189–211.
[BI]
Chambers, D. & Reisberg, D. (1992) What an image depicts depends on what an
image means. Cognitive Psychology 24:145–74. [aLWB]
Charland, L. C. (1995) Emotion as a natural kind: Towards a computational
foundation for emotion theory. Philosophical Psychology 8(1):59–84. [LCC]
(1996) Feeling and representing: Computational theory and the modularity of
affect. Synthese 105:273–301. [LCC]
(1997) Reconciling cognitive and perceptual theories of emotion. Philosophy of
Science 64:555–79. [LCC]
Cheng, P. W. & Novick, L. R. (1992) Covariation in natural causal induction.
Psychological Review 99:365–82. [aLWB]
Chomsky, N. (1957) Syntactic structures. Mouton. [arLWB]
(1975) Reflections on language. Pantheon Books. [FL]
Churchland, P. S. (1986) Neurophilosophy: Toward a unified science of the mind/
brain. MIT Press. [aLWB]
Clark, A. (1997) Being there: Putting brain, body, and world together again. MIT
Press. [arLWB]
Cohen, L. B. (1991) Infant attention: An information processing approach. In:
Newborn attention: Biological constraints and the influence of experience, ed.
M. J. Salomon Weiss & P. R. Zelazo. Ablex. [aLWB]
Cohen, P. R., Atkin, M. S., Oates, T. & Beal, C. R. (1997) Neo: Learning
conceptual knowledge by interacting with an environment. Proceedings of the
First International Conference on Autonomous Agents. ACM Press. [aLWB]
Compton, B. J. (1995) Retrieval strategies in memory-based automaticity. Doctoral
dissertation, University of Illinois, Champaign-Urbana. [aLWB]
Conway, M. A. (1990) Autobiographical memory: An introduction. Open
University Press. [aLWB]
Cornsweet, T. N. (1970) Visual perception. Academic Press. [AT]
Coulson, S. (1997) Semantic leaps: Frame-shifting and conceptual blending.
Doctoral dissertation, University of California, San Diego. [GF]
Cowey, A. & Stoerig, P. (1991) The neurobiology of blindsight. Trends in
Neuroscience 14:140–45. [aLWB]
Craik, F. I. M. & Lockhart, R. S. (1972) Levels of processing: A framework for
memory research. Journal of Verbal Learning and Verbal Behavior 11:671–
84. [aLWB]
Craik, K. (1943) The nature of explanation. Cambridge University Press. [aLWB]
Crammond, D. J. (1997) Motor imagery: Never in your wildest dreams. Trends in
Neuroscience 20:54–57. [aLWB]
Craver-Lemley, C. & Reeves, A. (1997) Effects of imagery on vernier acuity under
conditions of induced depth. Journal of Experimental Psychology: Human
Perception and Performance 23:3–13. [aLWB]
Cummins, R. (1996) Representations, targets, and attitudes. MIT Press. [aLWB]
Damasio, A. R. (1989) Time-locked multiregional retroactivation: A systems-level
proposal for the neural substrates of recall and recognition. Cognition 33:25–
62. [arLWB]
(1994) Descartes’ error. Grosset/Putnam. [arLWB, LCC]
Damasio, A. R. & Damasio, H. (1994) Cortical systems for retrieval of concrete
knowledge: The convergence zone framework. In: Large-scale neuronal
theories of the brain, ed. C. Koch & J. L. Davis. MIT Press. [aLWB]
D’Andrade, R. (1987) A folk model of the mind. In: Cultural models in language
and thought, ed. D. Holland & N. Quinn. Cambridge University Press.
[aLWB]
Danesi, M. (1998) The “dimensionality principle” and semiotic analysis. Sign
Systems Studies 26:42–60. [AT]
Davies, M. & Stone, T., eds. (1995) Mental simulation. Basil Blackwell. [aLWB]
Davis, M. (1998) Are different parts of the extended amygdala involved in fear
versus anxiety? Biological Psychiatry 44:1239–47. [rLWB]
DeJong, G. & Mooney, R. (1986) Explanation-based learning: An alternative view.
Machine Learning 1:145–76. [aLWB]
Dennett, D. C. (1969) The nature of images and the introspective trap. In:
Dennett, D. C. Content and consciousness. Routledge. [aLWB]
Dennett, D. C. & Kinsbourne, M. (1992) Time and the observer - the where and
when of consciousness in the brain. Behavioral and Brain Sciences 15:183–
247. [aLWB]
References/Barsalou: Perceptual symbol systems
BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4 653

Dent-Read, C. & Szokolszky, A. (1993) Where do metaphors come from?
Metaphor and Symbolic Activity 8(3):227–42. [BI]
DeRenzi, E. & Spinnler, H. (1967) Impaired performance on color tasks in
patients with hemispheric lesions. Cortex 3:194–217. [aLWB]
Deschaumes-Molinaro, C., Dittmer, A. & Vernet-Maury, E. (1992) Autonomic
nervous system response patterns correlate with mental imagery. Physiology
and Behavior 51:1021–27. [aLWB]
De Sousa, R. (1987) The rationality of emotion. Bradford Books, MIT Press. [LCC]
Diamond, A., Werker, J. F. & Lalonde, M. A. (1994) Toward understanding
commonalities in the development of object search, detour navigation,
categorization, and speech perception. In: Human behavior and the
developing brain, ed. G. Dawson & K. W. Fischer. Guilford. [LM]
Dietrich, E. & Markman, A. B., eds. (in press) Cognitive dynamics: Conceptual
change in humans and machines. Erlbaum. [rLWB]
Donald, M. (1991) Origins of the modern mind: Three stages in the evolution of
culture and cognition. Harvard University Press. [arLWB, LG, AJW]
(1993) Précis of Origins of the modern mind: Three stages in the evolution of
culture and cognition. Behavioral and Brain Sciences 16:739–91. [arLWB]
Dretske, F. (1981) Knowledge and the flow of information. MIT Press/Bradford
Books. [FA, MA]
(1988) Explaining behavior. MIT Press/Bradford Books. [FA]
(1995) Naturalizing the mind. MIT Press. [arLWB]
Droz, R. & Rahmy, M. (1974) Lire Piaget. Charles Dessart. [FL]
Edelman, G. M. (1987) Neural Darwinism: The theory of neuronal group selection.
Basic Books. [DAS]
(1992) Bright air, brilliant fire: On the matter of the mind. Basic Books.
[aLWB]
Edelman, S. (1994) Biological constraints and the representation of structure in
vision and language. Psycoloquy 5(57). ftp://ftp.princeton.edu/pub/harnad/
Psycoloquy/ 1994.volume.5/psyc.94.5.57.language-network.3.edelman. [SE]
(1998) Representation is representation of similarities. Behavioral and Brain
Sciences 21(4):449–98. [aLWB]
Ekman, P. (1992) An argument for basic emotions. Cognition and Emotion 6:169–
200. [LCC]
Elman, J. L., Bates, E. A., Johnson, M. H., Karmiloff-Smith, A., Parisi, D. &
Plunkett, K. (1996) Rethinking innateness: A connectionist perspective on
development. MIT Press. [arLWB]
Epicurus (1994) The Epicurus reader: Selected writings and testimonia, trans. B.
Inwood & L. P. Gerson. (Original work published in the 4th century B. C.)
Hackett. [aLWB]
Farah, M. J. (1988) Is visual imagery really visual? Overlooked evidence from
neuropsychology. Psychological Review 95:307–17. [aLWB]
(1995) The neural bases of mental imagery. In: The cognitive neurosciences, ed.
M. S. Gazzaniga. MIT Press. [aLWB]
Farah, M. J. & Feinberg, T. E. (1997) Perception and awareness. In: Behavioral
neurology and neuropsychology, ed. T. E. Feinberg & M. J. Farah. McGraw-
Hill. [aLWB]
Fauconnier, G. (1985) Mental spaces: Aspects of meaning construction in natural
language. MIT Press. [aLWB, RWL]
(1997) Mappings in thought and language. Cambridge University Press.
[aLWB, GF, RWL]
Fauconnier, G. & Sweetser, E., eds. (1996) Spaces, worlds, and grammar.
University of Chicago Press. [GF, RWL]
Fauconnier, G. & Turner, M. (1998) Conceptual integration networks. Cognitive
Science 22(2):133–87. [arLWB, GF, RWL]
Fillmore, C. J. (1985) Frames and the semantics of understanding. Quaderni di
Semantica 6:222–55. [aLWB]
Fincher-Kiefer, R. (1998) Visuospatial working memory in mental model
construction. Abstracts of the Psychonomic Society 3:79. [RAZ]
Finke, R. A. (1989) Principles of mental imagery. MIT Press. [aLWB]
Flavell, J. (1976) Metacognitive aspects of problem solving. In: The nature of
intelligence, ed. L. Resnick. Erlbaum. [RO]
Fodor, J. A. (1975) The language of thought. Harvard University Press. [aLWB,
DCD]
(1981a) Representations: Philosophical essays on the foundations of cognitive
science. MIT Press. [ABM]
(1981b) The present status of the innateness controversy. In: Representations,
J. A. Fodor. MIT Press. [BL]
(1983) The modularity of mind: An essay on faculty psychology. Bradford Books,
MIT Press. [aLWB]
(1987) Psychosemantics: The problem of meaning in the philosophy of mind.
MIT Press. [MA]
(1990) A theory of content (I and II). In: A theory of content and other essays,
J. A. Fodor. MIT Press. [MA]
Fodor, J. A. & LePore, E. (1992) Holism: A shopper’s guide. Blackwell. [aLWB]
Fodor, J. A. & McLaughlin, B. (1990) Connectionism and the problem of
systematicity: Why Smolensky’s solution doesn’t work. Cognition 35(2):183–
204. [ABM]
Fodor, J. A. & Pylyshyn, Z. W. (1988) Connectionism and cognitive architecture: A
critical analysis. Cognition 28:3–71. [FA, arLWB, ABM]
Fox, N. A. & Bell, M. A. (1993) Frontal functioning in cognitive and emotional
behaviors during infancy: Effects of maturation and experience. In: NATO
ASI Series: D. Developmental neurocognition: Speech and face processing in
the first year of life, ed. S. de Schonen, P. Jusczyk, P. McNeilage & J. Morton.
Kluwer. [LM]
Frege, G. (1892/1952) On sense and reference. In: Philosophical writings of
Gottlieb Frege, ed. P. T. Geach & M. Black. (Original work published in 1892).
Blackwell. [aLWB]
Freksa, C. (1991) Qualitative spatial reasoning. In: Cognitive and linguistic aspects
of geographic space, ed. D. M. Mark & A. U. Frank. Kluwer. [CF]
(1997) Spatial and temporal structures in cognitive processes. In: Foundations of
computer science: Potential - theory - cognition, ed. C. Freksa, M. Jantzen &
R. Valk. Springer. [CF]
Freksa, C. & Barkowsky, T. (1996) On the relation between spatial concepts and
geographic objects. In: Geographic objects with indeterminate boundaries, ed.
P. Burrough & A. Frank. Taylor & Francis. [CF]
Freyd, J. J. (1987) Dynamic mental representations. Psychological Review 94:427–
38. [aLWB]
Frith, C. & Dolan, R. J. (1997) Brain mechanisms associated with top-down
processes in perception. Philosophical Transactions of the Royal Society of
London, Series B: Biological Sciences 352:1221–30. [aLWB]
Gabora, L. (1998) Autocatalytic closure in a cognitive system: A tentative scenario
for the evolution of culture. Psycoloquy 9(67). www.cogsci.soton.ac.uk.psyc/
newpsy?9.67 ftp://ftp.princeton.edu/pub/harnard/ Psycoloquy/
1998.volume.9/page.98.9.XX.-.l.gabora. [LG]
(1999) Weaving, bending, patching, mending the fabric of reality: A cognitive
science perspective on worldview inconsistency. Foundations of Science 3(2).
www.vub.ac.be/clea/liane/fabric of reality/html. [LG]
Gainotti, G., Silveri, M. C., Daniele, A. & Giustolisi, L. (1995) Neuroanatomical
correlates of category-specific semantic disorders: A critical survey. Memory
3:247–64. [aLWB]
Garner, W. R. (1974) The processing of information and structure. Wiley. [aLWB]
(1978) Aspects of a stimulus: Features, dimensions, and configurations. In:
Cognition and categorization, ed. E. Rosch & B. B. Lloyd. Erlbaum.
[aLWB]
Gazzaniga, M. S. (1988) Brain modularity: Towards a philosophy of conscious
experience. In: Consciousness in contemporary science, ed. A. J. Marcel & E.
Bisiach. Clarendon Press. [aLWB]
Gazzaniga, M. S., Ivry, R. B. & Mangun, G. R. (1998) Cognitive neuroscience: The
biology of the mind. Norton. [aLWB]
Geach, P. T. (1957) Mental acts. Routledge. [aLWB]
Gelman, S. A., Coley, J. D., Rosengran, K. S., Hartman, E. & Pappas, A. (1998)
Beyond labeling: The role of maternal input in the acquisition of richly
structured categories. Monographs of the Society for Research in Child
Development 63, Serial No. 253. [rLWB]
Gelman, S. A. & Diesendruck, G. (in press) What’s in a concept? Context,
variability, and psychological essentialism. In: Theoretical perspectives in the
concept of representation, ed. I. E. Siegel. Erlbaum. [rLWB]
Gelman, S. A. & Tardiff, T. (1998) A cross-linguistic comparison of generic noun
phrases in English and Mandarin. Cognition 66:215–48. [rLWB]
Gentner, D. (1983) Structure-mapping: A theoretical framework for analogy.
Cognitive Science 7:155–70. [LG, ABM]
Gentner, D. & Markman, A. B. (1997) Structural alignment in analogy and
similarity. American Psychologist 52:45–56. [rLWB, LG, ABM]
Gentner, D. & Stevens, A. L., eds. (1983) Mental models. Erlbaum. [aLWB]
Gernsbacher, M. A., Varner, K. R. & Faust, M. E. (1990) Investigating differences
in general comprehension skill. Journal of Experimental Psychology:
Learning, Memory, and Cognition 16:430–45. [aLWB]
Gibbs, R. W., Jr. (1983) Do people always process the literal meanings of indirect
requests? Journal of Experimental Psychology: Learning, Memory, and
Cognition 9:524–33. [aLWB]
(1992) What do idioms really mean? Journal of Memory and Language 31:485–
506. [RWG]
(1994) The poetics of mind: Figurative thought, language, and understanding.
Cambridge University Press. [aLWB, GF, RWG]
Gibson, J. J. (1979) The ecological approach to visual perception. Houghton
Mifflin. [aLWB]
Gilbert, D. T. (1991) How mental systems believe. American Psychologist 46:107–
19. [rLWB]
Gilden, D., Blake, R. & Hurst, G. (1995) Neural adaptation of imaginary visual
motion. Cognitive Psychology 28:1–16. [aLWB]
Gineste, M.-D., Indurkhya, B. & Scart-Lhomme, V. (1997) Mental representations
in understanding metaphors. Notes et Documents Limsi No. 97–02, LIMSI-
CNRS, BP 133, F-91403, Orsay, Cedex, France. [BI]
Givón, T. (1978) Negation in language: Pragmatics, function, ontology. In: Syntax
and semantics 9: Pragmatics, ed. P. Cole. Academic Press. [aLWB]
References/Barsalou: Perceptual symbol systems
654 BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4

Jonides, J. & Smith, E. E. (1997) The architecture of working memory. In:
Cognitive neuroscience, ed. M. D. Rugg. MIT Press. [aLWB]
Jorion, P. (1990) Principes des systèmes intelligents. Collection “Sciences
cognitives.” Masson. (Summary in English: http://cogprints.soton.ac.uk/abs/
comp/199806039). [PJMJ]
(1999) Le secret de la chambre chinoise. L’Homme 150:1–26. [PJMJ]
Just, M. A. & Carpenter, P. A. (1987) The psychology of reading and language
comprehension. Allyn & Bacon. [aLWB]
Kahnemann, D. & Tversky, A. (1982) The simulation heuristic. In: Judgement
under uncertainty: Heuristics and biases, ed. D. Kahnemann, P. Slovic & A.
Tversky. Cambridge University Press. [aLWB]
Kanizsa, G. (1979) Organization in vision: Essays in Gestalt perception. Praeger
Press. [aLWB]
Kant, E. (1787/1965) The critique of pure reason, trans. N. K. Smith. St. Martin’s
Press. [aLWB]
Kaplan, S., Sonntag, M. & Chown, E. (1991) Tracing recurrent activity in cognitive
elements (TRACE): A model of temporal dynamics in a cell assembly.
Connection Science 3:179–206. [DAS]
Kaplan, S., Weaver, M. & French, R. (1990) Active symbols and internal models:
Towards a cognitive connectionism. A I and Society 4:51–71. [DAS]
Kauffman, S. A. (1993) Origins of order. Oxford University Press. [LG]
Keil, F. C. (1989) Concepts, kinds, and cognitive development. MIT Press.
[aLWB]
Kellman, P. (1993) Kinematic foundations of infant visual perception. In: Visual
perception and cognition in infancy, ed. C. Granrud. Erlbaum. [LM]
Kintsch, W. (1974) The representation of meaning in memory. Erlbaum. [aLWB]
Klatzky, R. L., Pellegrino, J. W., McCloskey, B. P. & Doherty, S. (1989) Can you
squeeze a tomato? The role of motor representations in semantic sensibility
judgments. Journal of Memory and Language 28:56–77. [RAZ]
Koestler, A. (1964) The act of creation. (2nd Danube edition, 1976). Hutchinsons
of London. [BI]
Kolers, P. A. & Roediger, H. L., III. (1984) Procedures of mind. Journal of Verbal
Learning and Verbal Behavior 23:425–49. [aLWB]
Kosslyn, S. M. (1976) Can imagery be distinguished from other forms of internal
representation? Evidence from studies of information retrieval time. Memory
and Cognition 4:291–97. [aLWB]
(1980) Image and mind. Harvard University Press. [aLWB, DCD]
(1987) Seeing and imaging in the cerebral hemispheres: A computational
approach. Psychological Review 94:148–75. [aLWB]
(1994) Image and brain. MIT Press. [aLWB]
Kosslyn, S. M., Cave, C. B., Provost, D. A. & von Gierke, S. M. (1988) Sequential
processes in image generation. Cognitive Psychology 20:319–43. [aLWB]
Kosslyn, S. M., Thompson, W. L., Kim, I. J. & Alpert, N. M. (1995) Topographical
representations of mental images in primary visual cortex. Nature 378:496–
98. [aLWB]
Kowler, E., Anderson, E., Dosher, B. & Blasser, E. (1995) The role of attention in
the programming of saccades. Vision Research 35:1897–916. [JH]
Lakoff, G. (1987) Women, fire, and dangerous things: What categories reveal about
the mind. University of Chicago Press. [aLWB, GF, RWL]
(1988) Cognitive semantics. In: Meaning and mental representations, ed. U.
Eco, M. Santambrogio & P. Violi. Indiana University Press. [aLWB]
(1990) The invariance hypothesis: Is abstract reason based on image-schemas?
Cognitive Linguistics 1:39–74. [RWL]
Lakoff, G. & Johnson, M. (1980) Metaphors we live by. University of Chicago
Press. [aLWB, RWL]
(1999) Philosophy in the flesh. Basic Books. [GF]
Lakoff, G. & Turner, M. (1989) More than cool reason: A field guide to poetic
metaphor. University of Chicago Press. [aLWB, RWL]
Landauer, T. K. & Dumais, S. T. (1997) A solution to Plato’s problem: The Latent
Semantic Analysis theory of acquisition, induction, and representation of
knowledge. Psychological Review 104:211–40. [aLWB, AMG, TKL]
Landauer, T. K., Foltz, P. W. & Laham, D. (1998a) An introduction to Latent
Semantic Analysis. Discourse Processes 25:259–84. [TKL]
Landauer, T. K., Laham, D. & Foltz, P. W. (1998b) Learning human-like
knowledge by singular value decomposition: A progress report. Advances in
Neural Information Processing Systems 10:45–51. [TKL]
Lang, P. J. (1979) A bio-informational theory of emotional imagery.
Psychophysiology 16:495–512. [RAZ]
Langacker, R. W. (1986) An introduction to cognitive grammar. Cognitive Science
10:1–40. [aLWB]
(1987) Foundations of cognitive grammar, vol. 1. Theoretical prerequisites.
Stanford University Press. [aLWB, GF, RWL]
(1990) Concept, image, and symbol: The cognitive basis of grammar. Mouton de
Gruyter. [RWL]
(1991) Foundations of cognitive grammar, vol. 2. Descriptive application.
Stanford University Press. [aLWB, GF, RWL]
(1993) Universals of construal. Proceedings of the Annual Meeting of the Annual
Meeting of the Berkeley Linguistics Society 19:447–63. [RWL]
(1995) Viewing in cognition and grammar. In: Alternative linguistics: Descriptive
and theoretical modes, ed. P. W. Davis. John Benjamins. [RWL]
(1997) Constituency, dependency, and conceptual grouping. Cognitive
Linguistics 8:1–32. [aLWB]
(1998) Conceptualization, symbolization, and grammar. In: The new psychology
of language: Cognitive and functional approaches to language structure, ed.
M. Tomasello. Erlbaum. [RWL]
Larbig, W., Montoya, P., Flor, H., Bilow, H., Weller, S. & Birbaumer, N. (1996)
Evidence for a change in neural processing in phantom limb pain patients.
Pain 67:275–83. [SE]
Lassaline, M. E. & Logan, G. D. (1993) Memory-based automaticity in the
discrimination of visual numerosity. Journal of Experimental Psychology:
Learning, Memory, and Cognition 19:561–81. [aLWB]
Lazarus, R. (1991) Emotion and adaptation. Oxford University Press. [rLWB,
LCC]
LeDoux, J. E. (1996) The emotional brain: The mysterious underpinnings of
emotional life. Simon & Schuster. [aLWB, LCC]
Lehrer, A. (1974) Semantic fields and lexical structure. American Elsevier.
[aLWB]
Lehrer, A. & Kittay, E. F., eds. (1992) Frames, fields, and contrasts: New essays in
lexical and semantic organization. Erlbaum. [aLWB]
Lehrer, K. (1989) Thomas Reid. Routledge. [aLWB]
Leslie, A. M. (1987) Pretence and representation: The origins of “theory of mind.”
Psychological Review 94:412–26. [RO]
(1994) ToMM, ToBy, and Agency: Core architecture and domain specificity. In:
Mapping the mind: Domain specificity in cognition and culture, ed. L. A.
Hirschfeld & S. A. Gelman. Cambridge University Press. [RO]
Leslie, A. M. & German, T. P. (1995) Knowledge and ability in “theory of mind”:
One- eyed overview of a debate. In: Mental simulation. Evaluations and
applications, ed. M. Davies & T. Stone. Blackwell. [RO]
Levin, B. (1995) English verb classes and alternations. University of Chicago Press.
[aLWB]
Levine, D. N., Warach, J. & Farah, M. J. (1985) Two visual systems in mental
imagery: Dissociation of “What: and “Where” in imagery disorders due to
bilateral posterior cerebral lesions. Neurology 35:1010–18. [aLWB]
Libet, B. (1982) Brain stimulation in the study of neuronal functions for conscious
sensory experiences. Human Neurobiology 1:235–42. [aLWB]
(1985) Unconscious cerebral initiative and the role of conscious will in voluntary
action. Behavioral and Brain Sciences 8:529–66. [aLWB]
Locke, J. (1690/1959) An essay concerning human understanding, vols. I and II.
1st edition. Dover. (Original work published in 1690). [aLWB]
Logan, G. (1988) Toward an instance theory of automatization. Psychological
Review 95:492–527. [aLWB]
(1995) Linguistic and conceptual control of visual spatial attention. Cognitive
Psychology 28:103–74. [aLWB]
Logan, G. D. & Etherton, J. L. (1994) What is learned during automatization? The
role of attention in constructing an instance. Journal of Experimental
Psychology: Learning, Memory, and Cognition 20:1022–50. [aLWB]
Logan, G. D., Taylor, S. W. & Etherton, J. L. (1996) Attention in the acquisition
and expression of automaticity. Journal of Experimental Psychology: Learning,
Memory, and Cognition 22:620–38. [aLWB]
Lotman, Y. M. (1983) Asimmetrija i dialog. [Asymmetry and a dialogue]. Trudy
po Znakovym Sistemam 16:15–30. [AT]
(1984) O semiosfere. [On a semiosphere]. Trudy po Znakovym Sistemam
17:5–23. [AT]
(1989) O roli sluchainykh faktorov v literaturnoi evoljutsii. [On the role of
chance factors in the evolution of literature]. Trudy po Znakovym Sistemam
23:39–48. [AT]
Lowenthal, F. (1978) Logic of natural language and games at primary school.
Revue de Phonètique Appliquée 46/47:133–40. [FL]
(1986) Non-verbal communication devices: Their relevance, their use and the
mental processes involved. In: Pragmatics and education, ed. F. Lowenthal &
F. Vandamme. Plenum Press. [FL]
(1992) Formalism for clinical observations. In: Advances in nonverbal
communication, ed. F. Poyatos. John Benjamins. [FL]
Lowenthal, F. & Saerens, J. (1982) Utilisation de formalismes logiques pour
l’examen d’enfants aphasiques. Acta Neurologica Belgica 16:275–80. [FL]
(1986) Evolution of an aphasic child after the introduction of NVCDs. In:
Pragmatics and education, ed. F. Lowenthal & F. Vandamme. Plenum Press.
[FL]
Luria, A. R. (1969) Vyshije korkovyje funktsii celoveka i ih narushenija pri
lokal’nyh porazenijah mozga. [Higher cortical functions in man and their
disturbances in local brain lesions]. Moscow University Press. [AT]
(1973) Osnovy neiropsikhologii. [Foundations of neuropsychology]. Moscow
University Press. [AT]
(1979) Jazyk i soznanije. [Language and consciousness]. Moscow University
Press. [AT]
Lyons, W. (1980) Emotion. Cambridge University Press. [LCC]
References/Barsalou: Perceptual symbol systems
656 BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4

MacDonald, M. C. & Just, M. A. (1989) Changes in activation level with negation.
Journal of Experimental Psychology: Learning, Memory, and Cognition
15:633–42. [RAZ]
MacWhinney, B. (1998) The emergence of language. Erlbaum. [arLWB]
Magnuson, J. S. & Nusbaum, H. C. (1993) Talker differences and perceptual
normalization. Journal of the Acoustical Society of America 93:2371. [aLWB]
Malt, B. C. (1994) Water is not H20. Cognitive Psychology 27:41–70. [rLWB]
Malt, B. C., Sloman, S. A., Gennari, S., Shi, M. & Wang, Y. (1999) Knowing versus
naming: Similarity and the linguistic categorization of artifacts. Journal of
Memory and Language 40:230–62. [rLWB]
Mandelblit, N. (1997) Creativity and schematicity in grammar and translation:
The cognitive mechanisms of blending. Doctoral dissertation, Cognitive
Science, University of California, San Diego. [GF]
Mandler, G.. (1975) Mind and body: Psychology of emotion and stress. W. W.
Norton. [aLWB, LCC]
Mandler, J. M. (1992) How to build a baby: II. Conceptual primitives.
Psychological Review 99:587–604. [aLWB]
Marcel, A. J. (1983a) Conscious and unconscious perception: Experiments on visual
masking and word recognition. Cognitive Psychology 15:197–237. [aLWB]
(1983b) Conscious and unconscious perception: An approach to the relations
between phenomenal experience and perceptual processes. Cognitive
Psychology 15:238–300. [aLWB]
Marcus, G. F. (1998) Can connectionism save constructivism? Cognition 66:153–
82. [ABM]
Markman, A. B. (1999) Knowledge representation. Erlbaum. [ABM]
Markman, A. B. & Gentner, D. (1993) Splitting the differences: A structural
alignment view. Journal of Memory and Language 32:517–35. [KW-H]
Markman, E. M. (1989) Categorization and naming in children. MIT Press.
[arLWB]
Markman, K. D., Gavinski, I., Sherman, S. J. & McMullen, M. N. (1993) The
mental simulation of better and worse possible worlds. Journal of
Experimental Social Psychology 29:87–109. [aLWB]
Marr, D. (1982) Vision. W. H. Freeman. [ABM]
Marschark, M., Katz, A. & Paivio, A. (1983) Dimensions of metaphors. Journal of
Psycholinguistic Research 12:17–40. [BI]
Marslen-Wilson, W. D. & Tyler, L. K. (1980) The temporal structure of spoken
language understanding. Cognition 8:1–71. [aLWB]
Martin, A., Haxby, J. V., Lalonde, F. M., Wiggs, C. L. & Ungerleider, L. G. (1995)
Discrete cortical regions associated with knowledge of color and knowledge of
action. Science 270:102–105. [aLWB]
Martin, A., Wiggs, C. L., Ungerleider, L. G. & Haxby, J. V. (1996) Neural correlates
of category-specific knowledge. Nature 379:649–52. [aLWB]
Martin, M. & Jones, G. V. (1998) Generalizing everyday memory: Signs and
handedness. Memory and Cognition 26:193–200. [aLWB]
Mauro, A. (1990) Observation de deux enfants polyhandicapés à l’aide de
manipulations concrètes de systèmes formels. Post-graduat en sciences
psychopédagogiques, Université de Mons-Hainaut. [FL]
McClelland, J. L., Rumelhart, D. E. & the PDP Research Group (1986) Parallel
distributed processing: Explorations in the microstructure of cognition: Vol. 2.
Psychological and biological models. MIT Press. [aLWB]
McCloskey, M. (1983) Naive theories of motion. In: Mental models, ed. D.
Gentner & A. L. Stevens. Erlbaum. [aLWB]
McCune, L. (1995) A normative study of representational play at the transition to
language. Developmental Psychology 31:198–206. [LM]
McCune-Nicolich, L. (1981) The cognitive basis of relational words. Journal of
Child Language 8:15–36. [LM]
McDermott, D. (1987) A critique of pure reason. Computational Intelligence
3:151–60. [aLWB]
McLaughlin, B. (1997) Classical constituents in Smolensky’s ICS architecture. In:
Structures and norms in science, ed. M. L. Dalla Chiara, K. Doets, D.
Mundici & J. van Benthem. Kluwer Academic Publishers. [ABM]
McNamara, T. P. & Sternberg, R. J. (1983) Mental models of word meaning.
Journal of Verbal Learning and Verbal Behavior 22:449–74. [KW-H]
Medin, D. L. & Ross, B. H. (1989) The specific character of abstract thought:
Categorization, problem-solving, and induction. In: Advances in the
psychology of human intelligence, vol. 5, ed. R. J. Sternberg. Erlbaum.
[aLWB]
Medin, D. L. & Schaffer, M. (1978) A context theory of classification learning.
Psychological Review 85:207–38. [aLWB]
Melara, R. D. & Marks, L. E. (1990) Dimensional interactions in language
processing: Investigating directions and levels of crosstalk. Journal of
Experimental Psychology: Learning, Memory, and Cognition 16:539–54.
[aLWB]
Melzack, R., Israel, R., Lacroix, R. & Schultz, G. (1997) Phantom limbs in people
with congenital limb deficiency or amputation in early childhood. Brain
120:1603–20. [SE]
Miikkulainen, R. (1993) Subsymbolic natural language processing: An integrated
model of scripts, lexicon, and memory. MIT Press. [SE]
Mill, J. S. (1965) On the permanent possibility of sensation. An examination of Sir
William Hamilton’s philosophy. In: A source book in the history of psychology,
ed. R. J. Herrnstein & E. G. Boring. Harvard University Press. [JH]
Miller, G. A. & Johnson-Laird, P. N. (1976) Language and perception. Harvard
University Press. [aLWB]
Millikan, R. G. (1998) A common structure for concepts of individuals, stuffs, and
real kinds: More Mama, more milk, and more mouse. Behavioral and Brain
Sciences 21:55–100. [arLWB]
Mills, D. L., Coffey-Corina, S. & Neville, H. J. (1994) Variability in cerebral
organization during primary language acquisition. In: Human behavior and
the developing brain, ed. G. Dawson & K. W. Fischer. Guilford Press. [LM]
Milner, A. D. & Goodale, M. A. (1995) The visual brain in action. Oxford
University Press. [aLWB]
Miner, A. & Reder, L. M. (1994) A new look at feeling-of-knowing: Its
metacognitive role in regulating question answering. In: Metacognition:
Knowing about knowing, ed. J. Metcalfe & A. P. Shimamura. Bradford Books/
MIT Press. [RO]
Minsky, M. L. (1977) A framework for representing knowledge. In: The psychology
of computer vision, ed. P. H. Winston. McGraw-Hill. [aLWB]
(1985) The society of mind. Simon & Schuster. [aLWB]
Mitchell, R. W. (1994) The evolution of primate cognition: Simulation, self-
knowledge, and knowledge of other minds. In: Hominid culture in primate
perspective, ed. D. Quiatt & J. Itani. University of Colorado Press. [RWM]
Mitchell, T. M., Keller, R. M. & Kedar-Cabelli, S. T. (1986) Explanation-based
generalization: A unifying view. Machine Learning 1:47–80. [aLWB]
Mitchell, W. T. (1990) The logic of architecture: Design, computation, and
cognition. MIT Press. [aLWB]
Montague, R. (1973) The proper treatment of quantification in ordinary language.
In: Approaches to natural language, ed. J. Hintikka. Reidel. [JRH]
Morris, C. D., Bransford, J. D. & Franks, J. J. (1977) Levels of processing versus
test-appropriate strategies. Journal of Verbal Learning and Verbal Behavior
16:519–33. [aLWB]
Morrow, D. G. & Clark, H. H. (1988) Interpreting words in spatial descriptions.
Language and Cognitive Processes 3:275–91. [RAZ]
Morrow, D. G., Greenspan, S. L. & Bower, G. H. (1987) Accessibility and situation
models in narrative comprehension. Journal of Memory and Language
26:165–87. [aLWB]
Muhlnickel, W., Elbert, T., Taub, E. & Flor, H. (1998) Reorganization of auditory
cortex in tinnitus. Proceedings of the National Academy of Science 95:10340–
43. [SE]
Murphy, G. L. (1996) On metaphoric representation. Cognition 60:173–204.
[aLWB]
Murphy, G. L. & Medin, D. L. (1985) The role of theories in conceptual
coherence. Psychological Review 92:289–316. [aLWB]
Narayanan, S. (1997) KARMA: Knowledge-based active representations for
metaphor and aspect. Ph. D. dissertation, Computer Science Division,
University of California, Berkeley. [GF]
Neisser, U. (1967) Cognitive psychology. Prentice-Hall. [arLWB]
(1976) Cognition and reality. Freeman. [aLWB]
Neisser, U. & Becklen, R. (1975) Selective looking: Attending to visually specified
events. Cognitive Psychology 7:480–94. [JH]
Nelson, D. L., Walling, J. R. & McEvoy, C. L. (1979) Doubts about depth. Journal
of Experimental Psychology: Human Learning and Memory 5:24–44.
[aLWB]
Nelson, K. (1996) Language in cognitive development: Emergence of the mediated
mind. Cambridge University Press. [arLWB]
Nelson, T. O. & Narens, L. (1990) Metamemory: A theoretical framework and new
findings. In: The psychology of learning and motivation, vol. 26, ed. G. Bower.
Academic Press. [RO]
Nersessian, N. J. (1992) In the theoretician’s laboratory: Thought experimenting as
mental modeling. Proceedings of the Philosophical Association of America
2:291–301. [aLWB]
Newell, A. (1990) Unified theories of cognition. Harvard University Press.
[aLWB]
Newell, A. & Simon, H. A. (1972) Human problem solving. Prentice-Hall.
[aLWB]
Newton, N. (1996) Foundations of understanding. John Benjamins. [arLWB,
AMG]
Nisbett, R. & Wilson, T. (1977) Telling more than we can know: Verbal reports on
mental processes. Psychological Review 84:231–59. [aLWB]
Norman, D. A. (1976) Memory and attention: An introduction to human
information processing. Wiley. [aLWB]
Norman, D. A., Rumelhart, D. E. & the LNR Research Group (1975) Explorations
in cognition. Freeman. [aLWB]
Nosofsky, R. M. (1984) Choice, similarity, and the context theory of classification.
Journal of Experimental Psychology: Learning, Memory, and Cognition
10:104–14. [aLWB]
Nueckles, M. & Jantezko, D. (1997) The role of semantic similarity in the
References/Barsalou: Perceptual symbol systems
BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4 657

Tarr, M. J. & Pinker, S. (1989) Mental rotation and orientation-dependence in
shape recognition. Cognitive Psychology 21:233–82. [aLWB]
Tetlock, P. & Belkin, A., eds. (1996) Counterfactual thought experiments in world
politics: Logical, methodological, and psychological perspectives. Princeton
University Press. [GF]
Thagard, P. (1992) Conceptual revolutions. Princeton University Press. [aLWB]
The Neural Theory of Language Project. (n.d.) http://www.icsi.berkeley.edu/
NTL/. [GF]
Theios, J. & Amhrein, P. C. (1989) Theoretical analysis of the cognitive processing
of lexical and pictorial stimuli: Reading, naming, and visual and conceptual
comparisons. Psychological Review 96:5–24. [aLWB]
Tomasello, M. (1992) First verbs: A case study of early grammatical development.
Cambridge University Press. [aLWB]
Tomasello, M., ed. (1998) The new psychology of language: Cognitive and
functional approaches to language structure. Erlbaum. [GF]
Tomasello, M. & Call, J. (1997) Primate cognition. Oxford University Press.
[rLWB]
Tomasello, M., Kruger, A. & Ratner, H. (1993) Cultural learning. Behavioral and
Brain Sciences 16:495–552. [aLWB]
Toomela, A. (1996) How culture transforms mind: A process of internalization.
Culture and Psychology 2:285–305. [AT]
Tootel, R. B. H., Silverman, M. S., Switkes, E. & De Valois, R. L. (1982)
Deoxyglucose analysis of retinotopic organization in primate striate cortex.
Science 218:902–904. [aLWB]
Tourangeau, R. & Ripe, L. (1991) Understanding and appreciating metaphors.
Cognition 11:203–44. [BI]
Trabasso, T. R. & Bower, G. H. (1968) Attention in learning: Theory and research.
Wiley. [aLWB]
Trehub, A. (1977) Neuronal models for cognitive processes: Networks for learning,
perception, and imagination. Journal of Theoretical Biology 65:141–69.
[rLWB]
(1991) The cognitive brain. MIT Press. [rLWB]
Treisman, A. M. (1969) Strategies and models of selective attention. Psychological
Review 76:282–99. [aLWB]
(1993) The perception of features and objects. In: Attention: Selection,
awareness, and control: A tribute to Donald Broadbent, ed. A. Baddeley & L.
Weiskrantz. Clarendon Press. [aLWB]
Treisman, A. M. & Geffen, G. (1967) Selective attention: Perception or response?
Quarterly Journal of Experimental Psychology 19:1–17. [JH]
Turner, M. (1987) Death is the mother of beauty: Mind, metaphor, criticism.
University of Chicago Press. [RWL]
(1996) The literary mind. Oxford University Press. [aLWB]
Turner, M. & Fauconnier, G. (1995) Conceptual integration and formal expression.
Metaphor and Symbolic Activity 10(3):183–204. [BI]
(1998) Conceptual integration in counterfactuals. In: Discourse and cognition,
ed. J.-P. Koenig. CSLI Publications. [GF]
Tversky, B. & Hemenway, K. (1985) Objects, parts, and categories. Journal of
Experimental Psychology: General 113:169–93. [aLWB]
Tye, M. (1991) The imagery debate. MIT Press. [aLWB]
(1993) Image indeterminacy. In: Spatial representation, ed. N. Eilen, R.
McCarthy & B. Brewer. Blackwell. [aLWB]
Tzourio, N. & Mellet, E. (1998) Imagerie des fonctions cognitives. Direction des
Sciences du Vivant, lettre d’information scientifique 16. URL: http://www-
dsv.cea.fr/dsv/ceabio/16/tzourio16—fr.htm. [FL]
Ullman, S. (1996) High-level vision. MIT Press. [ABM]
Ungerleider, L. G. & Mishkin, M. (1982) Two cortical visual systems. In: Analysis
of visual behavior, ed. D. J. Ingle, M. A. Goodale & R. W. J. Mansfield. MIT
Press. [aLWB]
Van Dijk, T. A. & Kintsch, W. (1983) Strategies of discourse comprehension.
Academic. [aLWB]
Van Essen, D. C. (1985) Functional organization of primate visual cortex. In:
Cerebral cortex, ed. A. Peters & E. G. Jones. Plenum. [aLWB]
Van Gelder, T. (1990) Compositionality: A connectionist variation on a classical
theme. Cognitive Science 14:355–84. [rLWB]
Vaughn, H. G. & Kurtzberg, D. (1992) Electrophysiologic indices of human brain
maturation and cognitive development. In: Developmental behavioral
neurosciences: The Minnesota Symposium, vol. 24, ed. M. R. Gunnar & C. A.
Nelson. Erlbaum. [LM]
Velmans, M. (1991) Is human information processing conscious? Behavioral and
Brain Sciences 14:651–726. [aLWB]
Von Eckardt, B. & Potter, M. C. (1985) Clauses and the semantic representation of
words. Memory and Cognition 13:371–76. [aLWB]
Vosniadou, S. & Brewer, W. F. (1992) Mental models of the earth: A study of
conceptual change in childhood. Cognitive Psychology 24:535–85. [WFB]
(1994) Mental models of the day/night cycle. Cognitive Science 18:123–83.
[WFB]
Vygotsky, L. S. (1925/1982) Soznanije kak problema psikhologii povedenija.
[Consciousness as a problem for a psychology of behaviour]. In: Sobranije
sochinenii, Tom 1 [Collected works, vol. 1], ed. L. S. Vygotsky. Pedagogika.
[AT]
(1934/1996) Myshlenije i rech. [Thinking and speech]. Labirint. [AT]
(1962) Thought and language. MIT Press. [FL]
Warren, R. M. (1970) Perceptual restoration of missing speech sounds. Science
167:392–93. [aLWB]
Warrington, E. K. & Shallice, T. (1984) Category-specific semantic impairments.
Brain 107:829–54. [aLWB]
Watson, J. B. (1913) Psychology as the behaviorist sees it. Psychological Review
20:158–77. [aLWB]
Wattenmaker, W. D. & Shoben, E. J. (1987) Context and the recallability of
concrete and abstract sentences. Journal of Experimental Psychology:
Learning, Memory, and Cognition 13:140–50. [rLWB]
Weiskrantz, L. (1986) Blindsight: A case study and implications. Oxford University
Press. [aLWB]
Weisstein, N. & Harris, C. S. (1974) Visual detection of line segments: An object-
superiority effect. Science 186:752–55. [aLWB]
(1980) Masking and unmasking of distributed representations in the visual
system. In: Visual coding and adaptability, ed. C. S. Harris. Erlbaum.
[aLWB]
Weisstein, N., Montalvo, F. S. & Ozog, G. (1972) Differential adaptation to
gratings blocked by cubes and gratings blocked by hexagons: A test of the
neural symbolic activity hypothesis. Psychonomic Science 27:89–91. [aLWB]
Weisstein, N. & Wong, E. (1986) Figure-ground organization and the spatial and
temporal responses of the visual system. In: Pattern recognition by humans
and machines: Vol. 2. Visual perception, ed. E. C. Schwab & H. C. Nusbaum.
Academic Press. [aLWB]
Wellman, H. & Bartsch, K. (1988) Young children’s reasoning about beliefs.
Cognition 30:239–77. [RO]
Wellman, H. M. & Gelman, S. A. (1988) Children’s understanding of the non-
obvious. In: Advances in the psychology of human intelligence, ed. R.
Sternberg. Erlbaum. [aLWB]
Wells, A. J. (1998) Turing’s analysis of computation and theories of cognitive
architecture. Cognitive Science 22(3):269–94. [AJW]
Wells, G. L. & Gavinski, I. (1989) Mental simulation of causality. Journal of
Personality and Social Psychology 56:161–69. [aLWB]
Werner, H. & Kaplan, B. (1963) Symbol formation. Wiley. [LM]
Wiemer-Hastings, K. (1998) Abstract noun classification: Using a neural network to
match word context and word meaning. Behavior Research Methods,
Instruments and Computers 30:264–71. [rLWB]
Wiemer-Hastings, K. & Graesser, A. C. (1998) Contextual representation of
abstract nouns: A neural network approach. Proceedings of the 20th Annual
Conference of the Cognitive Science Society:1036–42. Elrbaum. [rLWB]
Wilson, S. G., Rinck, M., McNamara, T. P., Bower, G. H. & Morrow, D. G. (1993)
Mental models and narrative comprehension: Some qualifications. Journal of
Memory and Language 32:141–54. [aLWB]
Wimmer, H. & Perner, J. (1983) Beliefs about beliefs: Representation and
constraining function of wrong beliefs in young children’s understanding of
deception. Cognition 13:103–28. [RO]
Winograd, T. & Flores, F. (1987) Understanding computers and cognition: A new
foundation for design. Addison-Wesley. [aLWB]
Wittgenstein, L. (1953) Philosophical investigations. Macmillan. [arLWB]
Wolfe, J. (1998) How quickly they forget: A modest alternative to blinks and
blindness. Abstracts of the Psychonomic Society 3:53. [JH]
Wu, L. L. (1995) Perceptual representation in conceptual combination. Doctoral
dissertation, University of Chicago. [aLWB]
Wu, L. L. & Barsalou, L. W. (1999) Grounding property listing in perceptual
simulation (under review). [arLWB]
Yeh, W. & Barsalou, L. W. (1999a) A review of situation effects in cognition (in
preparation). [aLWB]
(1999b) Situation effects in conceptual processing (in preparation). [aLWB]
Younger, B. A. & Cohen, L. B. (1985) How infants form categories. In: The
psychology of learning and motivation, vol. 19, ed. G. H. Bower. Academic
Press. [aLWB]
Zajonc, R. (1980) Feeling and thinking: Preferences need no inferences. American
Psychologist 39:151–75. [LCC]
Zatorre, R. J., Halpern, A. R., Perry, D. W., Meyer, E. & Evans, A. C. (1996)
Hearing in the mind’s ear: A PET investigation of musical imagery and
perception. Journal of Cognitive Neuroscience 8:29–46. [aLWB]
Zeki, S. (1993) A vision of the brain. Blackwell Scientific. [aLWB]
Zwaan, R. A. (1996) Processing narrative time shifts. Journal of Experimental
Psychology: Learning, Memory, and Cognition 22:1196–207. [RAZ]
(in press) Embodied cognition, perceptual symbols, and situation models.
Discourse Processes. [RAZ]
Zwaan, R. A. & Radvansky, G. A. (1998) Situation models in language
comprehension and memory. Psychological Bulletin 123:162–85. [RAZ]
References/Barsalou: Perceptual symbol systems
660 BEHAVIORAL AND BRAIN SCIENCES (1999) 22:4