COGNITIVE SCIENCE Vol 23 (2) 1999, pp. 207���245 ISSN 0364-0213 Copyright �� 1999 Cognitive Science Society, Inc. All rights of reproduction in any form reserved. Are Theories of Imagery Theories of Imagination? An Active Perception Approach to Conscious Mental Content. NIGEL J. T. THOMAS California State University, Fullerton Can theories of mental imagery, conscious mental contents, developed within cognitive science throw light on the obscure (but culturally very significant) concept of imagination? Three extant views of mental imagery are considered: quasi-pictorial, description, and perceptual activity theories. The first two face serious theoretical and empirical difficulties. The third is (for historically contingent reasons) little known, theoretically underdevel- oped, and empirically untried, but has real explanatory potential. It rejects the ���traditional��� symbolic computational view of mental contents, but is compatible with recent situated cognition and active vision approaches in robotics. This theory is developed and elucidated. Three related key aspects of imagination (non-discursiveness, creativity, and seeing as) raise difficul- ties for the other theories. Perceptual activity theory presents imagery as non-discursive and relates it closely to seeing as. It is thus well placed to be the basis for a general theory of imagination and its role in creative thought. ��1 INTRODUCTION Can extant scientific theories of mental imagery (quasi-perceptual experience) account for, or accommodate, the much wider functions of imagination as traditionally conceived? Such functions range from ordinary supposal to the most valued aspects of human insight and creativity (Thomas, 1997a, 1999a), but it is the latter that will be of particular concern here. The issue is raised not only to bring a fresh perspective to the notoriously recalcitrant scientific debate about the nature and cognitive importance of imagery, but also to try to illuminate the obscure topic of imagination itself. Prima facie imagery and imagination are intimately related��� certainly ���imagination��� is often used to name the faculty of image Direct all correspondence to: Nigel J. T. Thomas, 86 S. Sierra Madre Boulevard, #5, Pasadena, CA 91107 E-Mail: n.j.thomas70@members.leeds.ac.uk 207

production (or the mental arena in which images appear)���and this fact itself demands explanation. I will argue that neither of the better known theories of imagery, the ���quasi-pictorial��� theory (Kosslyn, 1980, 1994) and the ���description��� theory (Pylyshyn, 1973, 1981), are capable of meeting this challenge. I will develop and defend a theory that can, I believe, meet it: a version of the little known ���perceptual activity��� theory, related to recent ���situated��� approaches to cognition (Clancey, 1997). It seems likely that the association between imagery and creative imagination is not unrelated to the unusual passion (Dennett, 1978) surrounding the debates about imagery of the past 30 years or so. Except to the aficionado, mere visualization is surely not, in itself, a terribly exciting matter ���imagination���, by contrast is a highly value laden and culturally charged term (Kearney, 1988). If the relationship between imagery and imag- ination (in the relevant sense) can be clarified, perhaps the constraints which scientific results lay upon theories of imagery may help to bring some discipline to the more extreme claims made on behalf of the imagination. In particular, the concept of imagination has played a key role in anti-scientific attitudes and ideologies, at least since the writers and artists of the Romantic movement gave it a central place in their ideology (Daston, 1998). Imagination is presented as supremely valuable and forever beyond the reach of scientific understanding it is the guarantor and the embodiment of the (alleged) fact that science will never be able to illuminate ���what really matters��� in life. Romanticism was by no means unequivocally hostile to science (Abrams, 1953), but the strong anti-scientific strand of the 1960s ���counter culture��� movement clearly derived from it (Roszak, 1970), and although Gross and Levitt (1994) try to blame the contemporary prevalence of anti-science on the ���academic left���, they acknowledge that its intellectual roots are in Romanticism rather than Marx or the labor movement. Furthermore, despite the fact that the vital role of imagination in scientifically thinking has been urged repeatedly and convincingly (e.g. Tyndall, 1872 Van���t Hoff, 1878/1967 Brown, 1991 Holton, 1996) the apologists of science have frequently dem- onstrated a hostility to the claims of imagination that is quite comparable to the hostility to science shown by the ideologues of imagination (Daston, 1998). Nevertheless, the Romantic conception of imagination (which was heir to a long tradition) has had an enormous and ineradicable intellectual influence, and is deeply embedded in our folk psychology (Brann, 1991 Sutherland, 1971). There is every reason to think that it contains genuine and important insights, and that it addresses real socio-cultural, or even spiritual, needs. For science to dismiss the issues it raises would amount to the acceptance of a major limitation on the scope of the scientific approach to reality. In order to address these issues rationally and effectively we need a scientific theory of imagination that pays due respect to the folk/Romantic conception. In what follows, I will first outline the extant scientific theories of the nature and mechanisms of imagery, and will attempt to assess them purely as such. I will then try to delineate the relevant key components of the folk/Romantic conception of imagination, and finally I will try to assess to what degree the respective scientific theories of imagery are capable of throwing light upon them. 208 THOMAS

��2 IMAGERY AND THEORIES OF IMAGERY IN COGNITIVE SCIENCE Let us provisionally define mental imagery as quasi-perceptual experience: experience that significantly resembles perceptual experience (in any sense mode), but which occurs in the absence of appropriate external stimuli for the relevant perception (for discussion and defense of this definition see Thomas, 1999b). There are normally important expe- riential differences between imagery and perception���without them imagery may slide into hallucination��� but these need not concern us here it is the similarities which are definitive. ���Experience���, of course, implies consciousness, and this may raise a problem because there are reasons to think that imagery can play a role in cognition even when subjects are not consciously aware of it (Paivio, 1971, 1986 Baars, 1996). However, people often are consciously aware of quasi-perceptual experiences, and it is upon this phenomenological fact that the major cognitive research programs into imagery have been built (Thomas, 1987). I submit that we should think of occurrent mental images as, at the least, potential contents of consciousness (in something like the sense of Searle, 1992), even if it is the case that they may sometimes occur without the subject being explicitly aware of them. Imagery may, conceivably, play an important role in cognition even when people deny consciously experiencing it (Thomas, 1989, 1999b). Notoriously, many psychologists of the behaviorist era did deny, if not the experience, certainly the psychological significance of imagery. However, from the late 1950s interest in imagery gradually increased (Holt, 1964), and from the later 1960s a number of striking, now classic, experimental effects (for reviews see Morris and Hampson, 1983, and Finke, 1989) convinced many of its cognitive significance (Thomas, 1987). Imagery proved to have powerful mnemonic effects (Paivio, 1971, 1986, 1991), and it appeared that an image could be smoothly rotated (Shepard & Metzler, 1971 Shepard & Cooper, 1982), and scanned across (Kosslyn, 1973, 1980). It also appeared that when inspecting their images, subjects could find subjectively large details more quickly than subjectively small ones (Kosslyn, 1975, 1980), and that imagery and perceptual tasks in the same mode would often mutually interfere with one another (Brooks, 1968 Segal, 1971 Craver- Lemley & Reeves, 1992). A theory of the nature of imagery was clearly needed. With Morris and Hampson (1983), I take there to be three types of imagery theory extant in the Cognitive Science literature. I shall refer to them as ���description theory���, ���picture theory���, and ���perceptual activity [or PA] theory���. However, we might better understand these as paradigms (Kuhn, 1970), or, better still, as research programs (in the sense of Lakatos, 1978), rather than as static and unitary theories. Although research programs are characterized by certain core commitments, which suggest particular lines of investigation and explanatory strategies, different theoretical elaborations can be constructed around these cores to account (with varying degrees of strain) for almost any empirical evidence that is likely to arise. Thus, in the course of its development, a single vigorous research program may encompass a number of well elaborated and quite diverse theoretical positions whose detailed empirical commitments may differ considerably. In the case of the pictorialist research program we see this even within its relatively recent 209 IMAGERY AND IMAGINATION

evolution: Contrast the very different versions of picture theory given by Kosslyn in his two major statements on the subject (Kosslyn 1980, 1994). In the light of this, we should not expect to be able to make a definitive choice between the three imagery research programs solely on the basis of empirical evidence. Rational choice between programs (or paradigms) depends not just upon empirical adequacy, but also on judgments about factors like the elegance and consistency of the theoretical elaborations needed to assimilate the evidence, and the scope and promise of the core ideas (Lakatos, 1978). ��2.1 Picture Theory Picture theory has a very long history, going back to Plato or even Democritus (Thomas, 1987), and until quite recently it was almost universally accepted (White, 1990). All versions hold that having visual imagery involves having entities, in the head or in the mind, which are like, or functionally equivalent to, inner pictures. These pictures are thought of as being composed of copies or remnants of earlier sense impressions, complexes of visual sensations, which were themselves picture like. ���Picture��� is, of course, strictly appropriate only to copies in the visual mode (upon which most discus- sions of imagery concentrate), but, if this sort of account really works for the visual, then it seems at least initially plausible that similar ���copy��� theories could be devised for the other sensory modes (Matthews, 1969). ��2.1.1 Contemporary ���Quasi-Pictorial��� Theory. Picture theory came under severe philosophical attack in the middle years of this century (e.g. Sartre, 1936/1962 Ryle, 1949 Wittgenstein, 1953) for being committed to an implausible, Cartesian view of the mind. However, in the opinion of many, Kosslyn (1975, 1980, 1994) has succeeded, in the teeth of trenchant and sophisticated objections from such description theorists as Pylyshyn (1973, 1978, 1979, 1981) and Hinton (1979), in showing that his computational version of picture theory is both coherent and empirically credible. At least, Tye (1991) seems to have convincingly demonstrated that it is coherent given the assumption that computational data structures of some type are a proper model for conscious and intentionalistic mental contents. Kosslyn implicitly shares this assump- tion���let us call it computational mentalism, or CM���with the descriptionists. Indeed, most discussions of cognitive theory conflate the notion of a mental (intentional) content with that of a computational representation, failing to distinguish the CM dogma from the view that brain function may best be understood and simulated computationally.1 How- ever, one can consistently accept the latter position (and even a form of ���strong AI���) whilst rejecting CM, and there may be good reasons to do so (Slezak, 1999 Cummins 1989, 1996). CM faces the symbol grounding problem, the problem of understanding how computational states can acquire intrinsic intentional content (Harnad, 1990), and it is not at all clear that this problem is soluble.2 If CM is false (as I believe) then both the description theory of imagery and the computational version of picture theory defended by Kosslyn and Tye are profoundly compromised. However, the critiques of these theories that follow below will not depend upon prior rejection of CM. Many people do accept 210 THOMAS

CM, and it is at the foundations of the information processing paradigm, which views the task of a theory of perception as, essentially, to give an account of how the information implicit in states of the sense organs is converted into a canonically ���mental���, consciously available, data format or formats. Kosslyn���s fundamental disagreement with the descrip- tionists is over what sorts of format might be able to fill this role. Kosslyn draws his theory from an analogy with computer graphics, and its basic form is illustrated in Figure 1. Data from which images may be constructed are stored in long term memory in the form of ���deep representations���, structural descriptions not funda- mentally dissimilar to those envisaged by description theorists.3 But these are not directly available to consciousness. They are analogous to the files in which data is saved by a computer graphics program, and on the basis of which actual, viewable pictures are constructed on the computer���s CRT monitor. Kosslyn���s (1980) theory is explicitly based on this ���CRT metaphor���. He holds that ���quasi-pictures��� or ���surface representations��� are constructed on the basis of the information in deep representations. This construction takes place at a functionally defined neural locus that he calls the ���visual buffer���. Once the quasi-picture is established it is available to consciousness as an image, and, furthermore, information that was merely implicit in the deep representation (such as the pointedness of the fox���s ears, in the depicted example) can be extracted from it by the postulated ���mind���s eye function���. We should not think of this ���function��� as literally seeing the image, but it is needed to read and interpret the buffer���s ���surface display���. Clearly Kosslyn thinks of the visual buffer as also being a stage in perceptual information processing, and in more recent work he has explicitly identified it as composed out of the several retinotopic maps of the brain���s occipital cortex (Kosslyn, 1994). The evidence regarding this claim, however, is conflicting. Although Kosslyn and others (Kosslyn et al., 1993 Damasio et al., 1993 Kosslyn, Thompson, Kim, & Alpert, 1995) have detected activity in the relevant brain areas during visual imagery, other researchers (Roland & Gulya ��s, 1994 Mellet et al., 1996) find no such activity, and argue that imagery is more consistently associated with activity in other, non-retinotopically organized regions. Neurological patients who have lost the retinotopically mapped regions in one cerebral hemisphere, leaving them blind in the corresponding half of their visual field, show certain impaired imagery abilities in the blinded hemifield (Butter, Kosslyn, Mijovic-Prelec, & Riffle, 1997 Farah, Soso, & Dasheif, 1992). However, other patients suffering from cortical blindness due to damage in these areas seem to have relatively normal imagery (Chatterjee & Southwood, 1995). Furthermore, some patients with localized damage in the retinotopically mapped areas experience vivid, well-formed ���visual hallucinations��� (i.e. imagery that is outside of conscious control���they do not typically mistake it for reality) precisely in the affected (blind or ���blindsighted���) parts of their visual fields (Ramachandran & Hirstein, 1997 Weiskrantz, Warrington, Sanders, & Marshall, 1974). This suggests that activity in these brain areas cannot be essential for visual imagery. However, even if Kosslyn proves to be mistaken in equating the occipital retinotopic maps with his imagery buffer, this will not in itself entail the wholesale rejection of his research program. It could still quite reasonably be argued that the buffer may be located 211 IMAGERY AND IMAGINATION