I Lev Manovich The Language of New Media

II To Norman Klein / Peter Lunenfeld / Vivian Sobchack

III Table of Contents Prologue: Vertov���s Dataset.................................................................................VI Acknowledgments........................................................................................ XXVII Introduction.........................................................................................................30 A Personal Chronology...........................................................................30 Theory of the Present..............................................................................32 Mapping New Media: the Method..........................................................34 Mapping New Media: Organization .......................................................36 The Terms: Language, Object, Representation ......................................38 I. What is New Media?........................................................................................43 Principles of New Media..............................................................................49 1. Numerical Representation...................................................................49 2. Modularity ..........................................................................................51 3. Automation .........................................................................................52 4. Variability...........................................................................................55 5. Transcoding ........................................................................................63 What New Media is Not...............................................................................66 Cinema as New Media............................................................................66 The Myth of the Digital ..........................................................................68 The Myth of Interactivity........................................................................70 II. The Interface...................................................................................................75 The Language of Cultural Interfaces.........................................................80 Cultural Interfaces...................................................................................80 Printed Word...........................................................................................83 Cinema....................................................................................................87 HCI: Representation versus Control.......................................................94 The Screen and the User..............................................................................99 A Screen's Genealogy.............................................................................99 The Screen and the Body......................................................................105

IV Representation versus Simulation.........................................................111 III. The Operations ...........................................................................................115 Menus, Filters, Plug-ins.............................................................................120 The Logic of Selection..........................................................................120 ���Postmodernism��� and Photoshop .........................................................124 From Object to Signal...........................................................................126 Compositing................................................................................................130 From Image Streams to Modular Media...............................................130 The Resistance to Montage...................................................................134 Archeology of Compositing: Cinema...................................................138 Archeology of Compositing: Video......................................................141 Digital Compositing..............................................................................143 Compositing and New Types of Montage ...........................................145 Teleaction....................................................................................................150 Representation versus Communication ................................................150 Telepresence: Illusion versus Action....................................................152 Image-Instruments ................................................................................155 Telecommunication ..............................................................................156 Distance and Aura.................................................................................158 IV. The Illusions ................................................................................................162 Synthetic Realism and its Discontents......................................................168 Technology and Style in Cinema..........................................................168 Technology and Style in Computer Animation....................................171 The icons of mimesis............................................................................177 Synthetic Image and its Subject................................................................180 Georges M��li��s, the father of computer graphics.................................180 Jurassic Park and Socialist Realism......................................................181 Illusion, Narrative and Interactivity ........................................................185 V. The Forms .....................................................................................................190 Database......................................................................................................194 The Database Logic ..............................................................................194 Data and Algorithm ..............................................................................196 Database and Narrative.........................................................................199 Paradigm and Syntagm .........................................................................202

V A Database Complex ............................................................................205 Database Cinema: Greenaway and Vertov ...........................................207 Navigable space ..........................................................................................213 Doom and Myst ....................................................................................213 Computer Space....................................................................................219 The Poetics of Navigation.....................................................................223 The Navigator and the Explorer............................................................231 Kino-Eye and Simulators......................................................................234 EVE and Place ......................................................................................240 VI. What is Cinema?.........................................................................................244 Digital Cinema and the History of a Moving Image...............................249 Cinema, the Art of the Index ................................................................249 A Brief Archeology of Moving Pictures...............................................251 From Animation to Cinema..................................................................252 Cinema Redefined.................................................................................253 From Kino-Eye to Kino-Brush .............................................................259 New Language of Cinema..........................................................................260 Cinematic and Graphic: Cinegratography ............................................260 New Temporality: Loop as a Narrative Engine....................................264 Spatial Montage ....................................................................................269 Cinema as an Information Space ..........................................................273 Cinema as a Code .................................................................................276 NOTES ...............................................................................................................279

VI Prologue: Vertov���s Dataset The avant-garde masterpiece A Man With a Movie Camera completed by Russian director Dziga Vertov in 1929 will serve as our guide to the language of new media.This prologue consists of a number of stills from the film. Each still is accompanied by quote from the text summarizing a particular principle of new media. The number in brackets indicates a page from which the quote is taken. The prologue thus acts as a visual index to some of the book's ideas.

VII 1. [figure 1] (87) ���A hundred years after cinema's birth, cinematic ways of seeing the world, of structuring time, of narrating a story, of linking one experience to the next, are being extended to become the basic ways in which computer users access and interact with all cultural data. In this way, the computer fulfills the promise of cinema as a visual Esperanto which pre-occupied many film artists and critics in the 1920s, from Griffith to Vertov. Indeed, millions of computer users communicate with each other through the same computer interface. And, in contrast to cinema where most of its ���users��� were able to ���understand��� cinematic language but not ���speak��� it (i.e., make films), all computer users can ���speak��� the language of the interface. They are active users of the interface, employing it to perform many tasks: send email, organize their files, run various applications, and so on.���

VIII 2. [figure 2] [figure 3] [figure 4] [figure 5] (91) ���The incorporation of virtual camera controls into the very hardware of a game consoles is truly a historical event. Directing the virtual camera becomes as important as controlling the hero's actions��� the computer games are returning to "The New Vision" movement of the 1920s (Moholy-Nagy, Rodchenko, Vertov and others), which foregrounded new mobility of a photo and film camera, and made unconventional points of view the key part of their poetics.

IX 3. [figure 6] [figure 7] [figure 8] [figure 9] (140) ���Editing, or montage, is the key twentieth technology for creating fake realities. Theoreticians of cinema have distinguished between many kinds of montage but, for the purposes of sketching the archeology of the technologies of simulation leading to digital compositing, I will distinguish between two basic techniques. The first technique is temporal montage: separate realities form consecutive moments in time. The second technique is montage within a shot. It is the opposite of the first: separate realities form contingent parts of a single image��� examples [of montage within a shot] include the superimposition of a few images and multiple screens used by the avant-garde filmmakers in the 1920���s (for instance, superimposed images in Vertov���s Man with a Movie Camera and a three-part screen in Gance Abel���s 1927 Napol��on).

X 4. [figure 10] [figure 11] [figure 12] (140) ���As theorized by Vertov, through [temporal] montage, film can overcome its indexical nature, presenting a viewer with objects which never existed in reality.���

XI 5. [figure 13] [figure 14] (147) ���While the dominant use of digital compositing is to create a seamless virtual space, it does not have to be subordinated to this goal. The borders between different worlds do not have to be erased the different spaces do not have to be matched in perspective, scale and lighting the individual layers can retain their separate identity rather then being merged into a single space the different worlds can clash semantically rather than form a single universe.���

XII 6. [figure 15] [figure 16] [figure 17] [figure 18] [figure 19] (158) ���The cameraman, whom Benjamin compares to a surgeon, ���penetrates deeply into its [reality] web��� his camera zooms in order to ���pray an object from its shell.��� With its new mobility, glorified in such films as A Man with the Movie Camera, the camera can be anywhere, and, with its superhuman vision, it can obtain a close-up of any object... Along with disregarding the scale, the unique locations of the objects are discarded as well as their photographs brought together within a single picture magazine or a film newsreel, the forms which fit in with the demand of mass democratic society for ���the universal equality of things.������

XIII 7. [figure 20] [figure 21] (160) ���Modernization is accompanied by the process of disruption of physical space and matter, the process which privileges interchangeable and mobile signs over the original objects and relations���The concept of modernization fits equally well Benjamin's account of film and Virilio's account of telecommunication, the latter just being a more advanced stage in this continual process of turning objects into mobile signs. Before, different physical locations met within a single magazine spread or a film newsreel now, they meet within a single electronic screen.���

XIV 8. [figure 22] [figure 23] (183) ���Whose vision is it? It is the vision of a computer, a cyborg, a automatic missile. It is a realistic representation of human vision in the future when it will be augmented by computer graphics and cleansed from noise. It is the vision of a digital grid. Synthetic computer-generated image is not an inferior representation of our reality, but a realistic representation of a different reality.���

XV 9. [figure 24] (209) ���Along with Greenaway, Dziga Vertov can be thought of as a major ���database filmmaker��� of the twentieth century. Man with a Movie Camera is perhaps the most important example of database imagination in modern media art.���