Colloquium Complexity and robustness J. M. Carlson*��� and John Doyle��� *Department of Physics, University of California, Santa Barbara, CA 93106 and ���Control and Dynamical Systems, California Institute of Technology, Pasadena, CA 91125 Highly optimized tolerance (HOT) was recently introduced as a conceptual framework to study fundamental aspects of complex- ity. HOT is motivated primarily by systems from biology and engineering and emphasizes, (i) highly structured, nongeneric, self-dissimilar internal configurations, and (ii) robust yet fragile external behavior. HOT claims these are the most important fea- tures of complexity and not accidents of evolution or artifices of engineering design but are inevitably intertwined and mutually reinforcing. In the spirit of this collection, our paper contrasts HOT with alternative perspectives on complexity, drawing on real- world examples and also model systems, particularly those from self-organized criticality. Acertain vision shared by most researchers in complex systems is that intrinsic, perhaps even universal, features capture fundamental aspects of complexity in a manner that transcends specific domains. It is in identifying these features that sharp differences arise. In disciplines such as biology, engineering, sociology, economics, and ecology, individual complex systems are necessarily the objects of study, but there often appears to be little common ground between their models, abstractions, and methods. Highly optimized tolerance (HOT) (1���6) is one recent attempt, in a long history of efforts, to develop a general framework for studying complexity. The HOT view is motivated by examples from biology and engineering. Theoretically, it builds on mathematics and abstractions from control, commu- nications, and computing. In this paper, we retain the motivating examples but avoid theories and mathematics that may be unfamiliar to a nonengineering audience. Instead, we aim to make contact with the models, concepts, and abstractions that have been loosely collected under the rubric of a ������new science of complexity������ (NSOC) (7) or ������complex adaptive systems������ (CAS), and particularly the concept of self-organized criticality (SOC) (8, 9). SOC is only one element of NSOC CAS but is a useful representative, because it has a well-developed theory and broad range of claimed applications. In Table 1, we contrast HOT���s emphasis on design and rare configurations with the perspective provided by NSOC CAS SOC, which emphasizes structural complexity as ������emerging between order and disorder,������ (i) at a bifurcation or phase transition in an interconnection of components that is (ii) otherwise largely random. Advocates of NSOC CAS SOC are inspired by critical phenomena, fractals, self-similarity, pattern formation, and self-organization in statistical physics, and bifur- cations and deterministic chaos from dynamical systems. Moti- vating examples vary from equilibrium statistical mechanics of interacting spins on a lattice to the spontaneous formation of spatial patterns in systems far from equilibrium. This approach suggests a unity from apparently wildly different examples, because details of component behavior and their interconnec- tion are seen as largely irrelevant to system-wide behavior. Table 1 shows that SOC and HOT predict not just different but exactly opposite features of complex systems. HOT suggests that random interconnections of components say little about the complexity of real systems, that the details can matter enor- mously, and that generic (e.g., low codimension) bifurcations and phase transitions play a peripheral role. In principle, Table 1 could have a separate column for Data, by which we mean the observable features of real systems. Because HOT and Data turn out to be identical for these features, we can collapse the table as shown. This is a strong claim, and the remainder of this paper is devoted to justifying it in as much detail as space permits. What Do We Mean By Complexity? To motivate the theoretical discussion of complex systems, we briefly discuss concrete and hopefully reasonably familiar ex- amples and begin to fill in the ������Data������ part of Table 1. We start with biological cells and their modern technological counterparts such as very large-scale integrated central processing unit (CPU) chips. Each is a complex system, composed of many components, but is also itself a component in a larger system of organs or laptop or desktop personal computers or embedded in control systems of vehicles such as automobiles or commercial jet aircraft like the Boeing 777. These are again components of the even larger networks that make up organisms and ecosystems, This paper results from the Arthur M. Sackler Colloquium of the National Academy of Sciences, ������Self-Organized Complexity in the Physical, Biological, and Social Sciences,������ held March 23���24, 2001, at the Arnold and Mabel Beckman Center of the National Academies of Science and Engineering in Irvine, CA. Abbreviations: NSOC, new science of complexity CAS, complex adaptive systems SOC, self-organized criticality HOT, highly optimized tolerance CPU, central processing unit DC, data compression DDOF, design degree of freedom CF, California brushfires FF, U.S. Fish and Wildlife Service land fires PLR, probability-loss-resource WWW, World Wide Web. ���To whom reprint requests should be addressed. E-mail: carlson@physics.ucsb.edu. Table 1. Characteristics of SOC, HOT, and data Property SOC HOT and Data 1 Internal configuration Generic, homogeneous, self-similar Structured, heterogeneous, self-dissimilar 2 Robustness Generic Robust, yet fragile 3 Density and yield Low High 4 Max event size Infinitesimal Large 5 Large event shape Fractal Compact 6 Mechanism for power laws Critical internal fluctuations Robust performance 7 Exponent Small Large 8 vs. dimension d (d 1) 10 1 d 9 DDOFs Small (1) Large ( ) 10 Increase model resolution No change New structures, new sensitivities 11 Response to forcing Homogeneous Variable 2538���2545 PNAS February 19, 2002 vol. 99 suppl. 1 www.pnas.org cgi doi 10.1073 pnas.012582499

computer networks, and air and ground transportation. Al- though extremely complex, these systems have available reason- ably complete descriptions and thus make good starting exam- ples. Engineering systems are obviously better understood than biological systems, but the gap is closing. Engineers now build systems of almost bewildering levels of complexity, and biolo- gists are beginning to move beyond the components to charac- terizing the networks they create. Although no one person understands in complete detail how all these systems work, there is now a rich variety of accessible introductory material in each area that gives additional details well beyond what is discussed here. In each of the following paragraphs, we consider a critical question about complexity and the answers that these example systems suggest. What Distinguishes the Internal Configurations of Systems as Com- plex? It is not the mere number of component parts. Any macro- scopic material has a huge number of molecules. It is the extreme heterogeneity of the parts and their organization into intricate and highly structured networks, with hierarchies and multiple scales (Table 1.1). (Some researchers have suggested that ������complicated������ be used to describe this feature.) Even bacterial cells have thou- sands of genes, most coding for proteins that form elaborate regulatory networks. A modern CPU has millions of transistors and millions of supporting circuit elements many computers have billions of transistors, and the Internet will soon have billions of nodes. The 777 is fully ������fly-by-wire,������ with 150,000 different sub- systems, many of them quite complex, including roughly 1,000 CPUs that operate and automate all vehicle functions. Even automobiles have dozens of CPUs performing a variety of control functions. If self-similarity describes multiscale systems with similar structure at different scales, then these systems could be described as highly self-dissimilar, that is, extremely different at different scales and levels of abstraction. Just the design and manufacture of the 777 involved a global software and computing infrastructure with roughly 10,000 work stations, terabytes of data, and a one billion dollar price tag. What Does This Complexity Achieve? In each example, it is possible to build similar systems with orders of magnitude fewer com- ponents and much less internal complexity. The simplest bacteria have hundreds of genes. Much simpler CPUs, computers, net- works, jets, and cars can be and have been built. What is lost in these simpler systems is not their basic functionality but their robustness. By robustness, we mean the maintenance of some desired system characteristics despite fluctuations in the behav- ior of its component parts or its environment. Although we can loosely speak of robustness without reference to particular systems characteristics, or particular component or environmen- tal uncertainties, this can often be misleading, as we will see. All of our motivating examples illustrate this tradeoff between robustness and internal simplicity. Although it has become a cliche that greater complexity creates unreliability, the actual story is more complicated. What Robustness Would Be Lost in Simpler Systems? Simple bacteria with several hundred genes, like mycoplasma, require carefully controlled environments, whereas Escherichia coli, with almost 10 times the number of genes, can survive in highly fluctuating environments. Large internetworks do not change the basic capa- bilities of computers but instead improve their responsiveness to variations in a user���s needs and failures of individual computers. A jet with many fewer components and no very large-scale integrated chips or CPUs could be built with the same speed and payload as a 777, but it would be much less robust to component variations, failures, or fluctuations such as payload size and distribution or atmospheric conditions. Whereas older automobiles were simpler, new vehicles have elaborate control systems for air bags, ride control, antilock braking, antiskid turning, cruise control, satellite navigation, emergency notification, cabin temperature regulation, and automatic tuning of radios. At the same size and efficiency, they are safer, more robust, and require less maintenance. Thus robust- ness drives internal complexity and is the most striking feature of these complex systems. What Is the Price Paid for These Highly Structured Internal Configu- rations and the Resulting Robustness? Although there is the ex- pense of additional components, this is usually more than made up for by increased efficiency, manufacturability, evolvability of the system, and the ability to use sloppier and hence cheaper components. It is far more serious that these systems can be catastrophically disabled by cascading failures initiated by tiny perturbations. They are ������robust, yet fragile,������ that is, robust to what is common or anticipated but potentially fragile to what is rare or unanticipated and also to flaws in design, manufacturing, or maintenance (Table 1.2). Because robustness is achieved by very specific internal structures, when any of these systems is disassembled, there is very little latitude in reassembly if a working system is expected. Although large variations or even failures in components can be tolerated if they are designed for through redundancy and feedback regulation, what is rarely tolerated, because it is rarely a design requirement, is nontrivial rearrangements of the interconnection of internal parts. The fraction of all possible amino acid sequences or complementary metal oxide semiconductor circuits that yield functioning pro- teins or chips is vanishingly small. Portions of macromolecular networks as well as whole cells of advanced organisms can function in vitro, but we do not yet know how to reassemble them into fully functional cells and organisms. In contrast, when arbitrary interconnection is a specific design requirement, such as in routers in an internet protocol network, then this can be robustly designed for but with some added expense in resources. How Does ������Robust, Yet Fragile������ Manifest Itself in the Example Systems? Biological organisms are highly robust to uncertainty in their environments and component parts yet can be catastrophically disabled by tiny perturbations to genes or the presence of micro- scopic pathogens or trace amounts of toxins that disrupt structural elements or regulatory control networks. The 777 is robust to large-scale atmospheric disturbances, variations in cargo loads and fuels, turbulent boundary layers, and inhomogeneities and aging of materials, but could be catastrophically disabled by microscopic alterations in a handful of very large-scale integrated chips or by software failures. (Such a vulnerability is completely absent from a hypothetical simpler vehicle.) This scenario fortunately is vanish- ingly unlikely but illustrates the issue that this complexity can amplify small perturbations, and the design engineer must ensure that such perturbations are extremely rare. The 777 is merely a component in a large, highly efficient, and inexpensive air traffic network, but also one that can have huge cascading delays. Pro- cessor chips are similarly robust to large variations in the analog behavior of their CMOS circuit elements and can perform a literally ������universal������ array of computations but can fail completely if an element is removed or the circuit rearranged. Processor, memory, and other chips can be organized into highly fault-tolerant com- puters and networks, creating platforms for complex software systems with their own hierarchies of components. These software systems can perform a broad range of functions primarily limited only by the programmer���s imagination but can crash from a single line of faulty code. How Does NSOC CAS Differ from HOT with Respect to the Complexity of the Example Systems? As a specific, if somewhat whimsical, example, note that a 777 is sufficiently automated that it can fly without pilots, so we could quite fairly describe the mechanism by which it can transport people and material through the air Carlson and Doyle PNAS February 19, 2002 vol. 99 suppl. 1 2539