## The calculi of emergence: Computation, dynamics, and induction (1994)

Venue: | Physica D |

Citations: | 86 - 14 self |

### BibTeX

@ARTICLE{Crutchfield94thecalculi,

author = {James P. Crutchfield},

title = {The calculi of emergence: Computation, dynamics, and induction},

journal = {Physica D},

year = {1994},

volume = {75},

pages = {11--54}

}

### Years of Citing Articles

### OpenURL

### Abstract

Defining structure and detecting the emergence of complexity in nature are inherently subjective, though essential, scientific activities. Despite the difficulties, these problems can be analyzed in terms of how model-building observers infer from measurements the computational capabilities embedded in nonlinear processes. An observer’s notion of what is ordered, what is random, and what is complex in its environment depends directly on its computational resources: the amount of raw measurement data, of memory, and of time available for estimation and inference. The discovery of structure in an environment depends more critically and subtlely, though, on how those resources are organized. The descriptive power of the observer’s chosen (or implicit) computational model class, for example, can be an overwhelming determinant in finding regularity in data. This paper presents an overview of an inductive framework — hierarchical-machine reconstruction — in which the emergence of complexity is associated with the innovation of new computational model classes. Complexity metrics for detecting structure and quantifying emergence, along with an analysis of the constraints on the dynamics of innovation, are outlined. Illustrative examples are drawn from the onset of unpredictability in nonlinear systems, finitary nondeterministic processes, and

### Citations

6858 |
A mathematical theory of communication
- Shannon
- 1948
(Show Context)
Citation Context ... deterministic complexity are well understood. If the object in question is a string � v of v discrete symbols produced by an information source, such as a Markov chain, with Shannon entropy rate � ",=-=[48]-=- then the growth rate of the Kolmogorov-Chaitin complexity is u � v v v3I � " (1) The growth rate � " is independent of the particular choice of universal machine. In the modeling framework it can be ... |

4052 |
Introduction to Automata Theory, Languages and Computation
- Hopcroft, Ullman
- 1979
(Show Context)
Citation Context ...e classification powers. Figure 2 includes more detail than is necessary for the following discussion, but it does demonstrate some of the diversity of computational mechanisms that have been studied.=-=[31]-=- 2DSA U DTIME(n^cn) cn DTIME(n ) c 2NEDSA DSPACE(nlogn) 2DLBA DSPACE(n) PDA CFG DPDA DCFL Strictly Sofic 2NSA cn U DTIME(2 ) 2 c 2NENSA NSPACE(n ) 2 1DSA NLBA/CSG NSPACE(n) 1NnSA ICFG RICFG 1NSA DFA/N... |

2985 |
Adpatation in Natural and Artificial Systems
- Holland
- 1975
(Show Context)
Citation Context .... That is, all of the “substrates” indicated in Figure 1 are thrown out, leaving only those defined in terms of information processing. This more or less follows the spirit of computational evolution.=-=[83,84]-=- The intention is to expunge as much disciplinary semantic content as possible so that if novel structure emerges, it does so neither by overt design nor interpretation but (i) via the dynamics of int... |

2613 |
The Structure of Scientific Revolutions
- Kuhn
- 1962
(Show Context)
Citation Context ...far as the architecture of information processing is concerned, these questions for natural language have direct analogues in adaptation, evolution, and even in the development of scientific theories.=-=[70]-=- While conflating these fields suggests a possible general theory of the emergence of complexity, at the same time it reveals an ignorance of fundamental distinctions. Due to the sheer difficulty of t... |

1234 |
Modelling by shortest data description
- Rissanen
- 1978
(Show Context)
Citation Context ...rms of the dependence of the entropy on the topological complexity. The result, a lower bound, is that r@vA ag H C��� P P vP0g H The curved solid line in Figure 6(a) shows the relevant portion of Eq. =-=(21)-=-. Comparing the periodic and chaotic analyses — i.e. Eqs. (20) and (21) — provides a detailed picture of the complexity-entropy phase transition. The critical entropy r™ at each sequence length v is g... |

1158 |
The Origins of Order. Self-Organization and Selection in Evolution
- Kauffman
- 1993
(Show Context)
Citation Context ...scussing “emergence”. In spite of receiving much popular attention, the proposals for “computation at the edge of chaos”,[71] “adaptation toward the edge of chaos”,[72] and “life at the edge of chaos”=-=[73]-=- are recent examples of the problems engendered by this sort of blurring of basic distinctions. Despite the preceding sections’ review of the complexity versus entropy spectrum and their analyses of p... |

920 | Flocks, herds, and schools: A distributed behavioral model
- REYNOLDS
- 1987
(Show Context)
Citation Context ...ver time from the interaction of simple subsystems. Flocks of birds flying in lockstep formation and schools of fish swimming in coherent array abruptly turn together with no leader guiding the group.=-=[2]-=- Ants form complex societies whoses2 J. P. Crutchfield survival derives from specialized laborers, unguided by a central director.[3] Optimal pricing of goods in an economy appears to arise from agent... |

864 |
Theory of Recursive Functions and Effective Computability
- Rogers
- 1967
(Show Context)
Citation Context ... symbol manipulation, there are the (Lindenmayer) parallel-rewrite[40] and queue-based[41,42] computational models. There are also the arithmetic and analytic hierarchies of recursive function theory.=-=[43]-=- The list of discrete computation hierarchies seems large because it is and needs to be to capture the distinct types of symbolic information processing mechanisms. Although the discrete computation h... |

761 |
The chemical basis of morphogenesis
- Turing
- 1952
(Show Context)
Citation Context ...the statistical order parameters describing phase transitions, the divergent correlations and long-lived fluctuations in critical phenomena,[13–15] and the forms appearing in biological morphogenesis.=-=[10,16,17]-=- Although the behavior in these systems is readily described as “coherent”, “self-organizing”, and “emergent”, the patterns which appear are detected by the observers and analysts themselves. The role... |

531 |
Stochastic complexity
- Rissanen
- 1987
(Show Context)
Citation Context ...the inclusion of probability, inductive inference, and spatial extent. Each of these extensions has received some attention in theoretical computer science, coding theory, and mathematical statistics.=-=[23,55]-=- Each plays a prominent role in one of the examples to come later. More immediately the extension to probabilistic computation gives a unified comparison of the deterministic and statistical complexit... |

404 |
Introduction to Phase Transitions and Critical Phenomena
- Stanley
- 1971
(Show Context)
Citation Context ... as its left eigenvector �ƒ„ a �ƒ in which �ƒ is the causal states’ asymptotic probability distribution: �ƒ aI. From this we ƒPƒ have an informational quantity for the machine’s size (12) g " a r@�ƒA =-=(13)-=- This is the statistical complexity. If, as provided by machine reconstruction, the machine is minimal, then g " is the amount of memory (in bits) required for the agent to predict the environment at ... |

369 |
Models of Biological Pattern Formation
- Meinhardt
- 1982
(Show Context)
Citation Context ...the statistical order parameters describing phase transitions, the divergent correlations and long-lived fluctuations in critical phenomena,[13–15] and the forms appearing in biological morphogenesis.=-=[10,16,17]-=- Although the behavior in these systems is readily described as “coherent”, “self-organizing”, and “emergent”, the patterns which appear are detected by the observers and analysts themselves. The role... |

323 |
On growth and form
- THOMPSON
- 1942
(Show Context)
Citation Context ...the statistical order parameters describing phase transitions, the divergent correlations and long-lived fluctuations in critical phenomena,[13–15] and the forms appearing in biological morphogenesis.=-=[10,16,17]-=- Although the behavior in these systems is readily described as “coherent”, “self-organizing”, and “emergent”, the patterns which appear are detected by the observers and analysts themselves. The role... |

322 |
An information measure for classification
- Wallace, Boulton
- 1968
(Show Context)
Citation Context ...oped in Ref. [56] to explain the emergence of high computational capability between the ordered and disordered regimes. For processes with r@vAav ` r™ the entropy and complexity are equivalent g" a r =-=(20)-=- This is shown as a solid straight line on the left portion of Figure 6(a). For processes with r@vAav b r™ the dependence of complexity on entropy is more interesting. In fact, the solution is given i... |

314 |
Inductive Inference: Theory and Methods
- Angluin, Smith
- 1983
(Show Context)
Citation Context ...the inclusion of probability, inductive inference, and spatial extent. Each of these extensions has received some attention in theoretical computer science, coding theory, and mathematical statistics.=-=[23,55]-=- Each plays a prominent role in one of the examples to come later. More immediately the extension to probabilistic computation gives a unified comparison of the deterministic and statistical complexit... |

287 | Elements of the Theory of Computation - Lewis, Papadimitriou - 1981 |

255 |
The Geometry of Biological Time
- Winfree
- 2001
(Show Context)
Citation Context ...more immediate, measures of the environment’s structure. The topological complexity g H of a process is simply given in terms of the minimal number of causal states in the agent’s model g H a ��� P ƒ =-=(11)-=- It is an upper bound on the amount of information needed to specify which state the environment is in. There is also a probabilistic version of the “counting” topological complexity. It is formulated... |

243 |
Self-organized criticality
- Bak, Tang, et al.
- 1988
(Show Context)
Citation Context ...ly appropriate balance of evolutionary forces; there is a reason why houseflies and humans coexist in the same environment. It turns out that � has a simpler interpretation. First, note that from Eq. =-=(14)-=-, it can be rewritten � a ��� 3H P g� " @ aPA ��� P P g� " @ A (14) (15)s26 J. P. Crutchfield Expanding this, one finds � a ��� 3H �P†@ aPA � H P†@ A �� ��� P where †@ A is the sets of states in w�@ A... |

239 | On the Length of Programs for Computing Finite Binary Sequences - Chaitin - 1969 |

191 |
Efficient capital markets: II
- Fama
- 1991
(Show Context)
Citation Context .... P. Crutchfield survival derives from specialized laborers, unguided by a central director.[3] Optimal pricing of goods in an economy appears to arise from agents obeying the local rules of commerce.=-=[4]-=- Even in less manifestly complicated systems emergent global information processing plays a key role. The human perception of color in a small region of a scene, for example, can depend on the color c... |

182 |
Computation at the edge of chaos: Phase transitions and emergent computation
- Langton
- 1991
(Show Context)
Citation Context ...e must be particularly focused on concrete results and established theory in discussing “emergence”. In spite of receiving much popular attention, the proposals for “computation at the edge of chaos”,=-=[71]-=- “adaptation toward the edge of chaos”,[72] and “life at the edge of chaos”[73] are recent examples of the problems engendered by this sort of blurring of basic distinctions. Despite the preceding sec... |

136 |
A closed curve is much more than an incomplete one: effect of closure in figure-ground segmentation
- Kovacs, Julesz
- 1993
(Show Context)
Citation Context ...pectral response of spatially-localized retinal detectors.[5,6] Similarly, the perception of shape can be enhanced by global topological properties, such as whether or not curves are opened or closed.=-=[7]-=- How does global coordination emerge in these processes? Are common mechanisms guiding the emergence across these diverse phenomena? What languages do contemporary science and mathematics provide to u... |

118 | The strategy of the genes - Waddington - 1957 |

116 |
Inferring statistical complexity
- Crutchfield, Young
- 1989
(Show Context)
Citation Context ...ne recent approach is to adapt and extend ideas from the theory of discrete computation, which has developed measures of information-processing structure, to inferring complexity in dynamical systems.=-=[24]-=- Computation theory defines the notion of a “machine” — a device for encoding the structures in discrete processes. It has been argued that, due to the inherent limitations of scientific instruments, ... |

105 |
Computational theory of cellular automata
- Wolfram
- 1984
(Show Context)
Citation Context ... seen by first noting that the intermediate states e and f have the asymptotic probabilities €�@eA a ��€�@�A a H I P and, by symmetry, €�@fA a I P . The two inverse iterates of �™, �6 a �™ �™ I P � P =-=(25)-=- , delimit the interval segments corresponding to the occurrence of e f pairs. These then give the four state transition probabilities, such as €�@e eA a €�@eeA €�@eA aP �0 H ��€�@�A (26)s36 J. P. Cru... |

104 | Revisiting the edge of chaos: Evolving cellular automata to perform com
- Hraber, Crutchfield
(Show Context)
Citation Context ... that natural language is very far and must stay away from any “edge of chaos” (�" a H). A critique of the first two “edge of chaos” proposals, which concern cellular automata, can be found elsewhere.=-=[74]-=- Aside from the technical issues discussed there, all three proposals exhibit a fatal conceptual difficulty. To the extent that they presume to address the emergence of complexity, with reference neit... |

103 |
The theory of critical phenomena
- Binney, Dowrick, et al.
- 1993
(Show Context)
Citation Context ...seflies and humans coexist in the same environment. It turns out that � has a simpler interpretation. First, note that from Eq. (14), it can be rewritten � a ��� 3H P g� " @ aPA ��� P P g� " @ A (14) =-=(15)-=-s26 J. P. Crutchfield Expanding this, one finds � a ��� 3H �P†@ aPA � H P†@ A �� ��� P where †@ A is the sets of states in w�@ A. Thus, is the information gain in going from one model to a more accura... |

102 |
A History of Western Philosophy
- Russell
- 1945
(Show Context)
Citation Context ...oing process. That C H Hs46 J. P. Crutchfield process is the subtlety in the problem of emergence. For Heraclitus, the persistence of change was the by-product of an “attunement of opposite tensions”.=-=[69]-=- Natural language, to take one everpresent example, has come to be complex because utility required it to be highly informative and comprehensibility demanded a high degree of structure. For the proce... |

88 | Computation at the Onset of Chaos
- Crutchfield, Young
- 1990
(Show Context)
Citation Context ... terms of a Turing machine that can guess. Figure 3 shows a probabilistic generalization — the Bernoulli-Turing machine (BTM) — to the basic Turing machine model of the discrete computation hierarchy.=-=[56]-=- The equivalent of the road map shown in Figure 2 is a “stochastic” computation hierarchy, which will be the subject of a later section. With the Bernoulli-Turing machine in mind, the deterministic an... |

85 | The arrival of the fittest”: Toward a theory of biological organization - Fontana, Buss - 1994 |

84 |
Toward a quantitative theory of self-generated complexity
- Grassberger
- 1986
(Show Context)
Citation Context ...s how estimates of the Shannon entropy rate converge to the true value �". The excess entropy has been recoined twice, first as the “stored information” and then as the “effective measure complexity”.=-=[53,54]-=- Statistical complexity itself was introduced in Ref. [24]. Since it makes an explicit connection with computation and with inductive inference, g" will be the primary tool used here for quantifying s... |

76 |
Adaptation toward the edge of chaos, in
- Packard
- 1988
(Show Context)
Citation Context ...results and established theory in discussing “emergence”. In spite of receiving much popular attention, the proposals for “computation at the edge of chaos”,[71] “adaptation toward the edge of chaos”,=-=[72]-=- and “life at the edge of chaos”[73] are recent examples of the problems engendered by this sort of blurring of basic distinctions. Despite the preceding sections’ review of the complexity versus entr... |

62 | Indexed grammars — an extension of context-free grammars - Aho - 1968 |

58 | What would be conserved if "the tape were played twice - Fontana, Buss - 1994 |

55 |
The Linguistics of DNA
- Searls
- 1992
(Show Context)
Citation Context ...al still to be gleaned from the Chomsky hierarchy. The recent proposal to use context-free grammars to describe nonlocal nucleotide correlations associated with protein folding is one example of this.=-=[44]-=- 2 Complexity Randomness The main goal here is to detect and measure structure in nature. A computational road map only gives a qualitative view of computational capability and so, within the reconstr... |

51 |
Emergent Computation: Self-organization, collective, and cooperative phenomena in natural and artificial computing networks." Physica 42D
- Forrest
- 1990
(Show Context)
Citation Context ...er additional functionality which supports global information processing, such as the setting of optimal prices. Recently, examples of this sort have fallen under the rubric of “emergent computation”.=-=[18]-=- The approach here differs in that it is based on explicit methods of detecting computation embedded in nonlinear processes. More to the point, the hypothesis in the following is that during intrinsic... |

51 |
Entropy and the complexity of trajectories of dynamical systems
- Brudno
- 1983
(Show Context)
Citation Context ...hey describe, we have u � v � v3I "v 3H (2) where the continuous variables are coarse-grained at resolution into discrete “measurement” symbols � HY IY PY FFFY 0� I and � is the state space dimension.=-=[49]-=- Thus, there are aspects of deterministic complexity that relate directly to physical processes. This line of investigation has led to a deeper (algorithmic) understanding of randomness in physical sy... |

50 |
The attractor-basin portrait of a cellular automaton
- Hanson, Crutchfield
- 1991
(Show Context)
Citation Context ...ovation, shown schematically in Table 1, leads to an enlarged view of the evolutionary dynamic. This can be described from the agent’s view in terms of hierarchical -machine reconstruction as follows.=-=[28,58]-=- 1. Start at the lowest level of the computational hierarchy by building stochastic finite automata via -machine reconstruction. There are, in fact, transitions over three levels implicit in the previ... |

48 | E.: Turbulent pattern bases for cellular automata
- Crutchfield, Hanson
- 1993
(Show Context)
Citation Context ...resettable counter register,[59] at the onset of chaos as the innovation of string productions,[56] and in discrete spatial processes as the innovation of regular domains, domain walls, and particles.=-=[60]-=- It is also seen in the innovation of local state machines to break away from cellular automata look-up table representations; an example of this can be found elsewhere.[29] In each case it is quite s... |

44 | Equations of motion from a data series
- Crutchfield, McNamara
- 1987
(Show Context)
Citation Context ...— i.e. Eqs. (20) and (21) — provides a detailed picture of the complexity-entropy phase transition. The critical entropy r™ at each sequence length v is given P 0I r™@vA ag H @vA C ��� P ˜� P 0I (21) =-=(22)-=-sThe Calculi of Emergence 29 where gH@vA a ���Pv ���P ���P � is the complexity on the high entropy side at r™, � PXISSSQS is the solution of � ��� � � CP � 0I Q a H, and I ˜ Q P0 P IXHTHTT is a consta... |

43 |
The Dripping Faucet as a Model Chaotic System
- Shaw
- 1984
(Show Context)
Citation Context ...s how estimates of the Shannon entropy rate converge to the true value �". The excess entropy has been recoined twice, first as the “stored information” and then as the “effective measure complexity”.=-=[53,54]-=- Statistical complexity itself was introduced in Ref. [24]. Since it makes an explicit connection with computation and with inductive inference, g" will be the primary tool used here for quantifying s... |

39 |
Symbolic dynamics of noisy chaos
- Crutchfield, Packard
- 1983
(Show Context)
Citation Context ...ses the minimal representation w ���@�A.[51] Introduced as a practical alternative to the uncomputable logical depth, the “excess entropy” measures how an agent learns to predict successive bits of �.=-=[52]-=- It describes how estimates of the Shannon entropy rate converge to the true value �". The excess entropy has been recoined twice, first as the “stored information” and then as the “effective measure ... |

38 |
Transition from a uniform state to hexagonal striped Turing patterns. Nature 352
- OUYANG, SWINNEY
- 1991
(Show Context)
Citation Context ...ate probabilities as its left eigenvector �ƒ„ a �ƒ in which �ƒ is the causal states’ asymptotic probability distribution: �ƒ aI. From this we ƒPƒ have an informational quantity for the machine’s size =-=(12)-=- g " a r@�ƒA (13) This is the statistical complexity. If, as provided by machine reconstruction, the machine is minimal, then g " is the amount of memory (in bits) required for the agent to predict th... |

38 |
Universal behavior in nonlinear systems
- Feigenbaum
- 1983
(Show Context)
Citation Context ...t of chaos leads to an infinite machine. This is consonant with the view introduced by Feigenbaum that this onset of chaos can be viewed as a phase transition at which the correlation length diverges.=-=[61]-=- The computational analog of the latter is that the process intrinsically has an infinite memory capacity. But there is more that the computational analysis yields. As will now be shown, for example, ... |

36 |
J.: Quasiperiodicity in dissipative systems: a renormalization group analysis
- Feigenbaum, Kadanoff, et al.
- 1982
(Show Context)
Citation Context ...he logistic map, there is a signature to the path by which chaotic behavior is approached from periodic behavior. Furthermore, the circle map’s signature has the basic character of a phase transition.=-=[62]-=- The following will investigate one arc through @3Y�A-space that exhibits just such a phase transition to chaos. This is a path that includes the golden mean circle map — so-called since its winding n... |

36 |
Chance and Necessity: Essay on the Natural Philosophy of Modern Biology
- Monod
- 1977
(Show Context)
Citation Context ...ation.[75] The second, anarchistic camp consists of the Historicists who hold fast to the Darwinian mechanisms of selection and variation, but emphasize the accidental determinants of biological form.=-=[76,77]-=- What distinguishes this position from the Selectionists is the claim that major changes in structure can be and have been nonadaptive. Lastly, there are the Structuralists whose goal is to elucidate ... |

33 |
The retinex
- Land
- 1964
(Show Context)
Citation Context ...uman perception of color in a small region of a scene, for example, can depend on the color composition of the entire scene, not just on the spectral response of spatially-localized retinal detectors.=-=[5,6]-=- Similarly, the perception of shape can be enhanced by global topological properties, such as whether or not curves are opened or closed.[7] How does global coordination emerge in these processes? Are... |

31 |
Identifiability of hidden Markov information sources and their minimum degrees of freedom
- Amari, H, et al.
- 1992
(Show Context)
Citation Context ...e late 1950’s.[64] In fact, many questions about this class are very difficult. It is only recently, for example, that a procedure for determining the equivalence of two such processes has been given.=-=[65]-=- The awareness that this problem area bears on the complexity of observation and the result that finite complexity processes can appear infinitely complex is also recent.[59] The new notion of state h... |

27 |
E.: Computational Mechanics of Cellular Automata
- Hanson
- 1993
(Show Context)
Citation Context ...om an initial arbitrary pattern on a lattice of 200 sites. (b) The filtered version of the same space-time diagram that reveals a multiplicity of different particle types and interactions. (From Ref. =-=[67]-=-. Reprinted with permission of the author. Cf. [68].) -machine reconstruction at this “particle” level to find further regularity or stochasticity and ultimately to obtain a finite description. These ... |

26 |
Dissipation, information, computational complexity and the definition of organization
- Bennett
- 1985
(Show Context)
Citation Context ...introduced in order to capture the properties of statistical complexity. The “logical depth” of �, one of the first proposals, is the run time of the UTM that uses the minimal representation w ���@�A.=-=[51]-=- Introduced as a practical alternative to the uncomputable logical depth, the “excess entropy” measures how an agent learns to predict successive bits of �.[52] It describes how estimates of the Shann... |