Defining structure and detecting the emergence of complexity in nature are inherently subjective, though essential, scientific activities. Despite the difficulties, these problems can be analyzed in terms of how model-building observers infer from measurements the computational capabilities embedded in nonlinear processes. An observer’s notion of what is ordered, what is random, and what is complex in its environment depends directly on its computational resources: the amount of raw measurement data, of memory, and of time available for estimation and inference. The discovery of structure in an environment depends more critically and subtlely, though, on how those resources are organized. The descriptive power of the observer’s chosen (or implicit) computational model class, for example, can be an overwhelming determinant in finding regularity in data. This paper presents an overview of an inductive framework — hierarchical-machine reconstruction — in which the emergence of complexity is associated with the innovation of new computational model classes. Complexity metrics for detecting structure and quantifying emergence, along with an analysis of the constraints on the dynamics of innovation, are outlined. Illustrative examples are drawn from the onset of unpredictability in nonlinear systems, finitary nondeterministic processes, and

We introduce a new model of evolution on a fitness landscape possessing a tunable degree of neutrality. The model allows us to study the general properties of molecular species undergoing neutral evolution. We find that a number of phenomena seen in RNA sequence-structure maps are present also in our general model. Examples are the occurrence of “common ” structures which occupy a fraction of the genotype space which tends to unity as the length of the genotype increases, and the formation of percolating neutral networks which cover the genotype space in such a way that a member of such a network can be found within a small radius of any point in the space. We also describe a number of new phenomena which appear to be general properties of neutrally evolving systems. In particular, we show that the maximum fitness attained during the adaptive walk of a population evolving on such a fitness landscape increases with increasing degree of neutrality, and is directly related to the fitness of the most fit percolating network. 1

This brief essay reviews an approach to defining and then detecting the emergence of complexity in nonlinear processes. It is, in fact, a synopsis of Reference [1] that leaves out the technical details in an attempt to clarify the motivations behind the approach. The central puzzle

We present a systematic approach to mean field theory (MFT) in a general probabilistic setting without assuming a particular model. The mean field equations derived here may serve as a local and thus very simple method for approximate inference in probabilistic models such as Boltzmann machines or Bayesian networks. "Model-independent" means that we do not assume a particular type of dependencies; in a Bayesian network, for example, we allow arbitrary tables to specify conditional dependencies. In general, there are multiple solutions to the mean field equations. We show that improved estimates can be obtained by forming a weighted mixture of the multiple mean field solutions. Simple approximate expressions for the mixture weights are given. The general formalism derived so far is evaluated for the special case of Bayesian networks. The benefits of taking into account multiple solutions are demonstrated by using MFT for inference in a small and in a very large Bayesian network. The results are compared to the exact results.

In this article, we present a self-contained review of recent work on complex biological systems which exhibit no characteristic scale. This property can manifest itself with fractals (spatial scale invariance), flicker noise or 1}f-noise where f denotes the frequency of a signal (temporal scale invariance) and power laws (scale invariance in the size and duration of events in the dynamics of the system). A hypothesis recently put forward to explain these scale-free phenomomena is criticality, a notion introduced by physicists while studying phase transitions in materials, where systems spontaneously arrange themselves in an unstable manner similar, for instance, to a row of dominoes. Here, we review in a critical manner work which investigates to what extent this idea can be generalized to biology. More precisely, we start with a brief introduction to the concepts of absence of characteristic scale (power-law distributions, fractals and 1}fnoise) and of critical phenomena. We then review typical mathematical models exhibiting such properties : edge of chaos, cellular automata and self-organized critical models. These notions are then brought together to see to what extent they can account for the scale invariance observed in ecology, evolution of species, type III epidemics and some aspects of the central nervous system. This article also discusses how the notion of scale invariance can give important insights into the workings of biological systems.

We study the four-state antiferromagnetic Potts model on the triangular lattice. We show that the model has six types of defects which diuse and annihilate according to certain conservation laws consistent with their having a vector-valued topological charge. Using the properties of these defects, we deduce a 2 + 2-dimensional height representation for the model and hence show that the model is equivalent to the three-state Potts antiferromagnet on the Kagome lattice and to bond-coloring models on the triangular and honeycomb lattices. We also calculate critical exponents for the ground state ensemble of the model. We nd that the exponents governing the spin{spin correlation function and spin uctuations violate the Fisher scaling law because of constraints on path length which increase the eective wavelength of the spin operator on the height lattice. We conrm our predictions by extensive Monte Carlo simulations of the model using the Wang-SwendsenKoteck y cluster a...

‘Racial’ disparities among cancers, particularly of the breast and prostate, are something of a mystery. For the US, in the face of slavery and its sequelae, centuries of interbreeding has greatly leavened genetic differences between ‘Blacks ’ and ‘whites’, but marked contrasts in disease prevalence and progression persist. ‘Adjustment’ for socioeconomic status and lifestyle, while statistically accounting for much of the variance in breast cancer, only begs the question of ultimate causality. Here we propose a more basic biological explanation that

Extending recent information-theoretic phase transition approaches to evolutionary and cognitive process via the Rate Distortion Theorem in the circumstance of interaction with a structured environment suggests that learning plateaus in cognitive systems and punctuated equilibria in evolutionary process are formally analogous, even though evolution is most certainly not cognitive. The result is curiously direct, and implies that evolutionary theories which do not produce punctuation are likely to be seriously incomplete.

We compare phase transition(-like) phenomena in small model systems for both microcanonical and canonical ensembles. The model systems correspond to a few classical (non-quantum) point particles confined in a one-dimensional box and interacting via Lennard-Jones-type pair potentials. By means of these simple examples it can be shown already that the microcanonical thermodynamic functions of a small system may exhibit rich oscillatory behavior and, in particular, singularities (nonanalyticities) separating different microscopic phases. These microscopic phases may be identified as different microphysical dissociation states of the small system. The microscopic oscillations of microcanonical thermodynamic quantities (e.g. temperature, heat capacity, or pressure) should in principle be observable in suitably designed evaporation/dissociation experiments (which must realize the physical preconditions of the microcanonical ensemble). By contrast, singular phase transitions cannot occur, if a small system is embedded into an infinite heat bath (thermostat), corresponding to the canonical ensemble. For the simple model systems under consideration, it is nevertheless possible to identify a smooth canonical phase transition by studying the distribution of complex zeros of the canonical partition function. Key words: microscopic phase transitions, small systems, Lennard-Jones chains

A new general framework is presented for implementing complex a priori knowledge, having in mind especially situations where the number of available training data is small compared to the complexity of the learning task. A priori information is hereby decomposed into simple components represented by quadratic building blocks (quadratic concepts) which are then combined by conjunctions and disjunctions to built more complex, problem specific error functionals. While conjunction of quadratic concepts leads to classical quadratic regularization functionals, disjunctions, representing ambiguous priors, result in non-convex error functionals. These go beyond classical quadratic regularization approaches and correspond, in Bayesian interpretation, to non-gaussian processes. Numerical examples show that the resulting stationarity equations, despite being in general nonlinear, inhomogeneous (integro-)differential equations, are not necessarily difficult to solve. Appendix A relates the formalism of statistical