## Large Scale Simulations of Complex Systems Part I: Conceptual Framework (1997)

### Cached

### Download Links

### BibTeX

@MISC{Sloot97largescale,

author = {P. M. A. Sloot and A. Schoneveld and J. F. De Ronde and J. A. Ka},

title = {Large Scale Simulations of Complex Systems Part I: Conceptual Framework},

year = {1997}

}

### OpenURL

### Abstract

In this working document, we report on a new approach to high performance simulation. The main inspiration to this approach is the concept of complex systems: disparate elements with well defined interactions rules and non nonlinear emergent macroscopic behavior. We provide arguments and mechanisms to abstract temporal and spatial locality from the application and to incorporate this locality into the complete design cycle of modeling and simulation on parallel architectures. Although the main application area discussed here is physics, the presented Virtual Particle (VIP) paradigm in the context of Dynamic Complex Systems (DCS), is applicable to other areas of compute intensive applications. Part I deals with the concepts behind the VIP and DCS models. A formal approach to the mapping of application task-graphs to machine task-graphs is presented. The major part of section 3 has recently (July 1997) been accepted for publication in Complexity. In Part II we will elaborate on the execution behavior of

### Citations

11418 |
Computers and Intractability: A Guide to the Theory of NP -completeness
- Garey, Johnson
- 1979
(Show Context)
Citation Context ...e is able to mimic the computation of an arbitrary language in NP on a 69non-deterministic TM in polynomial time. The first problem for which this was done is the so-called Satisfiability citeBovet94=-=[33]-=-. In short, to show that every language in NP is reducible to Satisfiability, for each non-deterministic TMM that is time bounded by a polynomialp(n), construct a polynomial-time algorithm that takes ... |

6983 |
The Mathematical Theory of Communication
- SHANNON, WEAVER
- 1949
(Show Context)
Citation Context ... evolution, quantitative order parameters are needed. These order parameters need to distinguish between ordered and disordered states. A commonly used quantity for this purpose is the Shannon entropy=-=[96]-=-, defined on a discrete probability distributionpi: (1) HN=�kNXi=1pjlogpj This measureHcan be associated with the degree of uncertainty about a system. In a Cellular Automata, entropy can be defined o... |

3888 | Optimization by simulated annealing
- Kirkpatrick, Gelatt, et al.
- 1983
(Show Context)
Citation Context ...n problems. One approach to solve these problems, is to use numerical methods, another attractive approach is to use stochastic or natural solvers. Two typical natural solvers are Simulated Annealing =-=[52]-=- and Genetic Algorithms [43]. A fundamental problem is that both methods are difficult to parallelize to a level of high scalability. Classical GAs use global knowledge for their selection process. Th... |

3005 |
Adaptation in natural and artificial systems
- Holland
- 1975
(Show Context)
Citation Context ...solve these problems, is to use numerical methods, another attractive approach is to use stochastic or natural solvers. Two typical natural solvers are Simulated Annealing [52] and Genetic Algorithms =-=[43]-=-. A fundamental problem is that both methods are difficult to parallelize to a level of high scalability. Classical GAs use global knowledge for their selection process. There does not exist a spatial... |

2491 |
Equations of state calculations by fast computing machines
- Metroplis, Rosenbluth, et al.
- 1953
(Show Context)
Citation Context ...h of the phase space of a TAP instance. 3.1.12 Simulated Annealing For Optima Search In simulated physical systems, configurations at thermal equilibrium can be sampled using the Metropolis algorithm =-=[70]-=-. The determination of the loca37tion of the critical temperature, can be established by sampling at fixed temperatures over some temperature range. In the case of task allocation we are not interest... |

2026 |
Genetic algorithms in search, optimization and machine learning
- Goldberg
- 1989
(Show Context)
Citation Context ...dard GA using functionf4from Eq. 102. dard GA using functionf5from Eq. 103. 3.2.4 Experiments on the Convergence Behavior of C-GA In order to compare the convergence behavior of C-GA to a standard GA =-=[34]-=- with elitist strategy. It is known that standard GA algorithms are sensitive to f4(x)=30X1ixi4+Gauss(0;1)for�1:28xi1:28 the shape of the search space. To reduce the influence of shape dependency we u... |

783 | A fast algorithm for particle simulations
- Greengard, Rokhlin
- 1987
(Show Context)
Citation Context ...f VIPs is applied is inN-body problems, where the N2number of long-range interactions between the particles can be approximated byNlogNinteractions through the use of hierarchical tree methods[3] [6] =-=[38]-=-. In the hierarchical algorithm the overall complexity of the problem is reduced, which allows for the simulation of relatively larger problems, while the information about the error introduced by thi... |

632 |
The Origins of Order
- Kauffman
- 1993
(Show Context)
Citation Context ... system corresponds to a first order transition. A sudden, but continuous, change corresponds to a second order transition. At such a transition, the system is in a critical state. By some authors[57]=-=[50]-=- it is believed that the resulting infinite correlations can be interpreted as long-term memory needed to store information. We will review these notions briefly in section 2.1.2. Some non-equilibrium... |

479 |
Computation: Finite and Infinite Machines
- Minsky
- 1967
(Show Context)
Citation Context ...d by examining the dynamical behavior of a well known computational device: The Turing Machines (TM). A class of 7-state 4-symbol Turing machines, which also includes Minsky's universal Turing machine=-=[72]-=-, was used to address the question whether universal computation is found between order and chaos. A large number of randomly created TM's was used to generate three different sequences: a sequence of... |

416 |
Error-detecting and error-correcting codes
- Hamming
- 1950
(Show Context)
Citation Context ...ndex of a sequence letter corresponds to a task ID. The distance is given by the number of positions in which two sequencesAandB pair(A;B) differ; this metric distance measure is the Hamming distance =-=[39]-=-d(A;B). The Hamming graph can be constructed by connecting every sequence that hasd(A;B)=1. The number of configurations with a given distancedfrom an arbitrary reference pointN(P;n;d), the total numb... |

390 |
On a theory of computation and complexity over the real numbers: NP-completeness, recursive functions and universal machines
- Blum, Shub, et al.
- 1989
(Show Context)
Citation Context ...the Julia Set, which is the closure of the unstable equilibrium set of the following complex mapping: (124) It is undecidable whether a given point in the complex plane is an element of the Julia set =-=[9]-=-, it is however possible to just simulate the mapping and observe the membership of a certain point. 6.4 Computability: Cellular Automata Even in systems with a very simple structure, it is possible t... |

249 |
Self-organized criticality
- Bak, Tang, et al.
- 1988
(Show Context)
Citation Context ...ded for the chain reaction to stop or the total number of sites involved in the avalanche. This phenomenon of spontaneously developing critical scaling has been called Self Organized Criticality (SOC)=-=[5]-=-. A lot of naturally occurring systems exhibit this kind of scaling- or self-similar behavior. The basic model with which SOC behavior is demonstrated, is with the use of a special kind of Cellular Au... |

248 |
Universality and complexity in cellular automata
- Wolfram
- 1984
(Show Context)
Citation Context ...mputational complexity is given and we discuss NP-complete, PSPACE-complete and P-complete problems. A question to ask at this point is: ”When does a system display complex behavior?”. By some authors=-=[113]-=-[57][15] it is believed that when a system display complex behavior, universal computations can be performed. Mechanically speaking a computational system requires transmission, storage and modificati... |

245 |
Guy,Winning ways for your mathematical plays
- Berlekamp, Conway, et al.
- 1982
(Show Context)
Citation Context ... is merely meant as a proof of existence for computing universal cellular automata and differs from traditional universal CA proofs, in which usually logic gates acting on input signals are identified=-=[7]-=-. A Turing Machine works on a infinite tape on which symbols from an alphabet can be written. The TM can be in one of the states of a set�. Furthermore a TM executes instructions using a transition ta... |

243 | Theory and applications of cellular automata - Wolfram - 1986 |

183 |
Computation at the Edge of Chaos: Phase Transitions and Emergent
- Langton
- 1990
(Show Context)
Citation Context ...ly, behavior of the complex-type has been identified in dynamical systems[74][86][45][85]. It is assumed that dynamical systems which display complex behavior are capable of universal computation[113]=-=[58]-=-. In the previous section of this paper, some motivation supporting this system characteristic was given. This section further concentrates on CA, as a type of physical model with the ability to perfo... |

171 |
Lattice-gas automata for the navier-stokes equation
- Frisch, Hasslacher, et al.
- 1986
(Show Context)
Citation Context ...fferential equations b) molecular dynamics c) a lattice gas (after [56]) A very simple lattice gas method with which an incompressible fluid in 2D can be modeled is the FHP Lattice Gas Automata (LGA) =-=[31]-=-. In this method the fluid is represented by “particles” which can move from a lattice site to neighboring lattice site. The particles reside on a triangular lattice and are located in the centers, th... |

152 |
Fine-Grained Parallel Genetic Algorithms
- Manderick, Spiessens
- 1989
(Show Context)
Citation Context ...nherently sequential. Our approach to parallelize both methods is to introduce locality by using a mapping to a Cellular Automata. Examples in which a GA is mapped on a Cellular Automata are given in =-=[65]-=-, [36], and [107]. In the general case it is not possible to map SA on a Cellular Automata. However locality can be imposed to SA by applying a population based algorithm [35]. In simultaneous indepen... |

148 | An efficient program for many-body simulation
- Appel
- 1985
(Show Context)
Citation Context ...tering of VIPs is applied is inN-body problems, where the N2number of long-range interactions between the particles can be approximated byNlogNinteractions through the use of hierarchical tree methods=-=[3]-=- [6] [38]. In the hierarchical algorithm the overall complexity of the problem is reduced, which allows for the simulation of relatively larger problems, while the information about the error introduc... |

147 |
Correlated and Uncorrelated Fitness Landscapes and How to Tell the Difference
- Weinberger
- 1990
(Show Context)
Citation Context ...n the TAP. In section 3.1.11 the following experimental methods are presented: Simulated Annealing (SA) [53], for finding optima, and Weinberger correlation for phase space structure characterization =-=[110]-=-. In section 3.1.15 experimental results are presented, which are discussed in section 3.1.20. Finally, some concluding remarks and directions for future work are given in section 3.1.23. 3.1.2 Applic... |

125 |
Studying artificial life with cellular automata
- Langton
- 1986
(Show Context)
Citation Context ...tional complexity is given and we discuss NP-complete, PSPACE-complete and P-complete problems. A question to ask at this point is: ”When does a system display complex behavior?”. By some authors[113]=-=[57]-=-[15] it is believed that when a system display complex behavior, universal computations can be performed. Mechanically speaking a computational system requires transmission, storage and modification o... |

116 |
Inferring statistical complexity
- Crutchfield, Young
- 1989
(Show Context)
Citation Context ...in the thermodynamic limit. Other evidence for the ”Edge of chaos” hypothesis can be found in the work of Crutchfield on continuous dynamical systems[14] and the resulting -machine reconstruction. In =-=[16]-=- the so called intrinsic computation abilities of a continuous dynamical system are investigated. The output of the system (e.g. an iterative map,xn+1=f(xn)) in time is coarse grained into a sequence ... |

116 |
ASPARAGOS - An Asynchronous Parallel Genetic Optimization Strategy
- Gorges-Schleuter
- 1989
(Show Context)
Citation Context ...tly sequential. Our approach to parallelize both methods is to introduce locality by using a mapping to a Cellular Automata. Examples in which a GA is mapped on a Cellular Automata are given in [65], =-=[36]-=-, and [107]. In the general case it is not possible to map SA on a Cellular Automata. However locality can be imposed to SA by applying a population based algorithm [35]. In simultaneous independent s... |

101 | The evolution of emergent computation
- Crutchfield, Mitchell
- 1995
(Show Context)
Citation Context ...al complexity is given and we discuss NP-complete, PSPACE-complete and P-complete problems. A question to ask at this point is: ”When does a system display complex behavior?”. By some authors[113][57]=-=[15]-=- it is believed that when a system display complex behavior, universal computations can be performed. Mechanically speaking a computational system requires transmission, storage and modification of in... |

98 | Introduction to the Theory of Complexity
- Bovet, Crescenzi
- 1993
(Show Context)
Citation Context ...polynomial time non-deterministic TM. Furthermore, a language or a problem belongs to NP if and only if a given string or solution can be checked or verified in polynomial deterministic time, see also=-=[10]-=-. We can now define an NP-complete problemL`2NPas a problem to which every other problemL2NPcan be polynomially reduced. A problem Lcan be polynomially reduced to another problemL`if there exists a po... |

95 |
Unpredictability and Undecidability in Dynamical Systems
- Moore
- 1990
(Show Context)
Citation Context ...computation is called computationally irreducible [114]. Using computation- and complexity theory it can be shown that such systems have a stronger kind of unpredictability than chaotical systems[114]=-=[74]-=-. We can reformulate this notion in a particular physical version of the ChurchTuring Hypothesis: “Universal computers are as powerful in their computational capabilities as any physically realizable ... |

86 |
The algorithmic beauty of sea shells
- Meinhardt
- 1995
(Show Context)
Citation Context ...ombination of flow and diffusion, plays a fundamental role. An example is morphogenesis where the external distribution of nutrients [32] [47], as well as the internal distribution of morphogens [75] =-=[69]-=- by reactiondiffusion in the organism, can induce certain growth patterns. The Lattice Boltzmann method is especially suitable for modeling diffusion and flow phenomena in biological systems with high... |

76 |
Lattice gas hydrodynamics in two and three dimensions
- Frisch, d'Humieres, et al.
- 1987
(Show Context)
Citation Context ...y , and the pressurepare scaled with the factorg()(t0=g()t, 0==g()), the equation transforms into the Navier Stokes equation. 4.2.2 Lattice gases in 3D There also exist 3D extensions of the FHP-model =-=[30]-=-. In the 3D extension, the Face Centered hypercube (FCHC) geometry, the particles reside on a cubic lattice (see Fig. 33) where each node is connected with 24 other nodes. Along the dotted links betwe... |

76 |
Adaptation toward the edge of chaos
- Packard
- 1988
(Show Context)
Citation Context ...ng Minsky's universal TM. Mitchell et al, reviewed this idea of computation at the 'edge of chaos' and reported on experiments producing very different results from the original experiment by Packard =-=[80]-=-, they suggest that the interpretation of the original results is not correct [73]. 2.1.3 Computational Behavior as an Attractor: Self Organized Criticality Most of the time, equilibrium systems with ... |

66 | Mutual information functions versus correlation functions
- Li
- 1990
(Show Context)
Citation Context ...nd variable by knowing the first. Another definition of mutual information, =XXP(d)logP(d)�XXP(d)logPlogP PP block-to-block mutual information, is defined as the mutual information between twoL-blocks=-=[59]-=-. LetPbe the probability for anL-block andP(d)be the joint probability for two blocks separated by a distanced.: M(d)[L]=XXP(d)logP(d)�XPlogP�XPlogP (7) =H+H�H wherekis the number of states in a confi... |

65 |
Numerical Simulations of Particulate Suspensions via a Discretized Boltzmann-Equation. 1. Theoretical Foundation
- Ladd
(Show Context)
Citation Context ...Figure 29: Three different approaches to model the fluid surrounding the obstacles: the fluid is modeled using a) a set of partial differential equations b) molecular dynamics c) a lattice gas (after =-=[56]-=-) A very simple lattice gas method with which an incompressible fluid in 2D can be modeled is the FHP Lattice Gas Automata (LGA) [31]. In this method the fluid is represented by “particles” which can ... |

60 |
Simulated Annealing: Parallelization Techniques
- Azencott
- 1992
(Show Context)
Citation Context ...07]. In the general case it is not possible to map SA on a Cellular Automata. However locality can be imposed to SA by applying a population based algorithm [35]. In simultaneous independent searches =-=[4]-=- basically the same method is used without interactions. In [1] a generic algorithm, the so-called Abstract Genetic Algorithm, for both SA and GA was introduced. The AGA however was not designed to fa... |

58 |
A note on boltzmann tournament selection for genetic algorithms and population–oriented simulated annealing
- Goldberg
- 1990
(Show Context)
Citation Context ...ar Automata are given in [65], [36], and [107]. In the general case it is not possible to map SA on a Cellular Automata. However locality can be imposed to SA by applying a population based algorithm =-=[35]-=-. In simultaneous independent searches [4] basically the same method is used without interactions. In [1] a generic algorithm, the so-called Abstract Genetic Algorithm, for both SA and GA was introduc... |

52 | Gödel’s theorem and information
- Chaitin
- 1982
(Show Context)
Citation Context ...n. In turn Turing showed that there is no algorithm for deciding whether or not a program ever halts, i.e. the halt68ing problem is unsolvable. One can derive Gödel's theorem from the halting problem=-=[11]-=- Using a technique called diagonalisation it is possible to construct a language that is not accepted by any Turing Machine [44]. In this technique an undecidable language is constructed that is not i... |

48 |
Global convergence of genetic algorithms: A markov chain analysis
- Eiben, Aarts, et al.
- 1991
(Show Context)
Citation Context ...near diffusion equation. If the system has relaxed, another random perturbation can made. We have repeated the experiments with a continuous version of this model where randomly chosen energy quantah(=-=[0;1]-=-)are added to a randomly chosen site. If some site reaches the critical energy value1:0, the system is relaxed until every site is below this critical value. At some point the system reaches a station... |

45 |
Nonlinear systems
- DRAZIN
- 1992
(Show Context)
Citation Context ...e we are confronted with a kind of unpredictability: it is not possible to predict the future accurately, just because slight discrepancies in the initial configuration will grow exponentially in time=-=[22]-=-. This implies that, to accurately predict the systemtsteps in the future, we need approximatelytdigits in the specification of the initial configuration. So, the amount of information needed to predi... |

45 |
The Landscape of the Traveling Salesman Problem
- Stadler, Schnabl
- 1992
(Show Context)
Citation Context ...aying exponential, with correlation length . Such landscapes are classified as AR(1) landscapes and have been identified in various fields, e.g. (Bio)physics [110] and combinatorial optimization [103]=-=[104]-=-. It has been shown that incremental search methods like Simulated Annealing perform optimally on landscapes that show a self-similar structure [102]. We will derive expressions for the relaxation and... |

41 | Using deep structure to locate hard problems
- Williams, Hogg
- 1992
(Show Context)
Citation Context ...ation gives a rough estimate of the transition region. 33Many search methods have been shown to behave anomalytically for certain critical parameters of the instance of combinatorial search problems =-=[111]-=- (critical slowing down). We speculate on the existence of such an anomaly (often observable as a sudden increase in the search cost) in the spectrum of TAP instances. 3.1.8 Spin Glasses and Graph bi-... |

33 | Criticality and parallelism in combinatorial optimization. Science 261
- Macready, Siapas, et al.
- 1996
(Show Context)
Citation Context ...ate the effects of introducing locality in both the task graph as well as the processor topology. Also, we will study the critical behavior in more detail, by means of finite size scaling experiments =-=[64]-=-. Furthermore, our interest goes out to a thorough understanding of task allocation in dynamic and heterogeneous parallel applications and machines, in e.g. cluster computing [79]. 453.2 An Abstract ... |

31 |
On the Complexity of Scheduling Problems for Parallel/Pipelined Machines
- Bernstein, Rodeh, et al.
- 1989
(Show Context)
Citation Context ...s requires knowledge of the shape of the phase space. Great care has to be taken in selecting an optimization method, since searching the optimal solution to the TAP is known to be an NP-hard problem =-=[8]-=-. Hence, a study on the structure of the landscape of the TAP is necessary in order to identify effective optimization methods. Furthermore, the sensitivity of the TAP to a small set of machine and ap... |

30 | Dynamics, Computation, and the 'Edge Of Chaos': A Re-Examination
- Mitchell, Crutchfield, et al.
- 1994
(Show Context)
Citation Context ... 'edge of chaos' and reported on experiments producing very different results from the original experiment by Packard [80], they suggest that the interpretation of the original results is not correct =-=[73]-=-. 2.1.3 Computational Behavior as an Attractor: Self Organized Criticality Most of the time, equilibrium systems with short-range interactions, exhibit exponentially decaying correlations. Infinite co... |

28 | Cellular automata as a paradigm for ecological modeling
- Hogeweg
- 1988
(Show Context)
Citation Context ...encing scheme falls within the formal definition of a cellular automaton and can be indicated as a computational CA for an independently formulated model represented by partial differential equations =-=[41]-=-. This is a fundamental difference with the lattice gas methods (LGA and LBE) where the problem is directly modeled using a cellular automaton. By applying mass and momentum conserving rules in the CA... |

24 |
Lattice Gas Method for Partial Differential Equations
- Doolen
- 1990
(Show Context)
Citation Context ...rtifact can be partially avoided by taking spatial averages for the velocities. In practice the average velocityuin a 2D lattice gas model is determined in clusters with a minimal size of 64 64 nodes =-=[21]-=-. The limited range of velocities also restricts the allowed range of Reynolds numbers which can be simulated with the lattice gas method. Due to the large scale separations it is not necessary to use... |

24 |
Undecidability and intractability in theoretical physics
- Wolfram
- 1985
(Show Context)
Citation Context ...blem to which every other problem in PSPACE can be reduced, butL is not necessarily in PSPACE. 6.2 Difficulty of Predicting Behavior in Complex Systems Physical processes can be viewed as computations=-=[114]-=-, where the difficulty of answering questions about these processes is equivalent to performing the corresponding computations. In principle the behavior of many complex system can be calculated by ex... |

21 |
The parallel genetic cellular automata: Application to global function optimization
- Tomassini
- 1993
(Show Context)
Citation Context ...tial. Our approach to parallelize both methods is to introduce locality by using a mapping to a Cellular Automata. Examples in which a GA is mapped on a Cellular Automata are given in [65], [36], and =-=[107]-=-. In the general case it is not possible to map SA on a Cellular Automata. However locality can be imposed to SA by applying a population based algorithm [35]. In simultaneous independent searches [4]... |

20 | A Dynamic Load Balancing System for Parallel Cluster Computing. Future Generation Computer Systems
- Overeinder, Sloot, et al.
- 1996
(Show Context)
Citation Context ... scaling experiments [64]. Furthermore, our interest goes out to a thorough understanding of task allocation in dynamic and heterogeneous parallel applications and machines, in e.g. cluster computing =-=[79]-=-. 453.2 An Abstract Cellular Genetic Algorithm for solving optimization problems Many problems from the natural sciences can be considered as optimization problems. One approach to solve these proble... |

19 | Applications of statistical mechanics to combinatorial search problems
- Hogg
- 1995
(Show Context)
Citation Context ... optimal solutions. 3.1.13 Search Cost Estimation In comparable (NP-hard) problems the computational cost of determining the (optimal) solutions shows a dependence on problem specific parameters [112]=-=[42]-=-[12]. For example, in the case of graph coloring it has been observed that the “difficulty” of determining if a graph can be be colored, increases abruptly when the average connectivity in the graph i... |

19 | Elements of a Theory of Simulation
- Rasmussen, Barrett
(Show Context)
Citation Context ...ing the property we can simulate the system and “watch” whether a specific property emerges. Even non-computable problems can be “solved” by simulation, just by watching the system evolve. An example =-=[84]-=- is the problem of determining whether a certain point is a member of the Julia Set, which is the closure of the unstable equilibrium set of the following complex mapping: (124) It is undecidable whet... |

17 |
Lattice Boltzmann computational fluid dynamics in three dimensions
- Chen, Wang, et al.
- 1992
(Show Context)
Citation Context ...om slow converging iteration processes. For some problems, in which highly irregular geometries are involved as for example flows through porous media, the LBE method is the only available solver. In =-=[13]-=- the LBE method is compared to a numerical method, the spectral method, which is still considered a superior and efficient method for hydrodynamical problems. In many biological systems the distributi... |

17 |
The computational complexity of pattern formation
- Machta
- 1994
(Show Context)
Citation Context ...sion Limited Growth, especially in the 3D case, is computationally very expensive. The development of parallel growth models, especially in the case of DLA is not straightforward (see also references =-=[61]-=- [63] ). The computationally most expensive step, step 1 solving the Laplace equation, of the numerical solver can be done in parallel. In a parallel implementation of Eq. 106 using SOR, the update or... |