Results 1  10
of
277
Universal coalgebra: a theory of systems
, 2000
"... In the semantics of programming, nite data types such as finite lists, have traditionally been modelled by initial algebras. Later final coalgebras were used in order to deal with in finite data types. Coalgebras, which are the dual of algebras, turned out to be suited, moreover, as models for certa ..."
Abstract

Cited by 298 (31 self)
 Add to MetaCart
In the semantics of programming, nite data types such as finite lists, have traditionally been modelled by initial algebras. Later final coalgebras were used in order to deal with in finite data types. Coalgebras, which are the dual of algebras, turned out to be suited, moreover, as models for certain types of automata and more generally, for (transition and dynamical) systems. An important property of initial algebras is that they satisfy the familiar principle of induction. Such a principle was missing for coalgebras until the work of Aczel (NonWellFounded sets, CSLI Leethre Notes, Vol. 14, center for the study of Languages and information, Stanford, 1988) on a theory of nonwellfounded sets, in which he introduced a proof principle nowadays called coinduction. It was formulated in terms of bisimulation, a notion originally stemming from the world of concurrent programming languages. Using the notion of coalgebra homomorphism, the definition of bisimulation on coalgebras can be shown to be formally dual to that of congruence on algebras. Thus, the three basic notions of universal algebra: algebra, homomorphism of algebras, and congruence, turn out to correspond to coalgebra, homomorphism of coalgebras, and bisimulation, respectively. In this paper, the latter are taken
The induction of dynamical recognizers
 Machine Learning
, 1991
"... A higher order recurrent neural network architecture learns to recognize and generate languages after being "trained " on categorized exemplars. Studying these networks from the perspective of dynamical systems yields two interesting discoveries: First, a longitudinal examination of the learning pro ..."
Abstract

Cited by 214 (16 self)
 Add to MetaCart
A higher order recurrent neural network architecture learns to recognize and generate languages after being "trained " on categorized exemplars. Studying these networks from the perspective of dynamical systems yields two interesting discoveries: First, a longitudinal examination of the learning process illustrates a new form of mechanical inference: Induction by phase transition. A small weight adjustment causes a "bifurcation" in the limit behavior of the network. This phase transition corresponds to the onset of the network’s capacity for generalizing to arbitrarylength strings. Second, a study of the automata resulting from the acquisition of previously published training sets indicates that while the architecture is not guaranteed to find a minimal finite automaton consistent with the given exemplars, which is an NPHard problem, the architecture does appear capable of generating nonregular languages by exploiting fractal and chaotic dynamics. I end the paper with a hypothesis relating linguistic generative capacity to the behavioral regimes of nonlinear dynamical systems.
Metaheuristics in combinatorial optimization: Overview and conceptual comparison
 ACM COMPUTING SURVEYS
, 2003
"... The field of metaheuristics for the application to combinatorial optimization problems is a rapidly growing field of research. This is due to the importance of combinatorial optimization problems for the scientific as well as the industrial world. We give a survey of the nowadays most important meta ..."
Abstract

Cited by 169 (14 self)
 Add to MetaCart
The field of metaheuristics for the application to combinatorial optimization problems is a rapidly growing field of research. This is due to the importance of combinatorial optimization problems for the scientific as well as the industrial world. We give a survey of the nowadays most important metaheuristics from a conceptual point of view. We outline the different components and concepts that are used in the different metaheuristics in order to analyze their similarities and differences. Two very important concepts in metaheuristics are intensification and diversification. These are the two forces that largely determine the behaviour of a metaheuristic. They are in some way contrary but also complementary to each other. We introduce a framework, that we call the I&D frame, in order to put different intensification and diversification components into relation with each other. Outlining the advantages and disadvantages of different metaheuristic approaches we conclude by pointing out the importance of hybridization of metaheuristics as well as the integration of metaheuristics and other methods for optimization.
Adaptive Greedy Approximations
"... The problem of optimally approximating a function with a linear expansion over a redundant dictionary of waveforms is NPhard. The greedy matching pursuit algorithm and its orthogonalized variant produce suboptimal function expansions by iteratively choosing dictionary waveforms that best match the ..."
Abstract

Cited by 112 (0 self)
 Add to MetaCart
The problem of optimally approximating a function with a linear expansion over a redundant dictionary of waveforms is NPhard. The greedy matching pursuit algorithm and its orthogonalized variant produce suboptimal function expansions by iteratively choosing dictionary waveforms that best match the function's structures. A matching pursuit provides a means of quickly computing compact, adaptive function approximations. Numerical experiments show that the approximation errors from matching pursuits initially decrease rapidly, but the asymptotic decay rate of the errors is slow. We explain this behavior by showing that matching pursuits are chaotic, ergodic maps. The statistical properties of the approximation errors of a pursuit can be obtained from the invariant measure of the pursuit. We characterize these measures using group symmetries of dictionaries and by constructing a stochastic differential equation model. We derive a notion of the coherence of a signal with respect to a dict...
Dynamical systems, Measures and Fractals via Domain Theory
 Information and Computation
, 1995
"... We introduce domain theory in dynamical systems, iterated function systems (fractals) and measure theory. For a discrete dynamical system given by the action of a continuous map f:X X on a metric space X, we study the extended dynamical systems (l/X,l/f), (UX, U f) and (LX, Lf) where 1/, U and L ar ..."
Abstract

Cited by 69 (19 self)
 Add to MetaCart
We introduce domain theory in dynamical systems, iterated function systems (fractals) and measure theory. For a discrete dynamical system given by the action of a continuous map f:X X on a metric space X, we study the extended dynamical systems (l/X,l/f), (UX, U f) and (LX, Lf) where 1/, U and L are respectively the Vietoris hyperspace, the upper hyperspace and the lower hyperspace functors. We show that if (X, f) is chaotic, then so is (UX, U f). When X is locally compact UX, is a continuous bounded complete dcpo. If X is second countable as well, then UX will be omegacontinuous and can be given an effective structure. We show how strange attractors, attractors of iterated function systems (fractals) and Julia sets are obtained effectively as fixed points of deterministic functions on UX or fixed points of nondeterministic functions on CUX where C is the convex (Plotkin) power domain. We also show that the set, M(X), of finite Borel measures on X can be embedded in PUX, where P is the probabilistic power domain. This provides an effective framework for measure theory. We then prove that the invariant measure of an hyperbolic iterated function system with probabilities can be obtained as the unique fixed point of an associated continuous function on PUX.
Universal Computation and Other Capabilities of Hybrid and Continuous Dynamical Systems
, 1995
"... We explore the simulation and computational capabilities of hybrid and continuous dynamical systems. The continuous dynamical systems considered are ordinary differential equations (ODEs). For hybrid systems we concentrate on models that combine ODEs and discrete dynamics (e.g., finite automata). We ..."
Abstract

Cited by 68 (3 self)
 Add to MetaCart
We explore the simulation and computational capabilities of hybrid and continuous dynamical systems. The continuous dynamical systems considered are ordinary differential equations (ODEs). For hybrid systems we concentrate on models that combine ODEs and discrete dynamics (e.g., finite automata). We review and compare four such models from the literature. Notions of simulation of a discrete dynamical system by a continuous one are developed. We show that hybrid systems whose equations can describe a precise binary timing pulse (exact clock) can simulate arbitrary reversible discrete dynamical systems defined on closed subsets of R n . The simulations require continuous ODEs in R 2n with the exact clock as input. All four hybrid systems models studied here can implement exact clocks. We also prove that any discrete dynamical system in Z n can be simulated by continuous ODEs in R 2n+1 . We use this to show that smooth ODEs in R 3 can simulate arbitrary Turing machines, and henc...
Adaptive Nonlinear Approximations
, 1994
"... The problem of optimally approximating a function with a linear expansion over a redundant dictionary of waveforms is NPhard. The greedy matching pursuit algorithm and its orthogonalized variant produce suboptimal function expansions by iteratively choosing the dictionary waveforms which best matc ..."
Abstract

Cited by 52 (1 self)
 Add to MetaCart
The problem of optimally approximating a function with a linear expansion over a redundant dictionary of waveforms is NPhard. The greedy matching pursuit algorithm and its orthogonalized variant produce suboptimal function expansions by iteratively choosing the dictionary waveforms which best match the function's structures. Matching pursuits provide a means of quickly computing compact, adaptive function approximations. Numerical experiments show that the approximation errors from matching pursuits initially decrease rapidly, but the asymptotic decay rate of the errors is slow. We explain this behavior by showing that matching pursuits are chaotic, ergodic maps. The statistical properties of the approximation errors of a pursuit can be obtained from the invariant measure of the pursuit. We characterize these measures using group symmetries of dictionaries and using a stochastic differential equation model. These invariant measures define a noise with respect to a given dictionary. ...
Domains for Computation in Mathematics, Physics and Exact Real Arithmetic
 Bulletin of Symbolic Logic
, 1997
"... We present a survey of the recent applications of continuous domains for providing simple computational models for classical spaces in mathematics including the real line, countably based locally compact spaces, complete separable metric spaces, separable Banach spaces and spaces of probability dist ..."
Abstract

Cited by 48 (10 self)
 Add to MetaCart
We present a survey of the recent applications of continuous domains for providing simple computational models for classical spaces in mathematics including the real line, countably based locally compact spaces, complete separable metric spaces, separable Banach spaces and spaces of probability distributions. It is shown how these models have a logical and effective presentation and how they are used to give a computational framework in several areas in mathematics and physics. These include fractal geometry, where new results on existence and uniqueness of attractors and invariant distributions have been obtained, measure and integration theory, where a generalization of the Riemann theory of integration has been developed, and real arithmetic, where a feasible setting for exact computer arithmetic has been formulated. We give a number of algorithms for computation in the theory of iterated function systems with applications in statistical physics and in period doubling route to chao...
Finite State Machines and Recurrent Neural Networks  Automata and Dynamical Systems Approaches
 Neural Networks and Pattern Recognition
, 1998
"... We present two approaches to the analysis of the relationship between a recurrent neural network (RNN) and the finite state machine M the network is able to exactly mimic. First, the network is treated as a state machine and the relationship between the RNN and M is established in the context of alg ..."
Abstract

Cited by 29 (11 self)
 Add to MetaCart
We present two approaches to the analysis of the relationship between a recurrent neural network (RNN) and the finite state machine M the network is able to exactly mimic. First, the network is treated as a state machine and the relationship between the RNN and M is established in the context of algebraic theory of automata. In the second approach, the RNN is viewed as a set of discretetime dynamical systems associated with input symbols of M. In particular, issues concerning network representation of loops and cycles in the state transition diagram of M are shown to provide a basis for the interpretation of learning process from the point of view of bifurcation analysis. The circumstances under which a loop corresponding to an input symbol x is represented by an attractive fixed point of the underlying dynamical system associated with x are investigated. For the case of two recurrent neurons, under some assumptions on weight values, bifurcations can be understood in the geometrical c...
ContextFree and ContextSensitive Dynamics in Recurrent Neural Networks
, 2000
"... Continuousvalued recurrent neural networks can learn mechanisms for processing contextfree languages. The dynamics of such networks is usually based on damped oscillation around fixed points in state space and requires that the dynamical components are arranged in certain ways. It is shown tha ..."
Abstract

Cited by 28 (6 self)
 Add to MetaCart
Continuousvalued recurrent neural networks can learn mechanisms for processing contextfree languages. The dynamics of such networks is usually based on damped oscillation around fixed points in state space and requires that the dynamical components are arranged in certain ways. It is shown that qualitatively similar dynamics with similar constraints hold for a n b n c n , a contextsensitive language. The additional difficulty with a n b n c n , compared with the contextfree language a n b n , consists of "counting up" and "counting down" letters simultaneously. The network solution is to oscillate in two principal dimensions, one for counting up and one for counting down. This study focuses on the dynamics employed by the Sequential Cascaded Network, in contrast with the Simple Recurrent Network, and the use of Backpropagation Through Time. Found solutions generalize well beyond training data, however, learning is not reliable. The contribution of this ...