We present a solution for inferring hidden state from sensorimotor experience when the environment takes the form of a POMDP with deterministic transition and observation functions. Such environments can appear to be arbitrarily complex and non-deterministic on the surface, but are actually deterministic with respect to the unobserved underlying state. We show that there always exists a finite history-based representation that fully captures the unobserved world state, allowing for perfect prediction of action effects. This representation takes the form of a looping prediction suffix tree (PST). We derive a sound and complete algorithm for learning a looping PST from a sufficient sample of sensorimotor experience. We also give empirical illustrations of the advantages conferred by this approach, and characterize the approximations to the looping PST that are made by existing algorithms such as Variable

Complex Systems Science aims to understand concepts like complexity, self-organization, emergence and adaptation, among others. The inherent fuzziness in complex systems definitions is complicated by the unclear relation among these central processes: does self-organisation emerge or does it set the preconditions for emergence? Does complexity arise by adaptation or is complexity necessary for adaptation to arise? The inevitable consequence of the current impasse is miscommunication among scientists within and across disciplines. We propose a set of concepts, together with their information-theoretic interpretations, which can be used as a dictionary of Complex Systems Science discourse. Our hope is that the suggested information-theoretic baseline may facilitate consistent communications among practitioners, and provide new insights into the field.

Abstract. The concept of self-organization has been examined oftentimes for several domains such as physics, chemistry, mathematics, etc. However, the current technical development opens a new field of self-organizing applications by creating systems of networked and massively distributed hardware with self-organized control. Having this view in mind, this papers reviews the questions: What is a self-organizing system?, What is it not?, Should there be a separate field of science for self-organizing systems?, and What are possible approaches to engineer a self-organizing control system?. The presented ideas have been elaborated at the Lakeside Research Days’08 (University of Klagenfurt, Austria), a workshop that featured guided discussions between invited experts working in the field of selforganizing systems. 1

Resumen: En este artículo presentamos un nuevo sistema para el reconocimiento de nombres propios en español. Este sistema está basado en el algoritmo CSSR (Causal-States Splitting Reconstruction) (Shalizi and Shalizi, 2004) que aprende un autómata de estados finitos partiendo de datos sequenciales. Los resultados obtenidos son ligeramente peores que los mejores sistemas presentados en la “shared task ” del CoNLL 2002, pero dada la simplicidad de los atributos utilizados, estos resultados son realmente prometedores y creemos que pueden ser fácilmente mejorados introduciendo más información al sistema. Palabras clave: Reconocimiento de Nombres Propios, Automatas de Estados Finitos,

Self-organization, whereby through purely local interactions, global order and structure emerge, is studied broadly across many fields of science, economics, and engineering. We review several existing methods and modeling techniques used to understand self-organization in a general manner. We then present implementation concepts and case studies for applying these principles for the design and deployment of robust self-organizing networked systems. 1

Abstract. Many networks are important because they are substrates for dynamical systems, and their pattern of functional connectivity can itself be dynamic — they can functionally reorganize, even if their underlying anatomical structure remains fixed. However, the recent rapid progress in discovering the community structure of networks has overwhelmingly focused on that constant anatomical connectivity. In this paper, we lay out the problem of discovering functional communities, and describe an approach to doing so. This method combines recent work on measuring information sharing across stochastic networks with an existing and successful community-discovery algorithm for weighted networks. We illustrate it with an application to a large biophysical model of the transition from beta to gamma rhythms in the hippocampus. 1

Abstract: Much is now known about the consistency of Bayesian updating on infinite-dimensional parameter spaces with independent or Markovian data. Necessary conditions for consistency include the prior putting enough weight on the correct neighborhoods of the data-generating distribution; various sufficient conditions further restrict the prior in ways analogous to capacity control in frequentist nonparametrics. The asymptotics of Bayesian updating with mis-specified models or priors, or non-Markovian data, are far less well explored. Here I establish sufficient conditions for posterior convergence when all hypotheses are wrong, and the data have complex dependencies. The main dynamical assumption is the asymptotic equipartition (Shannon-McMillan-Breiman) property of information theory. This, along with Egorov’s Theorem on uniform convergence, lets me build a sieve-like structure for the prior. The main statistical assumption, also a form of capacity control, concerns the compatibility of the prior and the data-generating process, controlling the fluctuations in the loglikelihood when averaged over the sieve-like sets. In addition to posterior convergence, I derive a kind of large deviations principle for the posterior measure, extending in some cases to rates of convergence, and discuss the advantages of predicting using a combination of models known to be wrong. An appendix sketches connections between these results and the replicator dynamics of evolutionary theory.

POMDPs are the models of choice for reinforcement learning (RL) tasks where the environment cannot be observed directly. In many applications we need to learn the POMDP structure and parameters from experience and this is considered to be a difficult problem. In this paper we address this issue by modeling the hidden environment with a novel class of models that are less expressive, but easier to learn and plan with than POMDPs. We call these models deterministic Markov models (DMMs), which are deterministic-probabilistic finite automata from learning theory, extended with actions to the sequential (rather than i.i.d.) setting. Conceptually, we extend the Utile Suffix Memory method of McCallum to handle long term memory. We describe DMMs, give Bayesian algorithms for learning and planning with them and also present experimental results for some standard POMDP tasks and tasks to illustrate its efficacy. 1.

Intertemporal topic correlations in online media

A method for validating and discovering associations between multi-level emergent behaviours in agent-based simulations