A major problem in machine learning is that of inductive bias: how to choose a learner's hypothesis space so that it is large enough to contain a solution to the problem being learnt, yet small enough to ensure reliable generalization from reasonably-sized training sets. Typically such bias is supplied by hand through the skill and insights of experts. In this paper a model for automatically learning bias is investigated. The central assumption of the model is that the learner is embedded within an environment of related learning tasks. Within such an environment the learner can sample from multiple tasks, and hence it can search for a hypothesis space that contains good solutions to many of the problems in the environment. Under certain restrictions on the set of all hypothesis spaces available to the learner, we show that a hypothesis space that performs well on a sufficiently large number of training tasks will also perform well when learning novel tasks in the same environment. Exp...

Many useful descriptions of stochastic models can be obtained from functional limit theorems (invariance principles or weak convergence theorems for probability meastires on function spaces). These descriptions typically come from standard functional limit theorems via the o^ntinuous mapping theorem. This paper facilitates applications of the continuous mapping theorem by determining when several important ftmctions and sequences of functions preserve convergence. The functions considered are composition, addition, composition plus addition, multiplication, supremtun, reflecting barrier, first passage time and time reversal. These functions provide means for proving new functional limit theorems from previous ones. These functions are useful, for example, to establish the stability or continuity of queues and other stochastic models.

We study the a.s. exponential stability of the optimal filter w.r.t. its initial conditions. A bound is provided on the exponential rate (equivalently, on the memory length of the filter) for a general setting both in discrete and in continuous time, in terms of Birkhoff's contraction coefficient. Criteria for exponential stability and explicit bounds on the rate are given in the specific cases of a diffusion process on a compact manifold, and discrete time Markov chains on both continuous and discrete-countable state spaces. R'esum'e Nous 'etudions la stabilit'e du filtre optimal par raport `a ses conditions initiales. Le taux de d'ecroissance exponentielle est calcul'e dans un cadre g'en'eral, pour temps discret et temps continu, en terme du coefficient de contraction de Birkhoff. Des crit`eres de stabilit'e exponentielle et des bornes explicites sur le taux sont calcul'ees pour les cas particuliers d'une diffusion sur une vari'ete compacte, ainsi que pour des chaines de Markov sur ...

The notion of process equivalence of probabilistic processes is sensitive to the exact probabilities of transitions. Thus, a slight change in the transition probabilities will result in two equivalent processes being deemed no longer equivalent. This instability is due to the quantitative nature of probabilistic processes. In a situation where the process behaviour has a quantitative aspect there should be a more robust approach to process equivalence. This paper studies a metric between labelled Markov processes. This metric has the property that processes are at zero distance if and only if they are bisimilar. The metric is inspired by earlier work on logics for characterizing bisimulation and is related, in spirit, to the Hutchinson metric.

In this paper foundations are presented to a new systematic approach to analysis and geometry for an important class of infinite dimensional manifolds, namely, configuration spaces. More precisely, a differential geometry is introduced on the configuration space \Gamma X over a Riemannian manifold X. This geometry is "non-flat" even if X = IR d . It is obtained as a natural lifting of the Riemannian structure on X. In particular, a corresponding gradient r \Gamma , divergence div \Gamma , and Laplace-Beltrami operator H \Gamma = \Gammadiv \Gamma r \Gamma are constructed. The associated volume elements, i.e., all measures ¯ on \Gamma X w.r.t. which r \Gamma and div \Gamma become dual operators on L 2 (\Gamma X ; ¯), are identified as exactly the mixed Poisson measures with mean measure equal to a multiple of the volume element dx on X. In particular, all these measures obey an integration by parts formula w.r.t. vector fields on \Gamma X . The corresponding Dirichlet...

This paper is a contribution to the theory of countable Borel equivalence relations on standard Borel spaces. As usual, by a standard Borel space we mean a Polish (complete separable metric) space equipped with its #-algebra of Borel sets. An equivalence relation E on a standard Borel space X is Borel if it is a Borel subset of X². Given two

This paper introduces a new aspect of queueing theory, the study of systems that service customers with specic timing requirements (e.g. due dates or deadlines). Unlike standard queueing theory in which common performance measures are customer delay, queue length and server utilization, real-time queueing theory focuses on the ability of a queue discipline to meet customer timing requirements, e.g., the fraction of customers who meet their requirements and the distribution of customer lateness. It also focuses on queue control policies to reduce or minimize lateness, although these control aspects are not explicitly addressed in this paper. To study these measures, one must keep track of the lead-times (deadline minus current time) of each customer, hence the system state is of unbounded dimension. A heavy trac analysis is presented for the earliest deadline rst (EDF) scheduling policy. This analysis decomposes the behavior of the real-time queue into two parts: the number in the sys...

Abstract not found

In the paper we study interacting particle approximations of discrete time and measure valued dynamical systems. Such systems have arisen in such diverse scientific disciplines as physics and signal processing. We give conditions for the so-called particle density profiles to converge to the desired distribution when the number of particles is growing. The strength of our approach is that is applicable to a large class of measure valued dynamical system arising in engineering and particularly in nonlinear filtering problems. Our second objective is to use these results to solve numerically the nonlinear filtering equation. Examples arising in fluid mechanics are also given. 1 Introduction 1.1 Measure valued processes Let (E; fi(E)) be a locally compact and separable metric space, endowed with a Borel oe-field, state space. Denote by P(E) be the space of all probability measures on E with the weak topology. The aim of this work is the design of a stochastic particle system approach fo...

In this online supplement we provide results that we have omitted from the main paper. First, in Appendix A, we give a proof of Lemma 2.1. In Appendix B we give a proof of Theorem 6.1 using the technique described in [2]. Finally, in Appendix C, we give an alternative proof of Theorem 5.2 using stopped arrival processes as in the proof of Theorem 6.3.