Abstract—An overview of statistical and information-theoretic aspects of hidden Markov processes (HMPs) is presented. An HMP is a discrete-time finite-state homogeneous Markov chain observed through a discrete-time memoryless invariant channel. In recent years, the work of Baum and Petrie on finite-state finite-alphabet HMPs was expanded to HMPs with finite as well as continuous state spaces and a general alphabet. In particular, statistical properties and ergodic theorems for relative entropy densities of HMPs were developed. Consistency and asymptotic normality of the maximum-likelihood (ML) parameter estimator were proved under some mild conditions. Similar results were established for switching autoregressive processes. These processes generalize HMPs. New algorithms were developed for estimating the state, parameter, and order of an HMP, for universal coding and classification of HMPs, and for universal decoding of hidden Markov channels. These and other related topics are reviewed in this paper. Index Terms—Baum–Petrie algorithm, entropy ergodic theorems, finite-state channels, hidden Markov models, identifiability, Kalman filter, maximum-likelihood (ML) estimation, order estimation, recursive parameter estimation, switching autoregressive processes, Ziv inequality. I.

Abstract not found

The purpose of time series analysis via mechanistic models is to reconcile the known or hypothesized structure of a dynamical system with observations collected over time. We develop a framework for constructing nonlinear mechanistic models and carrying out inference. Our framework permits the consideration of implicit dynamic models, meaning statistical models for stochastic dynamical systems which are specified by a simulation algorithm to generate sample paths. Inference procedures that operate on implicit models are said to have the plug-and-play property. Our work builds on recently developed plug-and-play inference methodology for partially observed Markov models. We introduce a class of implicitly specified Markov chains with stochastic transition rates, and we demonstrate its applicability to open problems in statistical inference for biological systems. As one example, these models are shown to give a fresh perspective on measles transmission dynamics. As a second example, we present a mechanistic analysis of cholera incidence data, involving interaction between two competing strains of the pathogen Vibrio cholerae. 1. Introduction. A

We describe some stochastic control problems in financial engineering arising from the need to find investment strategies to optimize some goal. Typically, these problems are characterized by nonlinear Hamilton-Jacobi-Bellman partial differential equations, and often they can be reduced to linear PDEs with the Legendre transform of convex duality. One situation where this cannot be achieved...

Abstract. We apply the formulation of a stochastic mode reduction method developed in a recent paper of Majda, Timofeyev, and Vanden-Eijnden [Comm. Pure Appl. Math., 54 (2001), pp. 891– 974] (MTV) to obtain simplified equations for the dynamics of structures immersed in a thermally fluctuating fluid at low Reynolds (or Kubo) number, as simulated by a recent extension of the immersed boundary (IB) method by Kramer and Peskin [Proceedings of the Second MIT Conference on Computational Fluid and Solid Mechanics, Elsevier Science, Oxford, UK, 2003, pp. 1755–1758]. The effective dynamics of the immersed structures are not obvious in the primitive equations, which involve both fluid and structure dynamics, but the procedure of MTV allows the rigorous derivation of a reduced stochastic system for the immersed structures alone. We find, in the limit of small Reynolds (or Kubo) number, that the Lagrangian particle constituents of the immersed structures undergo a drift-diffusive motion with several physically correct features, including the coupling between dynamics of different particles. The MTV procedure is also applied to the spatially discretized form of the IB equations with thermal fluctuations to assist in the design and assessment of numerical algorithms.

Abstract not found

It is shown how the phase-damping master equation, either in Markovian and nonMarkovian regimes, can be obtained as an averaged random unitary evolution. This, apart from offering a common mathematical setup for both regimes, enables us to solve this equation in a straightforward manner just by solving the Schrödinger equation and taking the stochastic expectation value of its solutions after an adequate modification. Using the linear entropy as a figure of merit (basically the loss of quantum coherence) the distinction of four kinds of environments is suggested. 1