Results 11  20
of
538
Bayesian sequential inference for nonlinear multivariate diffusions
 Statistics and Computing
, 2006
"... In this paper, we adapt recently developed simulationbased sequential algorithms to the problem concerning the Bayesian analysis of discretely observed diffusion processes. The estimation framework involves the introduction of m −1 latent data points between every pair of observations. Sequential ..."
Abstract

Cited by 34 (4 self)
 Add to MetaCart
In this paper, we adapt recently developed simulationbased sequential algorithms to the problem concerning the Bayesian analysis of discretely observed diffusion processes. The estimation framework involves the introduction of m −1 latent data points between every pair of observations. Sequential MCMC methods are then used to sample the posterior distribution of the latent data and the model parameters online. The method is applied to the estimation of parameters in a simple stochastic volatility model (SV) of the U.S. shortterm interest rate. We also provide a simulation study to validate our method, using synthetic data generated by the SV model with parameters calibrated to match weekly observations of the U.S. shortterm interest rate. 1
Exact simulation of diffusions
 Annals of Applied Probability
, 2005
"... We describe a new, surprisingly simple algorithm, that simulates exact sample paths of a class of stochastic differential equations. It involves rejection sampling and, when applicable, returns the location of the path at a random collection of time instances. The path can then be completed without ..."
Abstract

Cited by 34 (15 self)
 Add to MetaCart
We describe a new, surprisingly simple algorithm, that simulates exact sample paths of a class of stochastic differential equations. It involves rejection sampling and, when applicable, returns the location of the path at a random collection of time instances. The path can then be completed without further reference to the dynamics of the target process. 1. Introduction. Exact
Cochlear modeling
 IEEE ASSP Magazine
, 1985
"... A comparison of three different stochastic population ..."
Abstract

Cited by 34 (2 self)
 Add to MetaCart
A comparison of three different stochastic population
On Bayesian Inference for Stochastic Kinetic Models Using Diffusion Approximations
, 2004
"... This paper is concerned with the Bayesian estimation of stochastic rate constants in the context of dynamic models of intracellular processes. The underlying discrete stochastic kinetic model is replaced by a di#usion approximation (or stochastic di#erential equation approach) where a white noise t ..."
Abstract

Cited by 31 (8 self)
 Add to MetaCart
This paper is concerned with the Bayesian estimation of stochastic rate constants in the context of dynamic models of intracellular processes. The underlying discrete stochastic kinetic model is replaced by a di#usion approximation (or stochastic di#erential equation approach) where a white noise term models stochastic behaviour and the model is identified using equispaced time course data. The estimation framework involves the introduction of m1 latent data points between every pair of observations. MCMC methods are then used to sample the posterior distribution of the latent process and the model parameters
A generalized path integral control approach to reinforcement learning
 The Journal of Machine Learning Research
, 2010
"... With the goal to generate more scalable algorithms with higher efficiency and fewer open parameters, reinforcement learning (RL) has recently moved towards combining classical techniques from optimal control and dynamic programming with modern learning techniques from statistical estimation theory. ..."
Abstract

Cited by 31 (5 self)
 Add to MetaCart
With the goal to generate more scalable algorithms with higher efficiency and fewer open parameters, reinforcement learning (RL) has recently moved towards combining classical techniques from optimal control and dynamic programming with modern learning techniques from statistical estimation theory. In this vein, this paper suggests to use the framework of stochastic optimal control with path integrals to derive a novel approach to RL with parameterized policies. While solidly grounded in value function estimation and optimal control based on the stochastic HamiltonJacobiBellman (HJB) equations, policy improvements can be transformed into an approximation problem of a path integral which has no open algorithmic parameters other than the exploration noise. The resulting algorithm can be conceived of as modelbased, semimodelbased, or even model free, depending on how the learning problem is structured. The update equations have no danger of numerical instabilities as neither matrix inversions nor gradient learning rates are required. Our new algorithm demonstrates interesting similarities with previous RL research in the framework of probability matching and provides intuition why the slightly heuristically motivated probability matching approach can actually perform well. Empirical evaluations demonstrate significant performance improvements over gradientbased policy learning and scalability to highdimensional control problems. Finally, a learning experiment on a simulated 12 degreeoffreedom robot dog illustrates the functionality of our algorithm in a complex robot learning scenario. We believe that Policy Improvement with Path Integrals (PI 2) offers currently one of the most efficient, numerically robust, and easy to implement algorithms for RL based on trajectory rollouts.
A framework for worstcase and stochastic safety verification using barrier certificates
 IEEE TRANSACTIONS ON AUTOMATIC CONTROL
, 2007
"... This paper presents a methodology for safety verification of continuous and hybrid systems in the worstcase and stochastic settings. In the worstcase setting, a function of state termed barrier certificate is used to certify that all trajectories of the system starting from a given initial set do ..."
Abstract

Cited by 29 (1 self)
 Add to MetaCart
This paper presents a methodology for safety verification of continuous and hybrid systems in the worstcase and stochastic settings. In the worstcase setting, a function of state termed barrier certificate is used to certify that all trajectories of the system starting from a given initial set do not enter an unsafe region. No explicit computation of reachable sets is required in the construction of barrier certificates, which makes it possible to handle nonlinearity, uncertainty, and constraints directly within this framework. In the stochastic setting, our method computes an upper bound on the probability that a trajectory of the system reaches the unsafe set, a bound whose validity is proven by the existence of a barrier certificate. For polynomial systems, barrier certificates can be constructed using convex optimization, and hence the method is computationally tractable. Some examples are provided to illustrate the use of the method.
Efficient collocational approach for parametric uncertainty analysis
 Commun. Comput. Phys
, 2007
"... Abstract. A numerical algorithm for effective incorporation of parametric uncertainty into mathematical models is presented. The uncertain parameters are modeled as random variables, and the governing equations are treated as stochastic. The solutions, or quantities of interests, are expressed as co ..."
Abstract

Cited by 29 (4 self)
 Add to MetaCart
Abstract. A numerical algorithm for effective incorporation of parametric uncertainty into mathematical models is presented. The uncertain parameters are modeled as random variables, and the governing equations are treated as stochastic. The solutions, or quantities of interests, are expressed as convergent series of orthogonal polynomial expansions in terms of the input random parameters. A highorder stochastic collocation method is employed to solve the solution statistics, and more importantly, to reconstruct the polynomial expansion. While retaining the high accuracy by polynomial expansion, the resulting “pseudospectral ” type algorithm is straightforward to implement as it requires only repetitive deterministic simulations. An estimate on error bounded is presented, along with numerical examples for problems with relatively complicated forms of governing equations. Key words: Collocation methods; pseudospectral methods; stochastic inputs; random differential equations; uncertainty quantification. 1
Information Aggregation in Dynamic Markets with Strategic Traders,” Working Paper
, 2009
"... This paper studies information aggregation in dynamic markets with a finite number of partially informed strategic traders. It shows that for a broad class of securities, information in such markets always gets aggregated. Trading takes place in a bounded time interval, and in every equilibrium, as ..."
Abstract

Cited by 29 (0 self)
 Add to MetaCart
This paper studies information aggregation in dynamic markets with a finite number of partially informed strategic traders. It shows that for a broad class of securities, information in such markets always gets aggregated. Trading takes place in a bounded time interval, and in every equilibrium, as time approaches the end of the interval, the market price of a “separable” security converges in probability to its expected value conditional on the traders ’ pooled information. If the security is “nonseparable, ” then there exists a common prior over the states of the world and an equilibrium such that information does not get aggregated. The class of separable securities includes, among others, ArrowDebreu securities, whose value is one in one state of the world and zero in all others, and “additive ” securities, whose value can be interpreted as the sum of traders ’ signals.
A Toolbox of HamiltonJacobi Solvers for Analysis of Nondeterministic Continuous and Hybrid Systems
 In HSCC 2005, LNCS 3414
, 2005
"... Submitted to HSCC 2005. Please do not redistribute Abstract. HamiltonJacobi partial differential equations have many applications in the analysis of nondeterministic continuous and hybrid systems. Unfortunately, analytic solutions are seldom available and numerical approximation requires a great de ..."
Abstract

Cited by 28 (3 self)
 Add to MetaCart
Submitted to HSCC 2005. Please do not redistribute Abstract. HamiltonJacobi partial differential equations have many applications in the analysis of nondeterministic continuous and hybrid systems. Unfortunately, analytic solutions are seldom available and numerical approximation requires a great deal of programming infrastructure. In this paper we describe the first publicly available toolbox for approximating the solution of such equations, and discuss three examples of how these equations can be used in systems analysis: cost to go, stochastic differential games, and stochastic hybrid systems. For each example we briefly summarize the relevant theory, describe the toolbox implementation, and provide results. 1
Strong regularities in online peer production
 In Proc. of the 9th ACM conf. on Electronic commerce, EC ’08
, 2008
"... Online peer production systems have enabled people to coactively create, share, classify, and rate content on an unprecedented scale. This paper describes strong macroscopic regularities in how people contribute to peer production systems, and shows how these regularities arise from simple dynamica ..."
Abstract

Cited by 27 (2 self)
 Add to MetaCart
Online peer production systems have enabled people to coactively create, share, classify, and rate content on an unprecedented scale. This paper describes strong macroscopic regularities in how people contribute to peer production systems, and shows how these regularities arise from simple dynamical rules. First, it is demonstrated that the probability a person stops contributing varies inversely with the number of contributions he has made. This rule leads to a power law distribution for the number of contributions per person in which a small number of very active users make most of the contributions. The rule also implies that the power law exponent is proportional to the effort required to contribute, as justified by the data. Second, the level of activity per topic is shown to follow a lognormal distribution generated by a stochastic reinforcement mechanism. A small number of very popular topics thus accumulate the vast majority of contributions. These trends are demonstrated to hold across hundreds of millions of contributions to four disparate peer production systems of differing scope, interface style, and purpose. 1.