Results 11  20
of
339
Exact simulation of diffusions
 Annals of Applied Probability
, 2005
"... We describe a new, surprisingly simple algorithm, that simulates exact sample paths of a class of stochastic differential equations. It involves rejection sampling and, when applicable, returns the location of the path at a random collection of time instances. The path can then be completed without ..."
Abstract

Cited by 28 (12 self)
 Add to MetaCart
We describe a new, surprisingly simple algorithm, that simulates exact sample paths of a class of stochastic differential equations. It involves rejection sampling and, when applicable, returns the location of the path at a random collection of time instances. The path can then be completed without further reference to the dynamics of the target process. 1. Introduction. Exact
A framework for worstcase and stochastic safety verification using barrier certificates
 IEEE TRANSACTIONS ON AUTOMATIC CONTROL
, 2007
"... This paper presents a methodology for safety verification of continuous and hybrid systems in the worstcase and stochastic settings. In the worstcase setting, a function of state termed barrier certificate is used to certify that all trajectories of the system starting from a given initial set do ..."
Abstract

Cited by 28 (1 self)
 Add to MetaCart
This paper presents a methodology for safety verification of continuous and hybrid systems in the worstcase and stochastic settings. In the worstcase setting, a function of state termed barrier certificate is used to certify that all trajectories of the system starting from a given initial set do not enter an unsafe region. No explicit computation of reachable sets is required in the construction of barrier certificates, which makes it possible to handle nonlinearity, uncertainty, and constraints directly within this framework. In the stochastic setting, our method computes an upper bound on the probability that a trajectory of the system reaches the unsafe set, a bound whose validity is proven by the existence of a barrier certificate. For polynomial systems, barrier certificates can be constructed using convex optimization, and hence the method is computationally tractable. Some examples are provided to illustrate the use of the method.
A generalized path integral control approach to reinforcement learning
 The Journal of Machine Learning Research
, 2010
"... With the goal to generate more scalable algorithms with higher efficiency and fewer open parameters, reinforcement learning (RL) has recently moved towards combining classical techniques from optimal control and dynamic programming with modern learning techniques from statistical estimation theory. ..."
Abstract

Cited by 28 (4 self)
 Add to MetaCart
With the goal to generate more scalable algorithms with higher efficiency and fewer open parameters, reinforcement learning (RL) has recently moved towards combining classical techniques from optimal control and dynamic programming with modern learning techniques from statistical estimation theory. In this vein, this paper suggests to use the framework of stochastic optimal control with path integrals to derive a novel approach to RL with parameterized policies. While solidly grounded in value function estimation and optimal control based on the stochastic HamiltonJacobiBellman (HJB) equations, policy improvements can be transformed into an approximation problem of a path integral which has no open algorithmic parameters other than the exploration noise. The resulting algorithm can be conceived of as modelbased, semimodelbased, or even model free, depending on how the learning problem is structured. The update equations have no danger of numerical instabilities as neither matrix inversions nor gradient learning rates are required. Our new algorithm demonstrates interesting similarities with previous RL research in the framework of probability matching and provides intuition why the slightly heuristically motivated probability matching approach can actually perform well. Empirical evaluations demonstrate significant performance improvements over gradientbased policy learning and scalability to highdimensional control problems. Finally, a learning experiment on a simulated 12 degreeoffreedom robot dog illustrates the functionality of our algorithm in a complex robot learning scenario. We believe that Policy Improvement with Path Integrals (PI 2) offers currently one of the most efficient, numerically robust, and easy to implement algorithms for RL based on trajectory rollouts.
On Bayesian Inference for Stochastic Kinetic Models Using Diffusion Approximations
, 2004
"... This paper is concerned with the Bayesian estimation of stochastic rate constants in the context of dynamic models of intracellular processes. The underlying discrete stochastic kinetic model is replaced by a di#usion approximation (or stochastic di#erential equation approach) where a white noise t ..."
Abstract

Cited by 25 (7 self)
 Add to MetaCart
This paper is concerned with the Bayesian estimation of stochastic rate constants in the context of dynamic models of intracellular processes. The underlying discrete stochastic kinetic model is replaced by a di#usion approximation (or stochastic di#erential equation approach) where a white noise term models stochastic behaviour and the model is identified using equispaced time course data. The estimation framework involves the introduction of m1 latent data points between every pair of observations. MCMC methods are then used to sample the posterior distribution of the latent process and the model parameters
Fourier’s law for a harmonic crystal with selfconsistent stochastic reservoirs
, 2004
"... We consider a ddimensional harmonic crystal in contact with a stochastic Langevin type heat bath at each site. The temperatures of the ‘‘exterior’ ’ left and right heat baths are at specified values TL and TR, respectively, while the temperatures of the ‘‘interior’ ’ baths are chosen selfconsisten ..."
Abstract

Cited by 25 (3 self)
 Add to MetaCart
We consider a ddimensional harmonic crystal in contact with a stochastic Langevin type heat bath at each site. The temperatures of the ‘‘exterior’ ’ left and right heat baths are at specified values TL and TR, respectively, while the temperatures of the ‘‘interior’ ’ baths are chosen selfconsistently so that there is no average flux of energy between them and the system in the steady state. We prove that this requirement uniquely fixes the temperatures and the self consistent system has a unique steady state. For the infinite system this state is one of local thermal equilibrium. The corresponding heat current satisfies Fourier’s law with a finite positive thermal conductivity which can also be computed using the Green–Kubo formula. For the harmonic chain (d=1) the conductivity agrees with the expression obtained by Bolsterli, Rich, and Visscher in 1970 who first studied this model. In the other limit, d ± 1, the stationary infinite volume heat conductivity behaves as (add) −1 where ad is the coupling to the intermediate reservoirs. We also analyze the effect of having a nonuniform distribution of the heat bath couplings. These results are proven rigorously by controlling the behavior of the correlations in the thermodynamic limit. KEY WORDS: Fourier’s law; harmonic crystal; nonequilibrium systems; thermodynamic limit; Green–Kubo formula.
A Toolbox of HamiltonJacobi Solvers for Analysis of Nondeterministic Continuous and Hybrid Systems
 In HSCC 2005, LNCS 3414
, 2005
"... Submitted to HSCC 2005. Please do not redistribute Abstract. HamiltonJacobi partial differential equations have many applications in the analysis of nondeterministic continuous and hybrid systems. Unfortunately, analytic solutions are seldom available and numerical approximation requires a great de ..."
Abstract

Cited by 24 (3 self)
 Add to MetaCart
Submitted to HSCC 2005. Please do not redistribute Abstract. HamiltonJacobi partial differential equations have many applications in the analysis of nondeterministic continuous and hybrid systems. Unfortunately, analytic solutions are seldom available and numerical approximation requires a great deal of programming infrastructure. In this paper we describe the first publicly available toolbox for approximating the solution of such equations, and discuss three examples of how these equations can be used in systems analysis: cost to go, stochastic differential games, and stochastic hybrid systems. For each example we briefly summarize the relevant theory, describe the toolbox implementation, and provide results. 1
Optimal Replication of Contingent Claims Under Portfolio Constraints
 Rev. of Financial Studies
, 1998
"... We study the problem of determining the minimum cost of superreplicating a nonnegative contingent claim when there are convex constraints on the portfolio weights. It is shown that the optimal cost with constraints is equal to the price of a related claim without constraints. The related claim is a ..."
Abstract

Cited by 24 (2 self)
 Add to MetaCart
We study the problem of determining the minimum cost of superreplicating a nonnegative contingent claim when there are convex constraints on the portfolio weights. It is shown that the optimal cost with constraints is equal to the price of a related claim without constraints. The related claim is a dominating claim, i.e., a claim whose payoffs are increased in an appropriate way relative to the original claim. The results hold for a wide variety of options, including standard European and American calls and puts, multiasset options, and some pathdependent options. We also provide a somewhat similar analysis when there are constraints on the gamma of the replicating portfolio. Constraints on portfolio amounts and constraints on number of shares of assets are also considered. Optimal Replication of Contingent Claims Under Portfolio Constraints 2 Since the pioneering option pricing work of Black and Scholes (1973) and Merton (1973), much research has focused on relaxing the assumptio...
Cochlear modeling
 IEEE ASSP Magazine
, 1985
"... A comparison of three different stochastic population ..."
Abstract

Cited by 22 (0 self)
 Add to MetaCart
A comparison of three different stochastic population
Bayesian sequential inference for stochastic kinetic biochemical network models J Comput Biol
, 2006
"... As postgenomic biology becomes more predictive, the ability to infer rate parameters of genetic and biochemical networks will become increasingly important. In this paper, we explore the Bayesian estimation of stochastic kinetic rate constants governing dynamic models of intracellular processes. The ..."
Abstract

Cited by 22 (6 self)
 Add to MetaCart
As postgenomic biology becomes more predictive, the ability to infer rate parameters of genetic and biochemical networks will become increasingly important. In this paper, we explore the Bayesian estimation of stochastic kinetic rate constants governing dynamic models of intracellular processes. The underlying model is replaced by a diffusion approximation where a noise term represents intrinsic stochastic behavior and the model is identified using discretetime (and often incomplete) data that is subject to measurement error. Sequential MCMC methods are then used to sample the model parameters online in several datapoor contexts. The methodology is illustrated by applying it to the estimation of parameters in a simple prokaryotic autoregulatory gene network.
Likelihood based inference for diffusion driven models, working paper
 In submission
, 2004
"... This paper provides methods for carrying out likelihood based inference for diffusion driven models, for example discretely observed multivariate diffusions, continuous time stochastic volatility models and counting process models. The diffusions can potentially be nonstationary. Although our method ..."
Abstract

Cited by 21 (1 self)
 Add to MetaCart
This paper provides methods for carrying out likelihood based inference for diffusion driven models, for example discretely observed multivariate diffusions, continuous time stochastic volatility models and counting process models. The diffusions can potentially be nonstationary. Although our methods are sampling based, making use of Markov chain Monte Carlo methods to sample the posterior distribution of the relevant unknowns, our general strategies and details are different from previous work along these lines. The methods we develop are simple to implement and simulation efficient. Importantly, unlike previous methods, the performance of our technique is not worsened, in fact it improves, as the degree of latent augmentation is increased to reduce the bias of the Euler approximation. In addition, our method is not subject to a degeneracy that afflicts previous techniques when the degree of latent augmentation is increased. We also discuss issues of model choice, model checking and filtering. The techniques and ideas are applied to both simulated and real data.