Results 1  10
of
113
Discrete Choice Methods with Simulation
, 2002
"... This book describes the new generation of discrete choice methods, focusing on the many advances that are made possible by simulation. Researchers use these statistical methods to examine the choices that consumers, households, firms, and other agents make. Each of the major models is covered: logi ..."
Abstract

Cited by 1022 (18 self)
 Add to MetaCart
This book describes the new generation of discrete choice methods, focusing on the many advances that are made possible by simulation. Researchers use these statistical methods to examine the choices that consumers, households, firms, and other agents make. Each of the major models is covered: logit, generalized extreme value (including nested and crossnested logits), probit, and mixed logit, plus a variety of specifications that build on these basics. Simulationassisted estimation procedures are investigated and compared, including maximum simulated likelihood, the method of simulated moments, and the method of simulated scores. Procedures for drawing from densities are described, including variance reduction techniques such as antithetics and Halton draws. Recent advances in Bayesian procedures are explored, including the use of the Metropolis– Hastings algorithm and its variant Gibbs sampling. No other book incorporates all these topics, which have risen in the past 20 years. The procedures are applicable in many fields, including energy, transportation, environmental studies, health, labor, and marketing.
Treebased batch mode reinforcement learning
 Journal of Machine Learning Research
, 2005
"... Reinforcement learning aims to determine an optimal control policy from interaction with a system or from observations gathered from a system. In batch mode, it can be achieved by approximating the socalled Qfunction based on a set of fourtuples (xt,ut,rt,xt+1) where xt denotes the system state a ..."
Abstract

Cited by 199 (35 self)
 Add to MetaCart
(Show Context)
Reinforcement learning aims to determine an optimal control policy from interaction with a system or from observations gathered from a system. In batch mode, it can be achieved by approximating the socalled Qfunction based on a set of fourtuples (xt,ut,rt,xt+1) where xt denotes the system state at time t, ut the control action taken, rt the instantaneous reward obtained and xt+1 the successor state of the system, and by determining the control policy from this Qfunction. The Qfunction approximation may be obtained from the limit of a sequence of (batch mode) supervised learning problems. Within this framework we describe the use of several classical treebased supervised learning methods (CART, Kdtree, tree bagging) and two newly proposed ensemble algorithms, namely extremely and totally randomized trees. We study their performances on several examples and find that the ensemble methods based on regression trees perform well in extracting relevant information about the optimal control policy from sets of fourtuples. In particular, the totally randomized trees give good results while ensuring the convergence of the sequence, whereas by relaxing the convergence constraint even better accuracy results are provided by the extremely randomized trees.
KernelBased Reinforcement Learning
 Machine Learning
, 1999
"... We present a kernelbased approach to reinforcement learning that overcomes the stability problems of temporaldifference learning in continuous statespaces. First, our algorithm converges to a unique solution of an approximate Bellman's equation regardless of its initialization values. Second ..."
Abstract

Cited by 138 (1 self)
 Add to MetaCart
(Show Context)
We present a kernelbased approach to reinforcement learning that overcomes the stability problems of temporaldifference learning in continuous statespaces. First, our algorithm converges to a unique solution of an approximate Bellman's equation regardless of its initialization values. Second, the method is consistent in the sense that the resulting policy converges asymptotically to the optimal policy. Parametric value function estimates such as neural networks do not possess this property. Our kernelbased approach also allows us to show that the limiting distribution of the value function estimate is a Gaussian process. This information is useful in studying the biasvariance tradeo in reinforcement learning. We find that all reinforcement learning approaches to estimating the value function, parametric or nonparametric, are subject to a bias. This bias is typically larger in reinforcement learning than in a comparable regression problem.
A stochastic mesh method for pricing highdimensional American options
 Journal of Computational Finance
, 1997
"... Highdimensional problems frequently arise in the pricing of derivative securities – for example, in pricing options on multiple underlying assets and in pricing term structure derivatives. American versions of these options, ie, where the owner has the right to exercise early, are particularly chal ..."
Abstract

Cited by 119 (8 self)
 Add to MetaCart
Highdimensional problems frequently arise in the pricing of derivative securities – for example, in pricing options on multiple underlying assets and in pricing term structure derivatives. American versions of these options, ie, where the owner has the right to exercise early, are particularly challenging to price. We introduce a stochastic mesh method for pricing highdimensional American options when there is a finite, but possibly large, number of exercise dates. The algorithm provides point estimates and confidence intervals; we provide conditions under which these estimates converge to the correct values as the computational effort increases. Numerical results illustrate the performance of the method. 1
Comparing Solution Methods for Dynamic Equilibrium Economies
 Journal of Economic Dynamics and Control
, 2006
"... This paper compares solution methods for dynamic equilibrium economies. We compute and simulate the stochastic neoclassical growth model with leisure choice using Undetermined Coefficients in levels and in logs, Finite Elements, Chebyshev Polynomials, Second and Fifth Order Perturbations and Value F ..."
Abstract

Cited by 91 (28 self)
 Add to MetaCart
This paper compares solution methods for dynamic equilibrium economies. We compute and simulate the stochastic neoclassical growth model with leisure choice using Undetermined Coefficients in levels and in logs, Finite Elements, Chebyshev Polynomials, Second and Fifth Order Perturbations and Value Function Iteration for several calibrations. We document the performance of the methods in terms of computing time, implementation complexity and accuracy and we present some conclusions about our preferred approaches based on the reported evidence.
Modeling Model Uncertainty
 Journal of the European Economic Assocation
, 2003
"... Recently there has been a great deal of interest in studying monetary policy under model uncertainty. We point out that different assumptions about the uncertainty may result in drastically different “robust ” policy recommendations. Therefore, we develop new methods to analyze uncertainty about the ..."
Abstract

Cited by 72 (5 self)
 Add to MetaCart
Recently there has been a great deal of interest in studying monetary policy under model uncertainty. We point out that different assumptions about the uncertainty may result in drastically different “robust ” policy recommendations. Therefore, we develop new methods to analyze uncertainty about the parameters of a model, the lag specification, the serial correlation of shocks, and the effects of real time data in one coherent structure. We consider both parametric and nonparametric specifications of this structure and use them to estimate the uncertainty in a small model of the US economy. We then use our estimates to compute robust Bayesian and minimax monetary policy rules, which are designed to perform well in the face of uncertainty. Our results suggest that the aggressiveness recently found in robust policy rules is likely to be caused by overemphasizing uncertainty about economic dynamics at low frequencies.
Bayes Meets Bellman: The Gaussian Process Approach to Temporal Difference Learning
 Proc. of the 20th International Conference on Machine Learning
, 2003
"... We present a novel Bayesian approach to the problem of value function estimation in continuous state spaces. We de ne a probabilistic generative model for the value function by imposing a Gaussian prior over value functions and assuming a Gaussian noise model. ..."
Abstract

Cited by 72 (8 self)
 Add to MetaCart
(Show Context)
We present a novel Bayesian approach to the problem of value function estimation in continuous state spaces. We de ne a probabilistic generative model for the value function by imposing a Gaussian prior over value functions and assuming a Gaussian noise model.
Learning and Value Function Approximation in Complex Decision Processes
, 1998
"... In principle, a wide variety of sequential decision problems  ranging from dynamic resource allocation in telecommunication networks to financial risk management  can be formulated in terms of stochastic control and solved by the algorithms of dynamic programming. Such algorithms compute and sto ..."
Abstract

Cited by 38 (4 self)
 Add to MetaCart
In principle, a wide variety of sequential decision problems  ranging from dynamic resource allocation in telecommunication networks to financial risk management  can be formulated in terms of stochastic control and solved by the algorithms of dynamic programming. Such algorithms compute and store a value function, which evaluates expected future reward as a function of current state. Unfortunately, exact computation of the value function typically requires time and storage that grow proportionately with the number of states, and consequently, the enormous state spaces that arise in practical applications render the algorithms intractable. In this thesis, we study tractable methods that approximate the value function. Our work builds on research in an area of artificial intelligence known as reinforcement learning. A point of focus of this thesis is temporaldifference learning  a stochastic algorithm inspired to some extent by phenomena observed in animal behavior. Given a selection of...
2009a): “A Dynamic Oligopoly Game of the US Airline Industry: Estimation and Policy Experiments
"... This paper estimates the contribution of demand, cost and strategic factors to explain why most companies in the US airline industry operate using a hubspoke network. We postulate and estimate a dynamic oligopoly model where airline companies decide, every quarter, which routes (directional citypa ..."
Abstract

Cited by 32 (9 self)
 Add to MetaCart
This paper estimates the contribution of demand, cost and strategic factors to explain why most companies in the US airline industry operate using a hubspoke network. We postulate and estimate a dynamic oligopoly model where airline companies decide, every quarter, which routes (directional citypairs) to operate, the type of product (direct flight vs. stopflight), and the fare of each routeproduct. The model incorporates three factors which may contribute to the profitability of hubspoke networks. First, consumers may value the scale of operation of an airline in the origin and destination airports (e.g., more convenient checkingin and landing facilities). Second, operating costs and entry costs may depend on the airline’s network because economies of density and scale. And third, a hubspoke network may be an strategy to deter the entry of non hubspoke carriers in some routes. We estimate our dynamic oligopoly model using panel data from the Airline Origin and Destination Survey with information on quantities, prices, and entry and exit decisions for every airline company over more than two thousand citypair markets and several years. Demand and variable cost parameters are estimated using demand equations and NashBertrand equilibrium conditions for prices. In a second step, we estimate fixed operating costs and sunk costs from the dynamic entryexit game. Counterfactual experiments show that hubsize effects on entry costs is, by far, the most important factor to explain hubspoke networks. Strategic entry deterrence is also significant and more important to explain hubspoke networks than hubsize effects on demand, variable costs or fixed costs.