Results 1  10
of
11
Solving factored MDPs with hybrid state and action variables
 J. Artif. Intell. Res. (JAIR
"... Efficient representations and solutions for large decision problems with continuous and discrete variables are among the most important challenges faced by the designers of automated decision support systems. In this paper, we describe a novel hybrid factored Markov decision process (MDP) model tha ..."
Abstract

Cited by 18 (4 self)
 Add to MetaCart
(Show Context)
Efficient representations and solutions for large decision problems with continuous and discrete variables are among the most important challenges faced by the designers of automated decision support systems. In this paper, we describe a novel hybrid factored Markov decision process (MDP) model that allows for a compact representation of these problems, and a new hybrid approximate linear programming (HALP) framework that permits their efficient solutions. The central idea of HALP is to approximate the optimal value function by a linear combination of basis functions and optimize its weights by linear programming. We analyze both theoretical and computational aspects of this approach, and demonstrate its scaleup potential on several hybrid optimization problems. 1.
Provably Efficient Learning with Typed Parametric Models
"... To quickly achieve good performance, reinforcementlearning algorithms for acting in large continuousvalued domains must use a representation that is both sufficiently powerful to capture important domain characteristics, and yet simultaneously allows generalization, or sharing, among experiences. ..."
Abstract

Cited by 9 (3 self)
 Add to MetaCart
(Show Context)
To quickly achieve good performance, reinforcementlearning algorithms for acting in large continuousvalued domains must use a representation that is both sufficiently powerful to capture important domain characteristics, and yet simultaneously allows generalization, or sharing, among experiences. Our algorithm balances this tradeoff by using a stochastic, switching, parametric dynamics representation. We argue that this model characterizes a number of significant, realworld domains, such as robot navigation across varying terrain. We prove that this representational assumption allows our algorithm to be probably approximately correct with a sample complexity that scales polynomially with all problemspecific quantities including the statespace dimension. We also explicitly incorporate the error introduced by approximate planning in our sample complexity bounds, in contrast to prior Probably Approximately Correct (PAC) Markov Decision Processes (MDP) approaches, which typically assume the estimated MDP can be solved exactly. Our experimental results on constructing plans for driving to work using real car trajectory data, as well as a small robot experiment on navigating varying terrain, demonstrate that our dynamics representation enables us to capture realworld dynamics in a sufficient manner to produce good performance.
Learning Basis Functions in Hybrid Domains
 In Proceedings of the 2006 National Conference on Artificial Intelligence (AAAI
, 2006
"... Markov decision processes (MDPs) with discrete and continuous state and action components can be solved efficiently by hybrid approximate linear programming (HALP). The main idea of the approach is to approximate the optimal value function by a set of basis functions and optimize their weights by li ..."
Abstract

Cited by 5 (2 self)
 Add to MetaCart
(Show Context)
Markov decision processes (MDPs) with discrete and continuous state and action components can be solved efficiently by hybrid approximate linear programming (HALP). The main idea of the approach is to approximate the optimal value function by a set of basis functions and optimize their weights by linear programming. The quality of this approximation naturally depends on its basis functions. However, basis functions leading to good approximations are rarely known in advance. In this paper, we propose a new approach that discovers these functions automatically. The method relies on a class of parametric basis function models, which are optimized using the dual formulation of a relaxed HALP. We demonstrate the performance of our method on two hybrid optimization problems and compare it to manually selected basis functions.
Planning in Hybrid Structured Stochastic Domains
, 2006
"... Efficient representations and solutions for large structured decision problems with continuous and discrete variables are among the important challenges faced by the designers of automated decision support systems. In this work, we describe a novel hybrid factored Markov decision process (MDP) mod ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
Efficient representations and solutions for large structured decision problems with continuous and discrete variables are among the important challenges faced by the designers of automated decision support systems. In this work, we describe a novel hybrid factored Markov decision process (MDP) model that allows for a compact representation of these problems, and a hybrid approximate linear programming (HALP) framework that permits their efficient solutions. The central idea of HALP is to approximate the optimal value function of an MDP by a linear combination of basis functions and optimize its weights by linear programming. We study both theoretical and practical aspects of this approach, and demonstrate its scaleup potential on several hybrid optimization problems
Clinical Time Series Prediction: Towards A Hierarchical Dynamical System Framework
, 2014
"... Developing machine learning and data mining algorithms for building temporal models of clinical time series is important for understanding of patient state, the dynamics of a disease, effect of various patient management interventions and clinical decision making. In this work, we propose and deve ..."
Abstract
 Add to MetaCart
(Show Context)
Developing machine learning and data mining algorithms for building temporal models of clinical time series is important for understanding of patient state, the dynamics of a disease, effect of various patient management interventions and clinical decision making. In this work, we propose and develop a novel hierarchical framework for modeling clinical time series that combines advantages of the two approaches: Linear Dynamical Systems (LDS) and Gaussian Processes (GP). The new framework is more flexible than the two approaches individually in that it can model and learn from time series data of varied length and with irregularly sampled observations. We test our framework on the problem of learning time series models for the complete blood count panel, and show that it outperforms the existing models in terms of its predictive accuracy.
unknown title
"... Probabilistic inference for structured planning in robotics Abstract — Realworld robotic environments are highly structured. The scalability of planning and reasoning methods to cope with complex problems in such environments crucially depends on exploiting this structure. We propose a new approach ..."
Abstract
 Add to MetaCart
(Show Context)
Probabilistic inference for structured planning in robotics Abstract — Realworld robotic environments are highly structured. The scalability of planning and reasoning methods to cope with complex problems in such environments crucially depends on exploiting this structure. We propose a new approach to planning in robotics based on probabilistic inference. The method uses structured Dynamic Bayesian Networks to represent the scenario and efficient inference techniques (loopy belief propagation) to solve planning problems. In principle, any kind of factored or hierarchical state representations can be accounted for. We demonstrate the approach on reaching tasks under collision avoidance constraints with a humanoid upper body. I.
On the Smoothness of Linear Value Function Approximations
, 2006
"... Markov decision processes (MDPs) with discrete and continuous state and action components can be solved efficiently by hybrid approximate linear programming (HALP). The main idea of the approach is to approximate the optimal value function by a set of basis functions and optimize their weights b ..."
Abstract
 Add to MetaCart
Markov decision processes (MDPs) with discrete and continuous state and action components can be solved efficiently by hybrid approximate linear programming (HALP). The main idea of the approach is to approximate the optimal value function by a set of basis functions and optimize their weights by linear programming. It is known that the solution to this convex optimization problem minimizes the L1 norm distance in between the optimal value function and its approximation. In this paper, we relate this measure to the maxnorm error of the same value function. We believe that this theoretical analysis may help to understand the quality of HALP approximations in continuous domains.
Learning Basis Functions in Hybrid Domains
 IN PROCEEDINGS OF THE 21ST NATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE
, 2006
"... Markov decision processes (MDPs) with discrete and continuous state and action components can be solved efficiently by hybrid approximate linear programming (HALP). The main idea of the approach is to approximate the optimal value function by a set of basis functions and optimize their weights b ..."
Abstract
 Add to MetaCart
Markov decision processes (MDPs) with discrete and continuous state and action components can be solved efficiently by hybrid approximate linear programming (HALP). The main idea of the approach is to approximate the optimal value function by a set of basis functions and optimize their weights by linear programming. The quality of this approximation naturally depends on its basis functions. However, basis functions leading to good approximations are rarely known in advance. In this paper,
Modeling Clinical Time Series Using Gaussian Process Sequences
"... Development of accurate models of complex clinical time series data is critical for understanding the disease, its dynamics, and subsequently patient management and clinical decision making. Clinical time series differ from other time series applications mainly in that observations are often missing ..."
Abstract
 Add to MetaCart
(Show Context)
Development of accurate models of complex clinical time series data is critical for understanding the disease, its dynamics, and subsequently patient management and clinical decision making. Clinical time series differ from other time series applications mainly in that observations are often missing and made at irregular time intervals. In this work, we propose and test a new probabilistic approach for modeling clinical time series data that is optimized to handle irregularly sampled observations. Our model is defined by a sequence of Gaussian processes (GPs), each restricted to a window of a finite size, where dependencies among two consecutive Gaussian processes are represented using a linear dynamical system. We develop algorithms supporting both model learning and inference. Experiments on realworld clinical time series data show that our model is better for modeling clinical time series and that it outperforms or is close to alternative time series prediction models. 1
Mining of predictive patterns in Electronic health records data
"... The emergence of largescale datasets in health care that record large amounts of information about the patients, their diseases and treatments and provide us with an opportunity to understand better the dynamics of the disease, efficacy of treatments, and various influences affecting the wellbei ..."
Abstract
 Add to MetaCart
(Show Context)
The emergence of largescale datasets in health care that record large amounts of information about the patients, their diseases and treatments and provide us with an opportunity to understand better the dynamics of the disease, efficacy of treatments, and various influences affecting the wellbeing of a patient. The development of computer