Results 1 
9 of
9
Optimally solving DecPOMDPs as continuousstate MDPs
 in Proceedings of the TwentyThird International Joint Conference on Artificial Intelligence
, 2013
"... Optimally solving decentralized partially observable Markov decision processes (DecPOMDPs) is a hard combinatorial problem. Current algorithms search through the space of full histories for each agent. Because of the doubly exponential growth in the number of policies in this space as the planning ..."
Abstract

Cited by 11 (4 self)
 Add to MetaCart
Optimally solving decentralized partially observable Markov decision processes (DecPOMDPs) is a hard combinatorial problem. Current algorithms search through the space of full histories for each agent. Because of the doubly exponential growth in the number of policies in this space as the planning horizon increases, these methods quickly become intractable. However, in real world problems, computing policies over the full history space is often unnecessary. True histories experienced by the agents often lie near a structured, lowdimensional manifold embedded into the history space. We show that by transforming a DecPOMDP into a continuousstate MDP, we are able to find and exploit these lowdimensional representations. Using this novel transformation, we can then apply powerful techniques for solving POMDPs and continuousstate MDPs. By combining a general search algorithm and dimension reduction based on feature selection, we introduce a novel approach to optimally solve problems with significantly longer planning horizons than previous methods. 1
Point Based Value Iteration with Optimal Belief Compression for DecPOMDPs
"... We present four major results towards solving decentralized partially observable Markov decision problems (DecPOMDPs) culminating in an algorithm that outperforms all existing algorithms on all but one standard infinitehorizon benchmark problems. (1) We give an integer program that solves collabo ..."
Abstract

Cited by 5 (0 self)
 Add to MetaCart
(Show Context)
We present four major results towards solving decentralized partially observable Markov decision problems (DecPOMDPs) culminating in an algorithm that outperforms all existing algorithms on all but one standard infinitehorizon benchmark problems. (1) We give an integer program that solves collaborative Bayesian games (CBGs). The program is notable because its linear relaxation is very often integral. (2) We show that a DecPOMDP with bounded belief can be converted to a POMDP (albeit with actions exponential in the number of beliefs). These actions correspond to strategies of a CBG. (3) We present a method to transform any DecPOMDP into a DecPOMDP with bounded beliefs (the number of beliefs is a free parameter) using optimal (not lossless) belief compression. (4) We show that the combination of these results opens the door for new classes of DecPOMDP algorithms based on previous POMDP algorithms. We choose one such algorithm, pointbased valued iteration, and modify it to produce the first tractable value iteration method for DecPOMDPs that outperforms existing algorithms. 1
Exploiting Separability in Multiagent Planning with
"... Recent years have seen significant advances in techniques for optimally solving multiagent problems represented as decentralized partially observable Markov decision processes (DecPOMDPs). A new method achieves scalability gains by converting DecPOMDPs into continuous state MDPs. This method reli ..."
Abstract
 Add to MetaCart
(Show Context)
Recent years have seen significant advances in techniques for optimally solving multiagent problems represented as decentralized partially observable Markov decision processes (DecPOMDPs). A new method achieves scalability gains by converting DecPOMDPs into continuous state MDPs. This method relies on the assumption of a centralized planning phase that generates a set of decentralized policies for the agents to execute. However, scalability remains limited when the number of agents or problem variables becomes large. In this paper, we show that, under certain separability conditions of the optimal value function, the scalability of this approach can increase considerably. This separability is present when there is locality of interaction, which — as other approaches (such as those based on the NDPOMDP subclass) have already shown — can be exploited to improve performance. Unlike most previous methods, the novel continuousstate MDP algorithm retains optimality and convergence guarantees. Results show that the extension using separability can scale to a large number of agents and domain variables while maintaining optimality.
IS
, 2014
"... HAL is a multidisciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L’archive ouverte p ..."
Abstract
 Add to MetaCart
(Show Context)
HAL is a multidisciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et a ̀ la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.
Proceedings of the TwentyThird International Joint Conference on Artificial Intelligence Optimally Solving DecPOMDPs as ContinuousState MDPs
"... Optimally solving decentralized partially observable Markov decision processes (DecPOMDPs) is a hard combinatorial problem. Current algorithms search through the space of full histories for each agent. Because of the doubly exponential growth in the number of policies in this space as the planning ..."
Abstract
 Add to MetaCart
Optimally solving decentralized partially observable Markov decision processes (DecPOMDPs) is a hard combinatorial problem. Current algorithms search through the space of full histories for each agent. Because of the doubly exponential growth in the number of policies in this space as the planning horizon increases, these methods quickly become intractable. However, in real world problems, computing policies over the full history space is often unnecessary. True histories experienced by the agents often lie near a structured, lowdimensional manifold embedded into the history space. We show that by transforming a DecPOMDP into a continuousstate MDP, we are able to find and exploit these lowdimensional representations. Using this novel transformation, we can then apply powerful techniques for solving POMDPs and continuousstate MDPs. By combining a general search algorithm and dimension reduction based on feature selection, we introduce a novel approach to optimally solve problems with significantly longer planning horizons than previous methods. 1
Exploiting Structure [AAMAS'13]
, 2013
"... incremental clustering incremental expansion sufficient plantime statistics [IJCAI'13] Other/current work ..."
Abstract
 Add to MetaCart
(Show Context)
incremental clustering incremental expansion sufficient plantime statistics [IJCAI'13] Other/current work
ABSTRACT a
"... Recent years have seen significant advances in techniques for optimally solving multiagent problems represented as decentralized partially observable Markov decision processes (DecPOMDPs). A new method achieves scalability gains by converting DecPOMDPs into continuous state MDPs. This method relie ..."
Abstract
 Add to MetaCart
(Show Context)
Recent years have seen significant advances in techniques for optimally solving multiagent problems represented as decentralized partially observable Markov decision processes (DecPOMDPs). A new method achieves scalability gains by converting DecPOMDPs into continuous state MDPs. This method relies on the assumption of a centralized planning phase that generates a set of decentralized policies for the agents to execute. However, scalability remains limited when the number of agents or problem variables becomes large. In this paper, we show that, under certain separability conditions of the optimal value function, the scalability of this approach can increase considerably. This separability is present when there is locality of interaction, which — as other approaches (such as those based on the NDPOMDP subclass) have already shown — can be exploited to improve performance. Unlike most previous methods, the novel continuousstate MDP algorithm retains optimality and convergence guarantees. Results show that the extension using separability can scale to a large number of agents and domain variables while maintaining optimality.