Results 1 
4 of
4
Efficient planning in Rmax
 In Proc. of AAMAS
, 2011
"... PACMDP algorithms are particularly efficient in terms of the number of samples obtained from the environment which are needed by the learning agents in order to achieve a near optimal performance. These algorithms however execute a time consuming planning step after each new stateaction pair becom ..."
Abstract

Cited by 3 (1 self)
 Add to MetaCart
PACMDP algorithms are particularly efficient in terms of the number of samples obtained from the environment which are needed by the learning agents in order to achieve a near optimal performance. These algorithms however execute a time consuming planning step after each new stateaction pair becomes known to the agent, that is, the pair has been sampled sufficiently many times to be considered as known by the algorithm. This fact is a serious limitation on broader applications of these kind of algorithms. This paper examines the planning problem in PACMDP learning. Value iteration, prioritized sweeping, and backward value iteration are investigated. Through the exploitation of the specific nature of the planning problem in the considered reinforcement learning algorithms, we show how these planning algorithms can be improved. Our extensions yield significant improvements in all evaluated algorithms, and standard value iteration in particular. The theoretical justification to all contributions is provided and all approaches are further evaluated empirically. With our extensions, we managed to solve problems of sizes which have never been approached by PACMDP learning in the existing literature.
Solving Shortest Paths Efficiently on Nearly Acyclic Directed Graphs
"... Shortest path problems can be solved very efficiently when a directed graph is nearly acyclic. Earlier results defined a graph decomposition, now called the 1dominator set, which consists of a unique collection of acyclic structures with each single acyclic structure dominated by a single associate ..."
Abstract

Cited by 2 (2 self)
 Add to MetaCart
Shortest path problems can be solved very efficiently when a directed graph is nearly acyclic. Earlier results defined a graph decomposition, now called the 1dominator set, which consists of a unique collection of acyclic structures with each single acyclic structure dominated by a single associated trigger vertex. In this framework, a specialised shortest path algorithm only spends deletemin operations on trigger vertices, thereby making the computation of shortest paths through nontrigger vertices easier. A previously presented algorithm computed the 1dominator set in O(mn) worstcase time, which allowed it to be integrated as part of an O(mn + nr log r) time allpairs algorithm. Here m and n respectively denote the number of edges and vertices in the graph, while r denotes the number of trigger vertices. A new algorithm presented in this paper computes the 1dominator set in just O(m) time. This can be integrated as part of the O(m+r log r) time spent solving singlesource, improving on the value of r obtained by the earlier treedecomposition singlesource algorithm. In addition, a new bidirectional form of 1dominator set is presented, which further improves the value of r by defining acyclic structures in both directions over edges in the graph. The bidirectional 1dominator set can similarly be computed in O(m) time and included as part of the O(m + r log r) time spent computing singlesource. This paper also presents a new allpairs algorithm under the more general framework where r is defined as the size of any predetermined feedback vertex set of the graph, improving the previous allpairs time complexity from O(mn + nr 2) to O(mn + r 3).
Thin Heaps, Thick Heaps
, 2006
"... The Fibonacci heap was devised to provide an especially efficient implementation of Dijkstra’s shortest path algorithm. Although asyptotically efficient, it is not as fast in practice as other heap implementations. Expanding on ideas of Høyer, we describe three heap implementations (two versions of ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
The Fibonacci heap was devised to provide an especially efficient implementation of Dijkstra’s shortest path algorithm. Although asyptotically efficient, it is not as fast in practice as other heap implementations. Expanding on ideas of Høyer, we describe three heap implementations (two versions of thin heaps and one of thick heaps) that have the same amortized efficiency as Fibonacci heaps but need less space and promise better practical performance. As part of our development, we fill in a gap in Høyer’s analysis.
Reflected MinMax Heaps
 Information Processing Letters 86
, 2003
"... In this paper we present a simple and e#cient implementation of a minmax priority queue, reflected minmax priority queues. The main merits of our construction are threefold. First, the space utilization of the reflected minmax heaps is much better than the naive solution of putting two heaps b ..."
Abstract
 Add to MetaCart
In this paper we present a simple and e#cient implementation of a minmax priority queue, reflected minmax priority queues. The main merits of our construction are threefold. First, the space utilization of the reflected minmax heaps is much better than the naive solution of putting two heaps backtoback. Second, the methods applied in this structure can be easily used to transform ordinary priority queues into minmax priority queues. Third, when considering only the setting of minmax priority queues, we support merging in constant worstcase time which is a clear improvement over the best worstcase bounds achieved by Hyer.