Results

**1 - 2**of**2**### © 2007 INFORMS A �2/3�n 3 Fast-Pivoting Algorithm for the Gittins Index and Optimal Stopping of a Markov Chain

"... This paper presents a new fast-pivoting algorithm that computes the n Gittins index values of an n-state bandit—in the discounted and undiscounted cases—by performing �2/3�n3 + O�n2 � arithmetic operations, thus attaining better complexity than previous algorithms and matching that of solving a corr ..."

Abstract
- Add to MetaCart

This paper presents a new fast-pivoting algorithm that computes the n Gittins index values of an n-state bandit—in the discounted and undiscounted cases—by performing �2/3�n3 + O�n2 � arithmetic operations, thus attaining better complexity than previous algorithms and matching that of solving a corresponding linearequation system by Gaussian elimination. The algorithm further applies to the problem of optimal stopping of a Markov chain, for which a novel Gittins-index solution approach is introduced. The algorithm draws on Gittins and Jones ’ (1974) index definition via calibration, on Kallenberg’s (1986) proposal of using parametric linear programming, on Dantzig’s simplex method, on the Varaiya et al. (1985) algorithm, and on the author’s earlier work. This paper elucidates the structure of parametric simplex tableaux. Special structure is exploited to reduce the computational effort of pivot steps, decreasing the operation count by a factor of three relative to conventional pivoting, and by a factor of 3/2 relative to recent state-elimination algorithms. A computational study demonstrates significant time savings against alternative algorithms.

### 1 Asymptotic optimal control of multi-class restless bandits

, 2013

"... We study the asymptotic optimal control of multi-class restless bandits. A restless bandit is a controllable process whose state evolution depends on whether or not the bandit is made active. The aim is to find a control that determines at each decision epoch which bandits to make active in order to ..."

Abstract
- Add to MetaCart

We study the asymptotic optimal control of multi-class restless bandits. A restless bandit is a controllable process whose state evolution depends on whether or not the bandit is made active. The aim is to find a control that determines at each decision epoch which bandits to make active in order to minimize the overall average cost associated to the states the bandits are in. Since finding the optimal control is typically intractable, we study an asymptotic regime instead that is obtained by letting the number of bandits that can be simultaneously made active grow proportionally with the population of bandits. We consider both a fixed population of bandits as well as a dynamic population of bandits where bandits can depart and new bandits can arrive over time to the system. We propose a class of priority policies, obtained by solving a linear program, that are proved to be asymptotically optimal under a global attractor property and a technical condition. Indexability of the bandits is not required for the result to hold. For a fixed population of bandits, the technical condition reduces to checking a unichain property. For a dynamic population of bandits we present a large class of restless bandit problems for which the technical condition is always satisfied. As an example, we present a multi-class M/M/S+M queue, which is inside this class of problems and satisfies the global attractor