## The Minimax Strategy for Gaussian Density Estimation (2000)

Venue: | PROC. 13TH ANNU. CONFERENCE ON COMPUT. LEARNING THEORY |

Citations: | 8 - 1 self |

### BibTeX

@INPROCEEDINGS{Takimoto00theminimax,

author = {Eiji Takimoto and Manfred Warmuth},

title = {The Minimax Strategy for Gaussian Density Estimation},

booktitle = {PROC. 13TH ANNU. CONFERENCE ON COMPUT. LEARNING THEORY},

year = {2000},

pages = {100--106},

publisher = {Morgan Kaufmann}

}

### Years of Citing Articles

### OpenURL

### Abstract

We consider on-line density estimation with a Gaussian of unit variance. In each trial t the learner predicts a mean t . Then it receives an instance x t chosen by the adversary and incurs loss 1 2 ( t x t ) 2 . The performance of the learner is measured by the regret defined as the total loss of the learner minus the total loss of the best mean parameter chosen off-line. We assume that the horizon T of the protocol is fixed and known to both parties. We give the optimal strategies for both the learner and the adversary. The value of the game is 1 2 X 2 (ln T ln ln T +O(ln ln T= ln T )), where X is an upper bound of the 2-norm of instances. We also consider the standard algorithm that predicts with t = P t 1 q=1 x q =(t 1 + a) for a fixed a. We show that the regret of this algorithm is 1 2 X 2 (ln T O(1)) regardless of the choice of a.

### Citations

275 |
Fisher information and stochastic complexity
- Rissanen
- 1996
(Show Context)
Citation Context ...s the same as the minimax regret for Bernoulli density estimation. Our work on the minimax regret is different from a large body of work that has its roots in the Minimum Description Length community =-=[6, 7, 12, 9, 10, 13]-=-. In short we require the learner to choose its on-line parameters from the same model class from which the best off-line parameter is chosen. We will discuss the differences in Section 3. In this pap... |

151 | Universal portfolios
- Cover
- 1991
(Show Context)
Citation Context ...izon T is X2 T∑ ct = 2 1 2 X2 (ln T − ln ln T + O(ln ln T/ ln T )). (1) t=1 The − ln ln T term is surprising because many on-line games were shown to have O(ln T ) upper bounds for the minimax regret =-=[2, 5, 3, 11, 1, 13, 14]-=-. There are some intriguing properties of the optimal strategies of both parties. First, the learner does not need to know the upper bound X on the 2-norm of the instances. Second, the strategies we g... |

126 |
Universal sequential coding of single messages
- Shtarkov
- 1987
(Show Context)
Citation Context ... it. For the analogous problem of density estimation over a discrete domain w.r.t. log loss, Shtarkov gave the minimax strategy and an implicit form of the value of the game called the minimax regret =-=[8]-=-. Freund [3] gives an explicit formula for the minimax regret for Bernoulli density estimation: (1/2) ln(T + 1) + ln(π/2) − O(1/ √ T ). The minimax strategy has also been computed for the universal po... |

115 | Relative loss bounds for on-line density estimation with the exponential family of distributions
- AZOURY, WARMUTH
- 2001
(Show Context)
Citation Context ...izon T is X2 T∑ ct = 2 1 2 X2 (ln T − ln ln T + O(ln ln T/ ln T )). (1) t=1 The − ln ln T term is surprising because many on-line games were shown to have O(ln T ) upper bounds for the minimax regret =-=[2, 5, 3, 11, 1, 13, 14]-=-. There are some intriguing properties of the optimal strategies of both parties. First, the learner does not need to know the upper bound X on the 2-norm of the instances. Second, the strategies we g... |

66 | Approximate Solutions to Markov Decision Processes
- Gordon
- 1999
(Show Context)
Citation Context ...e. The initial instance is chosen to be zero for Gaussian density estimation. This prediction algorithm is the forward algorithm of [1]. The same algorithm was investigated in parallel work by Gordon =-=[4]-=-. The forward algorithm was inspired by a similar related algorithm of Vovk for linear regression [11]. We show that the regret of the forward algorithm is larger than 1 2X2 (ln T − O(1)) regardless o... |

55 |
Asymptotic minimax regret for data compression, gambling, and prediction
- Xie, Barron
- 1996
(Show Context)
Citation Context ...s the same as the minimax regret for Bernoulli density estimation. Our work on the minimax regret is different from a large body of work that has its roots in the Minimum Description Length community =-=[6, 7, 12, 9, 10, 13]-=-. In short we require the learner to choose its on-line parameters from the same model class from which the best off-line parameter is chosen. We will discuss the differences in Section 3. In this pap... |

49 |
A decision-theoretic extension of stochastic complexity and its applications to learning
- Yamanishi
- 1998
(Show Context)
Citation Context ...s the same as the minimax regret for Bernoulli density estimation. Our work on the minimax regret is different from a large body of work that has its roots in the Minimum Description Length community =-=[6, 7, 12, 9, 10, 13]-=-. In short we require the learner to choose its on-line parameters from the same model class from which the best off-line parameter is chosen. We will discuss the differences in Section 3. In this pap... |

39 | Predicting a binary sequence almost as well as the optimal biased coin
- Freund
- 1996
(Show Context)
Citation Context ... analogous problem of density estimation over a discrete domain w.r.t. log loss, Shtarkov gave the minimax strategy and an implicit form of the value of the game called the minimax regret [8]. Freund =-=[3]-=- gives an explicit formula for the minimax regret for Bernoulli density estimation: (1/2) ln(T + 1) + ln(π/2) − O(1/ √ T ). The minimax strategy has also been computed for the universal portfolio prob... |

21 | Stochastic Complexity in Learning - Rissanen |

20 |
The cost of achieving the best portfolio in hindsight
- Ordentlich, Cover
- 1998
(Show Context)
Citation Context ...es an explicit formula for the minimax regret for Bernoulli density estimation: (1/2) ln(T + 1) + ln(π/2) − O(1/ √ T ). The minimax strategy has also been computed for the universal portfolio problem =-=[5]-=-. In this case the strategy is not ef£ciently computable, but the minimax regret for the universal portfolio problem is the same as the minimax regret for Bernoulli density estimation. Our work on the... |

17 |
Competitive on-line linear regression
- Vovk
- 1998
(Show Context)
Citation Context ...izon T is X2 T∑ ct = 2 1 2 X2 (ln T − ln ln T + O(ln ln T/ ln T )). (1) t=1 The − ln ln T term is surprising because many on-line games were shown to have O(ln T ) upper bounds for the minimax regret =-=[2, 5, 3, 11, 1, 13, 14]-=-. There are some intriguing properties of the optimal strategies of both parties. First, the learner does not need to know the upper bound X on the 2-norm of the instances. Second, the strategies we g... |

12 |
Asymptotically minimax regret by Bayes mixtures
- Takeuchi, Barron
- 1998
(Show Context)
Citation Context |

10 |
Asymptotically minimax regret for exponential and curved exponential families
- Takeuchi, Barron
- 1998
(Show Context)
Citation Context |

5 | Extended stochastic complexity and minimax relative loss analysis - Yamanishi - 1999 |

1 |
Extended stochastic complexity and minimax relative loss analysis
- Yamanshi
- 1999
(Show Context)
Citation Context |