Results 1 
3 of
3
A Universal Compression Perspective of Smoothing
"... We analyze smoothing algorithms from a universalcompression perspective. Instead of evaluating their performance on an empirical sample, we analyze their performance on the most inconvenient sample possible. Consequently the performance of the algorithm can be guaranteed even on unseen data. We sho ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
We analyze smoothing algorithms from a universalcompression perspective. Instead of evaluating their performance on an empirical sample, we analyze their performance on the most inconvenient sample possible. Consequently the performance of the algorithm can be guaranteed even on unseen data. We show that universal compression bounds can explain the empirical performance of several smoothing methods. We also describe a new interpolated additive smoothing algorithm, and show that it has lower training complexity and better compression performance than existing smoothing techniques. Key words: Language modeling, universal compression, smoothing 1
Asymptotic Minimax Regret for Data Compression, Gambling, and Prediction
"... Abstract—For problems of data compression, gambling, and prediction of individual sequences I the following questions arise. Given a target family of probability mass functions @ I A, how do we choose a probability mass function @ I A so that it approximately minimizes the maximum regret /belowdispl ..."
Abstract
 Add to MetaCart
Abstract—For problems of data compression, gambling, and prediction of individual sequences I the following questions arise. Given a target family of probability mass functions @ I A, how do we choose a probability mass function @ I A so that it approximately minimizes the maximum regret /belowdisplayskip10ptminus6pt @�� � I @ I A �� � I @ I ” AA and so that it achieves the best constant in the asymptotics of the minimax regret, which is of the form @ PA ��� @ P AC C
Universal Prediction and Universal Coding
"... Although prediction schemes which are named "universal" are now abundant, very little has been addressed as to the definition of universal prediction. This paper addresses for ffnary (ff 2) sequences the criteria of successful universal prediction and the prediction schemes which achieve ..."
Abstract
 Add to MetaCart
Although prediction schemes which are named "universal" are now abundant, very little has been addressed as to the definition of universal prediction. This paper addresses for ffnary (ff 2) sequences the criteria of successful universal prediction and the prediction schemes which achieve the goals. We propose the following criteria: for any probability measures in a given measure class, the error probability of prediction (deterministic prediction) and the conditional probability of the next outcome given the past sequence (stochastic prediction) should converge to the optimal values in probability (weakly universal) and almost surely (strongly universal). We prove several properties with respect to the criteria in which a novel proof for Cover's open problem, which seems to be more simplified and intuitively appealing compared to the previous proofs, is presented. The proposed criteria are derived from an analogy with Davisson's universal coding, just like Feder, Merhav, and Gutman'...