A Universal Compression Perspective of Smoothing
BibTeX
@MISC{Jevtić_auniversal,
author = {Nikola Jevtić and Alon Orlitsky},
title = {A Universal Compression Perspective of Smoothing},
year = {}
}
OpenURL
Abstract
We analyze smoothing algorithms from a universal-compression perspective. Instead of evaluating their performance on an empirical sample, we analyze their performance on the most inconvenient sample possible. Consequently the performance of the algorithm can be guaranteed even on unseen data. We show that universal compression bounds can explain the empirical performance of several smoothing methods. We also describe a new interpolated additive smoothing algorithm, and show that it has lower training complexity and better compression performance than existing smoothing techniques. Key words: Language modeling, universal compression, smoothing 1







