Abstract

We show several PAC-style concentration bounds for learning unigrams language model. One interesting quantity is the probability of all words appearing exactly k times in a sample of size m. A standard estimator for this quantity is the Good-Turing estimator. The existing analysis on its error shows a PAC bound of approximately O . We improve its dependency on k to O 4 # k . We also analyze the empirical frequencies estimator, showing that its PAC error bound is approximately . We derive a combined estimator, which has , for any k. A standard measure...

Citations