Language models are probability distributions over a set of unilingual natural language text used in many natural language processing tasks such as statistical machine trans-lation, information retrieval, and speech processing. Since more well-formed training data means a better model and the increased availability of text via the Internet, the size of language modelling n-gram data sets have grown exponentially the past few years. The latest data sets available can no longer fit on a single computer. A recent investi-gation reported first known use of a probabilistic data structure to create a randomised language model capable of storing probability information for massive n-gram sets in a fraction of the space normally needed. We report and compare the properties of lossy language models using two probabilistic data structures: the Bloom filter and lossy dictionary. The Bloom filter has exceptional space requirements and only one-sided, false positive error returns but it is computationally slow in scale which is a potential drawback for a structure being queried millions of times per sentence. Lossy dictionar-ies have low space requirements and are very fast but with two-sided error that returns

Discriminative models are a class of learning methods where the focus is on learning class memberships, as opposed to Generative models, where the interest is in full class densities. While several approaches to discriminative modelling exist, we concentrate on the Maximum Entropy Framework, based on a theoretical argument developed by Jaynes [1957]. Maximum Entropy methods are feature-based: in order to infer an empirical distribution from the data they encode relevant statistics using features. In general, the quality of the model grows with the number and scope of features: unfortunately, the computational and memory resources needed to manipulate them also grow accordingly, often to an unmanageable extent. We investigate the possibility of representing features using randomised tech-niques. Exploring one class of important one-sided error randomised data struc-tures derived from the Bloom Filter, our study concentrates on the logarithmic-frequency Bloom Filter [Talbot and Osborne, 2007a,b] and the Bloom Map [Talbot

Discriminative models are a class of learning methods where the focus is on learning class memberships, as opposed to Generative models, where the interest is in full class densities. While several approaches to discriminative modelling exist, we concentrate on the Maximum Entropy Framework, based on a theoretical argument developed by Jaynes [1957]. Maximum Entropy methods are featurebased: in order to infer an empirical distribution from the data they encode relevant statistics using features. In general, the quality of the model grows with the number and scope of features: unfortunately, the computational and memory resources needed to manipulate them also grow accordingly, often to an unmanageable extent. We investigate the possibility of representing features using randomised techniques. Exploring one class of important one-sided error randomised data structures derived from the Bloom Filter, our study concentrates on the logarithmicfrequency Bloom Filter [Talbot and Osborne, 2007a,b] and the Bloom Map [Talbot and Talbot, 2008]. Both are introduced and tested in a discriminative learning