Abstract—In certain contexts, maximum entropy (ME) modeling can be viewed as maximum likelihood (ML) training for exponential models, and like other ML methods is prone to overfitting of training data. Several smoothing methods for ME models have been proposed to address this problem, but previous results do not make it clear how these smoothing methods compare with smoothing methods for other types of related models. In this work, we survey previous work in ME smoothing and compare the performance of several of these algorithms with conventional techniques for smoothing-gram language models. Because of the mature body of research in-gram model smoothing and the close connection between ME and conventional-gram models, this domain is well-suited to gauge the performance of ME smoothing methods. Over a large number of data sets, we find that fuzzy ME smoothing performs as well as or better than all other algorithms under consideration. We contrast this method with previous-gram smoothing methods to explain its superior performance. Index Terms—Exponential models, language modeling, maximum entropy, minimum divergence,-gram models, smoothing.

A maximum entropy (ME) model is usually estimated so that it conforms to equality constraints on feature expectations.

Abstract. We present a unified and complete account of maximum entropy distribution estimation subject to constraints represented by convex potential functions or, alternatively, by convex regularization. We provide fully general performance guarantees and an algorithm with a complete convergence proof. As special cases, we can easily derive performance guarantees for many known regularization types, including ℓ1, ℓ2, ℓ 2 2 and ℓ1 + ℓ 2 2 style regularization. Furthermore, our general approach enables us to use information about the structure of the feature space or about sample selection bias to derive entirely new regularization functions with superior guarantees. We propose an algorithm solving a large and general subclass of generalized maxent problems, including all discussed in the paper, and prove its convergence. Our approach generalizes techniques based on information geometry and Bregman divergences as well as those based more directly on compactness. 1

al Examples " was the winner of the 1995 ASME Adaptive Structures "Best Paper Award in Structural Dynamics and Control." This is one of two annual awards issued in the field of Smart Structures and Materials by the ASME Committee on "Adaptive Structures and Material Systems." The paper first appeared as an ICASE report in April 1992. It addresses the computational and theoretical foundation for the actual experiments carried out at NASA Langley in 1995. The calculation predicted a 20db reduction in vibration and noise, versus the 18db reduction achieved in the experiment. Inside this Issue Manuel Salas : : : : : : : : : : : : : : : : : : : : : : : : : : : : page 2 Hans Mark : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : page 3 M.Y. Hussaini : : : : : : : : : : : : : : : : : : : : : : : : : : : page 4 Control volume mixed finite-element : : : : : page 6 LES of of periodic shear flow : : : : : :

A maximum entropy (ME) model is usually estimated so that it conforms to equality constraints on feature expectations. However, the equality constraint is inappropriate for sparse and therefore unreliable features. This study explores an ME model with box-type inequality constraints, where the equality can be violated to reflect this unreliability. We evaluate the inequality ME model using text categorization datasets. We also propose an extension of the inequality ME model, which results in a natural integration with the Gaussian MAP estimation. Experimental results demonstrate the advantage of the inequality models and the proposed extension. 1