The paper introduces a generalization for known probabilistic models such as log-linear and graphical models, called here multiplicative models. These models, that express probabilities via product of parameters are shown to capture multiple forms of contextual independence between variables, including decision graphs and noisy-OR functions. An inference algorithm for multiplicative models is provided and its correctness is proved. The complexity analysis of the inference algorithm uses a more refined parameter than the tree-width of the underlying graph, and shows the computational cost does not exceed that of the variable elimination algorithm in graphical models. The paper ends with examples where using the new models and algorithm is computationally beneficial.

Abstract. Recent studies in classification have proposed ways of exploiting the association rule mining paradigm. These studies have performed extensive experiments to show their techniques to be both efficient and accurate. However, existing studies in this paradigm either do not provide any theoretical justification behind their approaches or assume independence between some parameters. In this work, we propose a new classifier based on association rule mining. Our classifier rests on the maximum entropy principle for its statistical basis and does not assume any independence not inferred from the given dataset. We use the classical generalized iterative scaling algorithm (GIS) to create our classification model. We show that GIS fails in some cases when itemsets are used as features and provide modifications to rectify this problem. We show that this modified GIS runs much faster than the original GIS. We also describe techniques to make GIS tractable for large feature spaces – we provide a new technique to divide a feature space into independent clusters each of which can be handled separately. Our experimental results show that our classifier is generally more accurate than the existing classification methods. 1

We describe a maximum entropy approach for computing volumes and counting integer points in polyhedra. To estimate the number of points from a particular set X ⊂ R n in a polyhedron P ⊂ R n, by solving a certain entropy maximization problem, we construct a probability distribution on the set X such that a) the probability mass function is constant on the set P ∩X and b) the expectation of the distribution lies in P. This allows us to apply Central Limit Theorem type arguments to deduce computationally efficient approximations for the number of integer points, volumes, and the number of 0-1 vectors in the polytope. As an application, we obtain asymptotic formulas for volumes of multi-index transportation polytopes and for the number of multi-way contingency tables. 1.

This paper describes AUEB’s participation in TAC 2009. Specifically, we participated in the textual entailment recognition track for which we used string similarity measures applied to shallow abstractions of the input sentences, and a Maximum Entropy classifier to learn how to combine the resulting features. We also exploited Word-Net to detect synonyms and a dependency parser to measure similarity in the grammatical structure of T and H. 1

This paper presents three methods that can be used to recognize paraphrases. They all employ string similarity measures applied to shallow abstractions of the input sentences, and a Maximum Entropy classifier to learn how to combine the resulting features. Two of the methods also exploit WordNet to detect synonyms and one of them also exploits a dependency parser. We experiment on two datasets, the MSR paraphrasing corpus and a dataset that we automatically created from the MTC corpus. Our system achieves state of the art or better results. 1

Abstract. Kullback-Leibler relative-entropy, in cases involving distributions resulting from relative-entropy minimization, has a celebrated property reminiscent of squared Euclidean distance: it satisfies an analogue of the Pythagoras ’ theorem. And hence, this property is referred to as Pythagoras ’ theorem of relative-entropy minimization or triangle equality and plays a fundamental role in geometrical approaches of statistical estimation theory like information geometry. Equvalent of Pythagoras’ theorem in the generalized nonextensive formalism is established in (Dukkipati at

While production statistics are reported on a geopolitical – often national- basis we often need to know, for example, the status of production or productivity within specific sub-regions, watersheds, or agro-ecological zones. Such re-aggregations are typically made using expert judgments or simple area-weighting rules. We describe a new, entropy-based approach to the plausible estimates of the spatial distribution of crop production. Using this approach tabular crop production statistics are blended judiciously with an array of other secondary data to assess the production of specific crops within individual ‘pixels ’ – typically 1 to 25 square kilometers in size. The information utilized includes crop production statistics, farming system characterization, satellite-based interpretation of land cover, biophysical crop suitability assessments, and population density. An application is presented in which Brazilian state level production statistics are used to generate pixel level crop production data for eight crops. To validate the spatial allocation we aggregated the pixel estimates to obtain synthetic estimates of municipio level production in Brazil, and compared those estimates with actual municipio statistics. The approach produced extremely promising results. We then examined the robustness of these results compared to short-cut approaches to spatializing crop production statistics and showed that, while computationally intensive, the cross-entropy method does provide more reliable spatial allocations.

Abstract. Let R = (r1,..., rm) and C = (c1,..., cn) be positive integer vectors such that r1 +... + rm = c1 +... + cn. We consider the set Σ(R, C) of non-negative m × n integer matrices (contingency tables) with row sums R and column sums C as a finite probability space with the uniform measure. We prove that a random table D ∈ Σ(R, C) is close with high probability to a particular matrix (“typical table”) Z defined as follows. We let g(x) = (x + 1) ln(x + 1) − x ln x for x ≥ 0 and let g(X) = P ij g(xij) for a non-negative matrix X = (xij). Then g(X) is strictly concave and attains its maximum on the polytope of non-negative m × n matrices X with row sums R and column sums C at a unique point, which we call the typical table Z.

This research deals with experiments or surveys producing multivariate categorical data which is incomplete, in the sense that not all variables of interest are measured on every subject or element of the sample. For the most part, incompleteness is taken to arise by design, rather than by random failure of the measurement process. In these circumstances, one can often assume that counts derived from appropriate disjoint subsets of the data arise from independent multinomial distributions with linearly related parameters. Best asymptotically normal oJ estimates of these parameters may be determined by maximizing the likelihood of the observations or by minimizing Pearson's-x 2, Neyman's X~,

The author(s) shown below used Federal funds provided by the U.S. Department of Justice and prepared the following final report: