Abstract

We present the learning system Maccent which addresses the novel task of stochastic MAximum ENTropy modeling with Clausal Constraints. Maximum Entropy method is a Bayesian method based on the principle that the target stochastic model should be as uniform as possible, subject to known constraints. Maccent incorporates clausal constraints that are based on the evaluation of Prolog clauses in examples represented as Prolog programs. We build on an existing maximum-likelihood approach to maximum entropy modeling, which we upgrade along two dimensions: (1) Maccent can handle larger search spaces, due to a partial ordering defined on the space of clausal constraints, and (2) uses a richer firstorder logic format. In comparison with other inductive logic programming systems, Maccent seems to be the first that explicitly constructs a conditional probability distribution p(CjI) based on an empirical distribution ~ p(CjI) (where p(CjI) (~p(CjI)) gives the induced (observed) probability of ...

Citations

1654	B.: Building a large annotated corpus of english: The Penn treebank - Marcus, Marcinkiewicz, et al. - 1993
962	Efficient induction of logic programs - Muggleton, Feng - 1990
846	A Maximum Entropy Approach to Natural Language Processing - Berger, Pietra, et al. - 1996
464	Inducing Features of Random Fields - Pietra, Pietra, et al. - 1997
428	A note on inductive generalization - Plotkin - 1970
355	Generalized Iterative Scaling for Log-Linear Models - Darroch, Ratcliff - 1972
274	A Maximum Entropy Part-Of-Speech Tagger - Ratnaparkhi - 1996
141	Theories for mutagenicity: A study in first-order and feature-based induction - Srinivasan, Muggleton, et al. - 1996
135	PROLOG: Programming for artificial intelligence - Bratko - 2001
80	Inductive constraint logic - Raedt, Laer - 1995
68	The Application of Inductive Logic Programming to Finite Element Mesh Design - Dolsak, Muggleton - 1992
63	First order jk-clausal theories are PAClearnable - Raedt, Dˇzeroski - 1994
60	Structural regression trees - Kramer - 1996
43	Image reconstruction from incomplete and noisy data,” Nature (London - Gull, Daniell - 1978
35	First Order Regression - Karalič, Bratko - 1997
23	A note on approximations to discrete probability distributions - Brown - 1959
22	Naive Bayesian classifier within ILP-R - Pompe, Kononenko - 1995
22	Using Logical Decision Trees for Clustering - Raedt, Blockeel - 1997
17	DLAB: A declarative language bias formalism - Dehaspe - 1996
14	Induction in logic - Raedt - 1996
13	Notes on present status and future prospects - Jaynes - 1991
9	Raedt. Experiments with top-down induction of logical decision trees - Blockeel, De - 1997
9	Probabilistic First-Order Classification - Pompe, Kononenko - 1997
4	Bayesian Inductive Logic Programming with explicit probabilistic bias - Cussens - 1996
2	Maximum entropy toolkit, release 1.5 beta - Ristad - 1997