This paper discusses unsupervised learning of orthogonal concepts on relational data. Relational predicates, while formally equivalent to the features of the concept-learning literature, are not a good basis for defining concepts. Hence the currenttaskdemandsamuch larger search space than traditional concept learning algorithms, the sort of space explored by connectionist algorithms. However the intended application, using the discovered concepts in the Cyc knowledge base, requires that the concepts be interpretable byahuman, an ability not yet realized with connectionist algorithms. Interpretability is aided by including a characterization of simplicity in the evaluation function. For Hinton's Family Relations data, we do find cleaner, more intuitive features. Yet when the solutions are not known in advance, the difficultyofinterpreting even features meeting the simplicity criteria calls into question the usefulness of any reformulation algorithm that creates radically new primitives in a knowledge-based setting. At the very least, much more sophisticated explanation tools are needed.

After the initial relevance of deduction in AI, nowadays, inductive learning, and all its varieties (abductive reasoning, reasoning by analogy, connectionism, ILP, grammatical inference, HMM, EBR, etc.), are beginning to play a more central and still agglutinative role in the proper subset of AI devoted to make intelligent machines. The issue has been much clearer in discovery science, where induction has been the prominent inference process. In this trend, new unified frameworks for understanding reasoning have been appearing, with the aim of integrating all the different inference mechanims [25]. In particular, a radical approach has been undertaken by Wolff, with the claim that “all kinds of computing and formal reasoning may usefully be understood as information compression by pattern matching, unification and search ” [46]. In this paper, we will discuss critically a very influential and now classical issue in this line, which is based on the view of unsupervised learning as compression [38], its famous operative Minimum Description Length (MDL)