Abstract

This article describes an approach to combining symbolic and connectionist approaches to machine learning. A three-stage framework is presented and the research of several groups is reviewed with respect to this framework. The first stage involves the insertion of symbolic knowledge into neural networks, the second addresses the refinement of this prior knowledge in its neural representation, while the third concerns the extraction of the refined symbolic knowledge. Experimental results and open research issues are discussed.

Citations

2888	Induction of Decision Trees - Quinlan - 1986
2307	Learning internal representations by error propagation - Rumelhart, Hinton, et al. - 1986
1314	Finding structure in time - Elman - 1990
1117	Clustering Algorithms - Hartigan - 1975
962	Efficient Induction of Logic Programs - Muggleton, Feng
784	Learning logical definitions from relations - Quinlan - 1989
665	Adaptive mixtures of local experts - Jacobs, Jordan, et al. - 1991
537	Neural networks and the bias/variance dilemma - Geman, Bienenstock, et al. - 1992
413	Parallel networks that learn to pronounce English text - Sejnowski, Rosenberg - 1987
299	Recursive distributed representations - Pollack - 1990
290	Connectionist Learning Procedures - Hinton - 1990
256	Backpropagation applied to handwritten zip code recognition - LeCun, Boser, et al. - 1989
247	Forward models: Supervised learning with a distal teacher - Jordan, Rumelhart - 1992
228	Attractor dynamics and parallelism in a connectionist sequential machine - Jordan - 1986
163	Learning and extracted finite state automata with second-order recurrent neural networks. Neural Comput - Giles, Miller, et al. - 1992
154	Learning distributed representations of concepts - Hinton - 1986
113	Simplifying neural networks by soft weight-sharing - Nowlan, Hinton - 1992
107	Connectionist Expert Systems - Gallant - 1988
94	Explanation-Based Neural Network Learning: A Lifelong Learning Approach - Thrun - 1996
82	Learning from Hints in Neural Networks - Abu–Mostafa - 1990
82	An experimental comparison of symbolic and connectionist learning algorithms - Mooney, Shavlik, et al. - 1989
80	An Empirical Comparison of ID3 and Back-Propagation - Fisher, McKusick - 1986
77	Review of neural networks for speech recognition - Lippmann
73	Symbolic Knowledge and Neural Networks: Insertion, Refinement and Extraction - Towell
69	Rule learning by searching on adapted nets - Fu - 1991
66	Training knowledge-based neural networks to recognize genes in dna sequences - Noordewieer, Towell, et al. - 1991
55	Perceptron trees: A case study in hybrid concept representations - Utgoff - 1989
52	Interpretation of artificial neural networks: Mapping knowledge-based neural networks into rules - Towell, Shavlik - 1992
49	Integration of neural heuristics into knowledge-based inference - Fu - 1989
48	Combining explanation-based and neural learning: An algorithm and empirical results - Shavlik, Towell - 1989
40	Medical diagnostic expert system based on pdp model - Saito, Nakano - 1988
39	Higher order recurrent networks and grammatical inference - Giles, Sun, et al. - 1990
39	neural expert system with automated extraction of fuzzy if-then rules - Hayashi, “A - 1991
35	Refinement of approximately correct domain theories by knowledge-based neural networks - Towell, Shavlik, et al. - 1990
33	Compilation and analysis of Escherichia coli promoter DNA sequences - Hawley, McClure - 1983
32	Training second-order recurrent neural networks using hints - Omlin - 1992
30	Rule induction through integrated symbolic and subsymbolic processing - McMillan, Mozer, et al. - 1992
25	Heuristically expanding knowledge-based neural networks - Opitz, Shavlik - 1993
24	A unified approach for integrating explicit knowledge and learning by example in recurrent networks - Frasconi, Gori, et al. - 1991
21	Performance comparisons between backpropagation Network Classification Trees On Real Time World Application - Atlas - 1990
18	Refinement of approximate reasoning-based controllers by reinforcement learning - Berenji - 1990
18	R.J.: Combining neural and symbolic learning to revise probabilistic rule bases - Mahoney, Mooney - 1993
18	Locating protein coding regions in human DNA sequences by a multiple sensor - neural network approach - Uberbacher, Mural - 1991
16	Combining Artificial Neural Networks and Symbolic Processing for Autonomous Robot Guidance - Pomerleau, Gowdy, et al. - 1991
13	Neural control for rolling mills: Incorporating domain theories to overcome data deficiency - Roscheisen, Hofmann - 1991
13	Refining PID Controllers using Neural Networks - Scott, Shavlik, et al. - 1992
11	Preface to the special issue on connectionist symbol processing - Hinton - 1990
10	Escherichia coli promoters - O'Neill, Chiafari - 1989
5	Finite state automata and simple recurrent networks - Clecremans, Servan-Schreiber, et al. - 1989
4	Neural networks and the bias/variance dilemma - Gemen, Bienenstock, et al. - 1992