This chapter provides an introduction to the field of hybrid neural systems. Hybrid neural systems are computational systems which are based mainly on artificial neural networks but also allow a symbolic interpretation, or interaction with symbolic components. In this overview, we will describe recent results of hybrid neural systems. We will give a brief overview of the main methods used, outline the work that is presented here, and provide additional references. We will also highlight some important general issues and trends.

Kohonen's Self-Organizing Map (SOM) is one of the most popular artificial neural network algorithms. Word category maps are SOMs that have been organized according to word similarities, measured by the similarity of the short contexts of the words. Conceptually interrelated words tend to fall into the same or neighboring map nodes. Nodes may thus be viewed as word categories. Although no a priori information about classes is given, during the self-organizing process a model of the word classes emerges. The central topic of the thesis is the use of the SOM in natural language processing. The approach based on the word category maps is compared with the methods that are widely used in artificial intelligence research. Modeling gradience, conceptual change, and subjectivity of natural language interpretation are considered. The main application area is information retrieval and textual data mining for which a specific SOM-based method called the WEBSOM has been developed. The WEBSOM metho...

In this paper 1 we describe a new approach for learning dialog act processing. In this approach we integrate a symbolic semantic segmentation parser with a learning dialog act network. In order to support the unforeseeable errors and variations of spoken language we have concentrated on robust data-driven learning. This approach already compares favorably with the statistical average plausibility method, produces a segmentation and dialog act assignment for all utterances in a robust manner, and reduces knowledge engineering since it can be bootstrapped from rather small corpora. Therefore, we consider this new approach as very promising for learning dialog act processing. 1 This article has been submitted and accepted to the International Conference on Computational Linguistics 1996, Kopenhagen, Denmark. 1 Introduction For several decades, the pragmatic interpretation at a dialog act level belongs to the most difficult and challenging tasks for natural language processing and co...

Previously neural networks have shown interesting performance results for tasks such as classification, but they still suffer from an insufficient focus on the structure of the knowledge represented therein. In this paper, we analyze various knowledge extraction techniques in detail and we develop new transducer extraction techniques for the interpretation of recurrent neural network learning. First, we provide an overview of different possibilities to express structured knowledge using neural networks. Then, we analyze a type of recurrent network rigorously, applying a broad range of different techniques. We argue that analysis techniques, such as weight analysis using Hinton diagrams, hierarchical cluster analysis, and principal component analysis may be useful for providing certain views on the underlying knowledge. However, we demonstrate that these techniques are too static and too low-level for interpreting recurrent network classifications. The contribution of this paper is a particularly broad analysis of knowledge extraction techniques. Furthermore, we propose dynamic learning analysis and transducer extraction as two new dynamic interpretation techniques. Dynamic learning analysis provides a better understanding of how the network learns, while transducer extraction provides a better understanding of what the network represents.

In this paper we present the Deductive Object-oriented Approach (DOA), a new framework for natural language interface design. It makes use of the available powerful logic and object-oriented programming language of deductive object-oriented databases (DOOD) to develop the interface as component of the database system. This provides the basis for a consistent and efficient mapping of the user input to the target representation. Especially object-oriented inheritance mechanisms make it possible to structure the linguistic knowledge hierarchically, which guarantees a compact and natural representation. One of the main difficulties for the practicable use of natural language interfaces is the required high amount of manual knowledge acquisition. For each portation to a new application domain the developer has to repeat this elaborate process. Furthermore, the interfaces are often part of dynamic environments with constant changes concerning the addressed topics or the user population. Therefore, we propose adaptive techniques to deal efficiently with these high demands on dynamic knowledge engineering. This paper focuses on the modules of the lexical component for Japanese language. As first test of the feasibility of our approach we applied the interface architecture successfully to the question support facility of the VIENA Classroom collaborative hypermedia education system. 1

This paper describes preference classes and preference Moore machines as a basis for integrating different hybrid neural representations. Preference classes are shown to pro- vide a basic link between neural preferences and fuzzy representations at the preference class level. Preference Moore machines provide a link between recurrent neural networks and symbolic transducers at the preference Moore machine level. We demonstrate how the concepts of preference classes and preference Moore machines can be used to interpret neural network representations and to integrate knowledge from hybrid neural represen- tations. One main contribution of this paper is the introduction and analysis of neural preference Moore machines and their link to a fuzzy interpretation. Furthermore, we il- lustrate the interpretation and combination of various neural preference Moore machines with additional real world examples.

In this paper we outline hybrid approaches to arti cial neural network-based natural language processing. We start by motivating hybrid symbolic/connectionist processing. Then we suggest various types of symbolic/connectionist integration for language processing: connectionist structure architectures, hybrid transfer architectures, hybrid processing architectures. Furthermore, we focus particularly on loosely coupled, tightly coupled, and fully integrated hybrid processing architectures. We give particular examples of these hybrid processing architectures and argue that the hybrid approach to arti cial neural network-based language processing has a lot of potential to overcome the gap between a neural level and a symbolic conceptual level. ii 1 Motivation for hybrid symbolic/connectionist processing In recent years, the eld of hybrid symbolic/connectionist processing has seen a remarkable

In this paper 1 we describe a robust, learning approach to spoken language understanding. Since interactively spoken and computationally analyzed language often contains many errors, robust connectionist networks are used for providing a flat screening analysis. A screening analysis is a shallow flat analysis based on category sequences at various syntactic, semantic and dialog levels. Rather than using tree or graph representations a screening analysis uses category sequences in order to support robustness and learning. This flat screening analysis is examined in the context of the system SCREEN (Symbolic Connectionist Robust EnterprisE for Natural language). Starting with the word hypotheses generated by a speech recognizer, we give an overview of the architecture, and illustrate the flat robust processing at the levels of syntax, semantics, and dialog acts. While early connectionist models were often limited to a single network and a small task, the hybrid connectionist SCREEN sys...

In this paper we present a multilingual natural language interface architecture, which can be used for accessing on line product catalogs and lets users formulate their queries in their native languages. In our interface architecture a rule based machinelearning module replaces an elaborate semantic analysis component. The learning module learns the correct mappings of a user's input to the corresponding database command based on a collection of past input data.

The research project AgNeT develops Agents for Neural Text routing in the internet. Unrestricted potentially faulty text messages arrive at a certain delivery point (e.g. email address or world wide web address). These text messages are scanned and then distributed to one of several expert agents according to a certain task criterium. Possible specific scenarios within this framework include the learning of the routing of publication titles or news titles. In this paper we describe extensive experiments for semantic text routing based on classified library titles and newswire titles.