We present a new self-organizing neural network model having two variants. The first variant performs unsupervised learning and can be used for data visualization, clustering, and vector quantization. The main advantage over existing approaches, e.g., the Kohonen feature map, is the ability of the model to automatically find a suitable network structure and size. This is achieved through a controlled growth process which also includes occasional removal of units. The second variant of the model is a supervised learning method which results from the combination of the abovementioned self-organizing network with the radial basis function (RBF) approach. In this model it is possible - in contrast to earlier approaches - to perform the positioning of the RBF units and the supervised training of the weights in parallel. Therefore, the current classification error can be used to determine where to insert new RBF units. This leads to small networks which generalize very well. Results on the t...

Kohonen's Self-Organizing Map (SOM) is one of the most popular artificial neural network algorithms. Word category maps are SOMs that have been organized according to word similarities, measured by the similarity of the short contexts of the words. Conceptually interrelated words tend to fall into the same or neighboring map nodes. Nodes may thus be viewed as word categories. Although no a priori information about classes is given, during the self-organizing process a model of the word classes emerges. The central topic of the thesis is the use of the SOM in natural language processing. The approach based on the word category maps is compared with the methods that are widely used in artificial intelligence research. Modeling gradience, conceptual change, and subjectivity of natural language interpretation are considered. The main application area is information retrieval and textual data mining for which a specific SOM-based method called the WEBSOM has been developed. The WEBSOM metho...

The self-organizing feature maps introduced by T. Kohonen use a cell array of fixed size and structure. In many cases this array is not able to model a given signal distribution properly. We present a method to construct two-dimensional cell structures during a self-organization process which are specially adapted to the underlying distribution: Starting with a small number of cells new cells are added successively. Thereby signal vectors according to the (usually not explicitly known) probability distribution are used to determine where to insert or delete cells in the current structure. This process leads to problem dependent cell structures which model the given distribution with arbitrary high accuracy.

This thesis is organized in three parts. In the first part, the Self-Organizing Map algorithm is introduced. The discussion focuses on the analysis of the Self-Organizing Map algorithm. It is shown that the nonlinear nature of the algorithm makes it difficult to analyze the algorithm except in some trivial cases. In the second part the Self-Organizing Map algorithm is applied to several patterns sequence analysis tasks. The first application is a voice quality analysis system. It is shown that the Self-Organizing Map algorithm can be applied to voice analysis by providing the visualization of certain deviations. The key point in the applicability of Self-Organizing Map algorithm is the topological nature of the mapping; similar voice samples are mapped to nearby locations in the map. The second application is a speech recognition system. Through several experiments it is demonstrated that by collecting some time dependent features and using them in conjunction with the basic Self-Organ...

implementation Self-organization can be efficiently implemented based on Euclidian distance and global supervision. It is not necessary to explicitly model the connections between the units in the network. Every unit computes the distance between its weight vector and the input vector. An external supervisor finds the unit with the smallest distance, looks up the current neighborhood radius from a training schedule, and tells the units within this radius to modify their input weights. The weight adaptations are proportional to the Euclidian difference. The weights of unit (i; j) in a 2-D map are (a) 0 samples (b) 30 samples (c) 100 samples (d) 10,000 samples Figure 1: Abstract implementation of self-organization. The map consists of 20 \Theta 20 units in a 2-D array organization. The weight vector of each unit is shown as a point on the unit square 0 x; y 1. Each vector is connected with a line to the weight vectors of the four neighboring units. In other words, each intersection ...

Availability of large full-text document collections in electronic form has created a need for intelligent information retrieval techniques. Especially the expanding World Wide Web presupposes methods for systematic exploration of miscellaneous document collections. In this paper we introduce a new method, the WEBSOM, for this task. Self-Organizing Maps (SOMs) are used to represent documents on a map that provides an insightful view of the text collection. This view visualizes similarity relations between the documents, and the display can be utilized for orderly exploration of the material rather than having to rely on traditional search expressions. The complete WEBSOM method involves a two-level SOM architecture comprising of a word category map and a document map, and means for interactive exploration of the data base. 1. Introduction Full-text classification may be based on the assumption that the elementary textual features of documents that deal with similar topics are statist...

In connection to a speech recognizer, the aim of which is to produce phonemic transcriptions of arbitrary spoken utterances, we investigate the combined effect of several improvements at different stages of phoneme recognition. The core of the basic recognition system is Learning Vector Quantization (LVQ1) [1]. This algorithm was originally used to classify FFT-based short-time feature vectors into phonemic classes. The phonemic decoding stage was earlier based on simple durational rules [2] [3]. At the feature level, we now study the effect of using mel-scale cepstral features and concatenating consecutive feature vectors to include context. At the output of vector quantization, a comparison of three approaches to take into account the classifications of feature vectors in local context is presented. The rule-based phonemic decoding is compared to decoding employing Hidden Markov Models (HMMs). As earlier, an optional grammatical post-correction method (DEC) is applied. Experiments co...

Abstract. Advances in high quality digital wireless networks and differentiated services have enabled the development of mobile multimedia applications that can execute in global infrastructures. In this paper, we introduce a novel approach to supporting mobile multimedia services by effectively exploiting the intermittently available idle computing, storage and communication resources in a Grid infrastructure. Specifically, we develop efficient resource discovery policies that can ensure continuous access to information sources and maintain application Quality-of-Service (QoS) requirements, e.g. required network transmission bandwidth on the mobile clients. Our performance studies indicate that mobility patterns obtained via tracking or user-supplied itineraries assist in optimizing resource allocation. The proposed policies are also resilient to dynamic changes in the availability of grid resources. 1

An approach to automate knowledge acquisition from natural signals, such as speech, is presented. The knowledge is extracted in the form of context-sensitive production rules that can be used to map a signal or a sequence of events into another one, e.g., for correction or enhancement purposes. The rules are constructed automatically from examples by using a concept of Dynamically Focusing Context (DFC), which is an extension of Dynamically Expanding Context (DEC) introduced in [7]. An application is presented, in which vector quantized (VQ) speech is transcribed into phonemes. As vector quantizers, Kohonen's Feature Maps have been used. The proposed method is used to generate rules from VQ-codes that enhance phonemic differences and cancel coarticulation effects. A 38 per cent reduction in phonemic transcription error rate has been achieved. 1 The problem and an application The problem addressed in this paper is to transform a signal into a desired one aided by local context. The tr...

In spite of many statistical methods for outlier detection and for robust analysis, there is little work on further analysis of outliers themselves to determine their origins. For example, there are “good ” outliers that provide useful information that can lead to the discovery of new knowledge, or “bad ” outliers that include noisy data points. Successfully distinguishing between different types of outliers is an important issue in many applications, including fraud detection, medical tests, process analysis and scientific discovery. It requires not only an understanding of the mathematical properties of data but also relevant knowledge in the domain context in which the outliers occur. This thesis presents a novel attempt in automating the use of domain knowledge in helping distinguish between different types of outliers. Two complementary knowledge-based outlier analysis strategies are proposed: one using knowledge regarding how “normal data ” should be distributed in a domain of interest in order to identify “good ” outliers, and the other using the understanding of “bad ” outliers. This kind of knowledge-based outlier analysis is a useful extension to existing work in both statistical and computing communities on outlier detection.