We present a new self-organizing neural network model having two variants. The first variant performs unsupervised learning and can be used for data visualization, clustering, and vector quantization. The main advantage over existing approaches, e.g., the Kohonen feature map, is the ability of the model to automatically find a suitable network structure and size. This is achieved through a controlled growth process which also includes occasional removal of units. The second variant of the model is a supervised learning method which results from the combination of the abovementioned self-organizing network with the radial basis function (RBF) approach. In this model it is possible - in contrast to earlier approaches - to perform the positioning of the RBF units and the supervised training of the weights in parallel. Therefore, the current classification error can be used to determine where to insert new RBF units. This leads to small networks which generalize very well. Results on the t...

We offer three algorithms for the generation of topographic mappings to the practitioner of unsupervised data analysis. The algorithms are each based on the minimization of a cost function which is performed using an EM algorithm and deterministic annealing. The soft topographic vector quantization algorithm (STVQ) -- like the original Self-Organizing Map (SOM) -- provides a tool for the creation of self-organizing maps of Euclidean data. Its optimization scheme, however, offers an alternative to the heuristic stepwise shrinking of the neighborhood width in the SOM and makes it possible to use a fixed neighborhood function solely to encode desired neighborhood relations between nodes. The kernel-based soft topographic mapping (STMK) is a generalization of STVQ and introduces new distance measures in data space based on kernel functions. Using the new distance measures corresponds to performing the STVQ in a highdimensional feature space, which is related to data space by a nonlinear ma...

Understanding high-dimensional real world data usually requires learning the structure of the data space. The structure may contain high-dimensional clusters that are related in complex ways. Methods such as merge clustering and self-organizing maps are designed to aid the visualization and interpretation of such data. However, these methods often fail to capture critical structural properties of the input. Although self-organizing maps capture high-dimensional topology, they do not represent cluster boundaries or discontinuities. Merge clustering extracts clusters, but it does not capture local or global topology. This paper proposes an algorithm that combines the topology-preserving characteristics of self-organizing maps with a flexible, adaptive structure that learns the cluster boundaries in the data. 1 INTRODUCTION Real world data is often very high-dimensional, and often has a structure that is difficult both to recognize and describe. For instance, human blood can be tested fo...

The technique of local linear models is appealing for modeling complex time series due to the weak assumptions required and its intrinsic simplicity. Here, instead of deriving the local models from the data, we propose to estimate them directly from the weights of a self organizing map (SOM), which functions as a dynamic-preserving model of the dynamics. We introduce one modification to the Kohonen learning to ensure good representation of the dynamics and use weighted least squares to ensure continuity among the local models. The proposed scheme is tested using synthetic chaotic time series and real world data. The practicality of the method is illustrated in the identification and control of the NASA Langley wind tunnel during aerodynamic tests of model aircrafts. Modeling the dynamics with a SOM leads to a predictive multiple model control strategy (PMMC). Comparison of the new controller against the existing controller in test runs shows the superiority of our method. 1. Introducti...

In [5], a new incremental cascade network architecture has been presented. This paper discusses the properties of such cascade networks and investigates their generalization abilities under the particular constraint of small data sets. The evaluation is done for cascade networks consisting of local linear maps using the MackeyGlass time series prediction task as a benchmark. Our results indicate that to bring the potential of large networks to bear on the problem of extracting information from small data sets without running the risk of overfitting , deeply cascaded network architectures are more favorable than shallow broad architectures that contain the same number of nodes. 1 Introduction For many real-world applications, a major constraint for the successful learning from examples is the limited number of examples available. Thus, methods are required, that can learn from small data sets. This constraint makes the problem of generalization particularly hard. If the number of adjus...

Introduction It is well known that the classification and approximation capabilities of single layer perceptrons are restricted to linearly separable problems [8]. To overcome these limitations, various approaches have been made. The most popular one is training by error backpropagation [15, 13], at the expense of losing both the convergence guarantee [1, 12] of strictly feed--forward trained networks and the biological plausibility of the approach. The early successes of backpropagation networks led to extensive studies of the behavior of shallow networks with many units in few hidden layers. Fahlman [3] proposed networks with a complementary structure, trained by the cascade--correlation algorithm. Training is strictly feed--forward and the nonlinearity is achieved by incrementally adding units trained to maximize the covariance with the residual error. Thus, by constructing a narrow, deeply cascaded structure, the network can solve problems that are not linearly separable.

This paper deals with the problem of obtaining a rough estimate of three dimensional object position and orientation from a single two dimensional camera image. Such an estimate is required by most 3-D to 2-D registration and tracking methods that can efficiently refine an initial value by numerical optimization to precisely recover 3-D pose. However, the analytic computation of an initial pose guess requires the solution of an extremely complex correspondence problem that is due to the large number of topologically distinct aspects that arise when a three dimensional opaque object is imaged by a camera. Hence general analytic methods fail to achieve real-time performance and most tracking and registration systems are initialized interactively or by ad hoc heuristics. To overcome these limitations we present a novel method for approximate object pose estimation that is based on a neural net and that can easily be implemented in real-time. A modification of Kohonen's self-organizing fe...

. A system for the automatic segmentation of fluorescence micrographs is presented. In a first step positions of fluorescent cells are detected by a fast learning neural network, which acquires the visual knowledge from a set of training cell-image patches selected by the user. Guided by the detected cell positions the system extracts in the second step the contours of the cells. For contour extraction a recurrent neural network model is used to approximate the cell shapes. Even though the micrographs are noisy and the fluorescent cells vary in shape and size, the system detects at minimum 95% of the cells. 1 Introduction In the last decades experimental research in biomedicine was influenced by automation of sample preparation and digital microscopy imaging. Research groups in related fields are now enabled to produce large sets of digitized micrographs. Hence the problem of efficient evaluation of large datasets arises, for example in the case of high-throughput analysis of ...

This paper presents VisBoS, a visual system for detecting the 3D location of descriptive body landmarks (shoulder, elbow, wrist) out of stereo images using neural networks. The user is standing in front of a projection wall and interacts with a virtual construction scenario. Using whole arm gestures, the user can select and manipulate objects in the scene. Since we need a nearly dark room for displaying on the wall, the camera system had to make use of non-visible light. This is achieved by combining infrared illumination together with a suitable filter.

We present a new incremental cascade network architecture based on error minimization combined with "Local linear maps" (LLM) as cascaded units. The performance of the network is achieved by several layers of LLMs that are trained in a strictly feed-forward manner and one after the other. The properties of this and related cascade architectures are discussed. We report on extensive benchmarking results for various classification tasks, and time series prediction, and compare them with other results reported in the literature. Direct cascading is proposed as a promising approach to introduce context information in the approximation process. 1 Introduction Neural networks offer a wide range of architectural possibilities. So far, mainly shallow, broad architectures have been considered, while there has been very little research on the kind of architecture which is the focus of the present paper, namely narrow , but deeply cascaded networks. One of the few exceptions is the work of Fahl...