. We present a novel self-organizing network which is generated by a growth process. The application range of the model is the same as for Kohonen's feature map: generation of topologypreserving and dimensionality-reducing mappings, e.g., for the purpose of data visualization. The network structure is a rectangular grid which, however, increases its size during self-organization. By inserting complete rows or columns of units the grid may adapt its height/width ratio to the given pattern distribution. Both the neighborhood range used to co-adapt units in the vicinity of the winning unit and the adaptation strength are constant during the growth phase. This makes it possible to let the network grow until an application-specific performance criterion is fulfilled or until a desired network size is reached. A final approximation phase with decaying adaptation strength fine-tunes the network. 1 Introduction The self-organizing feature map [1] is a widely used method for generating topolog...

We propose a new scheme for enlarging generalized learning vector quantization (GLVQ) with weighting factors for the input dimensions. The factors allow an appropriate scaling of the input dimensions according to their relevance. They are adapted automatically during training according to the specific classification task whereby training can be interpreted as stochastic gradient descent on an appropriate error function. This method leads to a more powerful classifier and to an adaptive metric with little extra cost compared to standard GLVQ. Moreover, the size of the weighting factors indicates the relevance of the input dimensions. This proposes a scheme for automatically pruning irrelevant input dimensions. The algorithm is verified on artificial data sets and the iris data from the UCI repository.

. A new on-line criterion for identifying "useless" neurons of a self-organizing network is proposed. When this criterion is used in the context of the (formerly developed) growing neural gas model to guide deletions of units, the resulting method is able to closely track nonstationary distributions. Slow changes of the distribution are handled by adaptation of existing units. Rapid changes are handled by removal of "useless" neurons and subsequent insertions of new units in other places. 1 Non-stationary data is difficult to handle : : : Non-stationary data distributions can be found in many technical, biological or economical processes. Self-organizing neural networks have rarely been considered for tracking those distributions since many of the models, e.g. the selforganizing map [6], neural gas [8], or the hypercubical map [1], use decaying adaptation parameters 1 . Once the adaptation strength has decayed, the network is "frozen" and thus unable to react to subsequent changes i...

The Self-Organizing Map is a very popular unsupervised neural network model for the analysis of high-dimensional input data as in data mining applications. However, at least two limitations have to be noted, which are related, on the one hand, to the static architecture of this model, as well as, on the other hand, to the limited capabilities for the representation of hierarchical relations of the data. With our novel Growing Hierarchical SelfOrganizing Map presented in this paper we address both limitations. The Growing Hierarchical Self-Organizing Map is an arti cial neural network model with hierarchical architecture composed of independent growing self-organizing maps. The motivation was to provide a model that adapts its architecture during its unsupervised training process according to the particular requirements of the input data. Furthermore, by providing a global orientation of the independently growing maps in the individual layers of the hierarchy, navigation across branches is facilitated. The bene ts of this novel neural network are rst, a problem-dependent architecture, and second, the intuitive representation of hierarchical relations in the data. This is especially appealing in explorative data mining applications, allowing the inherent structure of the data to unfold in a highly intuitive fashion.

. The reasons to use growing self-organizing networks are investigated. First an overview of several models of this kind is given are they are related to other approaches. Then two examples are presented to illustrate the specific properties and advantages of incremental networks. In each case a non-incremental model is used for comparison purposes. The first example is pattern classification and compares the supervised growing neural gas model to a conventional radial basis function approach. The second example is data visualization and contrasts the growing grid model and the self-organizing feature map. 1. Introduction Growing (or incremental) network models have no pre-defined structure. Rather, they are generated by successive addition (and possibly occasional deletion) of elements. At first sight this makes them a lot more complicated than networks with static structure such as normal multi-layer-perceptrons or self-organizing maps the topology of which is chosen a priori and do...

Prototype based classi cation oers intuitive and sparse models with excellent generalization ability. However, these models usually crucially depend on the underlying Euclidian metric; moreover, online variants likely suer from the problem of local optima. We here propose a generalization of learning vector quantization with three additional features: (I) it directly integrates neighborhood cooperation, hence is less aected by local optima; (II) the method can be combined with any dierentiable similarity measure whereby metric parameters such as relevance factors of the input dimensions can automatically be adapted according to the given data; (III) it obeys a gradient dynamics hence shows very robust behavior, and the chosen objective is related to margin optimization.

The ability to grow extra nodes is a potentially useful facility for a self-organising neural network. A network that can add nodes into its map space can approximate the input space more accurately, and often more parsimoniously, than a network with predefined structure and size, such as the Self-Organising Map. In addition, a growing network can deal with dynamic input distributions. Most of the growing networks that have been proposed in the literature add new nodes to support the node that has accumulated the highest error during previous iterations or to support topological structures. This usually means that new nodes are added only when the number of iterations is an integer multiple of some pre-defined constant,

Neural maps combine the representation of data by codebook vectors, like a vector quantizer, with the property of topography, like a continuous function. While the quantization error is simple to compute and to compare between different maps, topography of a map is difficult to define and to quantify. Yet, topography of a neural map is an advantageous property, e.g. in the presence of noise in a transmission channel, in data visualization, and in numerous other applications. In this paper we review some conceptual aspects of definitions of topography, and some recently proposed measures to quantify topography. We apply the measures first to neural maps trained on synthetic data sets, and check the measures for properties like reproducability, scalability, systematic dependence of the value of the measure on the topology of the map etc. We then test the measures on maps generated for four real-world data sets, a chaotic time series, speech data, and two sets of image data. The measures ...

Abstract not found

For unsupervised sequence processing, standard self organizing maps can be naturally extended by using recurrent connections and explicit context representations. Models thereof are the temporal Kohonen map (TKM), recursive SOM, SOM for structured data (SOMSD), and HSOM for sequences (HSOM-S). Here, we discuss and compare the capabilities of exemplary approaches to store different types of sequences. We propose a new efficient model, the Merge-SOM (MSOM), which combines ideas of TKM and SOMSD and which is particularly suited for processing sequences with dynamic multimodal densities.