Results 11  20
of
51
Improved learning of Riemannian metrics for exploratory analysis
, 2004
"... We have earlier introduced a principle for learning metrics, which shows how metricbased methods can be made to focus on discriminative properties of data. The main applications are in supervising unsupervised learning to model interesting variation in data, instead of modeling all variation as pl ..."
Abstract

Cited by 11 (4 self)
 Add to MetaCart
We have earlier introduced a principle for learning metrics, which shows how metricbased methods can be made to focus on discriminative properties of data. The main applications are in supervising unsupervised learning to model interesting variation in data, instead of modeling all variation as plain unsupervised learning does. The metrics are derived by approximations to an informationgeometric formulation. In this paper, we review the theory, introduce better approximations to the distances, and show how to apply them in two different kinds of unsupervised methods: prototypebased and pairwise distancebased. The two examples are selforganizing maps and multidimensional scaling (Sammon’s mapping).
A TopographyPreserving Latent Variable Model with Learning Metrics
 Advances in SelfOrganizing Maps
, 2001
"... We introduce a new mapping model from a latent grid to the input space. The mapping preserves the topography but measures local distances in terms of auxiliary data that implicitly conveys information about the relevance or importance of local directions in the primary data space. Soft clusters corr ..."
Abstract

Cited by 6 (0 self)
 Add to MetaCart
(Show Context)
We introduce a new mapping model from a latent grid to the input space. The mapping preserves the topography but measures local distances in terms of auxiliary data that implicitly conveys information about the relevance or importance of local directions in the primary data space. Soft clusters corresponding to the map grid locations are defined into the primary data space, and a distortion measure is minimized for paired samples of primary and auxiliary data. The KullbackLeibler divergencebased distortion is measured between the conditional distributions of the auxiliary data given the primary data, and the model is optimized with stochastic approximation yielding an algorithm that resembles the SelfOrganizing Map, but in which distances are computed by taking into account the (local) relevance of directions.
Learning more accurate metrics for selforganizing maps
 in Artificial Neural Networks—ICANN 2002
, 2002
"... Abstract. Improved methods are presented for learning metrics that measure only important distances. It is assumed that changes in primary data are relevant only to the extent that they cause changes in auxiliary data, available paired with the primary data. The metrics are here derived from estimat ..."
Abstract

Cited by 6 (5 self)
 Add to MetaCart
(Show Context)
Abstract. Improved methods are presented for learning metrics that measure only important distances. It is assumed that changes in primary data are relevant only to the extent that they cause changes in auxiliary data, available paired with the primary data. The metrics are here derived from estimators of the conditional density of the auxiliary data. More accurate estimators are compared, and a more accurate approximation to the distances is introduced. The new methods improved the quality of SelfOrganizing Maps (SOMs) significantly for four of the five studied data sets. 1
Learning Metrics For Exploratory Data Analysis
 Neural Networks for Signal Processing XI, Proceedings of the 2001 IEEE Signal Processing Society Workshop
, 2001
"... . Visualization and cluster analysis of multivariate data is usually based on distances between samples in a data space. The distance measure is often heuristically chosen, for instance by choosing suitable features and then using a global Euclidean metric. We have developed methods that remove the ..."
Abstract

Cited by 6 (1 self)
 Add to MetaCart
. Visualization and cluster analysis of multivariate data is usually based on distances between samples in a data space. The distance measure is often heuristically chosen, for instance by choosing suitable features and then using a global Euclidean metric. We have developed methods that remove the arbitrariness by measuring distances only along important (local) directions. The metric is learned from auxiliary data that is paired with the primary data during the learning process. It is assumed that changes in the primary data are important or relevant if they cause changes in the auxiliary data; for example, in analysis of gene expression the auxiliary data can indicate the functional classes of the genes. The new distance measures can be used for instance in clustering and SelfOrganizing Mapbased data visualization. The methods have so far been applied in analysis of bankruptcy, text documents, and gene expression.
On the generalization ability of prototypebased classifiers with local relevance determination
, 2005
"... We extend a recent variant of the prototypebased classifier learning vector quantization to a scheme which locally adapts relevance terms during learning. We derive explicit dimensionalityindependent largemargin generalization bounds for this classifier and show that the method can be seen as ..."
Abstract

Cited by 5 (5 self)
 Add to MetaCart
We extend a recent variant of the prototypebased classifier learning vector quantization to a scheme which locally adapts relevance terms during learning. We derive explicit dimensionalityindependent largemargin generalization bounds for this classifier and show that the method can be seen as margin maximizer.
A General Framework for SelfOrganizing Structure Processing Neural Networks
, 2003
"... Selforganization constitutes an important paradigm in machine learning with successful applications e.g. for data and webmining. However, so far most approaches have been proposed for data contained in a fixed and finite dimensional vector space. We will focus on extensions for more general dat ..."
Abstract

Cited by 4 (4 self)
 Add to MetaCart
Selforganization constitutes an important paradigm in machine learning with successful applications e.g. for data and webmining. However, so far most approaches have been proposed for data contained in a fixed and finite dimensional vector space. We will focus on extensions for more general data structures like sequences and tree structures in this article. Various extensions of the standard selforganizing map (SOM) to sequences or tree structures have been proposed in the literature: the temporal Kohonen map, the recursive SOM, and SOM for structured data (SOMSD), for example. These methods enhance the standard SOM by recursive connections. We define in this article a general recursive dynamic which enables the recursive processing of complex data structures based on recursively computed internal representations of the respective context. The above mechanisms of SOMs for structures are special cases of the proposed general dynamic, furthermore, the dynamic covers the supervised case of recurrent and recursive networks, too. The general framework offers a uniform notation for training mechanisms such as Hebbian learning and the transfer of alternatives such as vector quantization or the neural gas algorithm to structure processing networks. The formal definition of the recursive dynamic for structure processing unsupervised networks allows the transfer of theoretical issues from the SOM literature to the structure processing case. One can formulate general cost functions corresponding to vector quantization, neural gas, and a modification of SOM for the case of structures. The cost functions can be compared to Hebbian learning which can be interpreted as an approximation of a stochastic gradient descent. We derive as an alternative the exact gradien...
Discriminative clustering in Fisher metrics
 in Artificial Neural Networks and Neural Information Processing  Supplementary prodceedings ICANN/ICONIP 2003
, 2003
"... Discriminative clustering (DC) finds a Voronoi partitioning of a primary data space that, while consisting of local partitions, simultaneously maximizes information about auxiliary data categories. DC is useful in exploration and in finding more coarse or refined versions of already existing categor ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
(Show Context)
Discriminative clustering (DC) finds a Voronoi partitioning of a primary data space that, while consisting of local partitions, simultaneously maximizes information about auxiliary data categories. DC is useful in exploration and in finding more coarse or refined versions of already existing categories. Theoretical results suggest that Voronoi partitions in the socalled Fisher metric would outperform partitions in the Euclidean metric. Here we use a local quadratic approximation of the Fisher metric, derived from a conditional density estimator, in defining the partitions and show that the resulting algorithms outperform the conventional variants.
SOMBased Exploratory Analysis of Gene Expression Data
 N, Yin H, Allinson L, and Slack J. London: Springer
, 2001
"... . Applications of new SOMbased exploratory data analysis methods to bioinformatics are described. Cluster structures are revealed in data describing the expression of a set of yeast genes in several experimental treatments. The structures are visualized in an intuitive manner with colors: The simil ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
(Show Context)
. Applications of new SOMbased exploratory data analysis methods to bioinformatics are described. Cluster structures are revealed in data describing the expression of a set of yeast genes in several experimental treatments. The structures are visualized in an intuitive manner with colors: The similarity of hue corresponds to the similarity of the multivariate data. The clusters can be interpreted by visualizing changes of the data variables (expression in dierent treatments) at the cluster borders. The relationship between the organization of the SOM and the functional classes of the proteins encoded by the genes may additionally reveal interesting relationships between the functional classes, and substructures within them.
Generalized Relevance LVQ with Correlation Measures for Biological Data
 European Symposium on Artificial Neural Networks (ESANN
, 2005
"... Abstract. Generalized Relevance Learning Vector Quantization (GRLVQ) is combined with correlationbased similarity measures. These are derived from the Pearson correlation coefficient in order to replace the adaptive squared Euclidean distance which is typically used for GRLVQ. Patterns can thus be ..."
Abstract

Cited by 2 (2 self)
 Add to MetaCart
(Show Context)
Abstract. Generalized Relevance Learning Vector Quantization (GRLVQ) is combined with correlationbased similarity measures. These are derived from the Pearson correlation coefficient in order to replace the adaptive squared Euclidean distance which is typically used for GRLVQ. Patterns can thus be used without further preprocessing and compared in a manner invariant to data shifting and scaling transforms. High accuracies are demonstrated for a reference experiment of handwritten character recognition and good discrimination ability is shown for the detection of systematic differences between gene expression experiments. Keywords. Prototypebased learning, adaptive metrics, correlation measure, Learning Vector Quantization, GRLVQ. 1
Relaxational metric adaptation and its application to semisupervised clustering and contentbased image retrieval
 Pattern Recognition
"... The performance of many supervised and unsupervised learning algorithms is very sensitive to the choice of an appropriate distance metric. Previous work in metric learning and adaptation has mostly been focused on classification tasks by making use of class label information. In standard clustering ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
(Show Context)
The performance of many supervised and unsupervised learning algorithms is very sensitive to the choice of an appropriate distance metric. Previous work in metric learning and adaptation has mostly been focused on classification tasks by making use of class label information. In standard clustering tasks, however, class label information is not available. In order to adapt the metric to improve the clustering results, some background knowledge or side information is needed. One useful type of side information is in the form of pairwise similarity or dissimilarity information. Recently, some novel methods (e.g., the parametric method proposed by Xing et al.) for learning global metrics based on pairwise side information have been shown to demonstrate promising results. In this paper, we propose a nonparametric method, called relaxational metric adaptation (RMA), for the same metric adaptation problem. While RMA is local in the sense that it allows locally adaptive metrics, it is also global because even patterns not in the vicinity can have longrange effects on the metric adaptation process. Experimental results for semisupervised clustering based on both simulated and realworld data sets show that RMA outperforms Xing Preprint submitted to Elsevier Science 28 July 2005 et al.’s method under most situations. Besides applying RMA to semisupervised learning, we have also used it to improve the performance of contentbased image retrieval systems through metric adaptation. Experimental results based on two realworld image databases show that RMA significantly outperforms other methods in improving the image retrieval performance.