Results 1  10
of
163
Divergence measures based on the Shannon entropy
 IEEE Transactions on Information theory
, 1991
"... AbstractA new class of informationtheoretic divergence measures based on the Shannon entropy is introduced. Unlike the wellknown Kullback divergences, the new measures do not require the condition of absolute continuity to be satisfied by the probability distributions involved. More importantly, ..."
Abstract

Cited by 404 (0 self)
 Add to MetaCart
AbstractA new class of informationtheoretic divergence measures based on the Shannon entropy is introduced. Unlike the wellknown Kullback divergences, the new measures do not require the condition of absolute continuity to be satisfied by the probability distributions involved. More importantly, their close relationship with the variational distance and the probability of misclassification error are established in terms of bounds. These bounds are crucial in many applications of divergence measures. The new measures are also well characterized by the properties of nonnegativity, finiteness, semiboundedness, and boundedness. Index TermsDivergence, dissimilarity measure, discrimination information, entropy, probability of error bounds. I.
Universal prediction
 IEEE Transactions on Information Theory
, 1998
"... Abstract — This paper consists of an overview on universal prediction from an informationtheoretic perspective. Special attention is given to the notion of probability assignment under the selfinformation loss function, which is directly related to the theory of universal data compression. Both th ..."
Abstract

Cited by 136 (11 self)
 Add to MetaCart
Abstract — This paper consists of an overview on universal prediction from an informationtheoretic perspective. Special attention is given to the notion of probability assignment under the selfinformation loss function, which is directly related to the theory of universal data compression. Both the probabilistic setting and the deterministic setting of the universal prediction problem are described with emphasis on the analogy and the differences between results in the two settings. Index Terms — Bayes envelope, entropy, finitestate machine, linear prediction, loss function, probability assignment, redundancycapacity, stochastic complexity, universal coding, universal prediction. I.
A tutorial introduction to the minimum description length principle
 in Advances in Minimum Description Length: Theory and Applications. 2005
"... ..."
Bayesian Clustering by Dynamics
, 2001
"... This paper introduces a Bayesian method for clustering dynamic processes. The method models dynamics as Markov chains and then applies an agglomerative clustering procedure to discover the most probable set of clusters capturing different dynamics. To increase efficiency, the method uses an entropy ..."
Abstract

Cited by 56 (7 self)
 Add to MetaCart
This paper introduces a Bayesian method for clustering dynamic processes. The method models dynamics as Markov chains and then applies an agglomerative clustering procedure to discover the most probable set of clusters capturing different dynamics. To increase efficiency, the method uses an entropybased heuristic search strategy. A controlled experiment suggests that the method is very accurate when applied to articial time series in a broad range of conditions and, when applied to clustering sensor data from mobile robots, it produces clusters that are meaningful in the domain of application.
Texture Analysis of SAR Sea Ice Imagery using Gray Level Cooccurrence Matrices
 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING
, 1999
"... This paper presents a preliminary study for mapping sea ice patterns (texture) with 100m ERS1 synthetic aperture radar (SAR) imagery. We used graylevel cooccurrence matrices (GLCM) to quantitatively evaluate textural parameters and representations and to determine which parameter values and rep ..."
Abstract

Cited by 39 (2 self)
 Add to MetaCart
This paper presents a preliminary study for mapping sea ice patterns (texture) with 100m ERS1 synthetic aperture radar (SAR) imagery. We used graylevel cooccurrence matrices (GLCM) to quantitatively evaluate textural parameters and representations and to determine which parameter values and representations are best for mapping sea ice texture. We conducted experiments on the quantization levels of the image and the displacement and orientation values of the GLCM by examining the effects textural descriptors such as entropy have in the representation of different sea ice textures. We showed that a complete graylevel representation of the image is not necessary for texture mapping, an eightlevel quantization representation is undesirable for textural representation, and the displacement factor in texture measurements is more important than orientation. In addition, we developed three GLCM implementations and
A Natural Law of Succession
, 1995
"... Consider the following problem. You are given an alphabet of k distinct symbols and are told that the i th symbol occurred exactly ni times in the past. On the basis of this information alone, you must now estimate the conditional probability that the next symbol will be i. In this report, we presen ..."
Abstract

Cited by 35 (3 self)
 Add to MetaCart
Consider the following problem. You are given an alphabet of k distinct symbols and are told that the i th symbol occurred exactly ni times in the past. On the basis of this information alone, you must now estimate the conditional probability that the next symbol will be i. In this report, we present a new solution to this fundamental problem in statistics and demonstrate that our solution outperforms standard approaches, both in theory and in practice.
Nonlinear unmixing of hyperspectral images using a generalized bilinear model
 IEEE Trans. Geosci. and Remote Sensing
"... Nonlinear models have recently shown interesting properties for spectral unmixing. This paper considers a generalized bilinear model recently introduced for unmixing hyperspectral images. Different algorithms are studied to estimate the parameters of this bilinear model. The positivity and sumtoon ..."
Abstract

Cited by 26 (21 self)
 Add to MetaCart
Nonlinear models have recently shown interesting properties for spectral unmixing. This paper considers a generalized bilinear model recently introduced for unmixing hyperspectral images. Different algorithms are studied to estimate the parameters of this bilinear model. The positivity and sumtoone constraints for the abundances are ensured by the proposed algorithms. The performance of the resulting unmixing strategy is evaluated via simulations conducted on synthetic and real data. Index Terms — hyperspectral imagery, spectral unmixing, bilinear model, Bayesian inference, MCMC methods, gradient descent algorithm, least square algorithm. 1.
Small Sample Statistics for Classification Error Rates I: Error Rate Measurements
 Dept. of Inf. and Comp. Sci
, 1996
"... Several methods (independent subsamples, leaveoneout, crossvalidation, and bootstrapping) have been proposed for estimating the error rates of classifiers. The rationale behind the various estimators and the causes of the sometimes conflicting claims regarding their bias and precision are explore ..."
Abstract

Cited by 25 (1 self)
 Add to MetaCart
Several methods (independent subsamples, leaveoneout, crossvalidation, and bootstrapping) have been proposed for estimating the error rates of classifiers. The rationale behind the various estimators and the causes of the sometimes conflicting claims regarding their bias and precision are explored in this paper. The biases and variances of each of the estimators are examined empirically. Crossvalidation, 10fold or greater, seems to be the best approach; the other methods are biased, have poorer precision, or are inconsistent. Though unbiased for linear discriminant classifiers, the 632b bootstrap estimator is biased for nearest neighbors classifiers, more so for single nearest neighbor than for three nearest neighbors. The 632b estimator is also biased for Cartstyle decision trees. Weiss' loo* estimator is unbiased and has better precision than crossvalidation for discriminant and nearest neighbors classifiers, but its lack of bias and improved precision for those classifiers do...
BCourse: A WebBased Tool For Bayesian And Causal Data Analysis
, 2002
"... this paper we discuss both the theoretical design principles underlying the BCourse tool, and the pragmatic methods adopted in the implementation of the software ..."
Abstract

Cited by 25 (6 self)
 Add to MetaCart
this paper we discuss both the theoretical design principles underlying the BCourse tool, and the pragmatic methods adopted in the implementation of the software
Symmetrizing the KullbackLeibler Distance
 IEEE Transactions on Information Theory
, 2000
"... We define a new distance measure the resistoraverage distance between two probability distributions that is closely related to the KullbackLeibler distance. While the KullbackLeibler distance is asymmetric in the two distributions, the resistoraverage distance is not. It arises from geometric ..."
Abstract

Cited by 24 (0 self)
 Add to MetaCart
We define a new distance measure the resistoraverage distance between two probability distributions that is closely related to the KullbackLeibler distance. While the KullbackLeibler distance is asymmetric in the two distributions, the resistoraverage distance is not. It arises from geometric considerations similar to those used to derive the Chernoff distance. Determining its relation to wellknown distance measures reveals a new way to depict how commonly used distance measures relate to each other. 1 Introduction The KullbackLeibler distance [15, 16] is perhaps the most frequently used informationtheoretic "distance" measure from a viewpoint of theory. If p 0 , p 1 are two probability densities, the KullbackLeibler distance is defined to be D(p 1 #p 0 )= # p 1 (x)log p 1 (x) p 0 (x) dx . (1) In this paper, log() has base two. The KullbackLeibler distance is but one example of the AliSilvey class of informationtheoretic distance measures [1], which are defined to ...