Results 1  10
of
15
Divergence measures based on the Shannon entropy
 IEEE Transactions on Information theory
, 1991
"... AbstractA new class of informationtheoretic divergence measures based on the Shannon entropy is introduced. Unlike the wellknown Kullback divergences, the new measures do not require the condition of absolute continuity to be satisfied by the probability distributions involved. More importantly, ..."
Abstract

Cited by 549 (0 self)
 Add to MetaCart
AbstractA new class of informationtheoretic divergence measures based on the Shannon entropy is introduced. Unlike the wellknown Kullback divergences, the new measures do not require the condition of absolute continuity to be satisfied by the probability distributions involved. More importantly, their close relationship with the variational distance and the probability of misclassification error are established in terms of bounds. These bounds are crucial in many applications of divergence measures. The new measures are also well characterized by the properties of nonnegativity, finiteness, semiboundedness, and boundedness. Index TermsDivergence, dissimilarity measure, discrimination information, entropy, probability of error bounds. I.
Jensenshannon boosting learning for object recognition
 In Proc. of IEEE Conf. on Computer Vision and Pattern Recognition (CVPR’05
"... In this paper, we propose a novel learning method, called JensenShannon Boosting (JSBoost) and demonstrate its application to object recognition. JSBoost incorporates JensenShannon (JS) divergence [2] into AdaBoost learning. JS divergence is advantageous in that it provides more appropriate measu ..."
Abstract

Cited by 20 (0 self)
 Add to MetaCart
In this paper, we propose a novel learning method, called JensenShannon Boosting (JSBoost) and demonstrate its application to object recognition. JSBoost incorporates JensenShannon (JS) divergence [2] into AdaBoost learning. JS divergence is advantageous in that it provides more appropriate measure of dissimilarity between two classes and it is numerically more stable than other measures such as KullbackLeibler (KL) divergence (see [2]). The best features are iteratively learned by maximizing the projected JS divergence, based on which best weak classifiers are derived. The weak classifiers are combined into a strong one by minimizing the recognition error. JSBoost learning is demonstrated with face object recognition using a local binary pattern (LBP) [13] based representation. JSBoost selects the best LBP features from thousands of candidate features and constructs a strong classifier based on the selected features. JSBoost empirically produces better face recognition results than other AdaBoost variants such as RealBoost [12], GentleBoost [5] and KLBoost [7], as demonstrated by experiments. 1
Bounds for entropy and divergence for distributions over a twoelement set
 J. Ineq. Pure & Appl. Math
"... set ..."
(Show Context)
2001): Large deviations of divergence measures on partitions
 Journal of Statistical Planning and Inference
"... ..."
Some Bayesian perspectives on statistical modelling
, 1988
"... I would like to thank my supervisor, Professor A. F. M. Smith, for all his advice and encourage ..."
Abstract

Cited by 5 (2 self)
 Add to MetaCart
I would like to thank my supervisor, Professor A. F. M. Smith, for all his advice and encourage
On McDiarmid’s concentration inequality
 ELECTRONIC COMMUNICATIONS IN PROBABILITY
, 2013
"... In this paper we improve the rate function in the McDiarmid concentration inequality for separately Lipschitz functions of independent random variables. In particular the rate function tends to infinity at the boundary. We also prove that in some cases the usual normalization factor is not adequate ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
In this paper we improve the rate function in the McDiarmid concentration inequality for separately Lipschitz functions of independent random variables. In particular the rate function tends to infinity at the boundary. We also prove that in some cases the usual normalization factor is not adequate and may be improved.
On Pinsker’s Type Inequalities and Csiszár’s fdivergences
 Part I: Second and FourthOrder Inequalities. arXiv:cs/0603097v2
, 2006
"... We study conditions on f under which an fdivergence Df will satisfy Df ≥ cfV 2 or Df ≥ c2,fV 2 +c4,fV 4, where V denotes variational distance and the coefficients cf, c2,f and c4,f are best possible. As a consequence, we obtain lower bounds in terms of V for many well known distance and divergence ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
(Show Context)
We study conditions on f under which an fdivergence Df will satisfy Df ≥ cfV 2 or Df ≥ c2,fV 2 +c4,fV 4, where V denotes variational distance and the coefficients cf, c2,f and c4,f are best possible. As a consequence, we obtain lower bounds in terms of V for many well known distance and divergence measures. For instance, let D (α)(P, Q) = [α(α − 1)] −1 [ ∫ qαp1−α dµ − 1] and Iα(P, Q) = (α−1) −1 log [ ∫ pαq1−α dµ] be respectively the relative information of type (1−α) and Rényi’s information gain of order α. We show that D (α) ≥ 1 2V 2 + 1 72 whenever −1 ≤ α ≤ 2, α = 0, 1 and that Iα ≥ α 2 V 2 + 1 Pinsker’s inequality D ≥ 1 of these. (α + 1)(2 − α)V 4 36α(1 + 5α − 5α2)V 4 for 0 < α < 1. 2 V 2 and its extension D ≥ 1 2 V 2 + 1 36 V 4 are special cases of each one
NEW INEQUALITIES FOR CSISZÁR DIVERGENCE AND APPLICATIONS
"... Abstract. In this paper we point out some new inequalities for Csiszár fdivergence and apply them for particular instances of distances between two probability distributions. 1. ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
Abstract. In this paper we point out some new inequalities for Csiszár fdivergence and apply them for particular instances of distances between two probability distributions. 1.
unknown title
"... Abstract We present a simple information theoretic approach for detecting corner points in gray level images based on a new concept of directional entropy. The gradient directions of the edge points are coded by a scheme similar to 8directional chain codes. Based on the coded gradient directions o ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract We present a simple information theoretic approach for detecting corner points in gray level images based on a new concept of directional entropy. The gradient directions of the edge points are coded by a scheme similar to 8directional chain codes. Based on the coded gradient directions of the edge pixels in a window, the entropy of an edge point is obtained which is referred to as the directional entropy of that edge point. Edge points with high directional entropy values are analyzed to obtain the detected corner points.