Results 1  10
of
158
Divergence measures based on the Shannon entropy
 IEEE Transactions on Information theory
, 1991
"... AbstractA new class of informationtheoretic divergence measures based on the Shannon entropy is introduced. Unlike the wellknown Kullback divergences, the new measures do not require the condition of absolute continuity to be satisfied by the probability distributions involved. More importantly, ..."
Abstract

Cited by 658 (0 self)
 Add to MetaCart
AbstractA new class of informationtheoretic divergence measures based on the Shannon entropy is introduced. Unlike the wellknown Kullback divergences, the new measures do not require the condition of absolute continuity to be satisfied by the probability distributions involved. More importantly, their close relationship with the variational distance and the probability of misclassification error are established in terms of bounds. These bounds are crucial in many applications of divergence measures. The new measures are also well characterized by the properties of nonnegativity, finiteness, semiboundedness, and boundedness. Index TermsDivergence, dissimilarity measure, discrimination information, entropy, probability of error bounds. I.
Mutualinformationbased registration of medical images: a survey
 IEEE Transcations on Medical Imaging
, 2003
"... Abstract—An overview is presented of the medical image processing literature on mutualinformationbased registration. The aim of the survey is threefold: an introduction for those new to the field, an overview for those working in the field, and a reference for those searching for literature on a s ..."
Abstract

Cited by 298 (3 self)
 Add to MetaCart
(Show Context)
Abstract—An overview is presented of the medical image processing literature on mutualinformationbased registration. The aim of the survey is threefold: an introduction for those new to the field, an overview for those working in the field, and a reference for those searching for literature on a specific application. Methods are classified according to the different aspects of mutualinformationbased registration. The main division is in aspects of the methodology and of the application. The part on methodology describes choices made on facets such as preprocessing of images, gray value interpolation, optimization, adaptations to the mutual information measure, and different types of geometrical transformations. The part on applications is a reference of the literature available on different modalities, on interpatient registration and on different anatomical objects. Comparison studies including mutual information are also considered. The paper starts with a description of entropy and mutual information and it closes with a discussion on past achievements and some future challenges. Index Terms—Image registration, literature survey, matching, mutual information. I.
Automatic Construction of Decision Trees from Data: A MultiDisciplinary Survey
 Data Mining and Knowledge Discovery
, 1997
"... Decision trees have proved to be valuable tools for the description, classification and generalization of data. Work on constructing decision trees from data exists in multiple disciplines such as statistics, pattern recognition, decision theory, signal processing, machine learning and artificial ne ..."
Abstract

Cited by 222 (1 self)
 Add to MetaCart
(Show Context)
Decision trees have proved to be valuable tools for the description, classification and generalization of data. Work on constructing decision trees from data exists in multiple disciplines such as statistics, pattern recognition, decision theory, signal processing, machine learning and artificial neural networks. Researchers in these disciplines, sometimes working on quite different problems, identified similar issues and heuristics for decision tree construction. This paper surveys existing work on decision tree construction, attempting to identify the important issues involved, directions the work has taken and the current state of the art. Keywords: classification, treestructured classifiers, data compaction 1. Introduction Advances in data collection methods, storage and processing technology are providing a unique challenge and opportunity for automated data exploration techniques. Enormous amounts of data are being collected daily from major scientific projects e.g., Human Genome...
An algebra for probabilistic databases
"... An algebra is presented for a simple probabilistic data model that may be regarded as an extension of the standard relational model. The probabilistic algebra is developed in such a way that (restricted to αacyclic database schemes) the relational algebra is a homomorphic image of it. Strictly prob ..."
Abstract

Cited by 148 (1 self)
 Add to MetaCart
An algebra is presented for a simple probabilistic data model that may be regarded as an extension of the standard relational model. The probabilistic algebra is developed in such a way that (restricted to αacyclic database schemes) the relational algebra is a homomorphic image of it. Strictly probabilistic results are emphasized. Variations on the basic probabilistic data model are discussed. The algebra is used to explicate a commonly used statistical smoothing procedure and is shown to be potentially very useful for decision support with uncertain information.
Transporting information and energy simultaneously
 in Proc. 2008 IEEE Int. Symposium on Inform. Theory
"... Abstract—The fundamental tradeoff between the rates at which energy and reliable information can be transmitted over a single noisy line is studied. Engineering inspiration for this problem is provided by powerline communication, RFID systems, and covert packet timing systems as well as communicatio ..."
Abstract

Cited by 95 (2 self)
 Add to MetaCart
(Show Context)
Abstract—The fundamental tradeoff between the rates at which energy and reliable information can be transmitted over a single noisy line is studied. Engineering inspiration for this problem is provided by powerline communication, RFID systems, and covert packet timing systems as well as communication systems that scavenge received energy. A capacityenergy function is defined and a coding theorem is given. The capacityenergy function is a nonincreasing concave ∩ function. Capacityenergy functions for several channels are computed. I.
Collecting User Access Patterns for Building User Profiles and Collaborative Filtering
 In Proceedings of the 1999 International Conference on Intelligent User Interfaces
, 1999
"... The paper proposes a new learning mechanism to extract user preferences transparently for a World Wide Web recommender system. The general idea is that we use the entropy of the page being accessed to determine its interestingness based on its occurrence probability following a sequence of pages acc ..."
Abstract

Cited by 50 (0 self)
 Add to MetaCart
The paper proposes a new learning mechanism to extract user preferences transparently for a World Wide Web recommender system. The general idea is that we use the entropy of the page being accessed to determine its interestingness based on its occurrence probability following a sequence of pages accessed by the user. The probability distribution of the pages is obtained by collecting the access patterns of users navigating on the Web. A finite contextmodel is used to represent the usage information. Based on our proposed model, we have developed an autonomous agent, named ProfBuilder, that works as an online recommender system for a Web site. ProfBuilder uses the usage information as a base for contentbased and collaborative filtering.
Fundamental properties of Tsallis relative entropy
 J. Math. Phys
"... Abstract. Fundamental properties for the Tsallis relative entropy in both classical and quantum systems are studied. As one of our main results, we give the parametric extension of the trace inequality between the quantum relative entropy and the minus of the trace of the relative operator entropy g ..."
Abstract

Cited by 39 (10 self)
 Add to MetaCart
(Show Context)
Abstract. Fundamental properties for the Tsallis relative entropy in both classical and quantum systems are studied. As one of our main results, we give the parametric extension of the trace inequality between the quantum relative entropy and the minus of the trace of the relative operator entropy given by Hiai and Petz. The monotonicity of the quantum Tsallis relative entropy for the trace preserving completely positive linear map is also shown. The generalized Tsallis relative entropy is defined and its subadditivity in the special case is shown by its joint convexity. As a byproduct, the superadditivity of the quantum Tsallis entropy for the independent systems in the case of 0 ≤ q < 1 is obtained. Moreover, the generalized PeierlsBogoliubov inequality is also proven.
A Counterexample to Theorems of Cox and Fine
 JOURNAL OF ARTIFICIAL INTELLIGENCE RESEARCH
, 1999
"... Cox's wellknown theorem justifying the use of probability is shown not to hold infinite domains. The counterexample also suggests that Cox's assumptions are insu cient to prove the result even in infinite domains. The same counterexample is used to disprove a result of Fine on comparative ..."
Abstract

Cited by 37 (2 self)
 Add to MetaCart
Cox's wellknown theorem justifying the use of probability is shown not to hold infinite domains. The counterexample also suggests that Cox's assumptions are insu cient to prove the result even in infinite domains. The same counterexample is used to disprove a result of Fine on comparative conditional probability.