Results 1  10
of
116
Divergence measures based on the Shannon entropy
 IEEE Transactions on Information theory
, 1991
"... AbstractA new class of informationtheoretic divergence measures based on the Shannon entropy is introduced. Unlike the wellknown Kullback divergences, the new measures do not require the condition of absolute continuity to be satisfied by the probability distributions involved. More importantly, ..."
Abstract

Cited by 451 (0 self)
 Add to MetaCart
AbstractA new class of informationtheoretic divergence measures based on the Shannon entropy is introduced. Unlike the wellknown Kullback divergences, the new measures do not require the condition of absolute continuity to be satisfied by the probability distributions involved. More importantly, their close relationship with the variational distance and the probability of misclassification error are established in terms of bounds. These bounds are crucial in many applications of divergence measures. The new measures are also well characterized by the properties of nonnegativity, finiteness, semiboundedness, and boundedness. Index TermsDivergence, dissimilarity measure, discrimination information, entropy, probability of error bounds. I.
Mutualinformationbased registration of medical images: a survey
 IEEE Transcations on Medical Imaging
, 2003
"... Abstract—An overview is presented of the medical image processing literature on mutualinformationbased registration. The aim of the survey is threefold: an introduction for those new to the field, an overview for those working in the field, and a reference for those searching for literature on a s ..."
Abstract

Cited by 204 (3 self)
 Add to MetaCart
(Show Context)
Abstract—An overview is presented of the medical image processing literature on mutualinformationbased registration. The aim of the survey is threefold: an introduction for those new to the field, an overview for those working in the field, and a reference for those searching for literature on a specific application. Methods are classified according to the different aspects of mutualinformationbased registration. The main division is in aspects of the methodology and of the application. The part on methodology describes choices made on facets such as preprocessing of images, gray value interpolation, optimization, adaptations to the mutual information measure, and different types of geometrical transformations. The part on applications is a reference of the literature available on different modalities, on interpatient registration and on different anatomical objects. Comparison studies including mutual information are also considered. The paper starts with a description of entropy and mutual information and it closes with a discussion on past achievements and some future challenges. Index Terms—Image registration, literature survey, matching, mutual information. I.
Automatic Construction of Decision Trees from Data: A MultiDisciplinary Survey
 Data Mining and Knowledge Discovery
, 1997
"... Decision trees have proved to be valuable tools for the description, classification and generalization of data. Work on constructing decision trees from data exists in multiple disciplines such as statistics, pattern recognition, decision theory, signal processing, machine learning and artificial ne ..."
Abstract

Cited by 164 (1 self)
 Add to MetaCart
(Show Context)
Decision trees have proved to be valuable tools for the description, classification and generalization of data. Work on constructing decision trees from data exists in multiple disciplines such as statistics, pattern recognition, decision theory, signal processing, machine learning and artificial neural networks. Researchers in these disciplines, sometimes working on quite different problems, identified similar issues and heuristics for decision tree construction. This paper surveys existing work on decision tree construction, attempting to identify the important issues involved, directions the work has taken and the current state of the art. Keywords: classification, treestructured classifiers, data compaction 1. Introduction Advances in data collection methods, storage and processing technology are providing a unique challenge and opportunity for automated data exploration techniques. Enormous amounts of data are being collected daily from major scientific projects e.g., Human Genome...
An algebra for probabilistic databases
"... An algebra is presented for a simple probabilistic data model that may be regarded as an extension of the standard relational model. The probabilistic algebra is developed in such a way that (restricted to αacyclic database schemes) the relational algebra is a homomorphic image of it. Strictly prob ..."
Abstract

Cited by 133 (1 self)
 Add to MetaCart
An algebra is presented for a simple probabilistic data model that may be regarded as an extension of the standard relational model. The probabilistic algebra is developed in such a way that (restricted to αacyclic database schemes) the relational algebra is a homomorphic image of it. Strictly probabilistic results are emphasized. Variations on the basic probabilistic data model are discussed. The algebra is used to explicate a commonly used statistical smoothing procedure and is shown to be potentially very useful for decision support with uncertain information.
Collecting User Access Patterns for Building User Profiles and Collaborative Filtering
 In Proceedings of the 1999 International Conference on Intelligent User Interfaces
, 1999
"... The paper proposes a new learning mechanism to extract user preferences transparently for a World Wide Web recommender system. The general idea is that we use the entropy of the page being accessed to determine its interestingness based on its occurrence probability following a sequence of pages acc ..."
Abstract

Cited by 45 (0 self)
 Add to MetaCart
The paper proposes a new learning mechanism to extract user preferences transparently for a World Wide Web recommender system. The general idea is that we use the entropy of the page being accessed to determine its interestingness based on its occurrence probability following a sequence of pages accessed by the user. The probability distribution of the pages is obtained by collecting the access patterns of users navigating on the Web. A finite contextmodel is used to represent the usage information. Based on our proposed model, we have developed an autonomous agent, named ProfBuilder, that works as an online recommender system for a Web site. ProfBuilder uses the usage information as a base for contentbased and collaborative filtering.
A Counterexample to Theorems of Cox and Fine
 JOURNAL OF ARTIFICIAL INTELLIGENCE RESEARCH
, 1999
"... Cox's wellknown theorem justifying the use of probability is shown not to hold infinite domains. The counterexample also suggests that Cox's assumptions are insu cient to prove the result even in infinite domains. The same counterexample is used to disprove a result of Fine on comparative ..."
Abstract

Cited by 35 (2 self)
 Add to MetaCart
Cox's wellknown theorem justifying the use of probability is shown not to hold infinite domains. The counterexample also suggests that Cox's assumptions are insu cient to prove the result even in infinite domains. The same counterexample is used to disprove a result of Fine on comparative conditional probability.
Fundamental properties of Tsallis relative entropy
 J. Math. Phys
"... Abstract. Fundamental properties for the Tsallis relative entropy in both classical and quantum systems are studied. As one of our main results, we give the parametric extension of the trace inequality between the quantum relative entropy and the minus of the trace of the relative operator entropy g ..."
Abstract

Cited by 24 (7 self)
 Add to MetaCart
(Show Context)
Abstract. Fundamental properties for the Tsallis relative entropy in both classical and quantum systems are studied. As one of our main results, we give the parametric extension of the trace inequality between the quantum relative entropy and the minus of the trace of the relative operator entropy given by Hiai and Petz. The monotonicity of the quantum Tsallis relative entropy for the trace preserving completely positive linear map is also shown. The generalized Tsallis relative entropy is defined and its subadditivity in the special case is shown by its joint convexity. As a byproduct, the superadditivity of the quantum Tsallis entropy for the independent systems in the case of 0 ≤ q < 1 is obtained. Moreover, the generalized PeierlsBogoliubov inequality is also proven.
Universal Coding with Minimum Probability of Codeword Length Overflow
 IEEE Trans. Information Theory
, 1991
"... AbstractLossless blocktovariable length source coding is studied for finitestate, finitealphabet sources. We aim to minimize the probability that the normalized length of the codeword will exceed a given threshold B, subject to the Kraft inequality. It is shown that the LempelZiv (LZ) algorit ..."
Abstract

Cited by 17 (3 self)
 Add to MetaCart
(Show Context)
AbstractLossless blocktovariable length source coding is studied for finitestate, finitealphabet sources. We aim to minimize the probability that the normalized length of the codeword will exceed a given threshold B, subject to the Kraft inequality. It is shown that the LempelZiv (LZ) algorithm asymptotically attains the optimal performance in the sense just defined, independently of the source and the value of B. For the subclass of unifilar Markov sources, faster convergence to the asymptotic optimum performance can be accomplished by using the minimum description length (MDL) universal code for this subclass. It is demonstrated that these universal codes are also nearly optimal in the sense of minimizing buffer overflow probability, and asymptotically optimal in a competitive sense. Index TemUniversal noiseless coding, LempelZiv algorithm, length overflow, buffer overflow, finitestate sources, large deviations, competitive optimality. I.
Objective classification of galaxy spectra using the information bottleneck method
 Monthly Notes of the Royal Astronomical Society
, 2001
"... method ..."
(Show Context)