Results 1  10
of
14
A Novel Measure for Independent Component Analysis (ICA)
"... Measures of independence (and dependence) are fundamental in many areas of engineering and signal processing. Shannon introduced the idea of Information Entropy which has a sound theoretical foundation but sometimes is not easy to implement in engineering applications. In this paper, Renyi's Entropy ..."
Abstract

Cited by 24 (10 self)
 Add to MetaCart
Measures of independence (and dependence) are fundamental in many areas of engineering and signal processing. Shannon introduced the idea of Information Entropy which has a sound theoretical foundation but sometimes is not easy to implement in engineering applications. In this paper, Renyi's Entropy is used and a novel independence measure is proposed. When integrated with a nonparametric estimator of the probability density function (Parzen Window), the measure can be related to the "potential energy of the samples" which is easy to understand and implement. The experimental results on Blind Source Separation confirm the theory. Although the work is preliminary, the "potential energy" method is rather general and will have many applications. 1. INTRODUCTION Information theory is a powerful tool in communication, signal processing, and even machine learning. The parallel between information and energy is well known and here their measures will be also linked. This paper shows that our...
InformationTheoretic Learning Using Renyi's Quadratic Entropy
 Proceedings of the First International Workshop on Independent Component Analysis and Signal Separation, Aussois
, 1999
"... Learning from examples has been traditionally based on correlation or on the mean square error (MSE) criterion, in spite of the fact that learning is intrinsically related with the extraction of information from examples. The problem is that Shannon's Information Entropy, which has a sound theoretic ..."
Abstract

Cited by 13 (1 self)
 Add to MetaCart
Learning from examples has been traditionally based on correlation or on the mean square error (MSE) criterion, in spite of the fact that learning is intrinsically related with the extraction of information from examples. The problem is that Shannon's Information Entropy, which has a sound theoretical foundation, is not easy to implement in a learning from examples scenario. In this paper, Renyi's Entropy definition is used and integrated with a nonparametric estimator of the probability density function (Parzen Window) . We develop a physical analogy for the estimation method which we call the information potential field. The "information potential" method is rather general and will have many applications. 1.0 Introduction Information theory is a powerful tool in communication, providing the basis for the design of efficient and reliable communication systems [18]. However, these advances are based on a priori assumptions about the data distributions and the form of the mapping, whic...
On Key Distribution via True Broadcasting
 In ACM Conference on Computer and Communications Security
, 1994
"... We consider true broadcast systems for the secure communication of session keys. These schemes provide for parallel rather than serial construction of broadcast messages, while avoiding selective broadcasting. We begin by introducing a conceptual framework for true broadcasting and illustrate its de ..."
Abstract

Cited by 12 (0 self)
 Add to MetaCart
We consider true broadcast systems for the secure communication of session keys. These schemes provide for parallel rather than serial construction of broadcast messages, while avoiding selective broadcasting. We begin by introducing a conceptual framework for true broadcasting and illustrate its design with a secure key broadcast scheme based on probabilistic encryption. The framework provides for a system requiring user anonymity, as a result of the absence of addressing for the broadcast message. We also illustrate how Shamir's threshold scheme can be altered to allow for parallel broadcasting. We then present a formal model and use information theoretic techiques to establish a lower bound on the size of the broadcast message for a class of true broadcast schemes. Finally, we improve upon the aforementioned threshold scheme such that it achieves the lower bound. 1991 AMS Classification: 94A60 CR Categories: D.4.6 Key Words and Phrases: Key distribution, broadcast encryption, sec...
Evaluating Information Transfer Between Auditory Cortical Neurons innovative methodology
, 2006
"... You might find this additional information useful... This article cites 68 articles, 24 of which you can access free at: ..."
Abstract

Cited by 9 (0 self)
 Add to MetaCart
You might find this additional information useful... This article cites 68 articles, 24 of which you can access free at:
Paraconsistent Reasoning as an Analytic Tool
 JOURNAL OF THE INTEREST GROUP IN PROPOSITIONAL LOGIC
, 2001
"... The study of logic usually focuses on either the proof theoretic or the model theoretic properties of logic. Yet the pragmatics of logic is often ignored. In this paper we would like to demonstrate that a logic can be practical in the sense that it can assist us in evaluating and measuring the amoun ..."
Abstract

Cited by 8 (0 self)
 Add to MetaCart
The study of logic usually focuses on either the proof theoretic or the model theoretic properties of logic. Yet the pragmatics of logic is often ignored. In this paper we would like to demonstrate that a logic can be practical in the sense that it can assist us in evaluating and measuring the amount of information in an inconsistent set of data. The underlying notion of information is inspired by Shannon's communication theory. It defines the amount of information of a message in terms of the probability of the message being true. The logic presented here is the paraconsistent logic QC. As such QC logic can be seen as an analytical tool for evaluating data.
A general approach to sparse basis selection: Majorization, concavity, and affine scaling
 IN PROCEEDINGS OF THE TWELFTH ANNUAL CONFERENCE ON COMPUTATIONAL LEARNING THEORY
, 1997
"... Measures for sparse best–basis selection are analyzed and shown to fit into a general framework based on majorization, Schurconcavity, and concavity. This framework facilitates the analysis of algorithm performance and clarifies the relationships between existing proposed concentration measures use ..."
Abstract

Cited by 6 (3 self)
 Add to MetaCart
Measures for sparse best–basis selection are analyzed and shown to fit into a general framework based on majorization, Schurconcavity, and concavity. This framework facilitates the analysis of algorithm performance and clarifies the relationships between existing proposed concentration measures useful for sparse basis selection. It also allows one to define new concentration measures, and several general classes of measures are proposed and analyzed in this paper. Admissible measures are given by the Schurconcave functions, which are the class of functions consistent with the socalled Lorentz ordering (a partial ordering on vectors also known as majorization). In particular, concave functions form an important subclass of the Schurconcave functions which attain their minima at sparse solutions to the best basis selection problem. A general affine scaling optimization algorithm obtained from a special factorization of the gradient function is developed and proved to converge to a sparse solution for measures chosen from within this subclass.
Assessment of an LKurtosisBased Criterion for Quantile Estimation
, 2001
"... The estimation of extreme quantiles corresponding to small probabilities of exceedance is commonly required in the risk analysis of flood protection structures. The usefulness of Lmoments has been well recognized in the statistical analysis of data, because they can be estimated with less uncerta ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
The estimation of extreme quantiles corresponding to small probabilities of exceedance is commonly required in the risk analysis of flood protection structures. The usefulness of Lmoments has been well recognized in the statistical analysis of data, because they can be estimated with less uncertainty than that associated with traditional moment estimates. The objective of the paper is to assess the effectiveness of Lkurtosis in the method of Lmoments for distribution fitting and quantile estimation from small samples. For this purpose, the performance of the proposed Lkurtosisbased criterion is compared against a set of benchmark measures of goodness of fit, namely, divergence, integratedsquare error, chi square, and probabilityplot correlation.
On the Semantics of Entropy Measures of Emergent Phenomena
 Cybernetics and Systems
, 1990
"... This paper is an attempt to perform some semantic analysis of the concept of "entropy" as it is used in both thermodynamic and nonthermodynamic contexts. We will first consider the relation between thermodynamic entropy as a semantic concept, a measured quantity of real systems; and statistical ent ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
This paper is an attempt to perform some semantic analysis of the concept of "entropy" as it is used in both thermodynamic and nonthermodynamic contexts. We will first consider the relation between thermodynamic entropy as a semantic concept, a measured quantity of real systems; and statistical entropy as a syntactic concept, a calculated quantity in formal systems. We will then consider the use of entropy as a measure of different types of systems, the origin and identification of emergent levels in physical systems, and the ontological status of nonthermodynamic entropies. 1 Syntactic vs. Semantic Definitions of Entropy The literature on the relation between thermodynamics and information theory is vast [8, 10, 23, 26, 51, 61, etc.]. While this debate is not the central concern of this paper, in any discussion of entropy it is wise to clearly express how it will be understood. We do so on the basis of the relation and difference between, on the one hand, quantities which result fr...
Coding. New York McGrawHill, 1979.
, 1981
"... [lo] L. Ozarow, “On a sourcecoding problem with two channels and three ..."
Broadcasting Session Keys
, 1994
"... This paper considers true broadcast systems for the secure communication of session keys. These are schemes that provide for parallel rather than serial construction of broadcast messages, while avoiding selective broadcasting. We improve upon a model implemented with Shamir's threshold scheme, and ..."
Abstract
 Add to MetaCart
This paper considers true broadcast systems for the secure communication of session keys. These are schemes that provide for parallel rather than serial construction of broadcast messages, while avoiding selective broadcasting. We improve upon a model implemented with Shamir's threshold scheme, and use information theoretic techniques to show that this enhancement is optimal. We then present a general foundation upon which true broadcasting can be realized and illustrate its design with two number theoretic implementations. 1991 AMS Classification: 94A60 CR Categories: D.4.6 Key Words and Phrases: Session key, Secure Broadcasting, True Broadcasting, Broadcast Encryption, Sealed Lock, Secret Sharing. Carleton University, School of Computer Science: SCSTR233 Note: A more recent version of this paper is included in the proceedings of the 2nd Annual ACM Conference on Computer and Communications Security, 1994. Carleton University, School of Computer Science, Ottawa, ON, Canada. K1...