An extension of the infomax algorithm of Bell and Sejnowski (1995) is presented that is able to blindly separate mixed signals with sub- and super-Gaussian source distributions. This was achieved by using a simple type of learning rule first derived by Girolami (1997) by choosing negentropy as a projection pursuit index. Parameterized probability distributions that have suband super-Gaussian regimes were used to derive a general learning rule that preserves the simple architecture proposed by Bell and Sejnowski (1995), is optimized using the natural gradient by Amari (1998), and uses the stability analysis of Cardoso and Laheld (1996) to switch between sub- and super-Gaussian regimes. We demonstrate that the extended infomax algorithm is able to easily separate 20 sources with a variety of source distributions. Applied to high-dimensional data from electroencephalographic (EEG) recordings, it is effective at separating artifacts such as eye blinks and line noise from weaker electrical ...

AbstractÐAn unsupervised classification algorithm is derived by modeling observed data as a mixture of several mutually exclusive classes that are each described by linear combinations of independent, non-Gaussian densities. The algorithm estimates the density of each class and is able to model class distributions with non-Gaussian structure. The new algorithm can improve classification accuracy compared with standard Gaussian mixture models. When applied to blind source separation in nonstationary environments, the method can switch automatically between classes, which correspond to contexts with different mixing properties. The algorithm can learn efficient codes for images containing both natural scenes and text. This method shows promise for modeling non-Gaussian structure in high-dimensional data and has many potential applications. Index TermsÐUnsupervised classification, Gaussian mixture model, independent component analysis, blind source separation, image coding, automatic context switching, maximum likelihood. æ 1

Source separation arises in a variety of signal processing applications, ranging from speech processing to medical image analysis. The separation of a superposition of multiple signals is accomplished by taking into account the structure of the mixing process and by making assumptions about the sources. When the information about the mixing process and sources is limited, the problem is called ‘blind’. By assuming that the sources can be represented sparsely in a given basis, recent research has demonstrated that solutions to previously problematic blind source separation problems can be obtained. In some cases, solutions are possible to problems intractable by previous non-sparse methods. Indeed, sparse methods provide a powerful approach to the separation of linear mixtures of independent data. This paper surveys the recent arrival of sparse blind source separation methods and the previously existing non-sparse methods, providing insights and appropriate hooks into the literature along the way.

Abstract — After summarizing typical approaches for solving independent component analysis (ICA) problems, advances on the ICA studies that consider hybrid sources of both subGaussians and super-Gaussians and the ICA extensions that consider noise and temporal dependence among observations have been overviewed from the perspective of Bayesian Ying-Yang independence learning. Not only new insights are provided on existing results in literature, but also a number of further results are presented.

We examine the relationship between the Bayesian and information-theoretic formulations of source separation algorithms. This work makes use of the relationship between the work of Claude E. Shannon and the ``Recent Contributions" by Warren Weaver (Shannon & Weaver 1949) as clarified by Richard T. Cox (1979) and expounded upon by Robert L. Fry (1996) as a duality between a logic of assertions and a logic of questions. Working with the logic of assertions requires the use of probability as a measure of degree of implication. This leads to a Bayesian formulation of the problem. Whereas, working with the logic of questions requires the use of entropy as a measure of the bearing of a question on an issue leading to an information-theoretic formulation of the problem.

In this thesis, theory and applications of machine learning systems based on information theoretic criteria as performance measures are studied. A new clustering algorithm based on maximizing the Cauchy-Schwarz (CS) divergence measure between probability density functions (pdfs) is proposed. The CS divergence is estimated non-parametrically using the Parzen window technique for density estimation. The problem domain is transformed from discrete 0/1 cluster membership values to continuous membership values. A constrained gradient descent maximization algorithm is implemented. The gradients are stochastically approximated to reduce computational complexity, making the algorithm more practical. Parzen window annealing is incorporated into the algorithm to help avoid convergence to a local maximum. The clustering results obtained on synthetic and real data are encouraging. The Parzen window-based estimator for the CS divergence is shown to have a dual expression as a measure of the cosine of the angle between cluster mean vectors in a feature space determined by the eigenspectrum of a Mercer kernel matrix. A spectral clustering

Abstract. We apply blind source separation techniques from statistical signal processing to separate the traffic in a mix network into either individual flows or groups of flows. This separation requires no a priori information about the individual flows. As a result, unlinkability can be compromised without ever observing individual flows. Our experiments show that this attack is effective and scalable. By correlating separated groups of flows across nodes, a passive attacker can get an accurate traffic map of the mix network. We use a non-trivial network to show that the combined attack works. The experiments also show that multicast traffic can be dangerous for anonymity networks. 1

We propose a methodology to identify nodes in fully anonymized wireless networks using collections of very simple sensors. Based on time series of counts of anonymous packets provided by the sensors, we estimate the number of nodes using Principal Component Analysis. We then proceed to separate the collected packet data into traffic flows that, with help of the spatial diversity in the available sensors, can be used to estimate the location of the wireless nodes. Our simulation experiments indicate that the estimators show high accuracy and high confidence for anonymized TCP traffic. Additional experiments indicate that the estimators perform very well in anonymous wireless networks that use traffic padding. 1

We propose two algorithms to estimate the number and location of nodes in a fully anonymized wireless network using a network of very simple sensors. Based on time series of counts of anonymous packets provided by the sensors, we estimate the density of nodes using Principal Component Analysis. We then proceed to separate the collected packet data into traffic flows that, with help of the spatial diversity in the sensor network, can be used to estimate the location of the wireless nodes. Our simulation experiments indicate that the estimators show high accuracy and high confidence for TCP traffic. Addition experiments indicate that the estimators perform very well in wireless networks that use traffic padding to prevent anonymity attacks. Based on these results, we believe that a new MAC protocol that considers both privacy and efficient use of bandwidth is needed. 1