Abstract not found

We show that different theories recently proposed for Independent Component Analysis (ICA) lead to the same iterative learning algorithm for blind separation of mixed independent sources. We review those theories and suggest that information theory can be used to unify several lines of research. Pearlmutter and Parra (1996) and Cardoso (1997) showed that the infomax approach of Bell and Sejnowski (1995) and the maximum likelihood estimation approach are equivalent. We show that negentropy maximization also has equivalent properties and therefore all three approaches yield the same learning rule for a fixed nonlinearity. Girolami and Fyfe (1997a) have shown that the nonlinear Principal Component Analysis (PCA) algorithm of Karhunen and Joutsensalo (1994) and Oja (1997) can also be viewed from information-theoretic principles since it minimizes the sum of squares of the fourth-order marginal cumulants and therefore approximately minimizes the mutual information (Comon, 1994). Lambert (19...

A general tool for multichannel and multipath problems is given in FIR matrix algebra. With Finite Impulse Response (FIR) filters (or polynomials) assuming the role played by complex scalars in traditional matrix algebra, we adapt standard eigenvalue routines, factorizations, decompositions, and matrix algorithms for use in multichannel /multipath problems. Using abstract algebra/group theoretic concepts, information theoretic principles, and the Bussgang property, methods of single channel filtering and source separation of multipath mixtures are merged into a general FIR matrix framework. Techniques developed for equalization may be applied to source separation and vice versa. Potential applications of these results lie in neural networks with feed-forward memory connections, wideband array processing, and in problems with a multi-input, multi-output network having channels between each source and sensor, such as source separation. Particular applications of FIR polynomial matrix alg...

A measure of temporal predictability is defined and used to separate linear mixtures of signals. Given any set of statistically independent source signals, it is conjectured here that a linear mixture of those signals has the following property: the temporal predictability of any signal mixture is less than (or equal to) that of any of its component source signals. It is shown that this property can be used to recover source signals from a set of linear mixtures of those signals by finding an un-mixing matrix that maximizes a measure of temporal predictability for each recovered signal. This matrix is obtained as the solution to a generalized eigenvalue problem; such problems have scaling characteristics of O(N 3), where N is the number of signal mixtures. In contrast to independent component analysis, the temporal predictability method requires minimal assumptions regarding the probability density functions of source signals. It is demonstrated that the method can separate signal mixtures in which each mixture is a linear combination of source signals with supergaussian, subgaussian, and gaussian probability density functions and on mixtures of voices and music.

Spatiotemporal connectionist networks (STCN's) comprise an important class of neural models that can deal with patterns distributed both in time and space. In this paper, we widen the application domain of the taxonomy for supervised STCN's recently proposed by Kremer (2001) to the unsupervised case. This is possible through a reinterpretation of the state vector as a vector of latent (hidden) variables, as proposed by Meinicke (2000). The goal of this generalized taxonomy is then to provide a nonlinear generative framework for describing unsupervised spatiotemporal networks, making it easier to compare and contrast their representational and operational characteristics. Computational properties, representational issues and learning are also discussed and a number of references to the relevant source publications are provided. It is argued that the proposed approach is simple and more powerful than the previous attempts, from a descriptive and predictive viewpoint. We also discuss the relation of this taxonomy with automata theory and state space modeling, and suggest directions for further work.

Blind separation and blind deconvolution are related problems in unsupervised learning. In blind separation [7], illustrated in Fig.1a, a set of sources, s 1 (t); : : : ; s N (t), (different people speaking, music etc) are mixed together linearly by a matrix a. We do not know anything about the sources, or the mixing process. All we receive are the N superpositions of them, x 1 (t); : : : ; xN (t). The task is to recover the original sources by finding a square matrix W which is a permutation of the inverse of the unknown matrix, A. The problem has also been called the `cocktail-party' problem. In blind deconvolution [6], illustrated in Fig.1b, an unknown signal s(t) is convolved with an unknown tapped delay-line filter, a 1 ; : : : ; aL , giving a corrupted signal x(t) = a(t) s(t) where a(t) is the impulse response of the filter. The task is to recover s(t) by convolving x(t) with a learnt filter w 1 ; : : : ; wL which reverses the effect of the filter a. Both problems are difficu...

This article reviews space-time modem technology for mobile radio applications. We begin with motivations for the use of space-time modems and then briefly discuss the challenges posed by wireless propagation. Next, we develop a signal model for the wireless environment. Channel estimation, equalization, and filtering techniques for space-time modems in the forward and reverse links are then discussed. Finally we review applications of space-time modems to cellular systems and discuss industry trends.

A computationally simple nonlinearity in the form of a threshold device is shown to serve as contrast function in blind signal separation. Convergence is shown to be robust, fast, and comparable with that of more complex polynomial nonlinearities. Together with the known signum nonlinearity for super-Gaussian distributions, which basically is a threshold device with the threshold set to zero, the general threshold nonlinearity (with an appropriate threshold) can separate any non-Gaussian signals. 1. INTRODUCTION Blind signal separation using higher-order statistics either explicitly or implicitly has attracted many researchers whose main goal is to separate a set of mixed signals as fast as possible with the smallest residual mixing. Throughout this paper we assume a linear mixing and separation process as depicted in Fig. 1. A W s x u separation process mixing process sensors separated sources sources Figure 1: Blind source separation model. The measured signals x = [x 1 , . . . ...

{ This paper presents extensions of stochastic gradient independent component analysis (ICA) methods to the blind deconvolution task. Of particular importance in these extensions are the constraints placed on the deconvolution system transfer function. While unit-norm constrained ICA approaches can be directly applied to the blind deconvolution task, an allpass lter constraint within the optimization procedure is more appropriate. We show how such constraints can be approximately imposed within gradient adaptive nite-impulse-response (FIR) lter implementations by proper extensions of gradient techniques within the Stiefel manifold of orthonormal matrices. Both on-line time-domain and block-based frequency-domain algorithms are described. Simulations verify the superior performance behaviors provided by our allpass-constrained algorithms over standard unit-norm-constrained ICA algorithms in blind deconvolution tasks. accepted for publication in Journal of VLSI Signal Processing Sys...

A computationally simple nonlinearity in the form of a threshold device for the blind separation of sub-Gaussian signals is derived. Convergence is shown to be robust, fast, and comparable to that of more complex polynomial nonlinearities. Together with the known signum nonlinearity for super-Gaussian distributions, which basically is a threshold device with the threshold set to zero, the general threshold nonlinearity (with an appropriate threshold) can separate any non-Gaussian signals. 1. INTRODUCTION Blind signal separation using higher-order statistics either explicitly or implicitly has attracted many researchers whose main goal is to separate a set of mixed signals as fast as possible with the smallest residual mixing. Throughout this paper we assume a linear mixing and separation process as depicted in Fig. 1. A W s x u separation process mixing process sensors separated sources sources Figure 1: Blind source separation model. The measured signals x = [x 1 , . . . , x M S ...