Results 1  10
of
16
Advances in nonlinear blind source separation
 In Proc. of the 4th Int. Symp. on Independent Component Analysis and Blind Signal Separation (ICA2003
, 2003
"... Abstract — In this paper, we briefly review recent advances in blind source separation (BSS) for nonlinear mixing models. After a general introduction to the nonlinear BSS and ICA (independent Component Analysis) problems, we discuss in more detail uniqueness issues, presenting some new results. A f ..."
Abstract

Cited by 41 (2 self)
 Add to MetaCart
(Show Context)
Abstract — In this paper, we briefly review recent advances in blind source separation (BSS) for nonlinear mixing models. After a general introduction to the nonlinear BSS and ICA (independent Component Analysis) problems, we discuss in more detail uniqueness issues, presenting some new results. A fundamental difficulty in the nonlinear BSS problem and even more so in the nonlinear ICA problem is that they are nonunique without extra constraints, which are often implemented by using a suitable regularization. Postnonlinear mixtures are an important special case, where a nonlinearity is applied to linear mixtures. For such mixtures, the ambiguities are essentially the same as for the linear ICA or BSS problems. In the later part of this paper, various separation techniques proposed for postnonlinear mixtures and general nonlinear mixtures are reviewed. I. THE NONLINEAR ICA AND BSS PROBLEMS Consider Æ samples of the observed data vector Ü, modeled by
Unsupervised variational Bayesian learning of nonlinear models
 In Advances in Neural Information Processing Systems 17
, 2005
"... In this paper we present a framework for using multilayer perceptron (MLP) networks in nonlinear generative models trained by variational Bayesian learning. The nonlinearity is handled by linearizing it using a Gauss–Hermite quadrature at the hidden neurons. This yields an accurate approximation fo ..."
Abstract

Cited by 28 (11 self)
 Add to MetaCart
(Show Context)
In this paper we present a framework for using multilayer perceptron (MLP) networks in nonlinear generative models trained by variational Bayesian learning. The nonlinearity is handled by linearizing it using a Gauss–Hermite quadrature at the hidden neurons. This yields an accurate approximation for cases of large posterior variance. The method can be used to derive nonlinear counterparts for linear algorithms such as factor analysis, independent component/factor analysis and statespace models. This is demonstrated with a nonlinear factor analysis experiment in which even 20 sources can be estimated from a real world speech data set. 1
Blind Separation of Postnonlinear Mixtures using Linearizing Transformations and Temporal Decorrelation
 JOURNAL OF MACHINE LEARNING RESEARCH
, 2003
"... We propose two methods that reduce the postnonlinear blind source separation problem (PNLBSS) to a linear BSS problem. The first method is based on the concept of maximal correlation: we apply the alternating conditional expectation (ACE) algorithma powerful technique from nonparametric stati ..."
Abstract

Cited by 26 (2 self)
 Add to MetaCart
We propose two methods that reduce the postnonlinear blind source separation problem (PNLBSS) to a linear BSS problem. The first method is based on the concept of maximal correlation: we apply the alternating conditional expectation (ACE) algorithma powerful technique from nonparametric statisticsto approximately invert the componentwise nonlinear functions. The second method is a Gaussianizing transformation, which is motivated by the fact that linearly mixed signals before nonlinear transformation are approximately Gaussian distributed. This heuristic, but simple and efficient procedure works as good as the ACE method. Using the framework provided by ACE, convergence can be proven. The optimal transformations obtained by ACE coincide with the soughtafter inverse functions of the nonlinearities. After equalizing the nonlinearities, temporal decorrelation separation (TDSEP) allows us to recover the source signals. Numerical simulations testing "ACETD" and "GaussTD" on realistic examples are performed with excellent results.
Variational learning and bitsback coding: an informationtheoretic view to Bayesian learning
 IEEE Transactions on Neural Networks
"... Abstract—The bitsback coding first introduced by Wallace in 1990 and later by Hinton and van Camp in 1993 provides an interesting link between Bayesian learning and informationtheoretic minimumdescriptionlength (MDL) learning approaches. The bitsback coding allows interpreting the cost function ..."
Abstract

Cited by 18 (7 self)
 Add to MetaCart
Abstract—The bitsback coding first introduced by Wallace in 1990 and later by Hinton and van Camp in 1993 provides an interesting link between Bayesian learning and informationtheoretic minimumdescriptionlength (MDL) learning approaches. The bitsback coding allows interpreting the cost function used in the variational Bayesian method called ensemble learning as a code length in addition to the Bayesian view of misfit of the posterior approximation and a lower bound of model evidence. Combining these two viewpoints provides interesting insights to the learning process and the functions of different parts of the model. In this paper, the problem of variational Bayesian learning of hierarchical latent variable models is used to demonstrate the benefits of the two views. The codelength interpretation provides new views to many parts of the problem such as model comparison and pruning and helps explain many phenomena occurring in learning. Index Terms—Bitsback coding, ensemble learning, hierarchical latent variable models, minimum description length, variational Bayesian learning. I.
Building Blocks For Variational Bayesian Learning Of Latent Variable Models
 JOURNAL OF MACHINE LEARNING RESEARCH
, 2006
"... We introduce standardised building blocks designed to be used with variational Bayesian learning. The blocks include Gaussian variables, summation, multiplication, nonlinearity, and delay. A large variety of latent variable models can be constructed from these blocks, including variance models a ..."
Abstract

Cited by 11 (8 self)
 Add to MetaCart
We introduce standardised building blocks designed to be used with variational Bayesian learning. The blocks include Gaussian variables, summation, multiplication, nonlinearity, and delay. A large variety of latent variable models can be constructed from these blocks, including variance models and nonlinear modelling, which are lacking from most existing variational systems. The introduced blocks are designed to fit together and to yield e#cient update rules. Practical implementation of various models is easy thanks to an associated software package which derives the learning formulas automatically once a specific model structure has been fixed. Variational Bayesian learning provides a cost function which is used both for updating the variables of the model and for optimising the model structure. All the computations can be carried out locally, resulting in linear computational complexity. We present
Approximating nonlinear transformations of probability distributions for nonlinear independent component analysis
 In Proc. 2004 IEEE Int. Joint Conf. on Neural Networks (IJCNN 2004
, 2004
"... Abstract — The nonlinear independent component analysis method introduced by Lappalainen and Honkela in 2000 uses a truncated Taylor series representation to approximate the nonlinear transformation from sources to observations. The approach uses information only at the single point of input mean an ..."
Abstract

Cited by 8 (3 self)
 Add to MetaCart
(Show Context)
Abstract — The nonlinear independent component analysis method introduced by Lappalainen and Honkela in 2000 uses a truncated Taylor series representation to approximate the nonlinear transformation from sources to observations. The approach uses information only at the single point of input mean and can produce poor results if the input variance is large. This feature has recently been identified to be the cause of instability of the algorithm with large source dimensionalities. In this paper, an improved approximation is presented. The derivatives used in the Taylor scheme are replaced with slopes evaluated by global GaussHermite quadrature. The resulting approximation is more accurate under high input variance and the new learning algorithm more stable with high source dimensionalities. I.
Partially observed values, in
 Proc. Int. Joint Conf. on Neural Networks (IJCNN 2004
, 2004
"... It is common to have both observed and missing values in data. This paper concentrates on the case where a value can be somewhere between those two ends, partially observed and partially missing. To achieve that, a method of using evidence nodes in a Bayesian network is studied. Different ways of ha ..."
Abstract

Cited by 7 (4 self)
 Add to MetaCart
(Show Context)
It is common to have both observed and missing values in data. This paper concentrates on the case where a value can be somewhere between those two ends, partially observed and partially missing. To achieve that, a method of using evidence nodes in a Bayesian network is studied. Different ways of handling inaccuracies are discussed in examples and the proposed approach is justified in the experiments with real image data.
Using kernel PCA for initialisation of variational Bayesian nonlinear blind source separation method
 Proc. of the Fifth Int. Conf. on Independent Component Analysis and Blind Signal Separation (ICA 2004), volume 3195 of Lecture Notes in Computer Science
, 2004
"... Abstract. The variational Bayesian nonlinear blind source separation method introduced by Lappalainen and Honkela in 2000 is initialised with linear principal component analysis (PCA). Because of the multilayer perceptron (MLP) network used to model the nonlinearity, the method is susceptible to loc ..."
Abstract

Cited by 7 (2 self)
 Add to MetaCart
(Show Context)
Abstract. The variational Bayesian nonlinear blind source separation method introduced by Lappalainen and Honkela in 2000 is initialised with linear principal component analysis (PCA). Because of the multilayer perceptron (MLP) network used to model the nonlinearity, the method is susceptible to local minima and therefore sensitive to the initialisation used. As the method is used for nonlinear separation, the linear initialisation may in some cases lead it astray. In this paper we study the use of kernel PCA (KPCA) in the initialisation. KPCA is a rather straightforward generalisation of linear PCA and it is much faster to compute than the variational Bayesian method. The experiments show that it can produce significantly better initialisations than linear PCA. Additionally, the model comparison methods provided by the variational Bayesian framework can be easily applied to compare different kernels. 1
Postnonlinear independent component analysis by variational Bayesian learning
 In Proc. 5th Int. Conf. on Independent Component Analysis and Blind Signal Separation (ICA 2004
, 2004
"... Abstract. Postnonlinear (PNL) independent component analysis (ICA) is a generalisation of ICA where the observations are assumed to have been generated from independent sources by linear mixing followed by componentwise scalar nonlinearities. Most previous PNL ICA algorithms require the postnonli ..."
Abstract

Cited by 5 (2 self)
 Add to MetaCart
(Show Context)
Abstract. Postnonlinear (PNL) independent component analysis (ICA) is a generalisation of ICA where the observations are assumed to have been generated from independent sources by linear mixing followed by componentwise scalar nonlinearities. Most previous PNL ICA algorithms require the postnonlinearities to be invertible functions. In this paper, we present a variational Bayesian approach to PNL ICA that also works for noninvertible postnonlinearities. The method is based on a generative model with multilayer perceptron (MLP) networks to model the postnonlinearities. Preliminary results with a difficult artificial example are encouraging. 1
Bayesian versus constrained structure approaches for source separation in postnonlinear mixtures
 in Proc. International Joint Conference on Neural Networks (IJCNN 2004
, 2004
"... This material is posted here with permission of the IEEE. Such permission of the IEEE does not in any way imply IEEE endorsement of any of Helsinki University of Technology's products or services. Internal or personal use of this material is permitted. However, permission to reprint/republish t ..."
Abstract

Cited by 5 (1 self)
 Add to MetaCart
(Show Context)
This material is posted here with permission of the IEEE. Such permission of the IEEE does not in any way imply IEEE endorsement of any of Helsinki University of Technology's products or services. Internal or personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution must be obtained from the IEEE by writing to