Citation Context

...linear parametric model, we propose to estimate using a multilayer perceptron. This classic neural architecture has many interesting properties, one of which is its universal approximation capability =-=[17]-=-. We use a one hidden-layer perceptron with a linear output neuron (54) Fig. 5. Score functions of a uniform distribution. Theoretical ( ),sGC Y (--),sMSE Y (-). where denotes the sigmoidal activ...

Citation Context

...ar matrix. Source separation consists then of estimating a square nonsingular matrix such that (4) has statistically independent components. is called a separating matrix. Denoting , it is well known =-=[8]-=- that using an independence criterion, we can only estimate sources, when at most one is Gaussian, up to any diagonal matrix (scale factor) and up to any permutation matrix . In other words, separatio...

Citation Context

... this reason, the problem is generally called blind source separation. The source separation problem has been intensively studied over the last 12 years for linear instantaneous (memoryless) mixtures =-=[2]-=-, [5], [12], [16], [18], [20], [22], [25], [29] and, more recently, since 1990, for linear convolutive mixtures [13], [15], [19], [24], [36], [37], [42]. Up to now, more realistic models, especially n...

Citation Context

...lem is generally called blind source separation. The source separation problem has been intensively studied over the last 12 years for linear instantaneous (memoryless) mixtures [2], [5], [12], [16], =-=[18]-=-, [20], [22], [25], [29] and, more recently, since 1990, for linear convolutive mixtures [13], [15], [19], [24], [36], [37], [42]. Up to now, more realistic models, especially nonlinear models, have b...

Citation Context

...T-NONLINEAR MIXTURES 2811 V. PDF AND SCORE FUNCTIONS The derivatives of with respect to (25) and to (29) point out the importance of the functions called score functions (30) Recently, Cardoso et al. =-=[6]-=- showed that the nonlinearities minimizing the rejection rates of equivariant algorithms are proportional to in the case of i.i.d. sources . Earlier, Pham et al. [29] showed for the linear instantaneo...

Citation Context

...o estimate the linear stage, i.e., matrix , we use an equivariant algorithm by a postmultiplication of (25) by ,1 1The effect of BTB on the algorithm is treated by Cardoso et al. [6] and Amari et al. =-=[1]-=-. TALEB AND JUTTEN: SOURCE SEPARATION IN POST-NONLINEAR MIXTURES 2815 Fig. 7. Source distribution (s 2 ; s 1 ) points out the source independence. which produces the algorithm (69) where the entries o...

Citation Context

...on, the problem is generally called blind source separation. The source separation problem has been intensively studied over the last 12 years for linear instantaneous (memoryless) mixtures [2], [5], =-=[12]-=-, [16], [18], [20], [22], [25], [29] and, more recently, since 1990, for linear convolutive mixtures [13], [15], [19], [24], [36], [37], [42]. Up to now, more realistic models, especially nonlinear mo...

Citation Context

...blind source separation. The source separation problem has been intensively studied over the last 12 years for linear instantaneous (memoryless) mixtures [2], [5], [12], [16], [18], [20], [22], [25], =-=[29]-=- and, more recently, since 1990, for linear convolutive mixtures [13], [15], [19], [24], [36], [37], [42]. Up to now, more realistic models, especially nonlinear models, have been sketched by only a f...

Citation Context

...components of its output become statistically independent. This is achieved if the joint density factorizes as the product of the marginal densities (15) As proposed by a few authors [8], [28], [38], =-=[41]-=-, statistical independence can be measured using the Kullback–Liebler (KL) divergence between and KL (16) which is equal to Shannon’s mutual information between the components of output vector . This ...

Citation Context

... reason, the problem is generally called blind source separation. The source separation problem has been intensively studied over the last 12 years for linear instantaneous (memoryless) mixtures [2], =-=[5]-=-, [12], [16], [18], [20], [22], [25], [29] and, more recently, since 1990, for linear convolutive mixtures [13], [15], [19], [24], [36], [37], [42]. Up to now, more realistic models, especially nonlin...

Citation Context

...last 12 years for linear instantaneous (memoryless) mixtures [2], [5], [12], [16], [18], [20], [22], [25], [29] and, more recently, since 1990, for linear convolutive mixtures [13], [15], [19], [24], =-=[36]-=-, [37], [42]. Up to now, more realistic models, especially nonlinear models, have been sketched by only a few authors. Burel [4] has proposed a neural-network based solution for the case of known nonl...

Citation Context

...e problem is generally called blind source separation. The source separation problem has been intensively studied over the last 12 years for linear instantaneous (memoryless) mixtures [2], [5], [12], =-=[16]-=-, [18], [20], [22], [25], [29] and, more recently, since 1990, for linear convolutive mixtures [13], [15], [19], [24], [36], [37], [42]. Up to now, more realistic models, especially nonlinear models, ...

Citation Context

...s for linear instantaneous (memoryless) mixtures [2], [5], [12], [16], [18], [20], [22], [25], [29] and, more recently, since 1990, for linear convolutive mixtures [13], [15], [19], [24], [36], [37], =-=[42]-=-. Up to now, more realistic models, especially nonlinear models, have been sketched by only a few authors. Burel [4] has proposed a neural-network based solution for the case of known nonlinearity dep...

Citation Context

...y, since 1990, for linear convolutive mixtures [13], [15], [19], [24], [36], [37], [42]. Up to now, more realistic models, especially nonlinear models, have been sketched by only a few authors. Burel =-=[4]-=- has proposed a neural-network based solution for the case of known nonlinearity depending Manuscript received January 6, 1998; revised March 12, 1999. The associate editor coordinating the review of ...

Citation Context

...write (35) and deduce (36) If has a standard deviation , then (36) may be simplified to (37) where is the th-order cumulant. This kind of expansion has already been used by Comon [8] and Gaeta et al. =-=[14]-=- in the linear case and by Taleb and Jutten [34] and Yang et al. [39] in the nonlinear source separation case. The main advantages of this approach are its simplicity and its low computational cost. A...

Citation Context

... generally called blind source separation. The source separation problem has been intensively studied over the last 12 years for linear instantaneous (memoryless) mixtures [2], [5], [12], [16], [18], =-=[20]-=-, [22], [25], [29] and, more recently, since 1990, for linear convolutive mixtures [13], [15], [19], [24], [36], [37], [42]. Up to now, more realistic models, especially nonlinear models, have been sk...

Citation Context

...alled blind source separation. The source separation problem has been intensively studied over the last 12 years for linear instantaneous (memoryless) mixtures [2], [5], [12], [16], [18], [20], [22], =-=[25]-=-, [29] and, more recently, since 1990, for linear convolutive mixtures [13], [15], [19], [24], [36], [37], [42]. Up to now, more realistic models, especially nonlinear models, have been sketched by on...

Citation Context

...ssian, up to any diagonal matrix (scale factor) and up to any permutation matrix . In other words, separation is achieved when . This property is a direct consequence of the Darmois–Skitovich theorem =-=[10]-=-, [31]. B. Nonlinear Mixtures In this case, statistical independence is not a strong enough assumption to recover the sources without distortion. In fact, when and are two independent random variables...

Citation Context

...the joint pdf of and is then otherwise. (11) Relation (11) shows that the two random variables and are statistically independent. Other examples can be found in the literature (see for example Lukacs =-=[23]-=-) or can be easily constructed. C. Conclusion Source separation in the nonlinear case is, in general, impossible. The source independence assumption, which is sufficient to restore a copy of the sourc...

Citation Context

...h, although simple and attractive, requires a huge number of neurons for good accuracy and is restricted to sources having probability density functions (pdf’s) with bounded supports. Deco and Brauer =-=[11]-=- have also addressed the problem, considering a volume conservation condition on the nonlinear transforms. This constraint leads to very restrictive transforms and will not be considered in this paper...

Citation Context

...inear distortions. For instance, systems with memoryless nonlinearities can be encountered in sensor arrays [27], in digital satellite and microwave communications [30], and in some biological models =-=[21]-=-. B. Separability Contrary to general nonlinear mixtures, the PNL mixtures have a favorable separability property. In fact, using the separation structure shown in Fig. 2, it can be demonstrated, unde...

Citation Context

...y be simplified to (37) where is the th-order cumulant. This kind of expansion has already been used by Comon [8] and Gaeta et al. [14] in the linear case and by Taleb and Jutten [34] and Yang et al. =-=[39]-=- in the nonlinear source separation case. The main advantages of this approach are its simplicity and its low computational cost. Although the methods based on this expansion work very well for linear...

Citation Context

...ibution, and is the th Chebyshev–Hermite polynomial. These polynomials are orthogonal and are defined by (32) The coefficients can be computed using the orthogonality property of the polynomials (see =-=[32]-=-). For a zero mean random variable . . . . . . (33) and (34) Assuming the terms of orders greater than 4 can be neglected, i.e., the distribution is close to Gaussian, we can write (35) and deduce (36...

Citation Context

...ed over the last 12 years for linear instantaneous (memoryless) mixtures [2], [5], [12], [16], [18], [20], [22], [25], [29] and, more recently, since 1990, for linear convolutive mixtures [13], [15], =-=[19]-=-, [24], [36], [37], [42]. Up to now, more realistic models, especially nonlinear models, have been sketched by only a few authors. Burel [4] has proposed a neural-network based solution for the case o...

Citation Context

...r the last 12 years for linear instantaneous (memoryless) mixtures [2], [5], [12], [16], [18], [20], [22], [25], [29] and, more recently, since 1990, for linear convolutive mixtures [13], [15], [19], =-=[24]-=-, [36], [37], [42]. Up to now, more realistic models, especially nonlinear models, have been sketched by only a few authors. Burel [4] has proposed a neural-network based solution for the case of know...

Citation Context

...ally called blind source separation. The source separation problem has been intensively studied over the last 12 years for linear instantaneous (memoryless) mixtures [2], [5], [12], [16], [18], [20], =-=[22]-=-, [25], [29] and, more recently, since 1990, for linear convolutive mixtures [13], [15], [19], [24], [36], [37], [42]. Up to now, more realistic models, especially nonlinear models, have been sketched...

Citation Context

...pproximations of must be such that (60) which, in general, is fulfilled neither by a GC approximation nor by a linear projection on an ad hoc basis of nonlinear functions. C. Conclusion We have shown =-=[33]-=- that this method can be applied in the generic problem of entropy optimization. It was also successfully applied to the linear source separation problem and leads to excellent results concerning the ...

Citation Context

...t the components of its output become statistically independent. This is achieved if the joint density factorizes as the product of the marginal densities (15) As proposed by a few authors [8], [28], =-=[38]-=-, [41], statistical independence can be measured using the Kullback–Liebler (KL) divergence between and KL (16) which is equal to Shannon’s mutual information between the components of output vector ....

Citation Context

...for the linear instantaneous mixtures using a maximum likelihood approach that the optimal nonlinear functions for source separation algorithms are the score functions of the sources. Charkani et al. =-=[7]-=- extended these results to the convolutive mixtures. Unfortunately, these score functions are unknown and must be estimated (adaptively) from the output vector . A first idea is to estimate the pdf , ...

Citation Context

...ndent components, have a diagonal Jacobian. The answer is negative because we can construct, in an infinite number of manners, transforms with nondiagonal Jacobian conserving statistical independence =-=[9]-=-. Such mappings satisfy the following differential functional equation: (7) This result is well known for linear mixtures, where it is restricted to the Gaussian distribution. In that case, if is Gaus...

Citation Context

...The authors are with LIS-INPG, Grenoble, France. (e-mail: taleb@lisviallet.inpg.fr; chris@lis-viallet.inpg.fr). Publisher Item Identifier S 1053-587X(99)07662-X. on unknown parameters. Pajunen et al. =-=[26]-=- have addressed the problem using self-organizing maps. This approach, although simple and attractive, requires a huge number of neurons for good accuracy and is restricted to sources having probabili...

Citation Context

...less mixtures of sources), whereas sensors and their preamplifiers introduce memoryless nonlinear distortions. For instance, systems with memoryless nonlinearities can be encountered in sensor arrays =-=[27]-=-, in digital satellite and microwave communications [30], and in some biological models [21]. B. Separability Contrary to general nonlinear mixtures, the PNL mixtures have a favorable separability pro...

Citation Context

...2 years for linear instantaneous (memoryless) mixtures [2], [5], [12], [16], [18], [20], [22], [25], [29] and, more recently, since 1990, for linear convolutive mixtures [13], [15], [19], [24], [36], =-=[37]-=-, [42]. Up to now, more realistic models, especially nonlinear models, have been sketched by only a few authors. Burel [4] has proposed a neural-network based solution for the case of known nonlineari...

Citation Context

... studied over the last 12 years for linear instantaneous (memoryless) mixtures [2], [5], [12], [16], [18], [20], [22], [25], [29] and, more recently, since 1990, for linear convolutive mixtures [13], =-=[15]-=-, [19], [24], [36], [37], [42]. Up to now, more realistic models, especially nonlinear models, have been sketched by only a few authors. Burel [4] has proposed a neural-network based solution for the ...

Citation Context

... up to any diagonal matrix (scale factor) and up to any permutation matrix . In other words, separation is achieved when . This property is a direct consequence of the Darmois–Skitovich theorem [10], =-=[31]-=-. B. Nonlinear Mixtures In this case, statistical independence is not a strong enough assumption to recover the sources without distortion. In fact, when and are two independent random variables, (i.e...

Citation Context

.... A usual approximation of the Hessian is the covariance of the gradient of the regressor, the inverse of which can be iteratively calculated using the matrix inversion lemma. 3) Discussion: Recently =-=[35]-=-, a derivation of a batch quasinonparametric procedure for source separation in PNL mixtures pointed out an interesting fact about the choice of score functions. In fact, it leads to two sets of estim...

Citation Context

... In particular, we modeled the complex source separation problem (common in narrowband communications) as an entropy optimization problem and applied the same method for complex random variables (see =-=[3]-=-). VI. ALGORITHMS AND EXPERIMENTS A. Algorithms As described in the previous sections, the separation architecture consists of two major stages: a nonlinear stage and a linear stage. In addition, at t...

Citation Context

...so that the components of its output become statistically independent. This is achieved if the joint density factorizes as the product of the marginal densities (15) As proposed by a few authors [8], =-=[28]-=-, [38], [41], statistical independence can be measured using the Kullback–Liebler (KL) divergence between and KL (16) which is equal to Shannon’s mutual information between the components of output ve...

Citation Context

...a nonlinear stage (see Fig. 1). TALEB AND JUTTEN: SOURCE SEPARATION IN POST-NONLINEAR MIXTURES 2809 Fig. 2. Separation structure. A. The PNL Model This model, which was introduced by Taleb and Jutten =-=[34]-=-, provides the observation , which is the unknown nonlinear mixture of the unknown statistically independent source (12) where are unknown invertible derivable nonlinear functions, and ( ) denote the ...