Citation Context

...each other, and thus the signals can be recovered from linear mixtures xi by finding a transformation in which the transformed signals are as independent as possible, as in ICA. In feature extraction =-=[4, 25]-=-, si is the coefficient of the i-th feature in the observed data vector x. The use of ICA for feature extraction is motivated by results in neurosciences that suggest that the similar principle of red...

Citation Context

... 4. Each window in this Figure corresponds to one of the columns ai of the mixing matrix A. Thus an observed image window is a superposition of these windows as in (5), with independent coefficients (=-=Bell and Sejnowski, 1997-=-; Olshausen and Field, 1996). Now, suppose a noisy image model holds: z = x+n (49) where n is uncorrelated noise, with elements indexed in the image window in the same way as x, and z is the measured ...

Citation Context

...sformation that maximizes the non-Gaussianity1 of the marginal responses, the result is a basis set of bandpass oriented filters of different sizes spanning roughly an octave in bandwidth, e.g., [8], =-=[9]-=-. Due to the combination of these qualitative properties, as well as an elegant mathematical framework, multiscale oriented subband decompositions have emerged as the representations of choice for man...

Citation Context

...ng techniques. Here, we tested the result of using additional sparseness constraints. Figure 5 shows the basis vectors obtained by putting a sparseness constraint on the coefficients (Sh = 0.85) but leaving the sparseness of the basis vectors unconstrained. In this case, NMF learns oriented features that represent edges and lines. Such oriented features are widely regarded as the best type of low-level features for representing natural images, and similar features are also used by the early visual system of the biological brain (Field, 1987; Simoncelli et al., 1992; Olshausen and Field, 1996; Bell and Sejnowski, 1997). This example illustrates that sparseness constrained NMF does not simply ‘sparsify’ the result of standard, unconstrained NMF, but rather can find qualitatively different parts-based representations that are more compatible with the sparseness assumptions. 4.3 Convergence of Algorithm Implementing the Projection Step To verify the performance of our projection method we performed extensive tests, varying the number of dimensions, the desired degree of sparseness, and the sparseness of the original vector. The desired and the initial degrees of sparseness were set to 0.1, 0.3, 0.5, 0.7, and 0...

Citation Context

...hoosing bases to optimize informationtheoretic criterion, which suggest that a basis of localized oriented operators at multiple scales is optimal for image representation (Olshausen and Field, 1996; =-=Bell and Sejnowski, 1997-=-). Many authors have used sets of multi-scale bandpass filters for texture synthesis (Cano and Minh, 1988; Porat and Zeevi, 1989; Popat and Picard, 1993; Heeger and Bergen, 1995; Portilla et al., 1996...

Citation Context

...e distribution of the strengths with which each elementary signal is present in a set of images is not a Gaussian). ICA on natural 1simages again produces receptive fields like those of simple cells (=-=Bell & Sejnowski 1997-=-a,b; Hurri et al. 1996, Hurri 1997). Although the components produced by ICA on natural images are not completely independent, they are as independent as possible by a linear transformation. It should...

Citation Context

...uces statistically independent outputs, U. The ICA unmixing matrix W was found using an unsupervised learning algorithm derived from the principle of optimal information transfer between neurons [9], =-=[10]-=-. The algorithm maximizes the mutual information between the input and the output of a nonlinear transfer function g. A discussion of how information maximization leads to independent outputs can be f...