Results 1  10
of
20
An empirical bayesian strategy for solving the simultaneous sparse approximation problem
 IEEE Trans. Sig. Proc
, 2007
"... Abstract—Given a large overcomplete dictionary of basis vectors, the goal is to simultaneously represent 1 signal vectors using coefficient expansions marked by a common sparsity profile. This generalizes the standard sparse representation problem to the case where multiple responses exist that were ..."
Abstract

Cited by 91 (16 self)
 Add to MetaCart
(Show Context)
Abstract—Given a large overcomplete dictionary of basis vectors, the goal is to simultaneously represent 1 signal vectors using coefficient expansions marked by a common sparsity profile. This generalizes the standard sparse representation problem to the case where multiple responses exist that were putatively generated by the same small subset of features. Ideally, the associated sparse generating weights should be recovered, which can have physical significance in many applications (e.g., source localization). The generic solution to this problem is intractable and, therefore, approximate procedures are sought. Based on the concept of automatic relevance determination, this paper uses an empirical Bayesian prior to estimate a convenient posterior distribution over candidate basis vectors. This particular approximation enforces a common sparsity profile and consistently places its prominent posterior mass on the appropriate region of weightspace necessary for simultaneous sparse recovery. The resultant algorithm is then compared with multiple response extensions of matching pursuit, basis pursuit, FOCUSS, and Jeffreys priorbased Bayesian methods, finding that it often outperforms the others. Additional motivation for this particular choice of cost function is also provided, including the analysis of global and local minima and a variational derivation that highlights the similarities and differences between the proposed algorithm and previous approaches. Index Terms—Automatic relevance determination, empirical Bayes, multiple response models, simultaneous sparse approximation, sparse Bayesian learning, variable selection. I.
A new view of automatic relevance determination
 In NIPS 20
, 2008
"... Automatic relevance determination (ARD) and the closelyrelated sparse Bayesian learning (SBL) framework are effective tools for pruning large numbers of irrelevant features leading to a sparse explanatory subset. However, popular update rules used for ARD are either difficult to extend to more gene ..."
Abstract

Cited by 70 (9 self)
 Add to MetaCart
(Show Context)
Automatic relevance determination (ARD) and the closelyrelated sparse Bayesian learning (SBL) framework are effective tools for pruning large numbers of irrelevant features leading to a sparse explanatory subset. However, popular update rules used for ARD are either difficult to extend to more general problems of interest or are characterized by nonideal convergence properties. Moreover, it remains unclear exactly how ARD relates to more traditional MAP estimationbased methods for learning sparse representations (e.g., the Lasso). This paper furnishes an alternative means of expressing the ARD cost function using auxiliary functions that naturally addresses both of these issues. First, the proposed reformulation of ARD can naturally be optimized by solving a series of reweighted ℓ1 problems. The result is an efficient, extensible algorithm that can be implemented using standard convex programming toolboxes and is guaranteed to converge to a local minimum (or saddle point). Secondly, the analysis reveals that ARD is exactly equivalent to performing standard MAP estimation in weight space using a particular feature and noisedependent, nonfactorial weight prior. We then demonstrate that this implicit prior maintains several desirable advantages over conventional priors with respect to feature selection. Overall these results suggest alternative cost functions and update procedures for selecting features and promoting sparse solutions in a variety of general situations. In particular, the methodology readily extends to handle problems such as nonnegative sparse coding and covariance component estimation. 1
A unified Bayesian framework for MEG/EEG source imaging
 Neuroimage
, 2009
"... The illposed nature of the MEG (or related EEG) source localization problem requires the incorporation of prior assumptions when choosing an appropriate solution out of an infinite set of candidates. Bayesian approaches are useful in this capacity because they allow these assumptions to be explicit ..."
Abstract

Cited by 46 (2 self)
 Add to MetaCart
(Show Context)
The illposed nature of the MEG (or related EEG) source localization problem requires the incorporation of prior assumptions when choosing an appropriate solution out of an infinite set of candidates. Bayesian approaches are useful in this capacity because they allow these assumptions to be explicitly quantified using postulated prior distributions. However, the means by which these priors are chosen, as well as the estimation and inference procedures that are subsequently adopted to affect localization, have led to a daunting array of algorithms with seemingly very different properties and assumptions. From the vantage point of a simple Gaussian scale mixture model with flexible covariance components, this paper analyzes and extends several broad categories of Bayesian inference directly applicable to source localization including empirical Bayesian approaches, standard MAP estimation, and multiple variational Bayesian (VB) approximations. Theoretical properties related to convergence, global and local minima, and localization bias are analyzed and fast algorithms are derived that improve upon existing methods. This perspective leads to explicit connections between many established algorithms and suggests natural extensions for handling unknown dipole orientations, extended source configurations, correlated sources, temporal smoothness, and computational expediency. Specific imaging methods elucidated under this paradigm include weighted minimum ℓ2norm, FOCUSS, MCE, VESTAL, sLORETA, ReML and covariance component estimation, beamforming, variational Bayes, the Laplace approximation, and automatic relevance determination (ARD). Perhaps surprisingly, all of these methods can be formulated as particular cases of covariance component estimation using different concave regularization terms and optimization rules, making general theoretical analyses and algorithmic extensions/improvements particularly relevant. I.
Latent variable bayesian models for promoting sparsity
 Transactions on Information Theory
, 2011
"... Abstract—Many practical methods for finding maximally sparse coefficient expansions involve solving a regression problem using a particular class of concave penalty functions. From a Bayesian perspective, this process is equivalent to maximum a posteriori (MAP) estimation using a sparsityinducing ..."
Abstract

Cited by 35 (15 self)
 Add to MetaCart
(Show Context)
Abstract—Many practical methods for finding maximally sparse coefficient expansions involve solving a regression problem using a particular class of concave penalty functions. From a Bayesian perspective, this process is equivalent to maximum a posteriori (MAP) estimation using a sparsityinducing prior distribution (Type I estimation). Using variational techniques, this distribution can always be conveniently expressed as a maximization over scaled Gaussian distributions modulated by a set of latent variables. Alternative Bayesian algorithms, which operate in latent variable space leveraging this variational representation, lead to sparse estimators reflecting posterior information beyond the mode (Type II estimation). Currently, it is unclear how the underlying cost functions of Type I and Type II relate, nor what relevant theoretical properties exist, especially with regard to Type II. Herein a common set of auxiliary functions is used to conveniently express both Type I and Type II cost functions in either coefficient or latent variable space facilitating direct comparisons. In coefficient space, the analysis reveals that Type II is exactly equivalent to performing standard MAP estimation using a particular class of dictionary and noisedependent, nonfactorial coefficient priors. One prior (at least) from this class maintains several desirable advantages over all possible Type I methods and utilizes a novel, nonconvex approximation to the norm with most, and in certain quantifiable conditions all, local minima smoothed away. Importantly, the global minimum is always left unaltered unlike standard norm relaxations. This ensures that any appropriate descent method is guaranteed to locate the maximally sparse solution. Index Terms—Bayesian learning, compressive sensing, latent variable models, source localization, sparse priors, sparse representations, underdetermined inverse problems. I.
Compressive MUSIC: revisiting the link between compressive sensing and array signal processing
 IEEE Trans. on Information Theory
, 2012
"... Abstract—The multiple measurement vector (MMV) problem addresses the identification of unknown input vectors that share common sparse support. Even though MMV problems have been traditionally addressed within the context of sensor array signal processing, the recent trend is to apply compressive sen ..."
Abstract

Cited by 23 (4 self)
 Add to MetaCart
(Show Context)
Abstract—The multiple measurement vector (MMV) problem addresses the identification of unknown input vectors that share common sparse support. Even though MMV problems have been traditionally addressed within the context of sensor array signal processing, the recent trend is to apply compressive sensing (CS) due to its capability to estimate sparse support even with an insufficient number of snapshots, in which case classical array signal processing fails. However, CS guarantees the accurate recovery in a probabilistic manner, which often shows inferior performance in the regime where the traditional array signal processing approaches succeed. The apparent dichotomy between the probabilistic CS and deterministic sensor array signal processing has not been fully understood. The main contribution of the present article is a unified approach that revisits the link between CS and array signal processing first unveiled in the mid 1990s by Feng and Bresler. The new algorithm, which we call compressive MUSIC, identifies the parts of support using CS, after which the remaining supports are estimated using a novel generalized MUSIC criterion. Using a large system MMV model, we show that our compressive MUSIC requires a smaller number of sensor elements for accurate support recovery than the existing CS methods and that it can approach the optimalbound with finite number of snapshots even in cases where the signals are linearly dependent. Index Terms—Compressive sensing, multiple measurement vector problem, joint sparsity, MUSIC, SOMP, thresholding. I.
Beamforming using the Relevance Vector Machine
"... Beamformers are spatial filters that pass source signals in particular focused locations while suppressing interference from elsewhere. The widelyused minimum variance adaptive beamformer (MVAB) creates such filters using a sample covariance estimate; however, the quality of this estimate deteriora ..."
Abstract

Cited by 12 (3 self)
 Add to MetaCart
(Show Context)
Beamformers are spatial filters that pass source signals in particular focused locations while suppressing interference from elsewhere. The widelyused minimum variance adaptive beamformer (MVAB) creates such filters using a sample covariance estimate; however, the quality of this estimate deteriorates when the sources are correlated or the number of samples n is small. Herein, a modified beamformer is derived that replaces this problematic sample covariance with a robust maximum likelihood estimate obtained using the relevance vector machine (RVM), a Bayesian method for learning sparse models from possibly overcomplete feature sets. We prove that this substitution has the natural ability to remove the undesirable effects of correlations or limited data. When n becomes large and assuming uncorrelated sources, this method reduces to the exact MVAB. Simulations using directionofarrival data support these conclusions. Additionally, RVMs can potentially enhance a variety of traditional signal processing methods that rely on robust sample covariance estimates. 1.
Compressive diffuse optical tomography: noniterative exact reconstruction using joint sparsity
 IEEE Trans. Med. Imag
, 2011
"... Abstract—Diffuse optical tomography (DOT) is a sensitive and relatively low cost imaging modality that reconstructs optical properties of a highly scattering medium. However, due to the diffusive nature of light propagation, the problem is severely illconditioned and highly nonlinear. Even though n ..."
Abstract

Cited by 10 (3 self)
 Add to MetaCart
(Show Context)
Abstract—Diffuse optical tomography (DOT) is a sensitive and relatively low cost imaging modality that reconstructs optical properties of a highly scattering medium. However, due to the diffusive nature of light propagation, the problem is severely illconditioned and highly nonlinear. Even though nonlinear iterative methods have been commonly used, they are computationally expensive especially for three dimensional imaging geometry. Recently, compressed sensing theory has provided a systematic understanding of high resolution reconstruction of sparse objects in many imaging problems; hence, the goal of this paper is to extend the theory to the diffuse optical tomography problem. The main contributions of this paper are to formulate the imaging problem as a joint sparse recovery problem in a compressive sensing framework and to propose a novel noniterative and exact inversion algorithm that achieves the 0 optimality as the rank of measurement increases to the unknown sparsity level. The algorithm is based on the recently discovered generalized MUSIC criterion, which exploits the advantages of both compressive sensing and array signal processing. A theoretical criterion for optimizing the imaging geometry is provided, and simulation results confirm that the new algorithm outperforms the existing algorithms and reliably reconstructs the optical inhomogeneities when we assume that the optical background is known to a reasonable accuracy. Index Terms—Diffuse optical tomography (DOT), generalized MUSIC criterion, joint sparsity, multiple measurement vector (MMV), pthresholding, simultaneous orthogonal matching pursuit (SOMP). I.
Compressive MUSIC: a missing link between compressive sensing and array signal processing
, 2011
"... ..."
(Show Context)
Performance limits of matching pursuit algorithms
 In Proc. IEEE Int. Symp. Inform. Th
, 2008
"... AbstractIn this paper, we examine the performance limits of the Orthogonal Matching Pursuit (OMP) algorithm, which has proven to be effective in solving for sparse solutions to inverse problem arising in overcomplete representations. To identify these limits, we exploit the connection between spar ..."
Abstract

Cited by 8 (1 self)
 Add to MetaCart
(Show Context)
AbstractIn this paper, we examine the performance limits of the Orthogonal Matching Pursuit (OMP) algorithm, which has proven to be effective in solving for sparse solutions to inverse problem arising in overcomplete representations. To identify these limits, we exploit the connection between sparse solution problem and multiple access channel (MAC) in wireless communication domain. The forward selective nature of OMP helps it to be recognized as a successive interference cancellation (SIC) scheme that decodes nonzero entries one at a time in a specific order. We leverage this SIC decoding order and utilize the criterion for successful decoding to develop the informationtheoretic performance limitation for OMP, which involves factors such as dictionary dimension, signaltonoiseratio, and importantly, the relative behavior of the nonzeros entries. Supported by computer simulations, our proposed criterion is demonstrated to be asymptotically effective in explaining the behavior of OMP.
Sparse Estimation with Structured Dictionaries
"... In the vast majority of recent work on sparse estimation algorithms, performance has been evaluated using ideal or quasiideal dictionaries (e.g., random Gaussian or Fourier) characterized by unit ℓ2 norm, incoherent columns or features. But in reality, these types of dictionaries represent only a s ..."
Abstract

Cited by 6 (2 self)
 Add to MetaCart
(Show Context)
In the vast majority of recent work on sparse estimation algorithms, performance has been evaluated using ideal or quasiideal dictionaries (e.g., random Gaussian or Fourier) characterized by unit ℓ2 norm, incoherent columns or features. But in reality, these types of dictionaries represent only a subset of the dictionaries that are actually used in practice (largely restricted to idealized compressive sensing applications). In contrast, herein sparse estimation is considered in the context of structured dictionaries possibly exhibiting high coherence between arbitrary groups of columns and/or rows. Sparse penalized regression models are analyzed with the purpose of finding, to the extent possible, regimes of dictionary invariant performance. In particular, a Type II Bayesian estimator with a dictionarydependent sparsity penalty is shown to have a number of desirable invariance properties leading to provable advantages over more conventional penalties such as the ℓ1 norm, especially in areas where existing theoretical recovery guarantees no longer hold. This can translate into improved performance in applications such as model selection with correlated features, source localization, and compressive sensing with constrained measurement directions.