## Sparse solutions to linear inverse problems with multiple measurement vectors (2005)

### Cached

### Download Links

- [dsp.rice.edu]
- [www.dsp.ece.rice.edu]
- [dsp.rice.edu]
- DBLP

### Other Repositories/Bibliography

Venue: | IEEE Trans. Signal Processing |

Citations: | 131 - 10 self |

### BibTeX

@ARTICLE{Cotter05sparsesolutions,

author = {Shane F. Cotter and Bhaskar D. Rao and Kjersti Engan and Kenneth Kreutz-delgado and Senior Member},

title = {Sparse solutions to linear inverse problems with multiple measurement vectors},

journal = {IEEE Trans. Signal Processing},

year = {2005},

pages = {2477--2488}

}

### Years of Citing Articles

### OpenURL

### Abstract

Abstract—We address the problem of finding sparse solutions to an underdetermined system of equations when there are multiple measurement vectors having the same, but unknown, sparsity structure. The single measurement sparse solution problem has been extensively studied in the past. Although known to be NP-hard, many single–measurement suboptimal algorithms have been formulated that have found utility in many different applications. Here, we consider in depth the extension of two classes of algorithms–Matching Pursuit (MP) and FOCal Underdetermined System Solver (FOCUSS)–to the multiple measurement case so that they may be used in applications such as neuromagnetic imaging, where multiple measurement vectors are available, and solutions with a common sparsity structure must be computed. Cost functions appropriate to the multiple measurement problem are developed, and algorithms are derived based on their minimization. A simulation study is conducted on a test-case dictionary to show how the utilization of more than one measurement vector improves the performance of the MP and FOCUSS classes of algorithm, and their performances are compared. I.

### Citations

1956 |
Matrix Computations
- Golub, VanLoan
- 1983
(Show Context)
Citation Context ...ut the residual matrix is computed as as opposed to . To obtain the new residual matrix, we need to first form the projection matrix . Once the column index has been selected, a Modified Gram–Schmidt =-=[50]-=- type of procedure is used on the vector . With the initialization , ,wehave , where The residual is updated via Equations (6), (8), and (9) define the M-OMP algorithm, and similar stopping rules to t... |

1652 | Atomic decomposition by basis pursuit
- Chen, Donoho, et al.
- 2001
(Show Context)
Citation Context ...d results [7], [8], [18]–[22]. Other approaches have also been suggested which are based on the use of optimization techniques to minimize diversity measures and, hence, promote sparsity. In [23] and =-=[24]-=-, the norm of the solution was used as the diversity measure, and consideration of the more general norm-like diversity measures led to the development of the FOCUSS (FOCal Underdetermined System Solv... |

1047 | Matching pursuits with time-frequency dictionaries
- Mallat, Zhang
- 1993
(Show Context)
Citation Context ...no). Digital Object Identifier 10.1109/TSP.2005.849172 A popular search technique for finding a sparse solution/representation is based on a suboptimal forward search through the dictionary [7], [8], =-=[16]-=-–[22]. These algorithms, termed Matching Pursuit (MP) [16], proceed by sequentially adding vectors to a set which will be used to represent the signal. Simple procedures were implemented initially [16... |

1040 |
Introduction to Linear and Nonlinear Programming
- Luenberger
- 1973
(Show Context)
Citation Context ...hosen. The choice of is dictated by the speed of convergence and the sparsity of the solution generated. Letting gives the norm solution. Values of give sparse solutions, but the order of convergence =-=[56]-=- is given by (2- ) [3], which implies that the algorithm converges more quickly for small values of . However, for small values of , it has been found to have a higher likelihood of getting trapped in... |

927 |
Emergence of simple-cell receptive field properties by learning a sparse code for natural images
- Olshausen, Field
(Show Context)
Citation Context ...ration [11], and stock market analysis [12]. It has also been argued that overcomplete representations and basis selection have a role in the coding of sensory information in biological systems [13], =-=[14]-=-. In all cases, the underlying linear inverse problem is the same and can be stated as follows: Represent a signal of interest using the minimum number of vectors from an overcomplete dictionary (set ... |

524 | Greed is good: Algorithmic results for sparse approximation - Tropp |

378 |
What Is the Goal of Sensory Coding
- Field
- 1994
(Show Context)
Citation Context ... restoration [11], and stock market analysis [12]. It has also been argued that overcomplete representations and basis selection have a role in the coding of sensory information in biological systems =-=[13]-=-, [14]. In all cases, the underlying linear inverse problem is the same and can be stated as follows: Represent a signal of interest using the minimum number of vectors from an overcomplete dictionary... |

365 | Elad.“Optimally sparse representation in general (nonorthogonal) dictionaries via 1 minimization - Donoho, Michael - 2003 |

359 | Uncertainty principles and ideal atomic decomposition
- Donoho, Huo
- 2001
(Show Context)
Citation Context ...ng the optimality of theCOTTER et al.: SPARSE SOLUTIONS TO LINEAR INVERSE PROBLEMS WITH MULTIPLE MEASUREMENT VECTORS 2479 obtained solution. Several results have been recently developed for the case =-=[43]-=-–[49]. The presence of multiple measurements will be shown to be helpful in this regard. In the case where the diversity of the solution is known to be bounded from above by the value , we look for an... |

350 | Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition - Pati, Rezaiifar, et al. - 1993 |

316 |
Sparse approximate solutions to linear systems
- Natarajan
- 1995
(Show Context)
Citation Context ...e algorithms have been applied to biomagnetic inverse problems [2], [3], bandlimited extrapolation and spectral estimation [4], [5], direction-of-arrival estimation [6], [3], functional approximation =-=[7]-=-, [8], channel equalization [9], echo cancellation [10], image restoration [11], and stock market analysis [12]. It has also been argued that overcomplete representations and basis selection have a ro... |

291 |
Adapted Wavelet Analysis from Theory to Software
- Wickerhauser
- 1994
(Show Context)
Citation Context ...nonsparse, we need to consider alternate functionals, referred to as diversity measures, which lead to sparse solutions when minimized. A popular diversity measure for vectors ( )is [11], [25], [31], =-=[51]-=-–[53], where Due to the close connection to norms, these measures are referred to as “ diversity measures” or “ -norm-like diversity measures.” The diversity measure for (or, equivalently, the numeros... |

216 | Sparse signal reconstruction from limited data using FOCUSS: a re-weighted norm minimization algorithm
- Gorodnitsky, Rao
- 1997
(Show Context)
Citation Context ...mall number of entries are nonzero) to linear inverse problems arises in a large number of application areas [1]. For instance, these algorithms have been applied to biomagnetic inverse problems [2], =-=[3]-=-, bandlimited extrapolation and spectral estimation [4], [5], direction-of-arrival estimation [6], [3], functional approximation [7], [8], channel equalization [9], echo cancellation [10], image resto... |

214 | Sparse representations in unions of bases - Gribonval, Nielsen |

179 |
Analysis of Discrete Ill-Posed Problems by Means of the L-Curve
- Hansen
- 1992
(Show Context)
Citation Context ...curve method described in [28] and [29] as a method of choosing the regularization parameter also performs well in this context. The modified -curve method is based on the -curve method introduced in =-=[54]-=- and [55] as a method for finding the parameter , and more details can be found in [28] and [29]. In the M-FOCUSS algorithm, the parameter has to be chosen. The choice of is dictated by the speed of c... |

174 | A generalized uncertainty principle and sparse representations of pairs of bases - Elad, Bruckstein |

157 | On sparse representations in arbitrary redundant bases - Fuchs |

123 | Basis Pursuit
- Chen, Donoho
- 1994
(Show Context)
Citation Context ...awbacks. First, the minimum 2-norm solutions which make up are based on a criterion that favors solutions with many small nonzero entries, which is contrary to the goal of sparsity/concentration [3], =-=[23]-=-. Second, the solution for each of the measurement vectors is computed independently so that a common sparsity structure is not enforced across the solutions. The first problem (i.e., the use of the 2... |

114 | Adaptive greedy approximations
- Davis, Mallat, et al.
- 1997
(Show Context)
Citation Context ... same and can be stated as follows: Represent a signal of interest using the minimum number of vectors from an overcomplete dictionary (set of vectors). This problem has been shown to be NP-hard [7], =-=[15]-=-. Much research effort has been invested in finding low complexity algorithms that yield solutions very close, in a chosen metric, to those obtained using an exhaustive search. Manuscript received Jun... |

110 |
The use of the L-curve in the regularization of discrete ill-posed problems
- Hansen, O’Leary
- 1993
(Show Context)
Citation Context ...hod described in [28] and [29] as a method of choosing the regularization parameter also performs well in this context. The modified -curve method is based on the -curve method introduced in [54] and =-=[55]-=- as a method for finding the parameter , and more details can be found in [28] and [29]. In the M-FOCUSS algorithm, the parameter has to be chosen. The choice of is dictated by the speed of convergenc... |

86 |
Introduction to Spectral Analysis, (Upper Saddle River
- Stoica, Moses
- 1997
(Show Context)
Citation Context ... sensors [3], [6]. Another important application of this formulation is in nonparametric spectrum analysis of time series where the data is often partitioned into segments for statistical reliability =-=[41]-=-. In this context, each segment corresponds to a measurement vector leading to the MMV problem. Recently, forward sequential search-based methods have been applied to the equalization of sparse channe... |

78 | An affine scaling methodology for best basis selection
- Rao, Kreutz-Delgado
- 1999
(Show Context)
Citation Context ...d as the diversity measure, and consideration of the more general norm-like diversity measures led to the development of the FOCUSS (FOCal Underdetermined System Solver) class of algorithms [2], [3], =-=[25]-=-–[27]. A robust version of the FOCUSS algorithm, called Regularized FOCUSS, handles noisy data and can also be used as an efficient representation for compression purposes [28], [29]. Yet another appr... |

75 | Adaptive time-frequency decompositions - Davis, Mallat, et al. - 1994 |

73 |
Neuromagnetic source imaging with focuss: a recursive weighted minimum norm algorithm, Electroencephalography and Clinical Neurophysiology 95
- Gorodnitsky, George, et al.
- 1995
(Show Context)
Citation Context ...ery small number of entries are nonzero) to linear inverse problems arises in a large number of application areas [1]. For instance, these algorithms have been applied to biomagnetic inverse problems =-=[2]-=-, [3], bandlimited extrapolation and spectral estimation [4], [5], direction-of-arrival estimation [6], [3], functional approximation [7], [8], channel equalization [9], echo cancellation [10], image ... |

59 | Sparse channel estimation via matching pursuit with application to equalization
- Cotter, Rao
- 2002
(Show Context)
Citation Context ...t vector leading to the MMV problem. Recently, forward sequential search-based methods have been applied to the equalization of sparse channels which are found in some communication environments [9], =-=[42]-=-. In this case, for a fast time-varying channel, oversampling at the receiver leads to the MMV problem. While these applications are meant to highlight the importance of the MMV problem, the framework... |

56 | Highly sparse representations from dictionaries are unique and independent of the sparseness measure
- Gribonval, Nielsen
- 2003
(Show Context)
Citation Context ...e optimality of theCOTTER et al.: SPARSE SOLUTIONS TO LINEAR INVERSE PROBLEMS WITH MULTIPLE MEASUREMENT VECTORS 2479 obtained solution. Several results have been recently developed for the case [43]–=-=[49]-=-. The presence of multiple measurements will be shown to be helpful in this regard. In the case where the diversity of the solution is known to be bounded from above by the value , we look for an exac... |

40 | Subset selection in noise based on diversity measure minimization
- Rao, Engan, et al.
- 2003
(Show Context)
Citation Context ...orithms [2], [3], [25]–[27]. A robust version of the FOCUSS algorithm, called Regularized FOCUSS, handles noisy data and can also be used as an efficient representation for compression purposes [28], =-=[29]-=-. Yet another approach was introduced in [30]–[32], where the search is based on a sequential backward elimination of elements from a complete or undercomplete (i.e., nonovercomplete) dictionary. This... |

36 |
Proportionate normalized least mean square adaptation in echo cancelers
- Duttweiler
- 2000
(Show Context)
Citation Context ...problems [2], [3], bandlimited extrapolation and spectral estimation [4], [5], direction-of-arrival estimation [6], [3], functional approximation [7], [8], channel equalization [9], echo cancellation =-=[10]-=-, image restoration [11], and stock market analysis [12]. It has also been argued that overcomplete representations and basis selection have a role in the coding of sensory information in biological s... |

30 |
Sparse decomposition of stereo signals with matching pursuit and application to blind separation of more than two sources from a stereo mixture
- Gribonval
- 2002
(Show Context)
Citation Context ...file. This work expands on some of the initial results presented in [35] and [36]. More recently, extensions of the matching pursuit framework to the MMV framework were also introduced and studied in =-=[37]-=-–[39]. It will be shown that we can greatly improve on our ability to provide sparse signal representations by utilizing MMV. As motivation for the study of this problem, we outline some applications ... |

30 |
MEG-based imaging of focal neuronal current sources
- Phillips, Leahy, et al.
- 1997
(Show Context)
Citation Context ...terest in solving the MMV problem was motivated by the need to solve the neuromagnetic inverse problem that arises in Magnetoencephalography (MEG), which is a modality for imaging the brain [2], [3], =-=[40]-=-. It is assumed that the MEG signal is the result of activity at a small number of possible activation regions in the brain. When several snapshots (measurement vectors) are obtained over a small time... |

29 | On the optimality of the Backward Greedy Algorithm for the subset selection problem - Couvreur, Bresler |

24 |
Reduced Complexity Decision Feedback Equalization for Multipath Channels with Large Delay Spreads
- Fevrier, Gelfand, et al.
- 1999
(Show Context)
Citation Context ...to biomagnetic inverse problems [2], [3], bandlimited extrapolation and spectral estimation [4], [5], direction-of-arrival estimation [6], [3], functional approximation [7], [8], channel equalization =-=[9]-=-, echo cancellation [10], image restoration [11], and stock market analysis [12]. It has also been argued that overcomplete representations and basis selection have a role in the coding of sensory inf... |

24 |
Forward sequential algorithms for best basis selection
- Cotter, Adler, et al.
- 1999
(Show Context)
Citation Context ...Digital Object Identifier 10.1109/TSP.2005.849172 A popular search technique for finding a sparse solution/representation is based on a suboptimal forward search through the dictionary [7], [8], [16]–=-=[22]-=-. These algorithms, termed Matching Pursuit (MP) [16], proceed by sequentially adding vectors to a set which will be used to represent the signal. Simple procedures were implemented initially [16], [1... |

22 |
Signal processing with the sparseness constraint
- Rao
- 1998
(Show Context)
Citation Context ...INTRODUCTION THE problem of computing sparse solutions (i.e., solutions where only a very small number of entries are nonzero) to linear inverse problems arises in a large number of application areas =-=[1]-=-. For instance, these algorithms have been applied to biomagnetic inverse problems [2], [3], bandlimited extrapolation and spectral estimation [4], [5], direction-of-arrival estimation [6], [3], funct... |

22 | Fast orthogonal least squares algorithm for efficient subset model selection
- Chen, Wigger
- 1995
(Show Context)
Citation Context ... a set which will be used to represent the signal. Simple procedures were implemented initially [16], [17], while more complex algorithms were developed later which yielded improved results [7], [8], =-=[18]-=-–[22]. Other approaches have also been suggested which are based on the use of optimization techniques to minimize diversity measures and, hence, promote sparsity. In [23] and [24], the norm of the so... |

21 |
Extrapolation and spectral estimation with iterative weighted norm modification
- Cabrera, Parks
- 1991
(Show Context)
Citation Context ...ms arises in a large number of application areas [1]. For instance, these algorithms have been applied to biomagnetic inverse problems [2], [3], bandlimited extrapolation and spectral estimation [4], =-=[5]-=-, direction-of-arrival estimation [6], [3], functional approximation [7], [8], channel equalization [9], echo cancellation [10], image restoration [11], and stock market analysis [12]. It has also bee... |

20 |
Remde, "A new model of LPC excitation for producing natural-sounding speech at low bit rates
- Atal, R
- 1982
(Show Context)
Citation Context ...2]. These algorithms, termed Matching Pursuit (MP) [16], proceed by sequentially adding vectors to a set which will be used to represent the signal. Simple procedures were implemented initially [16], =-=[17]-=-, while more complex algorithms were developed later which yielded improved results [7], [8], [18]–[22]. Other approaches have also been suggested which are based on the use of optimization techniques... |

17 | Simultaneous approximation by greedy algorithms
- Leviatan, Temlyakov
- 2003
(Show Context)
Citation Context ... This work expands on some of the initial results presented in [35] and [36]. More recently, extensions of the matching pursuit framework to the MMV framework were also introduced and studied in [37]–=-=[39]-=-. It will be shown that we can greatly improve on our ability to provide sparse signal representations by utilizing MMV. As motivation for the study of this problem, we outline some applications in wh... |

14 |
Sparse inverse solution methods for signal and image processing applications
- Jeffs
- 1998
(Show Context)
Citation Context ...ation areas [1]. For instance, these algorithms have been applied to biomagnetic inverse problems [2], [3], bandlimited extrapolation and spectral estimation [4], [5], direction-of-arrival estimation =-=[6]-=-, [3], functional approximation [7], [8], channel equalization [9], echo cancellation [10], image restoration [11], and stock market analysis [12]. It has also been argued that overcomplete representa... |

13 | Restoration of blurred star field images by maximally sparse optimization
- Jeffs, Gunsay
- 1993
(Show Context)
Citation Context ...imited extrapolation and spectral estimation [4], [5], direction-of-arrival estimation [6], [3], functional approximation [7], [8], channel equalization [9], echo cancellation [10], image restoration =-=[11]-=-, and stock market analysis [12]. It has also been argued that overcomplete representations and basis selection have a role in the coding of sensory information in biological systems [13], [14]. In al... |

13 |
The application of wave form dictionaries to stock market index data
- Ramsey, Zhang
- 1996
(Show Context)
Citation Context ...l estimation [4], [5], direction-of-arrival estimation [6], [3], functional approximation [7], [8], channel equalization [9], echo cancellation [10], image restoration [11], and stock market analysis =-=[12]-=-. It has also been argued that overcomplete representations and basis selection have a role in the coding of sensory information in biological systems [13], [14]. In all cases, the underlying linear i... |

13 |
On minimum entropy segmentation,” in Wavelets: Theory, Algorithms and Applications
- Donoho
- 1994
(Show Context)
Citation Context ...to the close connection to norms, these measures are referred to as “ diversity measures” or “ -norm-like diversity measures.” The diversity measure for (or, equivalently, the numerosity discussed in =-=[52]-=-) is of special interest because it is a direct measure of sparsity. It provides a count of the number of nonzero components in : Finding a global minimum to the numerosity measure requires an enumera... |

11 |
Amplitude optimization and pitch prediction in multipulse coders
- Singhal, Atal
- 1989
(Show Context)
Citation Context ...the solution matrix , a backsolve is necessary, as in the formulation [22]. C. MMV Order Recursive Matching Pursuit (M-ORMP) This method is an extension of the methodology developed in [7], [18], and =-=[21]-=-. In this method, the pursuit of the matching th basis vector conceptually involves solving ( ) order recursive least squares problems of the type , where we use the notation . The vector that reduces... |

10 |
Fast optimal and suboptimal algorithms for sparse solutions to linear inverse problems
- Harikumar, Couvreur, et al.
- 1998
(Show Context)
Citation Context ...s are nonsparse, we need to consider alternate functionals, referred to as diversity measures, which lead to sparse solutions when minimized. A popular diversity measure for vectors ( )is [11], [25], =-=[31]-=-, [51]–[53], where Due to the close connection to norms, these measures are referred to as “ diversity measures” or “ -norm-like diversity measures.” The diversity measure for (or, equivalently, the n... |

9 |
An efficient implementation of the backward greedy algorithm for sparse signal reconstruction
- Reeves
- 1999
(Show Context)
Citation Context ...he FOCUSS algorithm, called Regularized FOCUSS, handles noisy data and can also be used as an efficient representation for compression purposes [28], [29]. Yet another approach was introduced in [30]–=-=[32]-=-, where the search is based on a sequential backward elimination of elements from a complete or undercomplete (i.e., nonovercomplete) dictionary. This has recently been extended in [33] and [34] to th... |

9 |
Measures and algorithms for best basis selection
- Kreutz-Delgado, Rao
- 1998
(Show Context)
Citation Context ...arse, we need to consider alternate functionals, referred to as diversity measures, which lead to sparse solutions when minimized. A popular diversity measure for vectors ( )is [11], [25], [31], [51]–=-=[53]-=-, where Due to the close connection to norms, these measures are referred to as “ diversity measures” or “ -norm-like diversity measures.” The diversity measure for (or, equivalently, the numerosity d... |

8 | Improvement of discrete band-limited signal extrapolation by iterative subspace modification - Lee, Sullivan, et al. - 1987 |

8 |
Frame based signal representation and compression
- Engan
- 2001
(Show Context)
Citation Context ...of algorithms [2], [3], [25]–[27]. A robust version of the FOCUSS algorithm, called Regularized FOCUSS, handles noisy data and can also be used as an efficient representation for compression purposes =-=[28]-=-, [29]. Yet another approach was introduced in [30]–[32], where the search is based on a sequential backward elimination of elements from a complete or undercomplete (i.e., nonovercomplete) dictionary... |

8 |
Basis selection in the presence of noise
- Rao, Kreutz-Delgado
- 1998
(Show Context)
Citation Context ...on of sparse solutions when there are multiple measurement vectors (MMV) and the solutions are assumed to have a common sparsity profile. This work expands on some of the initial results presented in =-=[35]-=- and [36]. More recently, extensions of the matching pursuit framework to the MMV framework were also introduced and studied in [37]–[39]. It will be shown that we can greatly improve on our ability t... |

7 | Efficient computational schemes for the orthogonal least squares algorithm - Chng, Chen, et al. - 1995 |