Results 1  10
of
16
Sparse signal reconstruction from limited data using FOCUSS: A reweighted minimum norm algorithm
 IEEE Trans. Signal Processing
, 1997
"... Abstract—We present a nonparametric algorithm for finding localized energy solutions from limited data. The problem we address is underdetermined, and no prior knowledge of the shape of the region on which the solution is nonzero is assumed. Termed the FOcal Underdetermined System Solver (FOCUSS), t ..."
Abstract

Cited by 218 (12 self)
 Add to MetaCart
Abstract—We present a nonparametric algorithm for finding localized energy solutions from limited data. The problem we address is underdetermined, and no prior knowledge of the shape of the region on which the solution is nonzero is assumed. Termed the FOcal Underdetermined System Solver (FOCUSS), the algorithm has two integral parts: a lowresolution initial estimate of the real signal and the iteration process that refines the initial estimate to the final localized energy solution. The iterations are based on weighted norm minimization of the dependent variable with the weights being a function of the preceding iterative solutions. The algorithm is presented as a general estimation tool usable across different applications. A detailed analysis laying the theoretical foundation for the algorithm is given and includes proofs of global and local convergence and a derivation of the rate of convergence. A view of the algorithm as a novel optimization method which combines desirable characteristics of both classical optimization and learningbased algorithms is provided. Mathematical results on conditions for uniqueness of sparse solutions are also given. Applications of the algorithm are illustrated on problems in directionofarrival (DOA) estimation and neuromagnetic imaging. I.
An affine scaling methodology for best basis selection
 IEEE Trans. Signal Processing
, 1999
"... Abstract — A methodology is developed to derive algorithms for optimal basis selection by minimizing diversity measures proposed by Wickerhauser and Donoho. These measures include the pnormlike (`(p 1)) diversity measures and the Gaussian and Shannon entropies. The algorithm development methodolog ..."
Abstract

Cited by 79 (11 self)
 Add to MetaCart
Abstract — A methodology is developed to derive algorithms for optimal basis selection by minimizing diversity measures proposed by Wickerhauser and Donoho. These measures include the pnormlike (`(p 1)) diversity measures and the Gaussian and Shannon entropies. The algorithm development methodology uses a factored representation for the gradient and involves successive relaxation of the Lagrangian necessary condition. This yields algorithms that are intimately related to the Affine Scaling Transformation (AST) based methods commonly employed by the interior point approach to nonlinear optimization. The algorithms minimizing the `(p 1) diversity measures are equivalent to a recently developed class of algorithms called FOCal Underdetermined System Solver (FOCUSS). The general nature of the methodology provides a systematic approach for deriving this class of algorithms and a natural mechanism for extending them. It also facilitates a better understanding of the convergence behavior and a strengthening of the convergence results. The Gaussian entropy minimization algorithm is shown to be equivalent to a wellbehaved p =0normlike optimization algorithm. Computer experiments demonstrate that the pnormlike and the Gaussian entropy algorithms perform well, converging to sparse solutions. The Shannon entropy algorithm produces solutions that are concentrated but are shown to not converge to a fully sparse solution. I.
Enhancing Sparsity by Reweighted ℓ1 Minimization
, 2007
"... It is now well understood that (1) it is possible to reconstruct sparse signals exactly from what appear to be highly incomplete sets of linear measurements and (2) that this can be done by constrained ℓ1 minimization. In this paper, we study a novel method for sparse signal recovery that in many si ..."
Abstract

Cited by 76 (5 self)
 Add to MetaCart
It is now well understood that (1) it is possible to reconstruct sparse signals exactly from what appear to be highly incomplete sets of linear measurements and (2) that this can be done by constrained ℓ1 minimization. In this paper, we study a novel method for sparse signal recovery that in many situations outperforms ℓ1 minimization in the sense that substantially fewer measurements are needed for exact recovery. The algorithm consists of solving a sequence of weighted ℓ1minimization problems where the weights used for the next iteration are computed from the value of the current solution. We present a series of experiments demonstrating the remarkable performance and broad applicability of this algorithm in the areas of sparse signal recovery, statistical estimation, error correction and image processing. Interestingly, superior gains are also achieved when our method is applied to recover signals with assumed nearsparsity in overcomplete representations—not by reweighting the ℓ1 norm of the coefficient sequence as is common, but by reweighting the ℓ1 norm of the transformed object. An immediate consequence is the possibility of highly efficient data acquisition protocols by improving on a technique known as compressed sensing.
Theoretical results on sparse representations of multiplemeasurement vectors
 IEEE Trans. Signal Process
, 2006
"... Abstract — Multiple measurement vector (MMV) is a relatively new problem in sparse representations. Efficient methods have been proposed. Considering many theoretical results that are available in a simple case – single measure vector (SMV) – the theoretical analysis regarding MMV is lacking. In th ..."
Abstract

Cited by 69 (2 self)
 Add to MetaCart
Abstract — Multiple measurement vector (MMV) is a relatively new problem in sparse representations. Efficient methods have been proposed. Considering many theoretical results that are available in a simple case – single measure vector (SMV) – the theoretical analysis regarding MMV is lacking. In this paper, some known results of SMV are generalized to MMV. Some of these new results take advantages of additional information in the formulation of MMV. We consider the uniqueness under both an ℓ0norm like criterion and an ℓ1norm like criterion. The consequent equivalence between the ℓ0norm approach and the ℓ1norm approach indicates a computationally efficient way of finding the sparsest representation in an overcomplete dictionary. For greedy algorithms, it is proven that under certain conditions, orthogonal matching pursuit (OMP) can find the sparsest representation of an MMV with computational efficiency, just like in SMV. Simulations show that the predictions made by the proved theorems tend to be very conservative; this is consistent with some recent theoretical advances in probability. The connections will be discussed.
Enhacing sparsity by reweighted ℓ1 minimization
 Journal of Fourier Analysis and Applications
, 2008
"... It is now well understood that (1) it is possible to reconstruct sparse signals exactly from what appear to be highly incomplete sets of linear measurements and (2) that this can be done by constrained ℓ1 minimization. In this paper, we study a novel method for sparse signal recovery that in many si ..."
Abstract

Cited by 34 (1 self)
 Add to MetaCart
It is now well understood that (1) it is possible to reconstruct sparse signals exactly from what appear to be highly incomplete sets of linear measurements and (2) that this can be done by constrained ℓ1 minimization. In this paper, we study a novel method for sparse signal recovery that in many situations outperforms ℓ1 minimization in the sense that substantially fewer measurements are needed for exact recovery. The algorithm consists of solving a sequence of weighted ℓ1minimization problems where the weights used for the next iteration are computed from the value of the current solution. We present a series of experiments demonstrating the remarkable performance and broad applicability of this algorithm in the areas of sparse signal recovery, statistical estimation, error correction and image processing. Interestingly, superior gains are also achieved when our method is applied to recover signals with assumed nearsparsity in overcomplete representations—not by reweighting the ℓ1 norm of the coefficient sequence as is common, but by reweighting the ℓ1 norm of the transformed object. An immediate consequence is the possibility of highly efficient data acquisition protocols by improving on a technique known as compressed sensing.
Lowauthority controller design via convex optimization
 AIAA Journal of Guidance, Control, and Dynamics
, 1999
"... In this paper we address the problem of lowauthority controller (LAC) design. The premise is that the actuators have limited authority, and hence cannot significantly shift the eigenvalues of the system. As a result, the closedloop eigenvalues can be well approximated analytically using perturbati ..."
Abstract

Cited by 30 (14 self)
 Add to MetaCart
In this paper we address the problem of lowauthority controller (LAC) design. The premise is that the actuators have limited authority, and hence cannot significantly shift the eigenvalues of the system. As a result, the closedloop eigenvalues can be well approximated analytically using perturbation theory. These analytical approximations may suffice to predict the behavior of the closedloop system in practical cases, and will provide at least a very strong rationale for the first step in the design iteration loop. We will show that LAC design can be cast as convex optimization problems that can be solved efficiently in practice using interiorpoint methods. Also, we will show that by optimizing the ℓ1 norm of the feedback gains, we can arrive at sparse designs, i.e., designs in which only a small number of the control gains are nonzero. Thus, in effect, we can also solve actuator/sensor placement or controller architecture design problems. Keywords: Lowauthority control, actuator/sensor placement, linear operator perturbation theory, convex optimization, secondorder cone programming, semidefinite programming, linear matrix inequality. 1
On The Optimallity Of The Backward Greedy Algorithm For The Subset Selection Problem
, 1998
"... The following linear inverse problem is considered: given a full column rank m \Theta n data matrix A and a length m observation vector b, find the best least squares solution to Ax = b with at most r ! n nonzero components. The backward greedy algorithm computes a sparse solution to Ax = b by remo ..."
Abstract

Cited by 29 (0 self)
 Add to MetaCart
The following linear inverse problem is considered: given a full column rank m \Theta n data matrix A and a length m observation vector b, find the best least squares solution to Ax = b with at most r ! n nonzero components. The backward greedy algorithm computes a sparse solution to Ax = b by removing greedily columns from A until r columns are left. A simple implementation based on a QR downdating scheme by Givens rotations is described. The backward greedy algorithm is shown to be optimal for this problem in the sense that it selects the "correct" subset of columns from A if the perturbation of the data vector b is small enough.
On the application of the global matched filter to DOA estimation with uniform circular arrays
 IEEE Trans. Signal Process
, 2001
"... Abstract—The problem of estimating the direction of arrivals (DOA) of narrowband sources impinging on a uniform circular array is considered. We present a method that uses as input the values of a small number of uniformly spaced beams and apply a modelfitting approach taking into account the stati ..."
Abstract

Cited by 24 (14 self)
 Add to MetaCart
Abstract—The problem of estimating the direction of arrivals (DOA) of narrowband sources impinging on a uniform circular array is considered. We present a method that uses as input the values of a small number of uniformly spaced beams and apply a modelfitting approach taking into account the statistical properties of the beams. The approach, which is called the “global matched filter, ” fits simultaneously to the observations all the elements needed to explain them. It chooses, among all the representations satisfying a constraint with a sensible physical interpretation, the one with minimal energy. The method drastically improves upon the conventional beamformer and has performance comparable with the best highresolution (HR) techniques. It further applies when the number of sources exceeds the number of sensors: a situation that cannot be handled by standard HR techniques. Index Terms—Directionof arrival estimation, matched filters, redundant parametrizations, regularized estimation, sparse signal representation. I.
A general approach to sparse basis selection: Majorization, concavity, and affine scaling
 IN PROCEEDINGS OF THE TWELFTH ANNUAL CONFERENCE ON COMPUTATIONAL LEARNING THEORY
, 1997
"... Measures for sparse best–basis selection are analyzed and shown to fit into a general framework based on majorization, Schurconcavity, and concavity. This framework facilitates the analysis of algorithm performance and clarifies the relationships between existing proposed concentration measures use ..."
Abstract

Cited by 6 (3 self)
 Add to MetaCart
Measures for sparse best–basis selection are analyzed and shown to fit into a general framework based on majorization, Schurconcavity, and concavity. This framework facilitates the analysis of algorithm performance and clarifies the relationships between existing proposed concentration measures useful for sparse basis selection. It also allows one to define new concentration measures, and several general classes of measures are proposed and analyzed in this paper. Admissible measures are given by the Schurconcave functions, which are the class of functions consistent with the socalled Lorentz ordering (a partial ordering on vectors also known as majorization). In particular, concave functions form an important subclass of the Schurconcave functions which attain their minima at sparse solutions to the best basis selection problem. A general affine scaling optimization algorithm obtained from a special factorization of the gradient function is developed and proved to converge to a sparse solution for measures chosen from within this subclass.
Theoretical results about finding the sparsest representations of multiple measurement vectors (MMV) in an overcomplete dictionary using ℓ1 minimization and greedy algorithms
, 2004
"... Multiple Measurement Vectors (MMV) is a newly emerged problem in sparse overcomplete representation. Efficient methods have been designed. Considering many theoretical results that are available in a simple case— Single Measure Vector (SMV)—the theoretical analysis regarding MMV is lacking. In this ..."
Abstract

Cited by 5 (3 self)
 Add to MetaCart
Multiple Measurement Vectors (MMV) is a newly emerged problem in sparse overcomplete representation. Efficient methods have been designed. Considering many theoretical results that are available in a simple case— Single Measure Vector (SMV)—the theoretical analysis regarding MMV is lacking. In this paper, some known results of SMV are generalized to MMVs; new results particularly for MMV are also derived. Our theoretical results show under what conditions a sparse representation is unique. Moreover, the equivalence of the solutions to both ‘minimizing the ℓ0 norm ’ problems and ‘minimizing the ℓ1 norm ’ problems indicates a computationally efficient approach of finding the sparsest representations. It is proved that under certain conditions, Orthogonal Matching Pursuit (OMP) – which is a greedy algorithm – and a modified version of OMP can find the sparsest representations for MMVs as well. Interestingly, simulations show that the predictions made by the above theorems tend to be conservative; this is consistent with some recent advances, which will be discussed.