Results 11  20
of
119
Proximal Newtontype methods for convex optimization
"... We seek to solve convex optimization problems in composite form: minimize x∈R n f(x): = g(x) + h(x), where g is convex and continuously differentiable and h: R n → R is a convex but not necessarily differentiable function whose proximal mapping can be evaluated efficiently. We derive a generalizatio ..."
Abstract

Cited by 15 (0 self)
 Add to MetaCart
(Show Context)
We seek to solve convex optimization problems in composite form: minimize x∈R n f(x): = g(x) + h(x), where g is convex and continuously differentiable and h: R n → R is a convex but not necessarily differentiable function whose proximal mapping can be evaluated efficiently. We derive a generalization of Newtontype methods to handle such convex but nonsmooth objective functions. We prove such methods are globally convergent and achieve superlinear rates of convergence in the vicinity of an optimal solution. We also demonstrate the performance of these methods using problems of relevance in machine learning and statistics. 1
Efficient first order methods for linear composite regularizers
, 2011
"... A wide class of regularization problems in machine learning and statistics employ a regularization term which is obtained by composing a simple convex function ω with a linear transformation. This setting includes Group Lasso methods, the Fused Lasso and other total variation methods, multitask l ..."
Abstract

Cited by 14 (5 self)
 Add to MetaCart
(Show Context)
A wide class of regularization problems in machine learning and statistics employ a regularization term which is obtained by composing a simple convex function ω with a linear transformation. This setting includes Group Lasso methods, the Fused Lasso and other total variation methods, multitask learning methods and many more. In this paper, we present a general approach for computing the proximity operator of this class of regularizers, under the assumption that the proximity operator of the function ω is known in advance. Our approach builds on a recent line of research on optimal first order optimization methods and uses fixed point iterations for numerically computing the proximity operator. It is more general than current approaches and, as we show with numerical simulations, computationally more efficient
Lasso screening rules via dual polytope projection,” arXiv:1211.3966
, 2012
"... Lasso is a widely used regression technique to find sparse representations. When the dimension of the feature space and the number of samples are extremely large, solving the Lasso problem remains challenging. To improve the efficiency of solving largescale Lasso problems, El Ghaoui and his collea ..."
Abstract

Cited by 14 (5 self)
 Add to MetaCart
Lasso is a widely used regression technique to find sparse representations. When the dimension of the feature space and the number of samples are extremely large, solving the Lasso problem remains challenging. To improve the efficiency of solving largescale Lasso problems, El Ghaoui and his colleagues have proposed the SAFE rules which are able to quickly identify the inactive predictors, i.e., predictors that have 0 components in the solution vector. Then, the inactive predictors or features can be removed from the optimization problem to reduce its scale. By transforming the standard Lasso to its dual form, it can be shown that the inactive predictors include the set of inactive constraints on the optimal dual solution. In this paper, we propose an efficient and effective screening rule via Dual Polytope Projections (DPP), which is mainly based on the uniqueness and nonexpansiveness of the optimal dual solution due to the fact that the feasible set in the dual space is a convex and closed polytope. Moreover, we show that our screening rule can be extended to identify inactive groups in group Lasso. To the best of our knowledge, there is currently no exact screening rule for group Lasso. We have evaluated our screening rule using synthetic and real data sets. Results show that our rule is more effective in identifying inactive predictors than existing stateoftheart screening rules for Lasso. 1
MotionAdaptive SpatioTemporal Regularization (MASTeR) for accelerated dynamic
, 2012
"... Accelerated MRI techniques reduce signal acquisition time by undersampling kspace. A fundamental problem in accelerated MRI is the recovery of quality images from undersampled kspace data. Current stateoftheart recovery algorithms exploit the spatial and temporal structures in underlying image ..."
Abstract

Cited by 13 (2 self)
 Add to MetaCart
(Show Context)
Accelerated MRI techniques reduce signal acquisition time by undersampling kspace. A fundamental problem in accelerated MRI is the recovery of quality images from undersampled kspace data. Current stateoftheart recovery algorithms exploit the spatial and temporal structures in underlying images to improve the reconstruction quality. In recent years, compressed sensing theory has helped formulate mathematical principles and conditions that ensure recovery of (structured) sparse signals from undersampled, incoherent measurements. In this paper, a new recovery algorithm, motionadaptive spatiotemporal regularization (MASTeR), is presented. MASTeR, which uses compressed sensing principles to recover dynamic MR images from highly undersampled kspace data, takes advantage of spatial and temporal structured sparsity in MR images. In contrast to existing algorithms, MASTeR models temporal sparsity using motionadaptive linear transformations between neighboring images. The efficiency of MASTeR is demonstrated with experiments on cardiac MRI for a range of reduction factors. Results are also compared with kt FOCUSS with motion estimation and compensation—another recently proposed recovery algorithm for dynamic MRI.
Constrained Overcomplete Analysis Operator Learning for Cosparse Signal Modelling
"... We consider the problem of learning a lowdimensional signal model from a collection of training samples. The mainstream approach would be to learn an overcomplete dictionary to provide good approximations of thetraining samples using sparsesynthesis coefficients. This famous sparse model has a less ..."
Abstract

Cited by 12 (1 self)
 Add to MetaCart
(Show Context)
We consider the problem of learning a lowdimensional signal model from a collection of training samples. The mainstream approach would be to learn an overcomplete dictionary to provide good approximations of thetraining samples using sparsesynthesis coefficients. This famous sparse model has a less well known counterpart, in analysis form, called the cosparse analysis model. In this new model, signals are characterised by their parsimony in a transformed domain using an overcomplete (linear) analysis operator. We propose to learn an analysis operator from a training corpus using a constrained optimisation framework based on L1 optimisation. The reason for introducing a constraint in the optimisation framework is to exclude trivial solutions. Although there is no final answer here for which constraint is the most relevant constraint, we investigate some conventional constraints in the model adaptation field and use the uniformly normalised tight frame (UNTF) for this purpose. We then derive a practical learning algorithm, based on projected subgradients and DouglasRachford splitting technique, and demonstrate its ability to robustly recover a ground truth analysis operator, when provided with a clean training set, of sufficient size. We also find an analysis operator for images, using some noisy cosparse signals, which is indeed a more realistic experiment. As the derived optimisation problem is not a convex program, we often find a local minimum using such variational methods. For two different settings, we provide preliminary theoretical support for the wellposedness of the learning problem, which can be practically used to test the local identifiability conditions of learnt operators.
A proximalgradient homotopy method for the sparse leastsquares problem
 SIAM Journal on Optimization
"... ar ..."
A generalized leastsquare matrix decomposition
 Journal of the American Statistical Association
"... Variables in many massive highdimensional data sets are structured, arising for example from measurements on a regular grid as in imaging and time series or from spatialtemporal measurements as in climate studies. Classical multivariate techniques ignore these structural relationships often result ..."
Abstract

Cited by 12 (5 self)
 Add to MetaCart
Variables in many massive highdimensional data sets are structured, arising for example from measurements on a regular grid as in imaging and time series or from spatialtemporal measurements as in climate studies. Classical multivariate techniques ignore these structural relationships often resulting in poor performance. We propose a generalization of the singular value decomposition (SVD) and principal components analysis (PCA) that is appropriate for massive data sets with structured variables or known twoway dependencies. By finding the best low rank approximation of the data with respect to a transposable quadratic norm, our decomposition, entitled the Generalized least squares Matrix Decomposition (GMD), directly accounts for structural relationships. As many variables in highdimensional settings are often irrelevant or noisy, we also regularize our matrix decomposition by adding twoway penalties to encourage sparsity or smoothness. We develop fast computational algorithms using our methods to perform generalized PCA (GPCA), sparse GPCA, and functional GPCA on massive data sets. Through simulations and a whole brain functional MRI example we demonstrate the utility of our methodology for dimension reduction, signal recovery, and feature selection with highdimensional structured data.
TARGET ESTIMATION IN COLOCATED MIMO RADAR VIA MATRIX COMPLETION
"... We consider a colocated MIMO radar scenario, in which the receive antennas forward their measurements to a fusion center. Based on the received data, the fusion center formulates a matrix which is then used for target parameter estimation. When the receive antennas sample the target returns at Nyqui ..."
Abstract

Cited by 11 (11 self)
 Add to MetaCart
(Show Context)
We consider a colocated MIMO radar scenario, in which the receive antennas forward their measurements to a fusion center. Based on the received data, the fusion center formulates a matrix which is then used for target parameter estimation. When the receive antennas sample the target returns at Nyquist rate, and assuming that there are more receive antennas than targets, the data matrix at the fusion center is lowrank. When each receive antenna sends to the fusion center only a small number of samples, along with the sample index, the receive data matrix has missing elements, corresponding to the samples that were not forwarded. Under certain conditions, matrix completion techniques can be applied to recover the full receive data matrix, which can then be used in conjunction with array processing techniques, e.g., MUSIC, to obtain target information. Numerical results indicate that good target recovery can be achieved with occupancy of the receive data matrix as low as 50%. Index Terms — Array processing, compressed sensing, matrix completion, MIMO radar, MUSIC
An EmpiricalBayes Approach to Recovering Linearly Constrained NonNegative Sparse Signals
"... Abstract—We consider the recovery of an (approximately) sparse signal from noisy linear measurements, in the case that the signal is apriori known to be nonnegative and obeys certain linear equality constraints. For this, we propose a novel empiricalBayes approach that combines the Generalized App ..."
Abstract

Cited by 9 (6 self)
 Add to MetaCart
Abstract—We consider the recovery of an (approximately) sparse signal from noisy linear measurements, in the case that the signal is apriori known to be nonnegative and obeys certain linear equality constraints. For this, we propose a novel empiricalBayes approach that combines the Generalized Approximate Message Passing (GAMP) algorithm with the expectation maximization (EM) algorithm. To enforce both sparsity and nonnegativity, we employ an i.i.d Bernoulli nonnegative Gaussian mixture (NNGM) prior and perform approximate minimum meansquared error (MMSE) recovery of the signal using sumproduct GAMP. To learn the NNGM parameters, we use the EM algorithm with a suitable initialization. Meanwhile, the linear equality constraints are enforced by augmenting GAMP’s linear observation model with noiseless pseudomeasurements. Numerical experiments demonstrate the stateofthe art meansquarederror and runtime of our approach. 1 I.