## On the LASSO and Its Dual (1999)

Venue: | Journal of Computational and Graphical Statistics |

Citations: | 145 - 2 self |

### BibTeX

@ARTICLE{Osborne99onthe,

author = {Michael R. Osborne and Brett Presnell and Berwin A. Turlach},

title = {On the LASSO and Its Dual},

journal = {Journal of Computational and Graphical Statistics},

year = {1999},

volume = {9},

pages = {319--337}

}

### Years of Citing Articles

### OpenURL

### Abstract

Proposed by Tibshirani (1996), the LASSO (least absolute shrinkage and selection operator) estimates a vector of regression coe#cients by minimising the residual sum of squares subject to a constraint on the l 1 -norm of coe#cient vector. The LASSO estimator typically has one or more zero elements and thus shares characteristics of both shrinkage estimation and variable selection. In this paper we treat the LASSO as a convex programming problem and derive its dual. Consideration of the primal and dual problems together leads to important new insights into the characteristics of the LASSO estimator and to an improved method for estimating its covariance matrix. Using these results we also develop an e#cient algorithm for computing LASSO estimates which is usable even in cases where the number of regressors exceeds the number of observations. KEY WORDS AND PHRASES. Convex Programming, Dual Problem, Partial Least Squares, Quadratic Programming, Penalised Regression, Regression, Shrinkag...

### Citations

1675 | Atomic decomposition by basis pursuit - Chen, Donoho, et al. - 1998 |

235 |
Subset Selection in Regression
- Miller
- 1990
(Show Context)
Citation Context ...ndex added to the set # is the index of the variable that has maximal correlation with the residual vector (of the constrained subproblem (5.1)). This is not dissimilar to forward variable selection (=-=Miller, 1990-=-, Chapter 3.2). However, whenever an index is added to # we have to solve a new subproblem (5.1) and while solving this new problem it may happen that the indices of some variables are deleted from #.... |

232 |
A Statistical View of Some Chemometrics Regression Tools (with discussion)," Technometries
- Friedman
- 1993
(Show Context)
Citation Context ... on block coordinate relaxation techniques. A possible generalisation of (1.1) is to change the constraint (1.1b) to m # j=1 |# j | # # t for some # # 1. This was investigated by Fu (1998) (see also, =-=Frank and Friedman, 1993-=-). Fu (1998) also proposes an alternative algorithm to solve (1.2). However, his algorithm is again not applicable if m > n as it starts from the unconstrained least-squares solution of (1.1a). The re... |

105 | Penalized regressions: The bridge versus the Lasso - Fu - 1998 |

79 | Block coordinate relaxation methods for nonparametric wavelet denoising - Sardy, Brouce, et al. - 2000 |

33 |
Finite algorithms in optimization and data analysis
- Osborne
- 1985
(Show Context)
Citation Context ...nction in # and L(#, #) ## as ### 1 ##. Hence, L(, #) has at least one minimum and # minimises L(#, #) if and only if the m-dimensional null-vector 0 is an element of the subdi#erential # # L( #, #) (=-=Osborne, 1985-=-, p. 23). In the current problem, the subdi#erential is given by (Osborne, 1985, p. 20, Remark 4.4) # # L(#, #) = -X T r + #v, where v = (v 1 , . . . , v m ) T is of the following form: v i = 1 if # i... |

30 |
Measurement, Regression and Calibration
- Brown
- 1993
(Show Context)
Citation Context ...limitation, since problems in which the number of variables is of the same or larger order than the number of observations occur frequently in areas such as chemometrics, where partial least squares (=-=Brown, 1993-=-; Haagen et al., 1993) is often employed. In fact our own interest in the LASSO was initially motivated by the problem of knot selection for regression splines (see Osborne et al., 1998), which can be... |

18 | Prostate specific antigen in the diagnosis and treatment of adenocarcinoma of the prostate, II: Radical prostatectomy treated patients - Stamey, Kabalin, et al. - 1989 |

7 | An extension of the mixed primal-dual bases algorithm to the case of more constraints than dimensions - Meyer - 1999 |

6 | Knot selection for regression splines via the lasso
- Osborne, Presnell, et al.
- 1998
(Show Context)
Citation Context ... partial least squares (Brown, 1993; Haagen et al., 1993) is often employed. In fact our own interest in the LASSO was initially motivated by the problem of knot selection for regression splines (see =-=Osborne et al., 1998-=-), which can be formulated as a variable selection problem with m > n. In this paper, we treat (1.1) as a convex programming problem and derive the dual optimisation problem. By considering simultaneo... |

2 |
On the asymptotic performance of median smoothers in image analysis and nonparametric regression
- Koch
- 1996
(Show Context)
Citation Context ... first approximation that we consider is to minimise f(#) subject to g c (#) = t - m # j=1 # c (# j ) # 0. (I) Another well-known approximation to the absolute function |u| is # u 2 + c 2 (see, e.g., =-=Koch, 1996-=-). This leads to the second smooth approximation of the constraint (2.1b) by g c (#) = t - m # j=1 # # 2 j + c 2 # 0. (II) In what follows, any quantity associated with one of the smooth optimisation ... |

2 |
Linear Statistical Inference and Its Applications, 2 edn
- Rao
- 1973
(Show Context)
Citation Context ...s also interesting to note that though the matrix W - chosen by Tibshirani is a generalised inverse of the matrix W, one might also consider using, for example, the Moore--Penrose inverse (see, e.g., =-=Rao, 1973-=-, p. 26) in (7). The Moore--Penrose inverse places zero into those diagonal elements of W - which correspond to parameters that are estimated to be zero. The estimated standard errors that one obtains... |

1 | Nonlinear statistical models - tex - 1987 |

1 |
Statistical modelling and latent variables
- Haagen, Bartholomew, et al.
- 1993
(Show Context)
Citation Context ...ince problems in which the number of variables is of the same or larger order than the number of observations occur frequently in areas such as chemometrics, where partial least squares (Brown, 1993; =-=Haagen et al., 1993-=-) is often employed. In fact our own interest in the LASSO was initially motivated by the problem of knot selection for regression splines (see Osborne et al., 1998), which can be formulated as a vari... |