DMCA
Piecewise linear regularized solution paths (2007)
| Venue: | Ann. Statist |
| Citations: | 133 - 8 self |
BibTeX
@ARTICLE{Rosset07piecewiselinear,
author = {Saharon Rosset and Ji Zhu},
title = {Piecewise linear regularized solution paths},
journal = {Ann. Statist},
year = {2007},
pages = {1030}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
We consider the generic regularized optimization problem ˆ β(λ) = arg minβ L(y, Xβ) + λJ(β). Recently, Efron et al. (2004) have shown that for the Lasso – that is, if L is squared error loss and J(β) = ‖β‖1 is the l1 norm of β – the optimal coefficient path is piecewise linear, i.e., ∂ ˆ β(λ)/∂λ is piecewise constant. We derive a general characterization of the properties of (loss L, penalty J) pairs which give piecewise linear coefficient paths. Such pairs allow for efficient generation of the full regularized coefficient paths. We investigate the nature of efficient path following algorithms which arise. We use our results to suggest robust versions of the Lasso for regression and classification, and to develop new, efficient algorithms for existing problems in the literature, including Mammen & van de Geer’s Locally Adaptive Regression Splines. 1
Keyphrases
solution path generic regularized optimization problem efficient generation optimal coefficient path efficient algorithm error loss piecewise linear coefficient path arg min full regularized coefficient path locally adaptive regression spline general characterization efficient path l1 norm robust version
