DMCA
Structured sparsity-inducing norms through submodular functions (2010)
Cached
Download Links
| Venue: | IN ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS |
| Citations: | 60 - 10 self |
BibTeX
@INPROCEEDINGS{Bach10structuredsparsity-inducing,
author = {Francis Bach},
title = {Structured sparsity-inducing norms through submodular functions},
booktitle = {IN ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS},
year = {2010},
publisher = {}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Sparse methods for supervised learning aim at finding good linear predictors from as few variables as possible, i.e., with small cardinality of their supports. This combinatorial selection problem is often turnedinto a convex optimization problem byreplacing the cardinality function by its convex envelope (tightest convex lower bound), in this case the ℓ1-norm. In this paper, we investigate more general set-functions than the cardinality, that may incorporate prior knowledge or structural constraints which are common in many applications: namely, we show that for nonincreasing submodular set-functions, the corresponding convex envelope can be obtained from its Lovász extension, a common tool in submodular analysis. This defines a family of polyhedral norms, for which we provide generic algorithmic tools (subgradients and proximal operators) and theoretical results (conditions for support recovery or high-dimensional inference). By selecting specific submodular functions, we can give a new interpretation to known norms, such as those based on rank-statistics or grouped norms with potentially overlapping groups; we also define new norms, in particular ones that can be used as non-factorial priors for supervised learning.
Keyphrases
sparsity-inducing norm submodular function support recovery small cardinality sparse method good linear predictor general set-functions supervised learning aim new interpretation convex envelope proximal operator corresponding convex envelope polyhedral norm non-factorial prior many application high-dimensional inference submodular set-functions new norm lov sz extension generic algorithmic tool specific submodular function common tool cardinality function prior knowledge combinatorial selection problem particular one structural constraint theoretical result supervised learning submodular analysis convex optimization problem
