Results 1 
7 of
7
An exponential lower bound on the complexity of regularization paths. arXiv:0903.4817v2 [cs.LG
, 2009
"... For a variety of regularization methods, algorithms computing the entire solution path have been developed recently. Solution path algorithms do not only compute the solution for one particular value of the regularization parameter but the entire path of solutions, making the selection of an optimal ..."
Abstract

Cited by 6 (3 self)
 Add to MetaCart
For a variety of regularization methods, algorithms computing the entire solution path have been developed recently. Solution path algorithms do not only compute the solution for one particular value of the regularization parameter but the entire path of solutions, making the selection of an optimal parameter much easier. It has been assumed that these piecewise linear solution paths have only linear complexity, i.e. linearly many bends. We prove that for the support vector machine this complexity can indeed be exponential in the number of training points in the worst case.
A New Solution Path Algorithm in Support Vector Regression
, 2008
"... Regularization path algorithms were proposed as a novel approach to the model selection problem by exploring the path of possibly all solutions with respect to some regularization hyperparameter in an efficient way. This approach was later extended to a support vector regression (SVR) model called ɛ ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
Regularization path algorithms were proposed as a novel approach to the model selection problem by exploring the path of possibly all solutions with respect to some regularization hyperparameter in an efficient way. This approach was later extended to a support vector regression (SVR) model called ɛSVR. However, the method requires that the error parameter ɛ be set a priori. This is only possible if the desired accuracy of the approximation can be specified in advance. In this paper, we analyze the solution space for ɛSVR and propose a new solution path algorithm, called ɛpath algorithm, which traces the solution path with respect to the hyperparameter ɛ rather than λ. Although both two solution path algorithms possess the desirable piecewise linearity property, our ɛpath algorithm overcomes some limitations of the original λpath algorithm and has more advantages. It is thus more appealing for practical use.
Approximating Parameterized Convex Optimization Problems ∗
, 2010
"... We consider parameterized convex optimization problems over the unit simplex, that depend on one parameter. We provide a simple and efficient scheme for maintaining an εapproximate solution (and a corresponding εcoreset) along the entire parameter path. We prove correctness and parameterized optim ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
We consider parameterized convex optimization problems over the unit simplex, that depend on one parameter. We provide a simple and efficient scheme for maintaining an εapproximate solution (and a corresponding εcoreset) along the entire parameter path. We prove correctness and parameterized optimization problem are for example regularization paths of support vector machines, multiple kernel learning, and minimum enclosing balls of moving points. 1
Regularisation Path for Ranking SVM
"... Abstract. Ranking algorithms are often introduced with the aim of automatically personalising search results. However, most ranking algorithms developed in the machine learning community rely on a careful choice of some regularisation parameter. Building upon work on the regularisation path for kern ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
Abstract. Ranking algorithms are often introduced with the aim of automatically personalising search results. However, most ranking algorithms developed in the machine learning community rely on a careful choice of some regularisation parameter. Building upon work on the regularisation path for kernel methods, we propose a parameter selection algorithm for ranking SVM. Empirical results are promising. 1
The Feature Selection Path in Kernel Methods
"... The problem of automatic feature selection/weighting in kernel methods is examined. We work on a formulation that optimizes both the weights of features and the parameters of the kernel model simultaneously, using L1 regularization for feature selection. Under quite general choices of kernels, we pr ..."
Abstract
 Add to MetaCart
The problem of automatic feature selection/weighting in kernel methods is examined. We work on a formulation that optimizes both the weights of features and the parameters of the kernel model simultaneously, using L1 regularization for feature selection. Under quite general choices of kernels, we prove that there exists a unique regularization path for this problem, that runs from 0 to a stationary point of the nonregularized problem. We propose an ODEbased homotopy method to follow this trajectory. By following the path, our algorithm is able to automatically discard irrelevant features and to automatically go back and forth to avoid local optima. Experiments on synthetic and real datasets show that the method achieves low prediction error and is efficient in separating relevant from irrelevant features. 1
Solution Path for Manifold Regularized Semisupervised Classification
"... Abstract—Traditional learning algorithms use only labeled data for training. However, labeled examples are often difficult or time consuming to obtain since they require substantial human labeling efforts. On the other hand, unlabeled data are often relatively easy to collect. Semisupervised learnin ..."
Abstract
 Add to MetaCart
Abstract—Traditional learning algorithms use only labeled data for training. However, labeled examples are often difficult or time consuming to obtain since they require substantial human labeling efforts. On the other hand, unlabeled data are often relatively easy to collect. Semisupervised learning addresses this problem by using large quantities of unlabeled data with labeled data to build better learning algorithms. In this paper, we use the manifold regularization approach to formulate the semisupervised learning problem where a regularization framework which balances a tradeoff between loss and penalty is established. We investigate different implementations of the loss function and identify the methods which have the least computational expense. The regularization hyperparameter, which determines the balance between loss and penalty, is crucial to model selection. Accordingly, we derive an algorithm that can fit the entire path of solutions for every value of the hyperparameter. Its computational complexity after preprocessing is quadratic only in the number of labeled examples rather than the total number of labeled and unlabeled examples. Index Terms—manifold regularization, semisupervised classification, solution path. I.
SemiSupervised Novelty Detection using SVM entire solution path
"... Abstract — Very often, the only reliable information available to perform change detection is the description of some unchanged regions. Since sometimes these regions do not contain all the relevant information to identify their counterpart (the changes), we consider the use of unlabeled data to per ..."
Abstract
 Add to MetaCart
Abstract — Very often, the only reliable information available to perform change detection is the description of some unchanged regions. Since sometimes these regions do not contain all the relevant information to identify their counterpart (the changes), we consider the use of unlabeled data to perform SemiSupervised Novelty detection (SSND). SSND can be seen as an unbalanced classification problem solved using the CostSensitive Support Vector Machine (CSSVM), but this requires a heavy parameter search. We propose here to use entire solution path algorithms for the CSSVM in order to facilitate and accelerate the parameter selection for SSND. Two algorithms are considered and evaluated. The first one is an extension of the CSSVM algorithm that returns the entire solution path in a single optimization. This way, the optimization of a separate model for each hyperparameter set is avoided. The second forces the solution to be coherent through the solution path, thus producing classification boundaries that are nested (included in each other). We also present a low density criterion for selecting the optimal classification boundaries, thus avoiding the recourse to crossvalidation that usually requires information about the “change ” class. Experiments are performed on two multitemporal change detection datasets (flood and fire detection). Both algorithms tracing the solution path provide similar performances than the standard CSSVM while being significantly faster. The low density criterion proposed achieves results that are close to the ones obtained by crossvalidation, but without using information about the changes.