In this paper we argue that the choice of the SVM cost parameter can be critical. We then derive an algorithm that can fit the entire path of SVM solutions for every value of the cost parameter, with essentially the same computational cost as fitting one SVM model.

Logistic regression with ℓ1 regularization has been proposed as a promising method for feature selection in classification problems. In this paper we describe an efficient interior-point method for solving large-scale ℓ1-regularized logistic regression problems. Small problems with up to a thousand or so features and examples can be solved in seconds on a PC; medium sized problems, with tens of thousands of features and examples, can be solved in tens of seconds (assuming some sparsity in the data). A variation on the basic method, that uses a preconditioned conjugate gradient method to compute the search step, can solve very large problems, with a million features and examples (e.g., the 20 Newsgroups data set), in a few minutes, on a PC. Using warm-start techniques, a good approximation of the entire regularization path can be computed much more efficiently than by solving a family of problems independently.

In engineering and applied mathematics, polynomial systems arise whose solution sets contain components of different dimensions and multiplicities. In this article we present algorithms, based on homotopy continuation, that compute much of the geometric information contained in the primary decomposition of the solution set. In particular, ignoring multiplicities, our algorithms lay out the decomposition of the set of solutions into irreducible components, by finding, at each dimension, generic points on each component. As by-products, the computation also determines the degree of each component and an upper bound on itsmultiplicity. The bound issharp (i.e., equal to one) for reduced components. The algorithms make essential use of generic projection and interpolation, and can, if desired, describe each irreducible component precisely as the common zeroesof a finite number of polynomials.

Abstract. Interior methods are an omnipresent, conspicuous feature of the constrained optimization landscape today, but it was not always so. Primarily in the form of barrier methods, interior-point techniques were popular during the 1960s for solving nonlinearly constrained problems. However, their use for linear programming was not even contemplated because of the total dominance of the simplex method. Vague but continuing anxiety about barrier methods eventually led to their abandonment in favor of newly emerging, apparently more efficient alternatives such as augmented Lagrangian and sequential quadratic programming methods. By the early 1980s, barrier methods were almost without exception regarded as a closed chapter in the history of optimization. This picture changed dramatically with Karmarkar’s widely publicized announcement in 1984 of a fast polynomial-time interior method for linear programming; in 1985, a formal connection was established between his method and classical barrier methods. Since then, interior methods have advanced so far, so fast, that their influence has transformed both the theory and practice of constrained optimization. This article provides a condensed, selective look at classical material and recent research about interior methods for nonlinearly constrained optimization.

Many applications modeled by polynomial systems have positive dimensional solution components (e.g., the path synthesis problems for four-bar mechanisms) that are challenging to compute numerically by homotopy continuation methods. A procedure of A. Sommese and C. Wampler consists in slicing the components with linear subspaces in general position to obtain generic points of the components as the isolated solutions of an auxiliary system. Since this requires the solution of a number of larger overdetermined systems, the procedure is computationally expensive and also wasteful because many solution paths diverge. In this article an embedding of the original polynomial system is presented, which leads to a sequence of homotopies, with solution paths leading to generic points of all components as the isolated solutions of an auxiliary system. The new procedure significantly reduces the number of paths to solutions that need to be followed. This approach has been implemented and applied to...

Abstract not found

The problem of learning a sparse conic combination of kernel functions or kernel matrices for classification or regression can be achieved via the regularization by a block 1-norm [1]. In this paper, we present an algorithm that computes the entire regularization path for these problems.

Regularization plays a central role in the analysis of modern data, where non-regularized fitting is likely to lead to over-fitted models, useless for both prediction and interpretation. We consider the design of incremental algorithms which follow paths of regularized solutions, as the regularization varies. These approaches often result in methods which are both efficient and highly flexible. We suggest a general path-following algorithm based on second-order approximations, prove that under mild conditions it remains “very close ” to the path of optimal solutions and illustrate it with examples. 1

Abstract. We show how to use numerical continuation to compute the intersection C = A∩B of two algebraic sets A and B, where A, B, and C are numerically represented by witness sets. Enroute to this result, we first show how to find the irreducible decomposition of a system of polynomials restricted to an algebraic set. The intersection of components A and B then follows by considering the decomposition of the diagonal system of equations u − v = 0 restricted to {u, v} ∈ A × B. One offshoot of this new approach is that one can solve a large system of equations by finding the solution components of its subsystems and then intersecting these. It also allows one to find the intersection of two components of the two polynomial systems, which is not possible with any previous numerical continuation approach.

Homotopies for polynomial systems provide computational evidence for a challenging instance of a conjecture about whether all solutions are real. The implementation of SAGBI homotopies involves polyhedral continuation, flat deformation and cheater's homotopy. The numerical difficulties are overcome if we work in the true synthetic spirit of the Schubert calculus by selecting the numerically most favorable equations to represent the geometric problem. Since a well-conditioned polynomial system allows perturbations on the input data without destroying the reality of the solutions we obtain not just one instance, but a whole manifold of systems that satisfy the conjecture. Also an instance that involves totally positive matrices has been verified. The optimality of the solving procedure is a promising first step towards the development of numerically stable algorithms for the pole placement problem in linear systems theory.