Results 1  10
of
51
On the limited memory BFGS method for large scale optimization
 Mathematical Programming
, 1989
"... this paper has appeared in ..."
An interiorpoint method for largescale l1regularized logistic regression
 Journal of Machine Learning Research
, 2007
"... Logistic regression with ℓ1 regularization has been proposed as a promising method for feature selection in classification problems. In this paper we describe an efficient interiorpoint method for solving largescale ℓ1regularized logistic regression problems. Small problems with up to a thousand ..."
Abstract

Cited by 156 (5 self)
 Add to MetaCart
Logistic regression with ℓ1 regularization has been proposed as a promising method for feature selection in classification problems. In this paper we describe an efficient interiorpoint method for solving largescale ℓ1regularized logistic regression problems. Small problems with up to a thousand or so features and examples can be solved in seconds on a PC; medium sized problems, with tens of thousands of features and examples, can be solved in tens of seconds (assuming some sparsity in the data). A variation on the basic method, that uses a preconditioned conjugate gradient method to compute the search step, can solve very large problems, with a million features and examples (e.g., the 20 Newsgroups data set), in a few minutes, on a PC. Using warmstart techniques, a good approximation of the entire regularization path can be computed much more efficiently than by solving a family of problems independently.
Theory of Algorithms for Unconstrained Optimization
, 1992
"... this article I will attempt to review the most recent advances in the theory of unconstrained optimization, and will also describe some important open questions. Before doing so, I should point out that the value of the theory of optimization is not limited to its capacity for explaining the behavio ..."
Abstract

Cited by 83 (1 self)
 Add to MetaCart
this article I will attempt to review the most recent advances in the theory of unconstrained optimization, and will also describe some important open questions. Before doing so, I should point out that the value of the theory of optimization is not limited to its capacity for explaining the behavior of the most widely used techniques. The question
Trust region Newton method for largescale logistic regression
 In Proceedings of the 24th International Conference on Machine Learning (ICML
, 2007
"... Largescale logistic regression arises in many applications such as document classification and natural language processing. In this paper, we apply a trust region Newton method to maximize the loglikelihood of the logistic regression model. The proposed method uses only approximate Newton steps in ..."
Abstract

Cited by 66 (12 self)
 Add to MetaCart
Largescale logistic regression arises in many applications such as document classification and natural language processing. In this paper, we apply a trust region Newton method to maximize the loglikelihood of the logistic regression model. The proposed method uses only approximate Newton steps in the beginning, but achieves fast convergence in the end. Experiments show that it is faster than the commonly used quasi Newton approach for logistic regression. We also compare it with existing linear SVM implementations. 1
A Global Convergence Theory for General TrustRegionBased Algorithms for Equality Constrained Optimization
 SIAM Journal on Optimization
, 1992
"... This work presents a global convergence theory for a broad class of trustregion algorithms for the smooth nonlinear progro.mmln S problem with equality constraints. The main result generalizes Powell's 1975 result for unconstrained trustregion algorithms. ..."
Abstract

Cited by 42 (10 self)
 Add to MetaCart
This work presents a global convergence theory for a broad class of trustregion algorithms for the smooth nonlinear progro.mmln S problem with equality constraints. The main result generalizes Powell's 1975 result for unconstrained trustregion algorithms.
Automatic preconditioning by limited memory QuasiNewton updating
 SIAM J. Optim
"... The paper proposes a preconditioner for the conjugate gradient method (CG) that is designed for solving systems of equations Ax = bi with di erent right hand side vectors, or for solving a sequence of slowly varying systems Akx = bk. The preconditioner has the form of a limited memory quasiNewton m ..."
Abstract

Cited by 31 (2 self)
 Add to MetaCart
The paper proposes a preconditioner for the conjugate gradient method (CG) that is designed for solving systems of equations Ax = bi with di erent right hand side vectors, or for solving a sequence of slowly varying systems Akx = bk. The preconditioner has the form of a limited memory quasiNewton matrix and is generated using information from the CG iteration. The automatic preconditioner does not require explicit knowledge of the coe cient matrix A and is therefore suitable for problems where only products of A times avector can be computed. Numerical experiments indicate that the preconditioner has most to o er when these matrixvector products are expensive to compute, and when low accuracy in the solution is required. The e ectiveness of the preconditioner is tested within a Hessianfree Newton method for optimization, and by solving certain linear systems arising in nite element models.
A Quadratic Assignment Formulation of the Molecular Conformation Problem
 JOURNAL OF GLOBAL OPTIMIZATION
, 1994
"... The molecular conformation problem is discussed, and a concave quadratic global minimization approach for solving it is described. This approach is based on a quadratic assignment formulation of a discrete approximation to the original problem. ..."
Abstract

Cited by 21 (0 self)
 Add to MetaCart
The molecular conformation problem is discussed, and a concave quadratic global minimization approach for solving it is described. This approach is based on a quadratic assignment formulation of a discrete approximation to the original problem.
Where is the Nearest NonRegular Pencil?
 TO APPEAR IN LINEAR ALGEBRA AND ITS APPLICATIONS
, 1997
"... This is a first step toward the goal of finding a way to calculate a smallest norm deregularizing perturbation of a given square matrix pencil. Minimal deregularizing perturbations have geometric characterizations that include a variable projection linear least squares problem and a minimax charac ..."
Abstract

Cited by 18 (6 self)
 Add to MetaCart
This is a first step toward the goal of finding a way to calculate a smallest norm deregularizing perturbation of a given square matrix pencil. Minimal deregularizing perturbations have geometric characterizations that include a variable projection linear least squares problem and a minimax characterization reminiscent of the CourantFischer theorem. The characterizations lead to new, computationally attractive upper and lower bounds. We give a brief survey and illustrate strengths and weaknesses of several upper and lower bounds some of which are wellknown and some of which are new. The ultimate goal remains elusive.
Signal reconstruction in sensor arrays using sparse representations $
, 2005
"... We propose a technique of multisensor signal reconstruction based on the assumption, that source signals are spatially sparse, as well as have sparse representation in a chosen dictionary in time domain. This leads to a large scale convex optimization problem, which involves combined l1l2 norm mini ..."
Abstract

Cited by 16 (0 self)
 Add to MetaCart
We propose a technique of multisensor signal reconstruction based on the assumption, that source signals are spatially sparse, as well as have sparse representation in a chosen dictionary in time domain. This leads to a large scale convex optimization problem, which involves combined l1l2 norm minimization. The optimization is carried by the truncated Newton method, using preconditioned conjugate gradients in inner iterations. The byproduct of reconstruction is the estimation of source locations. r 2005 Published by Elsevier B.V.
Curvilinear Stabilization Techniques for Truncated Newton Methods in Large Scale Unconstrained Optimization: the . . .
 SIAM J. Optim
, 1998
"... The aim of this paper is to define a new class of minimization algorithms for solving large scale unconstrained problems. In particular we describe a stabilization framework, based on a curvilinear linesearch, which uses a combination of a Newtontype direction and a negative curvature direction. Th ..."
Abstract

Cited by 15 (4 self)
 Add to MetaCart
The aim of this paper is to define a new class of minimization algorithms for solving large scale unconstrained problems. In particular we describe a stabilization framework, based on a curvilinear linesearch, which uses a combination of a Newtontype direction and a negative curvature direction. The motivation for using negative curvature direction is that of taking into account local nonconvexity of the objective function. On the basis of this framework, we propose an algorithm which uses the Lanczos method for determining at each iteration both a Newtontype direction and an effective negative curvature direction. The results of an extensive numerical testing is reported together with a comparison with the LANCELOT package. These results show that the algorithm is very competitive and this seems to indicate that the proposed approach is promising. 1 Introduction In this work, we deal with the definition of new efficient unconstrained minimization algorithms for solving large scal...