Abstract. Every Newton step in an interior-point method for optimization requires a solution of a symmetric indefinite system of linear equations. Most of today’s codes apply direct solution methods to perform this task. The use of logarithmic barriers in interior point methods causes unavoidable ill-conditioning of linear systems and, hence, iterative methods fail to provide sufficient accuracy unless appropriately preconditioned. Two types of preconditioners which use some form of incomplete Cholesky factorization for indefinite systems are proposed in this paper. Although they involve significantly sparser factorizations than those used in direct approaches they still capture most of the numerical properties of the preconditioned system. The spectral analysis of the preconditioned matrix is performed: for convex optimization problems all the eigenvalues of this matrix are strictly positive. Numerical results are given for a set of public domain large linearly constrained convex quadratic programming problems with sizes reaching tens of thousands of variables. The analysis of these results reveals that the solution times for such problems on a modern PC are measured in minutes when direct methods are used and drop to seconds when iterative methods with appropriate preconditioners are used. Keywords: interior-point methods, iterative solvers, preconditioners 1.

We consider solving a sequence of weighted linear least squares problems where the changes from one problem to the next are the weights and the right hand side (or data). This is the case for primaldual interior-point methods. We derive a class of preconditioners based on a low rank correction to a Cholesky factorization of a weighted normal equation coefficient matrix with the previous weight. Key Words. Weighted linear least squares, Preconditioners, Preconditioned conjugate gradient for least squares, Linear programming, Primaldual infeasible-interior-point algorithms. 1 Introduction In this paper, we present a class of preconditioners based on low rank corrections to the Cholesky factorization of a weighted normal equation coefficient matrix. This class of preconditioners leads to good performance for interiorpoint methods for linear programming. Particularly, we have implemented primal-dual Newton method to test this class of preconditioners. The numerical results on large scale...

this paper we consider solving a sequence of weighted linear least squares problems where the only changes from one problem to the next are the weights and the right hand side (or data). We alternate between iterative and direct methods to solve the normal equations for the least squares problems. The direct method is the Cholesky factorization. For the iterative method we discuss a class of preconditioners based on a low rank correction of a Cholesky factorization of a weighted normal equation coefficient matrix. Different ways to compute the preconditioner are given. Further, a sparse algorithm for modifying the Cholesky factors by a low rank matrix is derived.

Convex quadratic programming (CQP) is an optimization problem of minimizing a convex quadratic objective function subject to linear constraints. We propose an adaptive constraint reduction primal-dual interior-point algorithm for convex quadratic programming with many more constraints than variables. We reduce the computational effort by assembling the normal equation matrix with a subset of the constraints. Instead of the exact matrix, we compute an approximate matrix for a well chosen index set which includes indices of constraints that seem to be most critical. Starting with a large portion of the constraints, our proposed scheme excludes more unnecessary constraints at later iterations. We provide proofs for the global convergence and the quadratic local convergence rate of an affine scaling variant. A similar approach can be applied to Mehrotra’s predictor-corrector type algorithms. An example of CQP arises in training a linear support vector machine (SVM), which is a popular tool for pattern recognition. The difficulty in training a supportvector machine (SVM) lies in the typically vast number of patterns used for the training process. In this work, we propose an adaptive constraint reduction primaldual interior-point method for training the linear SVM with l1 hinge loss. We reduce the computational effort by assembling the normal equation matrix with a subset of well-chosen patterns. Starting with a large portion of the patterns, our proposed scheme excludes more and more unnecessary patterns as the iteration proceeds. We extend our approach to training nonlinear SVMs through Gram matrix approximation methods. Promising numerical results are reported.

This is a collection of four conference proceedings on scientific computation. In the proceedings, we discuss solving a sequence of linear systems arising from the application of an interior point method to a linear programming problem. The sequence of linear systems is solved by alternating between a direct and an iterative method. The preconditioner is based on low-rank modifications of the coefficient matrix where a direct solution technique has been used. We compare two different techniques of forming the low-rank modification matrix; namely one by Wang and O'Leary [11] and the other by Baryamureeba, Steihaug and Zhang [3]. The theory and numerical testing strongly support the latter. We derive a sparse algorithm for modifying the Cholesky factors by a low-rank matrix, discuss the computational issues of this preconditioner, and finally give numerical results that show the approach of alternating between a direct and an iterative method to be promising. Key Words. Linear Programmi...

A sequence of weighted linear least squares problems arises from interior-point methods for linear programming where the changes from one problem to the next are the weights and the right hand side. One approach for solving such a weighted linear least squares problem is to apply a preconditioned conjugate gradient method to the normal equations where the preconditioner is based on a low-rank correction to the Cholesky factorization of a previous coefficient matrix. In this paper, we establish theoretical results for such preconditioners that provide guidelines for the construction of preconditioners of this kind. We also present preliminary numerical experiments to validate our theoretical results and to demonstrate the effectiveness of this approach. Key Words. Weighted linear least squares, Preconditioner, Preconditioned conjugate gradient method, Linear programming, interior-point algorithms. This is Technical Report No. 170, Department of Informatics, University of Bergen, N50...

. In every primal-dual interior point method, a sequence of weighted linear least squares problems are solved, where the only change from one problem to the next are the weights and the right hand side (or data). We propose a mixed primal-dual method where we solve the weighted least squares problems with a direct method for every even interior point iteration and an iterative method for every odd iteration. A class of preconditioners based on a low rank correction of a Cholesky factorization of a weighted normal equation coefficient matrix is introduced. Key Words. Weighted linear least squares, Preconditioners, Preconditioned conjugate gradient for least squares, Linear programming, Primal-dual infeasible interior point algorithms. 1 Introduction An interior point algorithm solves the linear programming problem by generating a sequence of interior points from an initial interior point. At every interior point iteration weighted linear least squares problems are solved. The new class...

. In most interior point methods for linear programming, a sequence of weighted linear least squares problems are solved, where the only changes from one iteration to the next are the weights and the right hand side. The weighted least squares problems are usually solved as weighted normal equations by the direct method of Cholesky factorization. In this paper, we consider solving the weighted normal equations by a preconditioned conjugate gradient method at every other iteration. We use a class of preconditioners based on a low rank correction to a Cholesky factorization obtained from the previous iteration. Numerical results show that when properly implemented, the approach of combining direct and iterative methods is promising. Key Words. Weighted linear least squares, Parallel processing, Preconditioners, Linear programming, Primal-dual infeasible interior point algorithms. 1 Introduction The class of preconditioners we will consider is a low rank correction of a Cholesky factoriz...

Abstract not found

Abstract. We consider a primal-dual short-step interior-point method for conic convex optimization problems for which exact evaluation of the gradient and Hessian of the primal and dual barrier functions is either impossible or prohibitively expensive. As our main contribution, we show that if approximate gradients and Hessians of the primal barrier function can be computed, and the relative errors in such quantities are not too large, then the method has polynomial worst-case iteration complexity. (In particular, polynomial iteration complexity ensues when the gradient and Hessian are evaluated exactly.) In addition, the algorithm requires no evaluation—or even approximate evaluation—of quantities related to the barrier function for the dual cone, even for problems in which the underlying cone is not self-dual.