Abstract not found

We present a framework for solving large-scale ℓ1-regularized convex minimization problem: min �x�1 + µf(x). Our approach is based on two powerful algorithmic ideas: operator-splitting and continuation. Operator-splitting results in a fixed-point algorithm for any given scalar µ; continuation refers to approximately following the path traced by the optimal value of x as µ increases. In this paper, we study the structure of optimal solution sets; prove finite convergence for important quantities; and establish q-linear convergence rates for the fixed-point algorithm applied to problems with f(x) convex, but not necessarily strictly convex. The continuation framework, motivated by our convergence results, is demonstrated to facilitate the construction of practical algorithms.

We consider the forward-backward splitting method for finding a zero of the sum of two maximal monotone mappings. This method is known to converge when the inverse of the forward mapping is strongly monotone. We propose a modification to this method, in the spirit of the extragradient method for monotone variational inequalities, under which the method converges assuming only the forward mapping is monotone and (Lipschitz) continuous on some closed convex subset of its domain. The modification entails an additional forward step and a projection step at each iteration. Applications of the modified method to decomposition in convex programming and monotone variational inequalities are discussed.

. Some methods which we call function decomposition methods and space decomposition methods are developed. These methods deal with a convex programming problem, i.e. a minimization problem of a convex function over a space or a convex set of a space. If the function can be decomposed into the sum of convex functions or the space can be decomposed into the sum of subspaces, then parallel methods can be used for the minimization. In practical problems, there are many different ways to decompose a function and to decompose a space. Many partial differential equations can be formulated as a minimization problem in some way. Therefore, we get some parallel methods for partial differential equations. The method is not restricted to linear problems, nonlinear problems are naturally included into the theory. In the paper, the applications to linear and quasi--linear self--adjoint elliptic equations, to strongly nonlinear elliptic equation like the p--Laplace equation, to the Stokes equation a...

We propose a modification of the classical extragradient and proximal point algorithms for finding a zero of a maximal monotone operator in a Hilbert space. At each iteration of the method, an approximate extragradient-type step is performed using information obtained from an approximate solution of a proximal point subproblem. The algorithm is of a hybrid type, as it combines steps of the extragradient and proximal methods. Furthermore, the algorithm uses elements in the enlargement (proposed by Burachik, Iusem and Svaiter [2]) of the operator defining the problem. One of the important features of our approach is that it allows significant relaxation of tolerance requirements imposed on the solution of proximal point subproblems. This yields a more practical proximal-algorithm-based framework. Weak global convergence and local linear rate of convergence are established under suitable assumptions. It is further demonstrated that the modified forward-backward splitting algorithm of Tseng [35]...

This paper applies splitting techniques developed for set-valued maximal monotone operators to monotone affine variational inequalities, including as a special case the classical linear complementarity problem. We give a unified presentation of several splitting algorithms for monotone operators, and then apply these results to obtain two classes of algorithms for affine variational inequalities. The second class resembles classical matrix splitting, but has a novel "underrelaxation " step, and converges under more general conditions. In particular, the convergence proofs do not require the affine operator to be symmetric. We specialize our matrix-splittinglike method to discrete-time optimal control problems formulated as extended linear-quadratic programs in the manner advocated by Rockafellar and Wets. The result is a highly parallel algorithm, which we implement and test on the Connection Machine CM--5 computer family. The affine variational inequality problem is to find a vector x...

Prompted by advances in computer technology and the increasing confidence of decision makers in large-scale market models, practitioners of operations research are now tackling problems of increasing detail, complexity and size. This necessitates the development of new solution algorithms that exploit problem structure as well as the properties of the target hardware, in order to minimize turnaround time and maximize model utilization. Many models in planning and scheduling exhibit a block-angular structure, that can represent spatial or temporal partial decomposability: decision variables can be broken down to largely independent blocks, that correspond to first-level decisions satisfying a subset of the constraints, which may represent a time period, or a geographical region, or a commodity. The blocks interact via coupling constraints related to second-level coordination of block decisions, such as shared resource allocation restrictions. In this thesis we construct three efficient decomposition algorithms for such block-angular problems. These algorithms belong to the family of alternating directions methods, and can be thought of as block Gauss-Seidel iterative schemes for an augmented Lagrangian, that exploit the block structure. Alternatively, they can be thought of as Douglas--Rachford schemes for calculating a zero of the maximal monotone subgradient operator. Our algorithms are of the "fork--join" type, alternating a local and a global computation phase. In the local phase, decoupled optimization subproblems corresponding to blocks are solved. In the global phase, solution information is combined and a coordination problem is solved, the results of which are used in modifying the objective function of the subproblems. The algorithms are thus similar to price-d...

We study a generalized version of the method of alternating directions as applied to the minimization of the sum of two convex functions subject to linear constraints. The method consists of solving consecutively in each iteration two optimization problems which contain in the objective function both Lagrangian and proximal terms. The minimizers determine the new proximal terms and a simple update of the Lagrangian terms follows. We prove a convergence theorem which extends existing results by relaxing the assumption of uniqueness of minimizers. Another novelty is that we allow penalty matrices, and these may vary per iteration. This can be beneficial in applications, since it allows additional tuning of the method to the problem and can lead to faster convergence relative to fixed penalties. As an application, we derive a decomposition scheme for block angular optimization and present computational results on a class of dual block angular problems. Keywords: parallel computing, alter...

Abstract not found

: We develop and compare three decomposition algorithms derived from the method of alternating directions. They may be viewed as block Gauss-Seidel variants of augmented Lagrangian approaches that take advantage of block-angular structure. From a parallel computation viewpoint, they are ideally suited to a data parallel environment. Numerical results for large-scale multicommodity flow problems are presented to demonstrate the effectiveness of these decomposition algorithms on the Thinking Machines CM-5 parallel supercomputer relative to the widely-used serial optimization package MINOS 5.4 . 1 Introduction Three decomposition algorithms are derived from the method of alternating directions [21] for block-angular optimization. The three methods will be categorized below according to the type of proximal terms that they contain. They may be viewed as block Gauss-Seidel variants of augmented Lagrangian approaches that take advantage of block-angular structure, and consequently are ideal...