Due to their extraordinary utility and broad applicability in many areas of classical mathematics and modern physical sciences (most notably, computerized tomography), algorithms for solving convex feasibility problems continue to receive great attention. To unify, generalize, and review some of these algorithms, a very broad and flexible framework is investigated . Several crucial new concepts which allow a systematic discussion of questions on behaviour in general Hilbert spaces and on the quality of convergence are brought out. Numerous examples are given. 1991 M.R. Subject Classification. Primary 47H09, 49M45, 65-02, 65J05, 90C25; Secondary 26B25, 41A65, 46C99, 46N10, 47N10, 52A05, 52A41, 65F10, 65K05, 90C90, 92C55. Key words and phrases. Angle between two subspaces, averaged mapping, Cimmino's method, computerized tomography, convex feasibility problem, convex function, convex inequalities, convex programming, convex set, Fej'er monotone sequence, firmly nonexpansive mapping, H...

We consider a class of subgradient methods for minimizing a convex function that consists of the sum of a large number of component functions. This type of minimization arises in a dual context from Lagrangian relaxation of the coupling constraints of large scale separable problems. The idea is to perform the subgradient iteration incrementally, by sequentially taking steps along the subgradients of the component functions, with intermediate adjustment of the variables after processing each component function. This incremental approach has been very successful in solving large di#erentiable least squares problems, such as those arising in the training of neural networks, and it has resulted in a much better practical rate of convergence than the steepest descent method. In this paper, we establish the convergence properties of a number of variants of incremental subgradient methods, including some that are stochastic. Based on the analysis and computational experiments, the methods appear very promising and e#ective for important classes of large problems. A particularly interesting discovery is that by randomizing the order of selection of component functions for iteration, the convergence rate is substantially improved. 1 Research supported by NSF under Grant ACI-9873339.

We review a class of recently-proposed linear-cost network flow methods which are amenable to distributed implementation. All the methods in the class use the notion of e-complementary slackness, and most do not explicitly manipulate any "global " objects such as paths, trees, or cuts. Interestingly, these methods have stimulated a large number of new serial computational complexity results. We develop the basic theory of these methods and present two specific methods, the e-relaxation algorithm for the minimum-cost flow problem, and the auction algorithm for the assignment problem. We show how to implement these methods with serial complexities of O(N 3 log NC) and O(NA log NC), respectively. We also discuss practical implementation issues and computational experience to date. Finally, we show how to implement e-relaxation in a completely asynchronous, "chaotic" environment in which some processors compute faster than others, some processors communicate faster than others, and there can be arbitrarily large communication delays.

Scheduling is one of the most basic but the most difficult problems encountered in the manufacturing industry. Generally, some degree of time-consuming and impractical enumeration is required to obtain optimal solutions. Industry has thus relied on a combination of heuristics and simulation to solve the problem, resulting in unreliable and often infeasible schedules. Yet, there is a great need for an improvement in scheduling operations in complex and turbulent manufacturing environments. The logical strategy is to find scheduling methods which consistently generate good schedules efficiently. However, it is often difficult to measure the quality of a schedule without knowing the optimum. In this paper, the practical scheduling of three manufacturing environments are examined in the increasing order of complexity. The first problem considers scheduling singleoperation jobs on parallel, identical machines; the second one is concerned with scheduling multiple-operation jobs with simple ...

We consider a wide class of iterative methods arising in numerical mathematics and optimization which are known to converge only weakly. Exploiting an idea originally proposed by Haugazeau, we present a simple modification of these methods which makes them strongly convergent without additional assumptions. Several applications are discussed. AMS 1991 subject classification. Primary: 65J15, 47N10; secondary 41A29, 47H05, 47H09, 65K10, 90C25. Key words. Convex feasibility, Fej'er-monotonicity, firmly nonexpansive mapping, fixed point, Haugazeau, maximal monotone operator, projection, proximal point algorithm, resolvent, subgradient algorithm. 1 Introduction Let H be a real Hilbert space with scalar product h\Delta j \Deltai, norm k \Delta k, and distance d. In 1965, Bregman [5] proposed a simple iterative method for finding a common point of m intersecting closed convex sets (S i ) 1im in H. He showed that, given an arbitrary starting point x 0 2 H, the sequence (x n ) n0 gene...

The formulation of a wide variety of image recovery problems leads to the minimization of a convex objective over a convex set representing the constraints derived from a priori knowledge and consistency with the observed signals. In recent years, nondifferentiable objectives have become popular due in part to their ability to capture certain features such as sharp edges. They also arise naturally in minimax inconsistent set theoretic recovery problems. At the same time, the issue of developing reliable numerical algorithms to solve such convex programs in the context of image recovery applications has received little attention. In this paper, we address this issue and propose an adaptive level set method for nondifferentiable constrained image recovery. The asymptotic properties of the method are analyzed and its implementation is discussed. Numerical experiments illustrate applications to total variation and minimax set theoretic image restoration and denoising problems.

We generalize the subgradient optimization method for nondifferentiable convex programming to utilize conditional subgradients. Firstly, we derive the new method and establish its convergence by generalizing convergence results for traditional subgradient optimization. Secondly, we consider a particular choice of conditional subgradients, obtained by projections, which leads to an easily implementable modification of traditional subgradient optimization schemes. To evaluate the subgradient projection method we consider its use in three applications: uncapacitated facility location, two-person zero-sum matrix games, and multicommodity network flows. Computational experiments show that the subgradient projection method performs better than traditional subgradient optimization; in some cases the difference is considerable. These results suggest that our simple modification may improve subgradient optimization schemes significantly. This finding is important as such schemes are very popula...

. We consider the solution of finite element discretized optimum sheet problems by an iterative algorithm. The problem is that of maximizing the stiffness of a sheet subject to constraints on the admissible designs and unilateral contact conditions on the displacements. The model allows for zero design volumes, and thus constitutes a true topology optimization problem. We propose and evaluate a subgradient optimization algorithm for a reformulation into a nondifferentiable, convex minimization problem in the displacement variables. The convergence of this method is proven, and its low computational complexity is established. An optimal design is derived through a simple averaging scheme which combines the solutions to the linear design problems solved within the subgradient method. To illustrate the efficiency of the algorithm and investigate the properties of the optimal designs, the algorithm is numerically tested on some medium and large scale problems. Key words: optimum sheet; un...

Lagrangean dualization and subgradient optimization techniques are frequently used within the field of computational optimization for finding approximate solutions to large, structured optimization problems. The dual subgradient scheme does not automatically produce primal feasible solutions; there is an abundance of techniques for computing such solutions (via penalty functions, tangential approximation schemes, or the solution of auxiliary primal programs) , all of which require a fair amount of computational effort. We consider a subgradient optimization scheme applied to a Lagrangean dual formulation of a convex program, and construct, at minor cost, an ergodic sequence of subproblem solutions which converges to the primal solution set. Numerical experiments performed on a traffic equilibrium assignment problem under road pricing show that the computation of the ergodic sequence results in a considerable improvement in the quality of the primal solutions obtained, compared to those...

this paper (see also [1]), an algorithm was developed to solve systems of linear inequalities in R by successive projections onto the half-spaces de ning the polyhedral solution set S. The concept of Fejer monotonicity was shown to be an adequate tool to study convergence of this algorithm. Basic facts such as (5) and (9) can already be found in [19] and [1], respectively