We discuss formulations of integer programs with a huge number of variables and their solution by column generation methods, i.e., implicit pricing of nonbasic variables to generate new columns or to prove LP optimality at a node of the branchand-bound tree. We present classes of models for which this approach decomposes the problem, provides tighter LP relaxations, and eliminates symmetry. Wethen discuss computational issues and implementation of column generation, branch-andbound algorithms, including special branching rules and e cient ways to solve the LP relaxation. We also discuss the relationship with Lagrangian duality. 1

A linear support vector machine formulation is used to generate a fast, finitely-terminating linear-programming algorithm for discriminating between two massive sets in n-dimensional space, where the number of points can be orders of magnitude larger than n. The algorithm creates a succession of sufficiently small linear programs that separate chunks of the data at a time. The key idea is that a small number of support vectors, corresponding to linear programming constraints with positive dual variables, are carried over between the successive small linear programs, each of which containing a chunk of the data. We prove that this procedure is monotonic and terminates in a finite number of steps at an exact solution that leads to a globally optimal separating plane for the entire dataset. Numerical results on fully dense publicly available datasets, numbering 20,000 to 1 million points in 32-dimensional space, confirm the theoretical results and demonstrate the ability to handle very l...

We investigate the problem of permuting a sparse rectangular matrix into block diagonal form. Block diagonal form of a matrix grants an inherent parallelism for solving the deriving problem, as recently investigated in the context of mathematical programming, LU factorization and QR factorization. We propose bipartite graph and hypergraph models to represent the nonzero structure of a matrix, which reduce the permutation problem to those of graph partitioning by vertex separator and hypergraph partitioning, respectively. Our experiments on a wide range of matrices, using state-of-the-art graph and hypergraph partitioning tools MeTiS and PaToH, revealed that the proposed methods yield very effective solutions both in terms of solution quality and runtime.

Dantzig-Wolfe decomposition and column generation, devised for linear programs, is a success story in large scale integer programming. We outline and relate the approaches, and survey mainly recent contributions, not found in textbooks, yet. We emphasize on the growing understanding of the dual point of view, which brought considerable progress to the column generation theory and practice. It stimulated careful initializations, sophisticated solution techniques for restricted master problem and subproblem, as well as better overall performance. Thus, the dual perspective is an ever recurring concept in our "selected topics."

We present a survey of nondifferentiable optimization problems and methods with special focus on the analytic center cutting plane method. We propose a self-contained convergence analysis, that uses the formalism of the theory of self-concordant functions, but for the main results, we give direct proofs based on the properties of the logarithmic function. We also provide an in depth analysis of two extensions that are very relevant to practical problems: the case of multiple cuts and the case of deep cuts. We further examine extensions to problems including feasible sets partially described by an explicit barrier function, and to the case of nonlinear cuts. Finally, we review several implementation issues and discuss some applications.

We describe sequential and parallel algorithms that approximately solve linear programs with no negative coefficients (a.k.a. mixed packing and covering problems). For explicitly given problems, our fastest sequential algorithm returns a solution satisfying all constraints within a ¦ ¯ factor in Ç Ñ � ÐÓ � Ñ � ¯ time, where Ñ is the number of constraints and � is the maximum number of constraints any variable appears in. Our parallel algorithm runs in time polylogarithmic in the input size times ¯   � and uses a total number of operations comparable to the sequential algorithm. The main contribution is that the algorithms solve mixed packing and covering problems (in contrast to pure packing or pure covering problems, which have only “� ” or only “� ” inequalities, but not both) and run in time independent of the so-called width of the problem. 1. Background Packing and covering problems are problems that can be formulated as linear programs using only non-negative coefficients and non-negative variables. Special cases include pure packing problems, which are of the form Ñ�Ü � ¡ Ü � �Ü � � � and pure covering problems, which are of the form Ñ�Ò � ¡ Ü � �Ü � ��. Lagrangian-relaxation algorithms are based on the following basic idea. Given an optimization problem specified as a collection of constraints, modify the problem by selecting some of the constraints and replacing them by a continuous “penalty ” function that, given a partial solution Ü, measures how close Ü is to violating the removed constraints. Construct a solution iteratively in small steps, making each choice to maintain the remaining constraints while minimizing the increase in the penalty function. While Lagrangian-relaxation algorithms have the disadvantage of producing only approximately optimal (or approximately feasible) solutions, the algorithms have the fol-

We provide a review and synthesis of polyhedral approaches to machine scheduling problems. The choice of decision variables is the prime determinant of various formulations for such problems. Constraints, such as facet inducing inequalities for corresponding polyhedra, are often needed, in addition to those just required for the validity of the initial formulation, in order to obtain useful lower bounds and structural insights. We review formulations based on time–indexed variables; on linear ordering, start time and completion time variables; on assignment and positional date variables; and on traveling salesman variables. We point out relationship between various models, and provide a number of new results, as well as simplified new proofs of known results. In particular, we emphasize the important role that supermodular polyhedra and greedy algorithms play in many formulations and we analyze the strength of the lower and upper bounds obtained from different formulations and relaxations. We discuss separation algorithms for several classes of inequalities, and their potential applicability in generating cutting planes for the practical solution of such scheduling problems. We also review some recent results on approximation algorithms based on some of these formulations.

Airline crew scheduling is concerned with finding a minimum cost assignment of flight crews to a given flight schedule while satisfying restrictions dictated by collective bargaining agreements and the Federal Aviation Administration. Traditionally, the problem has been modeled as a set partitioning problem. In this paper, we present a new model based on breaking the decision process into two stages. In the first stage we select a set of duty periods that cover the flights in the schedule. Then, in the second stage, we attempt to build pairings using those duty periods. We suggest a decomposition approach for solving the model and present computational results for test problems provided by a major carrier. Our formulation provides a tighter linear programming bound than that of the conventional set partitioning formulation but is more difficult to solve. 1 Introduction In this paper we present a new formulation and decomposition approach for the airline crew scheduling problem. The ...

In this paper we describe a cutting plane algorithm for the Steiner tree packing problem. We use our algorithm to solve some switchbox routing problems of VLSI-design and report on our computational experience. This includes a brief discussion of separation algorithms, a new LP-based primal heuristic and implementation details. The paper is based on the polyhedral theory for the Steiner tree packing polyhedron developed in our companion paper [GMW92] and meant to turn this theory into an algoritmic tool for the solution of practical problems.

A general decomposition framework for large convex optimization problems based on augmented Lagrangians is described. The approach is then applied to multistage stochastic programming problems in two different ways: by decomposing the problem into scenarios or decomposing it into nodes corresponding to stages. In both cases the method has favorable convergence properties and a structure which makes it convenient for parallel computing environments. Keywords: Stochastic Programming, Decomposition, Augmented Lagrangian, Jacobi Method, Parallel Computation. iii iv On Augmented Lagrangian Decomposition Methods For Multistage Stochastic Programs Andrzej Ruszczy'nski 1 Introduction Multistage stochastic optimization problems belong to the most difficult problems of mathematical programming. Their size grows very quickly with the number of stages and with the number of events (scenarios) incorporated into the model. Although problems of this type occur frequently in applications (like,...