## Large-Scale Optimization of Eigenvalues (1991)

Venue: | SIAM J. Optimization |

Citations: | 78 - 3 self |

### BibTeX

@ARTICLE{Overton91large-scaleoptimization,

author = {Michael L. Overton},

title = {Large-Scale Optimization of Eigenvalues},

journal = {SIAM J. Optimization},

year = {1991},

volume = {2},

pages = {88--120}

}

### Years of Citing Articles

### OpenURL

### Abstract

Optimization problems involving eigenvalues arise in many applications. Let x be a vector of real parameters and let A(x) be a continuously differentiable symmetric matrix function of x. We consider a particular problem which occurs frequently: the minimization of the maximum eigenvalue of A(x), subject to linear constraints and bounds on x. The eigenvalues of A(x) are not differentiable at points x where they coalesce, so the optimization problem is said to be nonsmooth. Furthermore, it is typically the case that the optimization objective tends to make eigenvalues coalesce at a solution point. There are three main purposes of the paper. The first is to present a clear and self-contained derivation of the Clarke generalized gradient of the max eigenvalue function in terms of a "dual matrix". The second purpose is to describe a new algorithm, based on the ideas of a previous paper by the author (SIAM J. Matrix Anal. Appl. 9 (1988) 256-268), which is suitable for solving l...

### Citations

3265 | Convex Analysis
- ROCKAFELLAR
- 1996
(Show Context)
Citation Context ...le can be used to show that the maximum eigenvalue is a convex function of the matrix elements. Because of this fact, it has been recognized for some time that the techniques of convex analysis (e.g. =-=[Roc70]-=-) are applicable to eigenvalue optimization problems; optimality conditions and/or first-order algorithms for various problem classes have been given by [CDW75,PW83,Doy82,Sha85b,Gol87,All89]. See also... |

1346 |
Practical Optimization
- E, Murray, et al.
- 1981
(Show Context)
Citation Context ...5), (28) on d; ffi , together with the active bound restrictions on d, are linearly independent. Proof. It follows immediately from the standard optimality conditions for quadratic programs (see e.g. =-=[GMW81]-=-). 2 Theorem 9 Assumes= 0, so thats1 (x) has exact multiplicity t. The quadratic program (24)--(29) yields a vector d which is a descent direction fors1 , unless d = 0. Furthermore, if ae ? 0, then (d... |

1139 |
Geometric Algorithms and Combinatorial Optimizations
- Grotschel, Lovász, et al.
- 1993
(Show Context)
Citation Context ...ls, namely the three main diagonals and the Nth sub- and super-diagonal. 11 A Graph Problem The following problem was communicated to us by H. Schramm and J. Zowe; its origin may be found in [Lov79], =-=[GLS88]-=-. Given an undirected graph G, with vertices 1; . . . ; n, let M be an n \Theta n symmetric matrix with the restriction that its diagonal elements are zero and its offdiagonal elements (i; j) are zero... |

1081 |
Practical Methods of Optimization
- Fletcher
- 1981
(Show Context)
Citation Context ...f the dual matrix U defined by the multipliers characterizing a PLP "solution" is indefinite, the multiplicity tolerancesis reduced. The updating of the trust radius ae is based on recommend=-=ations in [Fle87]-=-. SPLP Algorithm to solve (10)--(11). Step 0. Initialize the trust radius ae and the multiplicity tolerances. Define a convergence tolerance ffl. Set x to an initial value satisfying (11). Compute the... |

956 |
Optimization and nonsmooth analysis
- Clarke
- 1990
(Show Context)
Citation Context ...neral case of minimizing sums of largest eigenvalues (algebraically or in absolute value), are given by [OWa]. These optimality conditions are derived by characterizing Clarke's generalized gradient (=-=[Cla83]-=-) in terms of a dual matrix. Proofs of the local quadratic convergence of the successive quadratic programming algorithm used in [Ove88] are being developed in [OWb]. There are three main purposes of ... |

764 |
The Symmetric Eigenvalue Problem
- Parlett
- 1980
(Show Context)
Citation Context ...mportant to maintain orthogonality of the r vectors during the iteration. The orthogonalized block versions of the power and inverse power methods are generally called subspace iteration; see Parlett =-=[Par80]-=- for details. The block of eigenvectors computed for the previous matrix iterate is a very valuable starting block for each subspace iteration after the first few optimization steps. The simplest vari... |

488 | On the limited memory BFGS method for large scale optimization
- Liu, Nocedal
- 1989
(Show Context)
Citation Context ... note that an SLP method may be regarded as an SQP method with a zero rank approximation to the quadratic term. An alternative idea is to approximate W using a limited memory quasi-Newton method; see =-=[LN89]-=-. In either case it seems probable that a practical SQP method could be devised which would converge faster than the SPLP method unless it had difficulty identifying the optimal multiplicity t . We co... |

460 |
An analysis of the finite element method
- Strang, Fix
- 1973
(Show Context)
Citation Context ...iefly summarize some of the computational results. We discretized the problem, approximating oe(x) by a piecewise constant function oe h , where h is the mesh size. Following the standard approach in =-=[SF73]-=-, we approximated u by u h , using piecewise cubic Hermite finite elements, and constructed the corresponding finite-dimensional bending matrix A(oe h ) and stiffness matrix B such that the eigenvalue... |

338 |
On the shannon capacity of a graph
- Lovász
- 1979
(Show Context)
Citation Context ...o diagonals, namely the three main diagonals and the Nth sub- and super-diagonal. 11 A Graph Problem The following problem was communicated to us by H. Schramm and J. Zowe; its origin may be found in =-=[Lov79]-=-, [GLS88]. Given an undirected graph G, with vertices 1; . . . ; n, let M be an n \Theta n symmetric matrix with the restriction that its diagonal elements are zero and its offdiagonal elements (i; j)... |

320 |
Solution of sparse indefinite systems of linear equations
- Paige, Saunders
- 1975
(Show Context)
Citation Context ...n the shift is good, perhaps because of instability resulting from the near singularity of sI \Gamma A(x). An alternative idea, following Szyld [Szy83], is to use the Paige and Saunders method SYMMLQ =-=[PS78]-=-. This may give better results than conjugate gradient for nearly singular positive definite systems. Szyld gives an argument explaining why the near singularity does not cause difficulty for SYMMLQ; ... |

159 | Analysis of Feedback Systems with Structured Uncertainties - Doyle - 1982 |

82 | Theory of extremal problems M - Ioffe, Tikhomirov - 1974 |

74 |
A Short Introduction to Perturbation Theory for Linear Operators
- Kato
- 1982
(Show Context)
Citation Context ...r positive semi-definite matrices with trace one. The result therefore follows from Theorem 1. 2 The formula for the directional derivative may alternatively be obtained from the classical results in =-=[Kat82]-=-, which state that the multiple eigenvalues1 = \Delta \Delta \Delta =st of A(x) splits into t eigenvalues of A(x + ffd), for ff near 0, with corresponding derivatives equal to the eigenvalues of B(d).... |

71 |
On minimizing the maximum eigenvalue of a symmetric matrix
- Overton
- 1988
(Show Context)
Citation Context .../or first-order algorithms for various problem classes have been given by [CDW75,PW83,Doy82,Sha85b,Gol87,All89]. See also [OT83,Bra86] for discussion of problems arising in structural engineering. In =-=[Ove88], a quadra-=-tically convergent algorithm was given to solve the model problem, using a "dual matrix" formulation of the optimality conditions to fully exploit the nonsmooth problem structure. Two papers... |

62 | Optimality conditions and duality theory for minimizing sums of the Lqest eigenvalues of symmetric matrices. Mathemicd Programming
- Overton, Wamersley
- 1993
(Show Context)
Citation Context ...e course of obtaining solutions to the column problem. The work on optimal preconditioners is joint with Anne Greenbaum. The presentation of the optimality conditions in Theorems 1 -- 3 is related to =-=[OWa]-=-, which is joint work with Rob Womersley. I would like to thank Helga Schramm and Jochem Zowe for providing me with details of the graph problems and the related performance of their bundle trust regi... |

60 |
Numerical Methods in Finite Element Analysis
- Bathe, Wilson
- 1976
(Show Context)
Citation Context ...pply exactly as before provided only that the normalization (46) is consistently used for the eigenvectors. Note that the details of subspace iteration are well known for the generalized problem; see =-=[BW76]-=-,[Par80, Ch. 15]. If a shift s is used, it is of course understood that A(x) is to be shifted by sB instead of sI. 8 Several Matrix Families Suppose it is desired to minimize OE(x) = max 1lp (l) 1 (x)... |

46 | The formulation and analysis of numerical methods for inverse eigenvalue problems
- Friedland, Nocedal, et al.
- 1987
(Show Context)
Citation Context ...ing the problem into a standard nonlinear programming form by means of determinants have been made [GT88], but these methods perform poorly [Pan89]; for other comments on the use of determinants, see =-=[FNO87]-=-. In the example given above, the maximum eigenvalue is convex in x. This is true in general when A depends linearly on x, since the Rayleigh principle can be used to show that the maximum eigenvalue ... |

44 | The minimization of certain nondifferentiable sums of eigenvalues of symmetric matrices - Cullum, Donath, et al. |

39 |
Applications of parametric programming and eigenvalue maximization to the quadratic assignment problem
- RENDL, WOLKOWICZ
- 1992
(Show Context)
Citation Context ...understood. The SQP algorithm of [Ove88], on which the new algorithm is based, has also been applied to some other applications not discussed in this paper, including the quadratic assignment problem =-=[RW91]-=-, the stability of Runge-Kutta methods for ordinary differential equations [Mul90] and optimal diagonal scaling of nonsymmetric matrices [Wat91]. Another application to which we hope to apply our larg... |

34 |
Finite algorithms in optimization and data analysis
- Osborne
- 1985
(Show Context)
Citation Context ...particularly useful for computation, since it does not involve taking a convex hull. Indeed, it characterizes the generalized gradient using structure functionals, to use a term introduced by Osborne =-=[Osb85]-=- for some other nonsmooth optimization problems. In our case, the structure functionals may be taken to be the t(t+1)=2 quantities q T i A(x)q j ; 1sisjst; (8) assuming the eigenvectors q 1 ; . . . ; ... |

26 |
Second derivatives for optimizing eigenvalues of symmetric matrices
- Overton, Womersley
- 1995
(Show Context)
Citation Context ...g Clarke's generalized gradient ([Cla83]) in terms of a dual matrix. Proofs of the local quadratic convergence of the successive quadratic programming algorithm used in [Ove88] are being developed in =-=[OWb]-=-. There are three main purposes of the present paper. The first is to present a clear and self-contained derivation of the generalized gradient of the max eigenvalue functional in terms of a dual matr... |

19 | On eigenvalues of matrices dependent on a parameter - Lancaster |

13 |
On minimizing the spectral radius of a nonsymmetric matrix function — optimality conditions and duality theory
- Overton, Wormley
- 1987
(Show Context)
Citation Context ...on is to minimize the maximum eigenvalue in absolute value. (We avoid the term spectral radius, since this suggests complex eigenvalues; nonsymmetric matrices are not discussed in this paper, but see =-=[OW88]-=-.) The model problem is directly applicable to many applications, including the first two mentioned above, while for other problems, e.g. those where only the constraints involve the eigenvalues, it i... |

12 | private communication - Boyd, Grinstein |

12 |
A projected Lagrangian algorithm for nonlinear ` 1 optimization
- Murray, Overton
- 1981
(Show Context)
Citation Context ...m of two smooth functions at x = 0; in other words, the eigenvalues themselves cannot be labeled, say,s1 ands2 , each a smooth function of x 2 ! 2 . Thus standard minmax optimization techniques (e.g. =-=[MO80]-=-) cannot be applied. Suggestions 2 for transforming the problem into a standard nonlinear programming form by means of determinants have been made [GT88], but these methods perform poorly [Pan89]; for... |

12 | Extremal problems on the set of nonnegative definite matrices - Shapiro - 1985 |

11 | On the optimal design of columns against buckling
- Cox, Overton
(Show Context)
Citation Context ...ied mathematicians and structural engineers, including [TK62], [OR77]. Our work on this problem is joint with Steve Cox; the details of our theoretical and computational contributions may be found in =-=[CO91]-=-. Here we briefly summarize some of the computational results. We discretized the problem, approximating oe(x) by a piecewise constant function oe h , where h is the mesh size. Following the standard ... |

11 | Eigenvalue perturbations and nonlinear parametric optimization - Gollan |

11 |
Optimal preconditioners of a given sparsity pattern
- GREENBAUM, RODRIGUE
- 1989
(Show Context)
Citation Context ...ely. Typically most of the active bounds were identified in just a few steps, with fine tuning of the active set taking place subsequently. 10 Design of Optimal Preconditioners Greenbaum and Rodrigue =-=[GR90]-=- have used our optimization programs to solve the following problem: given a positive definite symmetric matrix B, find the positive definite symmetric matrix M with prescribed sparsity pattern which ... |

10 |
Sequential Quadratic Programming Methods Based on Approximating a Projected Hessian
- Gurwitz
- 2001
(Show Context)
Citation Context ...of the trust radius is simply to restrict d so that its norm is not too large. In view of these remarks we have implemented the following "partial linear programming" solver. (For a related =-=idea, see [GO89]-=-.) PLP Algorithm to partially solve the LP ming Tsd (35) subject to Esd = e (36) F dsf (37) d k = 0; k 2 K (38) `sdsu (39) ksd k1 ae; (40) wheresd = (d T ; ffi ) T 2 ! m+1 , K is an index set, and g; ... |

10 |
Private communication
- Benjamini
- 2002
(Show Context)
Citation Context ...for the Shannon capacity of the graph [Lov79]. (The upper bound is sometimes called the Lovasz number of the graph.) 32 We applied our eigenvalue optimization algorithm to a test problem suggested by =-=[Sch89a]-=-. Given integers ffs1 and !s3, let n = ff! + 1 and define G to have the property that vertices i and j are adjacent if j \Gamma i ! ! or i + n \Gamma j ! !. The class of graphs with this property is d... |

10 | Private communication
- Zhou
(Show Context)
Citation Context ...sults from linearizing a differentiable system of t(t + 1)=2 nonlinear equations characterizing the conditions1 (x) = \Delta \Delta \Delta =st (x) = !, for some ! 2 !. Actually, as was pointed out by =-=[Zho88]-=-, the form of the nonlinear system given by (4.1) of [Ove88] is not 12 correct. The correct system uses a matrix exponential formulation based on Theorem 3.1 of [FNO87], as is explained in more detail... |

8 |
On single and bimodal optimum buckling loads of clamped columns
- Olhoff, Rasmussen
(Show Context)
Citation Context ...pplied at the ends of the column, in the direction of its axis.) The problem is a controversial one which has been addressed by many applied mathematicians and structural engineers, including [TK62], =-=[OR77]-=-. Our work on this problem is joint with Steve Cox; the details of our theoretical and computational contributions may be found in [CO91]. Here we briefly summarize some of the computational results. ... |

7 | Characterization and efficient computation of the structured singular value - Fan, Tits - 1986 |

6 |
Four quadratically convergent methods for solving inverse eigenvalue problems
- Friedland, Nocedal, et al.
- 1986
(Show Context)
Citation Context ... iterative method, since the basis Q 1 would tend to be little changed at each iteration. By contrast, when Eispack is used the basis Q 1 (x) does not generally converge as x ! x (see the examples in =-=[FNO86]), perhaps-=- giving a bundle which is more "rich" in the various possible values for the subgradients. Finally, we note that the dual matrix itself appears in the references [Lov79], [GLS88]. Indeed, th... |

6 |
User’s guide for LSSOL: A Fortran package for constrained linear least-squares and convex quadratic programming
- Gill, Hammarling, et al.
- 1986
(Show Context)
Citation Context ...entioned in later sections of the paper. Our Fortran implementations use Eispack subroutines [S + 67] to obtain the eigenvalues and eigenvectors of each matrix A(x) and either the Stanford code LSSOL =-=[GMSW86]-=- or the equivalent NAG routine [NAG] to solve the quadratic programs. Using current workstation technology, only a moderate amount of computer time is typically required to obtain a very accurate solu... |

6 | Eine Kombination von Bundle- und Trust-RegionVerfahren zur L"osung nichtdifferenzierbarer Optimierungsprobleme. Bayreuther mathematische schriften, 30, Fakult"at f"ur Mathematik und Physik, Universit"at - Schramm - 1989 |

6 |
Strongest columns and isoperimetric inequalities for eigenvalues
- Tadjbakhsh, Keller
(Show Context)
Citation Context ...oad is applied at the ends of the column, in the direction of its axis.) The problem is a controversial one which has been addressed by many applied mathematicians and structural engineers, including =-=[TK62]-=-, [OR77]. Our work on this problem is joint with Steve Cox; the details of our theoretical and computational contributions may be found in [CO91]. Here we briefly summarize some of the computational r... |

5 | On maximizing the minimum eigenvalue of a linear combination of symmetric matrices - Allwright - 1989 |

5 |
Semidefinite constraints in optimization
- Fletcher
- 1985
(Show Context)
Citation Context ... (4) is well known; see [CDW75,PW83,Cla83]. The equivalent form (6) is much less known and much more useful, as we shall see shortly; the earliest reference we know for this explicit form is Fletcher =-=[Fle85]-=-, where a different proof was given. Equation (7) was given in the case that A(x) is affine in [Ove88], using a proof based on Fletcher's work. The proofs given here make more use of the machinery of ... |

5 | The BT-algorithm for minimizing a non-smooth functional subject to linear constraints. non-smooth Optimization and Related - Zowe - 1989 |

4 | On structural optimization - OLHOFF, TAYLOR - 1983 |

4 | A nondifferentiable optimization algorithm for structural problems with eigenvalue inequality constraints - Polak, Wardi - 1983 |

4 |
An algorithm for optimal l 2 scaling of matrices
- Watson
- 1991
(Show Context)
Citation Context ...n this paper, including the quadratic assignment problem [RW91], the stability of Runge-Kutta methods for ordinary differential equations [Mul90] and optimal diagonal scaling of nonsymmetric matrices =-=[Wat91]-=-. Another application to which we hope to apply our large-scale algorithm is the computation of structured singular values in control [FT86,Wat90]. Perhaps the most important feature of our algorithms... |

3 |
On minimax eigenvalue problems via constrained optimization
- Goh, Teo
(Show Context)
Citation Context ... standard minmax optimization techniques (e.g. [MO80]) cannot be applied. Suggestions 2 for transforming the problem into a standard nonlinear programming form by means of determinants have been made =-=[GT88]-=-, but these methods perform poorly [Pan89]; for other comments on the use of determinants, see [FNO87]. In the example given above, the maximum eigenvalue is convex in x. This is true in general when ... |

3 |
Nonsmooth optimization
- Lemaréchal, Mifflin
- 1977
(Show Context)
Citation Context ... gradient which is estimated to apply at the limit point, not just the gradient information at the current iterate. This observation is the basis for the so-called "ffl -subgradient" methods=-= found in [LM78]-=-, and similarly it is the estimated optimal active constraint structure which underlies successive quadratic programming (SQP) methods for nonlinear programming. In the latter case this estimated stru... |

3 |
On the need for special purpose algorithms for minimax eigenvalue problems
- Panier
- 1989
(Show Context)
Citation Context ...(e.g. [MO80]) cannot be applied. Suggestions 2 for transforming the problem into a standard nonlinear programming form by means of determinants have been made [GT88], but these methods perform poorly =-=[Pan89]-=-; for other comments on the use of determinants, see [FNO87]. In the example given above, the maximum eigenvalue is convex in x. This is true in general when A depends linearly on x, since the Rayleig... |

3 | Optimal block diagonal l 2 -scaling of matrices - Shapiro - 1985 |

3 |
A two-level iterative method for large sparse generalized eigenvalue calculations
- Szyld
- 1983
(Show Context)
Citation Context ...ersion suggest that the method may give poor results when the shift is good, perhaps because of instability resulting from the near singularity of sI \Gamma A(x). An alternative idea, following Szyld =-=[Szy83]-=-, is to use the Paige and Saunders method SYMMLQ [PS78]. This may give better results than conjugate gradient for nearly singular positive definite systems. Szyld gives an argument explaining why the ... |

2 | Multiple eigenvalues in problems of optimizing the spectral properties of systems with a finite number of degrees of freedom, Zh. vychisl - Bratus |