this paper is as follows. In Section 2, we present some background material on general Krylov subspace methods, of which CGtype algorithms are a special case. We recall the outstanding properties of CG and discuss the issue of optimal extensions of CG to non-Hermitian matrices. We also review GMRES and related methods, as well as CG-like algorithms for the special case of Hermitian indefinite linear systems. Finally, we briefly discuss the basic idea of preconditioning. In Section 3, we turn to Lanczos-based iterative methods for general non-Hermitian linear systems. First, we consider the nonsymmetric Lanczos process, with particular emphasis on the possible breakdowns and potential instabilities in the classical algorithm. Then we describe recent advances in understanding these problems and overcoming them by using look-ahead techniques. Moreover, we describe the quasi-minimal residual algorithm (QMR) proposed by Freund and Nachtigal (1990), which uses the look-ahead Lanczos process to obtain quasi-optimal approximate solutions. Next, a survey of transposefree Lanczos-based methods is given. We conclude this section with comments on other related work and some historical remarks. In Section 4, we elaborate on CGNR and CGNE and we point out situations where these approaches are optimal. The general class of Krylov subspace methods also contains parameter-dependent algorithms that, unlike CG-type schemes, require explicit information on the spectrum of the coefficient matrix. In Section 5, we discuss recent insights in obtaining appropriate spectral information for parameter-dependent Krylov subspace methods. After that, 4 R.W. Freund, G.H. Golub and N.M. Nachtigal

Let A be a real symmetric positive definite matrix. We consider three particular questions, namely estimates for the error in linear systems Ax = b, minimizing quadratic functional min x (x T Ax \Gamma 2b T x) subject to the constraint k x k= ff, ff !k A \Gamma1 b k, and estimates for the entries of the matrix inverse A \Gamma1 . All of these questions can be formulated as a problem of finding an estimate or an upper and lower bound on u T F (A)u, where F (A) = A \Gamma1 resp. F (A) = A \Gamma2 , u is a real vector. This problem can be considered in terms of estimates in the Gauß-type quadrature formulas which can be effectively computed exploiting the underlying Lanczos process. Using this approach, we first recall the exact arithmetic solution of the questions formulated above and then analyze the effect of rounding errors in the quadrature calculations. It is proved that the basic relation between the accuracy of Gauß quadrature for f() = \Gamma1 and the rate of ...

The 3-term Lanczos process leads, for a symmetric matrix, to bases for Krylov subspaces of increasing dimension. The Lanczos basis, together with the recurrence coefficients, can be used for the solution of symmetric indefinite linear systems, by solving the reduced system in one way or another. This leads to well-known methods: MINRES, GMRES, and SYMMLQ. We will discuss in what way and to what extent these approaches differ in their sensitivity to rounding errors. In our analysis we will assume that the Lanczos basis is generated in exactly the same way for the different methods, and we will not consider the errors in the Lanczos process itself. We will show that the method of solution may lead, under certain circumstances, to large additional errors, that are not corrected by continuing the iteration process. Our findings are supported and illustrated by numerical examples. 1 Introduction We will consider iterative methods for the construction of approximate solutions, starting with...

This paper discusses iterative methods for the solution of very large severely ill-conditioned linear systems of equations that arise from the discretization of linear ill-posed problems. The right-hand side vector represents the given data and is assumed to be contaminated by errors. Solution methods proposed in the literature employ some form of filtering to reduce the influence of the error in the right-hand side on the computed approximate solution. The amount of filtering is determined by a parameter, often referred to as the regularization parameter. We discuss how the filtering affects the computed approximate solution and consider the selection of regularization parameter. Methods in which a suitable value of the regularization parameter is determined during the computation, without user intervention, are emphasized. New iterative solution methods based on expanding explicitly chosen filter functions in terms of Chebyshev polynomials are presented. The properties of these methods are illustrated with applications to image restoration.

We capitalize upon the known relationship between pairs of orthogonal and minimal residual methods (or, biorthogonal and quasi-minimal residual methods) in order to estimate how much smaller the residuals or quasi-residuals of the minimizing methods can be compared to the those of the corresponding Galerkin or Petrov-Galerkin method. Examples of such pairs are the conjugate gradient (CG) and the conjugate residual (CR) methods, the full orthogonalization method (FOM) and the generalized minimal residual (GMRes) method, the CGNE and CGNR versions of applying CG to the normal equations, as well as the biconjugate gradient (BiCG) and the quasi-minimal residual (QMR) methods. Also the pairs consisting of the (bi)conjugate gradient squared (CGS) and the transpose-free QMR (TFQMR) methods can be added to this list if the residuals at half-steps are included, and further examples can be created easily. The analysis is more generally applicable to the minimal residual (MR) and quasi-minimal residual (QMR) smoothing processes, which are known to provide the transition from the results of the first method of such a pair to those of the second one. By an interpretation of these smoothing processes in coordinate space we deepen the understanding of some of the underlying relationships and introduce a unifying framework for minimal residual and quasi-minimal residual smoothing. This framework includes the general notion of QMR-type methods.

We are concerned with the minimal residual method combined with polynomial preconditioning for solving large linear systems Ax.. b with indefinite Hermitian coefficient matrices A. The standard approach for choosing the polynomial preconditioner leads to preconditioned systems which are postive definite. Here, we investigate a different strategy which leaves the preconditioned coefficient matrix indefinite. More precisely, the polynomial preconditioner is designed to cluster the positive, resp. negative eigenvalues of A around 1, resp. around some negative constant. In particular, it is shown that such indefinite polynomial preconditioners can be obtained as the optimal solutions of a certain two-parameter family of Chebyshev approximation problems. We establish some basic results for these approximation problems and sketch a Remez type algorithm for their numerical solution. The problem of selecting the parameters such that the resulting indefinite polynomial preconditioner speeds up the convergence of minimal residual method optimally is alsoaddressed.Foxthistask,we proposeanapproachbased ontheconcept ofasymptoticonvergencefactors.Finally, some numericalexamplesofindefinite polynomial

The 3-term Lanczos process leads, for a symmetric matrix, to bases for Krylov subspaces of increasing dimension. The Lanczos basis, together with the recurrence coefficients, can be used for the solution of linear systems, by solving the reduced system in one way or another. This leads to well-known methods: MINRES (GMRES), CG, CR, and SYMMLQ. We will discuss in what way and to what extent the various approaches are sensitive to rounding errors. In our analysis we will assume that the Lanczos basis is generated in exactly the same way for the different methods (except CR), and we will not consider the errors in the Lanczos process itself. These errors may lead to large perturbations with respect to the exact process, but convergence takes still place. Our attention is focussed to what happens in the solution phase. We will show that the way of solution may lead, under circumstances, to large additional errors, that are not corrected by continuing the iteration process. Our findings are...

Mixed-hybrid nite element approximation of the potential uid ow problem leads to the solution of a large symmetric indenite system for the velocity and potential head vector components. Such discretization gives rise to a very accurate approximation of the continuity equation in every element, and for low order discretizations, the structural properties of the discrete matrix blocks allow cheap block elimination of the positive denite diagonal block and subsequent reduction to the Schur complement system for the pressure and Lagrangian vector components. This system is then frequently solved by the iterative conjugate gradient-type method. Whereas this approach is well known considerable less attention has been paid to the numerical stability aspects of such transformation. It was shown in [5] that block LU factorization can be unstable even if the system matrix is symmetric positive denite. In this paper we examine this type of conditional stability for a particular application ...

Large systems of linear equations arise frequently in numerical analysis and are the basis of many models in engineering and other applied sciences. This note provides a study for the solution of Hermitian linear systems. One particular feature which distinguishes this paper from the usual literature on polynomial based iteration methods is its emphasis on the properties of the underlying polynomials rather than more conventional matrix manipulations. In particular, a development and discussion of the properties of orthogonal polynomials leads to unified analysis of the state-of-the-art methods conjugate gradient and conjugate residual, respectively.

. Many iterative methods for solving linear systems, in particular the biconjugate gradient (BiCG) method and its \squared" version CGS (or BiCGS), produce often residuals whose norms decrease far from monotonously, but uctuate rather strongly. Large intermediate residuals are known to reduce the ultimately attainable accuracy of the method, unless special measures are taken to counteract this eect. One measure that has been suggested is residual smoothing: by application of simple recurrences, the iterates xn and the corresponding residuals rn : b Axn are replaced by smoothed iterates yn and corresponding residuals sn : b Ayn . We address the question whether the smoothed residuals can ultimately become markedly smaller than the primary ones. To investigate this, we present a roundo error analysis of the smoothing algorithms. It shows that the ultimately attainable accuracy of the smoothed iterates, measured in the norm of the corresponding residuals, is, in general, not higher t...