## Analysis of the finite precision Bi-Conjugate Gradient algorithm for nonsymmetric linear systems (1995)

### Cached

### Download Links

- [www.ams.org]
- [www.ams.org]
- [www.ams.org]
- [www-sccm.stanford.edu]
- DBLP

### Other Repositories/Bibliography

Venue: | Math. Comp |

Citations: | 10 - 4 self |

### BibTeX

@TECHREPORT{Tong95analysisof,

author = {Charles H. Tong and Qiang Ye},

title = {Analysis of the finite precision Bi-Conjugate Gradient algorithm for nonsymmetric linear systems},

institution = {Math. Comp},

year = {1995}

}

### OpenURL

### Abstract

Abstract. In this paper we analyze the bi-conjugate gradient algorithm in finite precision arithmetic, and suggest reasons for its often observed robustness. By using a tridiagonal structure, which is preserved by the finite precision bi-conjugate gradient iteration, we are able to bound its residual norm by a minimum polynomial of a perturbed matrix (i.e. the residual norm of the exact GMRES applied to a perturbed matrix) multiplied by an amplification factor. This shows that occurrence of near-breakdowns or loss of biorthogonality does not necessarily deter convergence of the residuals provided that the amplification factor remains bounded. Numerical examples are given to gain insights into these bounds. 1.

### Citations

353 | QMR: A quasi-minimal residual method for non-Hermitian linear systems
- Freund, Nachtigal
- 1991
(Show Context)
Citation Context ...nce its introduction by Lanczos [16] and later re-discovery by Fletcher [7] in its present form, the bi-conjugate gradient (BiCG) algorithm has evolved many variations (e.g. CGS, BiCGSTAB, QMR, CSBCG =-=[22, 25, 8, 2]-=-), each of which was specially designed to overcome some of its inherent difficulties (the need for adjoint matrix vector product, potential breakdowns, erratic convergence behavior, etc.). However, i... |

351 |
der Vorst, Bi-CGSTAB: a fast and smoothly converging variant of Bi-CG for the solution of nonsymmetric linear systems
- van
- 1992
(Show Context)
Citation Context ...nce its introduction by Lanczos [16] and later re-discovery by Fletcher [7] in its present form, the bi-conjugate gradient (BiCG) algorithm has evolved many variations (e.g. CGS, BiCGSTAB, QMR, CSBCG =-=[22, 25, 8, 2]-=-), each of which was specially designed to overcome some of its inherent difficulties (the need for adjoint matrix vector product, potential breakdowns, erratic convergence behavior, etc.). However, i... |

198 |
A fast Lanczos-type solver for nonsymmetric linear systems
- Sonneveld, CGS
- 1989
(Show Context)
Citation Context ...nce its introduction by Lanczos [16] and later re-discovery by Fletcher [7] in its present form, the bi-conjugate gradient (BiCG) algorithm has evolved many variations (e.g. CGS, BiCGSTAB, QMR, CSBCG =-=[22, 25, 8, 2]-=-), each of which was specially designed to overcome some of its inherent difficulties (the need for adjoint matrix vector product, potential breakdowns, erratic convergence behavior, etc.). However, i... |

186 |
Conjugate gradient methods for indefinite systems
- Fletcher
- 1976
(Show Context)
Citation Context ...at the amplification factor remains bounded. Numerical examples are given to gain insights into these bounds. 1. Introduction Since its introduction by Lanczos [16] and later re-discovery by Fletcher =-=[7]-=- in its present form, the bi-conjugate gradient (BiCG) algorithm has evolved many variations (e.g. CGS, BiCGSTAB, QMR, CSBCG [22, 25, 8, 2]), each of which was specially designed to overcome some of i... |

185 | Solution of systems of linear equations by minimized iterations
- Lanczos
- 1952
(Show Context)
Citation Context ...convergence of the residuals provided that the amplification factor remains bounded. Numerical examples are given to gain insights into these bounds. 1. Introduction Since its introduction by Lanczos =-=[16]-=- and later re-discovery by Fletcher [7] in its present form, the bi-conjugate gradient (BiCG) algorithm has evolved many variations (e.g. CGS, BiCGSTAB, QMR, CSBCG [22, 25, 8, 2]), each of which was s... |

86 | Analysis of some Krylov subspace approximations to the matrix exponential operator
- Saad
- 1992
(Show Context)
Citation Context ...(12). Note that the above lemma also holds if Tn is Hessenberg. In particular, if p is a polynomial of degree n − 1 (i.e. ψn = 0), then (12) becomes p(A)z1 = Znp(Tn)e1, a case that has been proved in =-=[17, 21]-=-. From this lemma, we have the following identity concerning rn+1. Lemma 3.3. Assume (14) AZn = ZnTn − 1 α ′ rn+1 n �r1� eTn with eT −1 n Tn e1 = α ′ n and r1 = �r1�z1, and assume that V T ∈ Rn×N is a... |

67 |
Analysis and Implementation of an Implicitly Restarted Iteration
- Lehoucq
- 1995
(Show Context)
Citation Context ...(12). Note that the above lemma also holds if Tn is Hessenberg. In particular, if p is a polynomial of degree n − 1 (i.e. ψn = 0), then (12) becomes p(A)z1 = Znp(Tn)e1, a case that has been proved in =-=[17, 21]-=-. From this lemma, we have the following identity concerning rn+1. Lemma 3.3. Assume (14) AZn = ZnTn − 1 α ′ rn+1 n �r1� eTn with eT −1 n Tn e1 = α ′ n and r1 = �r1�z1, and assume that V T ∈ Rn×N is a... |

49 |
Approximation theory and numerical linear algebra, in Algorithms for Approximation
- Trefethen
- 1990
(Show Context)
Citation Context ...ach n, ɛn =min p∈Pn,p(0)=1 �p(A + δAn)r1� is the nth residual norm of exact GMRES applied to the perturbed matrix A + δAn. Explicit bounds on ɛn have been discussed extensively in the literature; see =-=[24]-=- for example. Our numerical experiments show that the perturbation term δAn has little effect, i.e. usually ɛn ∼ min p∈Pn,p(0)=1 �p(A)r1�. Remark. If Tn is not too close to being singular, and the ele... |

46 |
Error analysis of the Lanczos algorithm for tridiagonalizing asymmetric matrix
- Paige
- 1976
(Show Context)
Citation Context ...al conditions. Finite precision analyses of conjugate gradient-type and Lanczos-type algorithms have played an important role in understanding these algorithms. The pioneering work is due to C. Paige =-=[19, 20]-=- and A. Greenbaum [12]. Paige showed in [19, 20] that the loss of orthogonality comes with but does not prevent convergence of the Ritz values, i.e., useful results can still be obtained from the algo... |

41 |
The convergence of inexact Chebyshev and Richardson iterative methods for solving linear systems
- Golub, Overton
- 1988
(Show Context)
Citation Context ...or Gene Golub for providing this opportunity and for his great hospitality. 1559 c○2000 American Mathematical Societys1560 CHARLES H. TONG AND QIANG YE hand, it has been observed by Golub and Overton =-=[9, 10]-=- that the preconditioned conjugate gradient method with inexact preconditioner, which amounts to relatively large perturbations to the CG recurrence, could still converge. These phenomena suggest that... |

33 |
Behavior of slightly perturbed Lanczos and conjugate-gradient recurrences
- Greenbaum
- 1989
(Show Context)
Citation Context ...als of degree not exceeding n. Lemma 3.2. Assume AZn = ZnTn − 1 α ′ rn+1 n �r1� eTn with eT n T −1 n e1 = α ′ n and r1 = �r1�z1. Then, for any polynomial p(x) = �n k=0 ψkxk of degree not exceeding n, =-=(12)-=- p(A)z1 = Znp(Tn)e1 + cnrn+1, where cn = −ψn(α1 ···αn�r1�) −1 .sTHE FINITE PRECISION BI-CONJUGATE GRADIENT ALGORITHM 1565 Proof. We first prove by induction that for any k with 1 ≤ k ≤ n − 1, A k Zne1... |

32 | An analysis of the composite step biconjugate gradient method
- Bank, Chan
- 1993
(Show Context)
Citation Context |

30 |
Accuracy and effectiveness of the Lanczos algorithm for the symmetric eigenproblem
- Paige
- 1980
(Show Context)
Citation Context ...al conditions. Finite precision analyses of conjugate gradient-type and Lanczos-type algorithms have played an important role in understanding these algorithms. The pioneering work is due to C. Paige =-=[19, 20]-=- and A. Greenbaum [12]. Paige showed in [19, 20] that the loss of orthogonality comes with but does not prevent convergence of the Ritz values, i.e., useful results can still be obtained from the algo... |

27 |
Predicting the behavior of finite precision Lanczos and conjugate gradient computations
- Greenbaum, Strakos
- 1992
(Show Context)
Citation Context ...e note that a recent work by Greenbaum, Druskin and Knizhnerman [15] on the finite precision CG also uses the approach of bounding the residuals. Other recent works on the finite precision CG include =-=[13, 14, 18, 25]-=-. The paper is organized as follows. In section 2, we review the BiCG algorithm and discuss its theoretical properties. We then present our results in section 3, withsTHE FINITE PRECISION BI-CONJUGATE... |

20 | Error analysis of the Lanczos algorithm for the nonsymmetric eigenvalue problem
- Bai
- 1994
(Show Context)
Citation Context ...an still be obtained from the algorithm even when the iterates deviate significantly from what would have been produced in exact arithmetic. A generalization to the nonsymmetric case was given by Bai =-=[1]-=-, and the near-breakdowns and loss of biorthogonality were discussed by Day [5]. Greenbaum established backward stability results in a generalized sense [12], showing that the iterative residuals prod... |

15 | Variable metric conjugate gradient methods
- Barth, Manteuffel
- 1994
(Show Context)
Citation Context ...1 −1 · · · · 1 −1 1 =[0,...,0, 1] and ⎞ ⎛ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ , Un = ⎜ ⎟ ⎜ ⎠ ⎝ 1 −β2 1 −β3 · · · · 1 −βn 1 Combining the two equations in (1), we obtain the governing equation ARn = Rn ˆ Tn − 1 rn+1e T n , =-=(3)-=- αn where ˆ Tn = LnΛ−1 n Un is an invertible tridiagonal matrix such that e T n ˆ T −1 n e1 = e T −1 −1 n Un ΛnLn e1 = e T n Λnv = αn, where v =[1 1 ··· 1] T . Similar results hold for the dual sequen... |

15 |
A comparative study of preconditioned lanczos methods for nonsymetric linear systems
- Tong
- 1992
(Show Context)
Citation Context ...e some of its inherent difficulties (the need for adjoint matrix vector product, potential breakdowns, erratic convergence behavior, etc.). However, it has been observed by Bank and Chan [2] and Tong =-=[23]-=- that, in many cases, BiCG may still be competitive (in terms of convergence and convergence rates), especially when coupled with no or relatively poor preconditioners. One major concern in using BiCG... |

14 |
A convergence analysis for nonsymmetric Lanczos algorithms
- Ye
- 1991
(Show Context)
Citation Context ... prove a posteriori residual bounds similar to those in [2] for the exact case, using an approach that is based on a tridiagonal structure implicit in the algorithm. This approach was also used by Ye =-=[27]-=- to analyze convergence of the Lanczos algorithms for eigenvalue problems. An advantage of analyzing BiCG using its tridiagonal structure is that our results include the finite precision case and the ... |

13 | der Vorst, The convergence behaviour of preconditioned CG and CGS in the presence of rounding errors, in Preconditioned conjugate gradient methods - van - 1990 |

9 |
On the convergence rate of the Conjugate Gradients in the presence of rounding errors
- NOTAY
- 1993
(Show Context)
Citation Context ...e note that a recent work by Greenbaum, Druskin and Knizhnerman [15] on the finite precision CG also uses the approach of bounding the residuals. Other recent works on the finite precision CG include =-=[13, 14, 18, 25]-=-. The paper is organized as follows. In section 2, we review the BiCG algorithm and discuss its theoretical properties. We then present our results in section 3, withsTHE FINITE PRECISION BI-CONJUGATE... |

8 |
Convergence of a two-stage Richardson iterative procedure for solving systems of linear equations
- Golub, Overton
- 1981
(Show Context)
Citation Context ...or Gene Golub for providing this opportunity and for his great hospitality. 1559 c○2000 American Mathematical Societys1560 CHARLES H. TONG AND QIANG YE hand, it has been observed by Golub and Overton =-=[9, 10]-=- that the preconditioned conjugate gradient method with inexact preconditioner, which amounts to relatively large perturbations to the CG recurrence, could still converge. These phenomena suggest that... |

6 |
Accuracy of computed solutions from conjugate-gradient-like methods, in Advances in Numerical Methods for Large Sparse Sets of Linear Systems (M. Natori and T. Nodera, eds), number 10 in 'Parallel Processing for Scientific Computing
- Greenbaum
- 1994
(Show Context)
Citation Context ...e note that a recent work by Greenbaum, Druskin and Knizhnerman [15] on the finite precision CG also uses the approach of bounding the residuals. Other recent works on the finite precision CG include =-=[13, 14, 18, 25]-=-. The paper is organized as follows. In section 2, we review the BiCG algorithm and discuss its theoretical properties. We then present our results in section 3, withsTHE FINITE PRECISION BI-CONJUGATE... |

4 | Semi-duality in the two-sided Lanczos algorithm
- Day
- 1993
(Show Context)
Citation Context ...antly from what would have been produced in exact arithmetic. A generalization to the nonsymmetric case was given by Bai [1], and the near-breakdowns and loss of biorthogonality were discussed by Day =-=[5]-=-. Greenbaum established backward stability results in a generalized sense [12], showing that the iterative residuals produced by the finite precision conjugate gradient algorithm are equivalent to wha... |

4 |
personal communication
- GREENBAUM
- 2003
(Show Context)
Citation Context ...e are not aware of any such generalization. One analysis is given by Cullum and Greenbaum [4], which relates BiCG type methods to QMR. We note that a recent work by Greenbaum, Druskin and Knizhnerman =-=[15]-=- on the finite precision CG also uses the approach of bounding the residuals. Other recent works on the finite precision CG include [13, 14, 18, 25]. The paper is organized as follows. In section 2, w... |

4 |
Predicting the behaviour of finite precision Lanczos and conjugate gradient computations
- Greenbaum, Strakoˇs
- 1992
(Show Context)
Citation Context ...) are equivalent to what would have been produced by applying the exact CG to a larger matrix. Thus the behavior of the finite precision case can be understood from what we know in the exact case (cf =-=[12]-=-). Some estimates on the larger matrix were given but may vary from step to step. We remark that it is necessary to consider the backward stability in this new sense because CG is not backward stable ... |

1 |
Relation between Galerkian and norm-minimizing iterative methods for solving linear systems
- Cullum, Greenbaum
- 1996
(Show Context)
Citation Context ...o step. It would be interesting to see if Greenbaum’s backward stability results can be generalized to BiCG; we are not aware of any such generalization. One analysis is given by Cullum and Greenbaum =-=[4]-=-, which relates BiCG type methods to QMR. We note that a recent work by Greenbaum, Druskin and Knizhnerman [15] on the finite precision CG also uses the approach of bounding the residuals. Other recen... |

1 |
A Talk given at the Linear and Nonlinear
- Manteuffel, Barth
- 1995
(Show Context)
Citation Context ...act arithmetic, approximation bounds on errors (or residuals) in the quantities computed by the BiCG process have been obtained by Bank and Chan [2]. It was also shown recently by Barth and Manteufel =-=[15]-=- that the BiCG residual indeed gives optimal approximation from Krylov subspaces considered in certain norm. Since proving these results [2] relies on the Galerkin condition (i.e., bi-orthogonality), ... |