Many advances in the development of Krylov subspace methods for the iterative solution of linear systems during the last decade and a half are reviewed. These new developments include different versions of restarted, augmented, deflated, flexible, nested, and inexact methods. Also reviewed are methods specifically tailored to systems with special properties such as special forms of symmetry and those depending on one or more parameters.

We present a general analytical model which describes the superlinear convergence of Krylov subspace methods. We take an invariant subspace approach, so that our results apply also to inexact methods, and to non-diagonalizable matrices. Thus, we provide a unified treatment of the superlinear convergence of GMRES, Conjugate Gradients, block versions of these, and inexact subspace methods. Numerical experiments illustrate the bounds obtained.

Many machine learning algorithms require the summation of Gaussian kernel functions, an expensive operation if implemented straightforwardly. Several methods have been proposed to reduce the computational complexity of evaluating such sums, including tree and analysis based methods. These achieve varying speedups depending on the bandwidth, dimension, and prescribed error, making the choice between methods difficult for machine learning tasks. We provide an algorithm that combines tree methods with the Improved Fast Gauss Transform (IFGT). As originally proposed the IFGT suffers from two problems: (1) the Taylor series expansion does not perform well for very low bandwidths, and (2) parameter selection is not trivial and can drastically affect performance and ease of use. We address the first problem by employing a tree data structure, resulting in four evaluation methods whose performance varies based on the distribution of sources and targets and input parameters such as desired accuracy and bandwidth. To solve the second problem, we present an online tuning approach that results in a black box method that automatically chooses the evaluation method and its parameters to yield the best performance for the input data, desired accuracy, and bandwidth. In addition, the new IFGT parameter selection approach allows for tighter error bounds. Our approach chooses the fastest method at negligible additional cost, and has superior performance in comparisons with previous approaches. 1

There are classes of linear problems for which the matrix-vector product is a time consuming operation because an expensive approximation method is required to compute it to a given accuracy. In recent years di#erent authors have investigated the use of, what is called, relaxation strategies for various Krylov subspace methods. These relaxation strategies aim to minimize the amount of work that is spent in the computation of the matrix-vector product without compromising the accuracy of the method or the convergence speed too much. In order to achieve this goal, the accuracy of the matrix-vector product is decreased when the iterative process comes closer to the solution. In this paper we show that a further significant reduction in computing time can be obtained by combining a relaxation strategy with the nesting of inexact Krylov methods. Flexible Krylov subspace methods allow variable preconditioning and therefore can be used in the outer most loop of our overall method. We analyze for several flexible Krylov methods strategies for controlling the accuracy of both the inexact matrix-vector products and of the inner iterations. The results of our analysis will be illustrated with an example that models global ocean circulation.

By using a combination of 32-bit and 64-bit floating point arithmetic the performance of many sparse linear algebra algorithms can be significantly enhanced while maintaining the 64-bit accuracy of the resulting solution. These ideas can be applied to sparse multifrontal and supernodal direct techniques and sparse iterative techniques such as Krylov subspace methods. The approach presented here can apply not only to conventional processors but also to exotic technologies such as

Abstract. We consider a preconditioned Krylov subspace iterative algorithm presented by Faul, Goodsell, and Powell (IMA J. Numer. Anal. 25 (2005), pp. 1–24) for computing the coefficients of a radial basis function interpolant over N data points. This preconditioned Krylov iteration has been demonstrated to be extremely robust to the distribution of the points and the iteration rapidly convergent. However, the iterative method has several steps whose computational and memory costs scale as O(N 2), both in preliminary computations that compute the preconditioner and in the matrix-vector product involved in each step of the iteration. We effectively accelerate the iterative method to achieve an overall cost of O(N log N). The matrix vector product is accelerated via the use of the fast multipole method. The preconditioner requires the computation of a set of closest points to each point. We develop an O(N log N) algorithm for this step as well. Results are presented for multiquadric interpolation in R 2 and biharmonic interpolation in R 3. A novel FMM algorithm for the evaluation of sums involving multiquadric functions in R 2 is presented as well.

Gaussian processes (GP) allow the treatment of non-linear non-parametric regression problems in a Bayesian framework [6]. Unfortunately its nonparametric nature causes computational problems for large data sets, due to an unfavorable O(N 3) time and O(N 2) memory scaling for training. The key computational task involves inversion of an N × N covariance matrix K + σ 2 I, where [K]ij = K(xi, xj), K is the covariance function of the GP, and σ 2 is the noise variance. Direct computation of the inverse requires O(N 3) operations and O(N 2) storage, which is impractical even for problems of moderate size (typically a few thousands). An important subfield of work in GP has attempted to bring this scaling down to O � m 2 N � by making sparse

Abstract not found

This work is the follow-up of the experimental study presented in [3]. It is based on and extends some theoretical results in [15, 18]. In a backward error framework we study the convergence of GMRES when the matrixvector products are performed inaccurately. This inaccuracy is modeled by a perturbation of the original matrix. We prove the convergence of GMRES when the perturbation size is proportional to the inverse of the computed residual norm; this implies that the accuracy can be relaxed as the method proceeds which gives rise to the terminology relaxed GMRES. As for the exact GMRES we show under proper assumptions that only happy breakdowns can occur. Furthermore the convergence can be detected using a by-product of the algorithm. We explore the links between relaxed right-preconditioned GMRES and flexible GMRES. In particular this enables us to derive a proof of convergence of FGMRES. Finally we report results on numerical experiments to illustrate the behaviour of the relaxed GMRES monitored by the proposed relaxation strategies. 1

Recent computational and theoretical studies have shown that the matrix-vector product occurring at each step of a Krylov subspace method can be relaxed as the iterations proceed, i.e., it can be computed in a less exact manner, without degradation of the overall performance. In the present paper a general operator treatment of this phenomenon is provided and a new result further explaining its behavior is presented. 1