## Fast Linear Iterations for Distributed Averaging (2003)

### Cached

### Download Links

Venue: | Systems and Control Letters |

Citations: | 193 - 11 self |

### BibTeX

@ARTICLE{Xiao03fastlinear,

author = {Lin Xiao and Stephen Boyd},

title = {Fast Linear Iterations for Distributed Averaging},

journal = {Systems and Control Letters},

year = {2003},

volume = {53},

pages = {65--78}

}

### Years of Citing Articles

### OpenURL

### Abstract

We consider the problem of finding a linear iteration that yields distributed averaging consensus over a network, i.e., that asymptotically computes the average of some initial values given at the nodes. When the iteration is assumed symmetric, the problem of finding the fastest converging linear iteration can be cast as a semidefinite program, and therefore efficiently and globally solved. These optimal linear iterations are often substantially faster than several common heuristics that are based on the Laplacian of the associated graph.

### Citations

3667 |
L.: Convex Optimization
- BOYD, VANDENBERGHE
- 2004
(Show Context)
Citation Context ...− 11 T /n W T − 11 T /n sI W ∈ S, 1 T W = 1 T , W 1 = 1. Here the symbol � denotes matrix inequality, i.e., X � Y means that X − Y is positive semidefinite. For background on SDP and LMIs, see, e.g., =-=[1, 2, 11, 12, 16, 41, 42, 43]-=-. Related background on eigenvalue optimization can be found in, e.g., [10, 22, 36]. Similarly, the symmetric FDLA problem (15) can be expressed as the SDP with variables s ∈ R and W ∈ R n×n . minimiz... |

3267 | Convex Analysis
- Rockafellar
- 1970
(Show Context)
Citation Context ...ient; but when r is not differentiable at w, it can have multiple subgradients. Subgradients play a key role in convex analysis, and are used in several algorithms for convex optimization (see, e.g., =-=[36, 38, 19, 4, 7]-=-). We can compute a subgradient of r at w as follows. If r(w) = 2(W) and u is the associated unit eigenvector, then a subgradient g is given by g!=-(ui-uj) 2, l{i,j}, l=l,...,m. Similarly, if r(w) = A... |

1533 |
Distributed Algorithms
- Lynch
- 1996
(Show Context)
Citation Context ...s of autonomous agents, in particular, the consensus or agreement problem among the agents. Distributed consensus problems have been studied extensively in the computer science literature (see, e.g., =-=[22]-=-). Recently it has found a wide range of applications, in areas such as formation flight of unmanned air vehicles and clustered satellites, and coordination of mobile robots. The recent paper [31] stu... |

770 | Semidefinite Programming
- Vandenberghe, Boyd
- 1996
(Show Context)
Citation Context ... subject to W T - 11T/r sI - W$, 1TW--I T, W1--1. o (16) Here the symbol _ denotes matrix inequality, i.e., X _ Y means that X - Y is positive semidefinite. For background on SDP and LMIs, see, e.g., =-=[10, 1, 40, 15, 41, 2, 39, 11]-=-. Related background on eigenvalue optimization can be found in, e.g., [34, 9, 21]. Similarly, the symmetric FDLA problem (15) can be expressed as the SDP minimize s subject to-sI _ W - 11T/r _ sI (17... |

764 |
The Symmetric Eigenvalue Problem
- Parlett
- 1980
(Show Context)
Citation Context ...e formulas can be found in [8]. For large sparse symmetric matrices W, we can compute a few extreme eigenvalues and their corresponding eigenvectots very efficiently using Lanczos methods (see, e.g., =-=[35, 37]-=-). Thus, we can compute a subgradient of r very efficiently. The subgradient method is very simple: given a feasible w (1) (e.g., from the maximum-degree or local-degree heuristics) k:--1 repeat 1. Co... |

745 |
Nonlinear Programming
- Bertsekas
- 2004
(Show Context)
Citation Context ...ient; but when r is not differentiable at w, it can have multiple subgradients. Subgradients play a key role in convex analysis, and are used in several algorithms for convex optimization (see, e.g., =-=[36, 38, 19, 4, 7]-=-). We can compute a subgradient of r at w as follows. If r(w) = 2(W) and u is the associated unit eigenvector, then a subgradient g is given by g!=-(ui-uj) 2, l{i,j}, l=l,...,m. Similarly, if r(w) = A... |

608 | Coordination of groups of mobile autonomous agents using nearest neighbor rules
- Jadbabaie, Lin, et al.
- 2003
(Show Context)
Citation Context ...cent paper [31] studies linear and nonlinear consensus protocols in these new applications with fixed network topology. Related coordination problems with time-varying topologies have been studied in =-=[20]-=- using a switched linear system model. In these previous works, the edge weights used in the linear consensus protocols are either constant or only dependent on the degrees of their incident nodes. Wi... |

529 |
Algebraic Graph Theory
- Biggs
- 1993
(Show Context)
Citation Context ... protocols are either constant or only dependent on the degrees of their incident nodes. With these simple methods of choosing edge weights, many concepts and tools from algebraic graph theory (e.g., =-=[5, 16]-=-), in particular the Laplacian matrix of the associated graph, appear to be very usetiff in the convergence analysis of consensus protocols [31]. The graph Laplacian has also been used in control of d... |

473 | Primal-dual interior-point methods for semidefinite programming: convergence rates, stability and numerical results
- Alizadeh, Haeberly, et al.
- 1998
(Show Context)
Citation Context ... subject to W T - 11T/r sI - W$, 1TW--I T, W1--1. o (16) Here the symbol _ denotes matrix inequality, i.e., X _ Y means that X - Y is positive semidefinite. For background on SDP and LMIs, see, e.g., =-=[10, 1, 40, 15, 41, 2, 39, 11]-=-. Related background on eigenvalue optimization can be found in, e.g., [34, 9, 21]. Similarly, the symmetric FDLA problem (15) can be expressed as the SDP minimize s subject to-sI _ W - 11T/r _ sI (17... |

430 |
Convex Analysis and Minimization Algorithms
- Hiriart-Urruty, Lemaréchal
- 1993
(Show Context)
Citation Context ...ient; but when r is not differentiable at w, it can have multiple subgradients. Subgradients play a key role in convex analysis, and are used in several algorithms for convex optimization (see, e.g., =-=[36, 38, 19, 4, 7]-=-). We can compute a subgradient of r at w as follows. If r(w) = 2(W) and u is the associated unit eigenvector, then a subgradient g is given by g!=-(ui-uj) 2, l{i,j}, l=l,...,m. Similarly, if r(w) = A... |

427 |
Numerical Methods for Large Eigenvalue Problems
- Saad
- 1992
(Show Context)
Citation Context ...e formulas can be found in [8]. For large sparse symmetric matrices W, we can compute a few extreme eigenvalues and their corresponding eigenvectots very efficiently using Lanczos methods (see, e.g., =-=[35, 37]-=-). Thus, we can compute a subgradient of r very efficiently. The subgradient method is very simple: given a feasible w (1) (e.g., from the maximum-degree or local-degree heuristics) k:--1 repeat 1. Co... |

394 | Algebraic graph theory - Godsil, Royle - 2001 |

341 | Interior point polynomial algorithms in convex programming - Nesterov, Nemirovskii - 1994 |

273 |
Minimization Methods for Nondifferentiable Functions
- Shor
- 1985
(Show Context)
Citation Context |

249 |
Information flow and cooperative control of vehicle formations”, submitted to
- Fax, Murray
- 2003
(Show Context)
Citation Context ...f the associated graph, appear to be very usetiff in the convergence analysis of consensus protocols [31]. The graph Laplacian has also been used in control of distributed dynamic systems (see, e.g., =-=[12, 13, 24]-=-). The FDLA problem (4) is closely related to the problem of finding the fastest mixing Markov chain on a graph [8]; the only difference is that in the FDLA problem, the weights can be (and the optima... |

221 |
Stability of multiagent systems with timedependent communication links
- Moreau
- 2005
(Show Context)
Citation Context ...nsus protocols in these new applications with fixed network topology. Related coordination problems with time-varying topologies have been studied in [21] using a switched linear system model, and in =-=[27]-=- using set-valued Lyapunov theory. In previous work, the edge weights used in the linear consensus protocols are either constant or only dependent on the degrees of their incident nodes. With these si... |

218 |
Interior Point Algorithms, Theory and Analysis
- Ye
- 1997
(Show Context)
Citation Context ...any weights are negative. 11 6 Computational methods 6.1 Interior-point method Standard interior-point algorithms for solving SDPs work well for problems with up to a thousand or so edges (see, e.g., =-=[29, 1, 40, 42, 41, 39, 11]-=-). The particular structure of the SDPs encountered in FDLA problems can be exploited for some gain in efficiency, but problems with more than a few thousand edges are probably beyond the capabilities... |

153 | A rank minimization heuristic with application to minimum order system approximation
- Fazel, Hindi, et al.
- 2001
(Show Context)
Citation Context ...is also possible to assign weights to the edges, to achieve (hopefully) some desired sparsity pattern. More sophisticated heuristics for sparse design and minimum rank problems can be found in, e.g., =-=[14]-=-. 15 To demonstrate this idea, we applied thesheuristic (29) to the example described in 5.1. We set the guaranteed convergence factor r TM z 0.910, which is only slightly larger than the minimum fact... |

138 | A spectral bundle method for semidefinite programming
- Helmberg, Rendl
(Show Context)
Citation Context ... lim_/3 -- 0 and oos/3 z oo. The convergence of this algorithm is proved in [38, 2.2]. Some closely related methods for solving large-scale SDPs and eigenvalue problems are the spectral bundle method =-=[18]-=- and a prox-method [28]; see also [32]. To demonstrate the subgradient method, we apply it to a large-scale network with 10000 nodes and 100000 edges. The graph is generated as follows. First we gener... |

136 | Convex Analysis and Nonlinear Optimization, Theory and Examples
- Borwein, Lewis
- 2000
(Show Context)
Citation Context |

114 | Solving large-scale sparse semidefinite programs for combinatorial optimization
- Benson, Ye, et al.
(Show Context)
Citation Context ...ese formulas are derived using equation (27). The structure exploited here is similar to the methods used in the dual-scaling algorithm for large-scale combinatorial optimization problems, studied in =-=[3]-=-. The total costs of this step (number of flops) is on the order of m 2 (negligible compared with step I and 3). 3. Compute the Newton step-H-lg by Cholesky factorization and back substitution. The co... |

104 | Semidefinite optimization
- Todd
- 2001
(Show Context)
Citation Context ... subject to W T - 11T/r sI - W$, 1TW--I T, W1--1. o (16) Here the symbol _ denotes matrix inequality, i.e., X _ Y means that X - Y is positive semidefinite. For background on SDP and LMIs, see, e.g., =-=[10, 1, 40, 15, 41, 2, 39, 11]-=-. Related background on eigenvalue optimization can be found in, e.g., [34, 9, 21]. Similarly, the symmetric FDLA problem (15) can be expressed as the SDP minimize s subject to-sI _ W - 11T/r _ sI (17... |

89 | Eigenvalue optimization
- Lewis, Overton
- 1996
(Show Context)
Citation Context ... i.e., X _ Y means that X - Y is positive semidefinite. For background on SDP and LMIs, see, e.g., [10, 1, 40, 15, 41, 2, 39, 11]. Related background on eigenvalue optimization can be found in, e.g., =-=[34, 9, 21]-=-. Similarly, the symmetric FDLA problem (15) can be expressed as the SDP minimize s subject to-sI _ W - 11T/r _ sI (17) W$, W--W T, W1--1, with variables s G R and W G R nXn. 4 Heuristics based on the... |

89 | Fastest mixing markov chain on a graph - Boyd, Diaconis, et al. |

78 | Large-scale optimization of eigenvalues
- Overton
- 1992
(Show Context)
Citation Context ...rgence of this algorithm is proved in [38, 2.2]. Some closely related methods for solving large-scale SDPs and eigenvalue problems are the spectral bundle method [18] and a prox-method [28]; see also =-=[32]-=-. To demonstrate the subgradient method, we apply it to a large-scale network with 10000 nodes and 100000 edges. The graph is generated as follows. First we generate a 10000 by 10000 symmetric matrix ... |

69 |
Laplacian matrices of graphs: A survey, Linear Algebra Appl
- Merris
- 1994
(Show Context)
Citation Context ...s not depend on the choice of reference directions). The Laplacian matrix is a useful tool in algebraic graph theory, and its eigenstructure reveals many important properties of the graph (see, e.g., =-=[17, 24]-=-). We note for future use that L is positive semidefinite, and since our graph is assumed connected, L has a simple eigenvalue zero, with corresponding eigenvector 1. We can use the incidence matrix t... |

67 | Consensus protocols for networks of dynamic agents
- Olfati-Saber, Murray
- 2003
(Show Context)
Citation Context ...g., [22]). Recently it has found a wide range of applications, in areas such as formation flight of unmanned air vehicles and clustered satellites, and coordination of mobile robots. The recent paper =-=[31]-=- studies linear and nonlinear consensus protocols in these new applications with fixed network topology. Related coordination problems with time-varying topologies have been studied in [20] using a sw... |

64 | Nonlinear programming, 2nd edition. Athena Scientific - Bertsekas |

63 | Algebraic Graph Theory, Graduate Texts - Godsil, Royle |

62 | Optimality conditions and duality theory for minimizing sums of the Lqest eigenvalues of symmetric matrices. Mathemicd Programming
- Overton, Wamersley
- 1993
(Show Context)
Citation Context ... i.e., X _ Y means that X - Y is positive semidefinite. For background on SDP and LMIs, see, e.g., [10, 1, 40, 15, 41, 2, 39, 11]. Related background on eigenvalue optimization can be found in, e.g., =-=[34, 9, 21]-=-. Similarly, the symmetric FDLA problem (15) can be expressed as the SDP minimize s subject to-sI _ W - 11T/r _ sI (17) W$, W--W T, W1--1, with variables s G R and W G R nXn. 4 Heuristics based on the... |

60 | Ghaoui, \Method of centers for minimizing generalized eigenvalues
- Boyd, El
- 1993
(Show Context)
Citation Context ... i.e., X � Y means that X − Y is positive semidefinite. For background on SDP and LMIs, see, e.g., [1, 2, 11, 12, 16, 41, 42, 43]. Related background on eigenvalue optimization can be found in, e.g., =-=[10, 22, 36]-=-. Similarly, the symmetric FDLA problem (15) can be expressed as the SDP with variables s ∈ R and W ∈ R n×n . minimize s subject to −sI � W − 11 T /n � sI W ∈ S, W = W T , W 1 = 1, 6 � � 0 (13) (14) (... |

58 | NP-hardness of some linear control design problems
- Blondel, Tsitsiklis
- 1997
(Show Context)
Citation Context ....e., the spectral radius of a matrix, is in general not a convex function; indeed it is not even Lipschitz continuous (see, e.g., [33]). Some related spectral radius minimization problems are NP-hard =-=[6, 27]-=-. We can also formulate the FDLA problem, with per-step convergence factor, as the following spectral norm minimization problem: minimize IlW- 11T/II subject to W,S, lrW--1 r, W1--1. In contrast to th... |

58 |
Algebraic graph theory, volume 207 of Graduate Texts in Mathematics
- Godsil, Royle
- 2001
(Show Context)
Citation Context ... protocols are either constant or only dependent on the degrees of their incident nodes. With these simple methods of choosing edge weights, many concepts and tools from algebraic graph theory (e.g., =-=[5, 16]-=-), in particular the Laplacian matrix of the associated graph, appear to be very usetiff in the convergence analysis of consensus protocols [31]. The graph Laplacian has also been used in control of d... |

42 |
Laplacian matrices of graphs: a survey
- Merris
- 1994
(Show Context)
Citation Context ...s not depend on the choice of reference directions). The Laplacian matrix is a useful tool in algebraic graph theory, and its eigenstructure reveals many important properties of the graph (see, e.g., =-=[23, 16]-=-). We note for future use that L is positive semidefinite, and since our graph is assumed connected, L has a simple eigenvalue zero, with corresponding eigenvector 1. We can use the incidence matrix t... |

42 | The Symmetric Eigenvalue Problem, Prentice-Hall - Parlett - 1980 |

41 | Algebraic Graph Theory, 2nd edition - Biggs - 1993 |

37 |
Several NP-hard problems arising in robust stability analysis. Mathematics of Control, Signals and Systems
- Nemirovskii
- 1993
(Show Context)
Citation Context ....e., the spectral radius of a matrix, is in general not a convex function; indeed it is not even Lipschitz continuous (see, e.g., [33]). Some related spectral radius minimization problems are NP-hard =-=[6, 27]-=-. We can also formulate the FDLA problem, with per-step convergence factor, as the following spectral norm minimization problem: minimize IlW- 11T/II subject to W,S, lrW--1 r, W1--1. In contrast to th... |

34 | Semide nite programming - Vandenberghe, Boyd - 1996 |

33 | Bayesian analysis for reversible Markov chains
- Diaconis, Rolles
(Show Context)
Citation Context ...d = 1 , (25) provided the graph is not bipartite. Compared with the optimal weights, the maximum-degree weights often lead to much slower convergence when there are bottle-neck links in the graph. In =-=[8]-=-, we give an example of two complete graphs connected by a bridge, where the optimal weight matrix W ⋆ can perform arbitrarily better than the maximum-degree weights, in the sense that the ratio (1 − ... |

31 | S.: ‘Low-authority controller design via convex optimization
- HASSIBI, HOW, et al.
- 1999
(Show Context)
Citation Context ...he convergence factor. This is a difficult combinatorial problem, but one very effective heuristic to achieve this goal is to minimize the 1 norm of the vector of edge weights; see, e.g., [11, 6] and =-=[17]-=-. For example, given the maximum allowed asymptotic convergence factor r TM, the 1 heuristic for the sparse graph design problem (with symmetric edge weights) can be posed as the convex problem minimi... |

24 | Minimization Methods for Nondi#erentiable Functions - Shor - 1985 |

19 |
Minimum Entropy H∞ Control
- Mustafa, Glover
- 1990
(Show Context)
Citation Context ... those that result in asymptotic convergence and minimize the logarithmic barrier function log det(I - 11T/n + W) - + log det(I + 11,/ - W) -. (28) (The terminology follows control theory; see, e.g., =-=[26]-=-.) In terms of the eigenvalues hi of W, the central weights minimize the objective 1og 1 14 0.75 0.7 0.65 0.6 0.55 0.5 0.45 0 ' A' W 1 O0 150 200 250 300 350 400 iteration number Figure 5: Progress of... |

14 | Interior point methods in semide nite programming with applications to combinatorial optimization - Alizadeh - 1995 |

13 |
Convergent powers of a matrix with applications to iterative methods for singular linear systems
- Meyer, Plemmons
- 1977
(Show Context)
Citation Context ...r matrix T such that W = T � Iκ 0 0 Z where Iκ is the κ-dimensional identity matrix (0 ≤ κ ≤ n) and Z is a convergent matrix, i.e., ρ(Z) < 1. (This can be derived using the Jordan canonical form; see =-=[32, 26]-=-.) Let u1, . . . , un be the columns of T and vT 1 , . . . , vT n be the rows of T −1 . Then we have lim t→∞ W t � Iκ 0 = lim T t→∞ 0 Zt � T −1 � � Iκ 0 = T T 0 0 −1 κ� = uiv i=1 T i . (12) Since each... |

13 |
On minimizing the spectral radius of a nonsymmetric matrix function — optimality conditions and duality theory
- Overton, Wormley
- 1987
(Show Context)
Citation Context ... in general is very hard. The reason is that the objective function, i.e., the spectral radius of a matrix, is in general not a convex function; indeed it is not even Lipschitz continuous (see, e.g., =-=[33]-=-). Some related spectral radius minimization problems are NP-hard [6, 27]. We can also formulate the FDLA problem, with per-step convergence factor, as the following spectral norm minimization problem... |

11 |
On a dynamic extension of the theory of graphs
- Mesbahi
- 2002
(Show Context)
Citation Context ...f the associated graph, appear to be very usetiff in the convergence analysis of consensus protocols [31]. The graph Laplacian has also been used in control of distributed dynamic systems (see, e.g., =-=[12, 13, 24]-=-). The FDLA problem (4) is closely related to the problem of finding the fastest mixing Markov chain on a graph [8]; the only difference is that in the FDLA problem, the weights can be (and the optima... |

11 | Fast linear iterations for distributed averaging,” Systems and - Xiao, Boyd - 2004 |

7 |
Graph Laplacians and vehicle formation stabilization
- Fax, Murray
- 2002
(Show Context)
Citation Context ...f the associated graph, appear to be very usetiff in the convergence analysis of consensus protocols [31]. The graph Laplacian has also been used in control of distributed dynamic systems (see, e.g., =-=[12, 13, 24]-=-). The FDLA problem (4) is closely related to the problem of finding the fastest mixing Markov chain on a graph [8]; the only difference is that in the FDLA problem, the weights can be (and the optima... |

7 |
Infinite power of matrice and characteristic root
- Oldenburger
- 1940
(Show Context)
Citation Context ...r matrix T such that W = T � Iκ 0 0 Z where Iκ is the κ-dimensional identity matrix (0 ≤ κ ≤ n) and Z is a convergent matrix, i.e., ρ(Z) < 1. (This can be derived using the Jordan canonical form; see =-=[32, 26]-=-.) Let u1, . . . , un be the columns of T and vT 1 , . . . , vT n be the rows of T −1 . Then we have lim t→∞ W t � Iκ 0 = lim T t→∞ 0 Zt � T −1 � � Iκ 0 = T T 0 0 −1 κ� = uiv i=1 T i . (12) Since each... |

6 | Minimum Entropy H - Mustafa, Glover - 1990 |