## Fast SDP relaxations of graph cut clustering, transduction, and other combinatorial problems (2006)

### Cached

### Download Links

Venue: | JMLR |

Citations: | 21 - 3 self |

### BibTeX

@ARTICLE{Bie06fastsdp,

author = {Tijl De Bie and Nello Cristianini and P. Bennett and Emilio Parrado-hernández},

title = {Fast SDP relaxations of graph cut clustering, transduction, and other combinatorial problems},

journal = {JMLR},

year = {2006},

volume = {7},

pages = {1409--1436}

}

### OpenURL

### Abstract

The rise of convex programming has changed the face of many research fields in recent years, machine learning being one of the ones that benefitted the most. A very recent developement, the relaxation of combinatorial problems to semi-definite programs (SDP), has gained considerable attention over the last decade (Helmberg, 2000; De Bie and Cristianini, 2004a). Although SDP problems can be solved in polynomial time, for many relaxations the exponent in the polynomial complexity bounds is too high for scaling to large problem sizes. This has hampered their uptake as a powerful new tool in machine learning. In this paper, we present a new and fast SDP relaxation of the normalized graph cut problem, and investigate its usefulness in unsupervised and semi-supervised learning. In particular, this provides a convex algorithm for transduction, as well as approaches to clustering. We further propose a whole cascade of fast relaxations that all hold the middle between older spectral relaxations and the new SDP relaxation, allowing one to trade off computational cost versus relaxation accuracy. Finally, we discuss how the methodology developed in this paper can be applied to other combinatorial problems in machine learning, and we treat the max-cut problem as an example.

### Citations

3923 | Convex Optimization - Boyd, Vandenberghe - 2004 |

2717 | Normalized cuts and image segmentation - SHI, MALIK |

1161 | On spectral clustering: analysis and an algorithm’, NIPS
- Ng, Jordan, et al.
- 2001
(Show Context)
Citation Context ...NCut costs are NP-complete problems (Shi and Malik, 2000). To get around this, spectral relaxations of the ACut and NCut optimization problems have been proposed in a clustering (Shi and Malik, 2000; =-=Ng et al., 2002-=-; Cristianini et al., 2002) and more recently also in a transduction setting (Kamvar et al., 2003; Joachims, 2003; De Bie et al., 2004). Xing and Jordan (2003) also proposed an interesting SDP relaxat... |

965 | Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming
- Goemans, Williamson
- 1995
(Show Context)
Citation Context ...ly the well-known max-cut problem. For this problem we will also discuss a specific choice of W in the cascade of SDP relaxations. 4.1 The Max-Cut Problem The SDP-relaxed max-cut problem is given by (=-=Goemans and Williamson, 1995-=-; Helmberg, 2000): P max-cut ⎧ ⎨ maxΓ ⎩ 1 4 〈Γ, D − A〉 s.t. Γ � 0, D diag(Γ) = 1. max-cut � minλ 1 ′ λ, s.t. − 1 4 (D − A) + diag(λ) � 0. where again Γ ≈ yy ′ is the label matrix, with y ∈ {−1, 1} n .... |

801 | Semidefinite programming - Vandenberghe, Boyd - 1996 |

197 | Transductive learning via spectral graph partitioning
- Joachims
- 2003
(Show Context)
Citation Context ...d NCut optimization problems have been proposed in a clustering (Shi and Malik, 2000; Ng et al., 2002; Cristianini et al., 2002) and more recently also in a transduction setting (Kamvar et al., 2003; =-=Joachims, 2003-=-; De Bie et al., 2004). Xing and Jordan (2003) also proposed an interesting SDP relaxation for the NCut optimization problem in a multiclass clustering setting, however, the computational cost to solv... |

142 | A Spectral Bundle Method for Semidefinite Programming - Helmberg, Rendl |

112 | A nonlinear programming algorithm for solving semide programs via low-rank factorization, Working paper
- Burer, Monteiro
- 2001
(Show Context)
Citation Context ...m), reducing the worst case complexity down to O(m2n2.5 ). Hence, m is a 3. Other software tools that are in practice often faster exist, notably SDPLR, which we used for the large-scale experiments (=-=Burer and Monteiro, 2003-=-, 2005). 1420sFast SDP Relaxations of Graph Cut Clustering parameter trading off the tightness of the relaxation with the computational complexity, and can be adapted according to the available comput... |

82 | Computing gaussian mixture models with em using equivalence constraints
- Shental, Bar-Hillel, et al.
(Show Context)
Citation Context ...cut to graph cuts that do not violate the label information. Even more generally, one can consider the case where labels are not exactly specified, but where equivalence or inequivalence constraints (=-=Shental et al., 2004-=-) are given instead, specifying equality or non-equality of the labels respectively. 1.1 Cut, Average Cut and Normalized Cut Cost Functions Several graph cut cost functions have been proposed in liter... |

77 | Spectral learning
- Kamvar, Klein, et al.
- 2003
(Show Context)
Citation Context ...ations of the ACut and NCut optimization problems have been proposed in a clustering (Shi and Malik, 2000; Ng et al., 2002; Cristianini et al., 2002) and more recently also in a transduction setting (=-=Kamvar et al., 2003-=-; Joachims, 2003; De Bie et al., 2004). Xing and Jordan (2003) also proposed an interesting SDP relaxation for the NCut optimization problem in a multiclass clustering setting, however, the computatio... |

64 |
Semide Programming for Combinatorial Optimization, ZIB-Report ZR-00-34, Konrad-Zuse-Zentrum
- Helmberg
- 2000
(Show Context)
Citation Context ...one of the ones that benefitted the most. A very recent developement, the relaxation of combinatorial problems to semi-definite programs (SDP), has gained considerable attention over the last decade (=-=Helmberg, 2000-=-; De Bie and Cristianini, 2004a). Although SDP problems can be solved in polynomial time, for many relaxations the exponent in the polynomial complexity bounds is too high for scaling to large problem... |

39 | N.: Convex methods for transduction - Bie, Cristianini - 2004 |

24 |
Spectral Kernel Methods for Clustering
- Cristianini, Shawe-Taylor, et al.
- 2002
(Show Context)
Citation Context ...-complete problems (Shi and Malik, 2000). To get around this, spectral relaxations of the ACut and NCut optimization problems have been proposed in a clustering (Shi and Malik, 2000; Ng et al., 2002; =-=Cristianini et al., 2002-=-) and more recently also in a transduction setting (Kamvar et al., 2003; Joachims, 2003; De Bie et al., 2004). Xing and Jordan (2003) also proposed an interesting SDP relaxation for the NCut optimizat... |

22 | Local minima and convergence in lowrank semidefinite programming - Burer, Monteiro - 2003 |

14 | Finding good nearly balanced cuts in power law graphs - Lang - 2004 |

11 | Learning from general label constraints - Bie, Suykens, et al. - 2004 |

3 | Kernel methods for exploratory data analysis: a demonstration on text data - Bie, Cristianini - 2004 |

1 | Fast SDP Relaxations of Graph Cut Clustering - Anjos, Wolkowicz - 2002 |