Abstract One of the central problems in machine learning and pattern recognition is to develop appropriate representations for complex data. We consider the problem of constructing a representation for data lying on a low dimensional manifold embedded in a high dimensional space. Drawing on the correspondence between the graph Laplacian, the Laplace Beltrami operator on the manifold, and the connections to the heat equation, we propose a geometrically motivated algorithm for representing the high dimensional data. The algorithm provides a computationally efficient approach to non-linear dimensionality reduction that has locality preserving properties and a natural connection to clustering. Some potential applications and illustrative examples are discussed. 1 Introduction In many areas of artificial intelligence, information retrieval and data mining, one is often confronted with intrinsically low dimensional data lying in a very high dimensional space. Consider, for example, gray scale images of an object taken under fixed lighting conditions with a moving camera. Each such image would typically be represented by a brightness value at each pixel. If there were n 2

Abstract not found

The graph partitioning problem is to divide the vertices of a graph into disjoint clusters to minimize the total cost of the edges cut by the clusters. A spectral partitioning heuristic uses the graph's eigenvectors to construct a geometric representation of the graph (e.g., linear orderings) which are subsequently partitioned. Our main result shows that when all the eigenvectors are used, graph partitioning reduces to a new vector partitioning problem. This result implies that as many eigenvectors as are practically possible should be used to construct a solution. This philosophy isincontrast to that of the widely-used spectral bipartitioning (SB) heuristic (which uses a single eigenvector to construct a 2-way partitioning) and several previous multiway partitioning heuristics [7][10][16][26][37] (which usek eigenvectors to construct a k-way partitioning). Our result motivates a simple ordering heuristic that is a multiple-eigenvector extension of SB. This heuristic not only signi cantly outperforms SB, but can also yield excellent multi-way VLSI circuit partitionings as compared to [1] [10]. Our experiments suggest that the vector partitioning perspective opens the door to new and effective heuristics.

Let G = (N; E) be an edge-weighted undirected graph. The graph partitioning problem is the problem of partitioning the node set N into k disjoint subsets of specified sizes so as to minimize the total weight of the edges connecting nodes in distinct subsets of the partition. We present a numerical study on the use of eigenvalue-based techniques to find upper and lower bounds for this problem. Results for bisecting graphs with up to several thousand nodes are given, and for small graphs some trisection results are presented. We show that the techniques are very robust and consistently produce upper and lower bounds having a relative gap of typically a few percentage points.

We give new, effective algorithms for k-way circuit partitioning in the two regimes of k ø n and k = \Theta(n), where n is the number of modules in the circuit. We show that partitioning an appropriately designed geometric embedding of the netlist, rather than a traditional graph representation, yields improved results as well as large speedups. We derive d- dimensional geometric embeddings of the netlist via (i) a new "partitioning-specific" net model for constructing the Laplacian of the netlist, and (ii) computation of d eigenvectors of the netlist Laplacian; we then apply (iii) fast top-down and bottom-up geometric clustering methods. 1 Preliminaries In top-down layout synthesis of complex VLSI systems, the goal of partitioning/clustering is to reveal the natural circuit structure, via a decomposition into k subcircuits which minimizes connectivity between subcircuits. A generic problem statement is as follows: k-Way Partitioning: Given a circuit netlist G = (V; E) with jV j...

A new semidefinite programming, SDP, relaxation for the general graph partitioning problem, GP, is derived. The relaxation arises from the dual of the (homogenized) Lagrangian dual of an appropriate quadratic representation of GP. The quadratic representation includes a representation of the 0,1 constraints in GP. The special structure of the relaxation is exploited in order to project onto the minimal face of the cone of positive semidefinite matrices which contains the feasible set. This guarantees that the Slater constraint qualification holds, which allows for a numerically stable primal-dual interior-point solution technique. A gangster operator is the key to providing an efficient representation of the constraints in the relaxation. An incomplete preconditioned conjugate gradient method is used for solving the large linear systems which arise when finding the Newton direction. Only dual feasibility is enforced, which results in the desired lower bounds, but avoids the expensive primal feasibility calculations.

Decomposition of large engineering system models is desirable since increased model size reduces reliability and speed of numerical solution algorithms. The article presents a methodology for optimal model-based decomposition (OMBD) of design problems, whether or not initially cast as optimization problems. The overall model is represented by a hypergraph and is optimally partitioned into weakly connected subgraphs that satisfy decomposition constraints. Spectral graph-partitioning methods together with iterative improvement techniques are proposed for hypergraph partitioning. A known spectral K-partitioning formulation, which accounts for partition sizes and edge weights, is extended to graphs with also vertex weights. The OMBD formulation is robust enough to account for computational demands and resources and strength of interdependencies between the computational modules contained in the model. KEYWORDS: Model decomposition, multidisciplinary design, hypergraph partitioning, larges...

Optimal design of large engineering systems modeled as nonlinear programming problems remains a challenge because increased size reduces reliability and speed of numerical optimization algorithms. Decomposition of the original model into smaller coordinated submodels is desirable or even necessary. The article presents a methodology for optimal model-based decomposition of design problems, whether or not initially cast as optimization models. The overall model is represented by a hypergraph that is optimally partitioned into weakly-connected subgraphs satisfying partitioning constraints. The formulation is robust enough to account for computational demands and resources, and the strength of interdependencies between the design relations contained in the model. This decomposition methodology is applied to a vehicle powertrain system design model consisting of engine, torque converter, transmission, and wheel-tire assemblies, with 87 design relations and 119 design and state/behavior variables.

This paper presents effective algorithms for multi-way partitioning. Confirming ideas originally due to Hall [27], we demonstrate that geometric embeddings of the circuit netlist can lead to high-quality k-way partitionings. The netlist embeddings are derived via the computation of d eigenvectors of the Laplacian for a graph representation of the netlist. As [27] did not specify how to partition such geometric embeddings, we explore various geometric partitioning objectives and algorithms, and find that they are limited because they do not integrate topological information from the netlist. Thus, we also present a new partitioning algorithm that exploits both the geometric embedding and netlist information, as well as a Restricted Partitioning formulation that requires each cluster of the k-way partitioning to be contiguous in a given linear ordering. We begin with a d-dimensional spectral embedding and construct a heuristic 1-dimensional ordering of the modules (combining spacefillin...

Semidefinite programming, SDP, is an extension of linear programming, LP, where the nonnegativity constraints are replaced by positive semidefiniteness constraints on matrix variables. SDP has proven successful in obtaining tight relaxations for NP -hard combinatorial optimization problems of simple structure such as the maxcut and graph bisection problems. In this work, we try to solve more complicated combinatorial problems such as the quadratic assignment, general graph partitioning and set partitioning problems. A tight SDP relaxation can be obtained by exploiting the geometrical structure of the convex hull of the feasible points of the original combinatorial problem. The analysis of the structure enables us to find the so-called "minimal face" and "gangster operator" of the SDP. This plays a significant role in simplifying the problem and enables us to derive a unified SDP relaxation for the three different problems. We develop an efficient "partial infeasible" primal-dual inter...