Results 1  10
of
16
Algebraic Transformations of Objective Functions
 Neural Networks
, 1994
"... Many neural networks can be derived as optimization dynamics for suitable objective functions. We show that such networks can be designed by repeated transformations of one objective into another with the same fixpoints. We exhibit a collection of algebraic transformations which reduce network cost ..."
Abstract

Cited by 26 (11 self)
 Add to MetaCart
Many neural networks can be derived as optimization dynamics for suitable objective functions. We show that such networks can be designed by repeated transformations of one objective into another with the same fixpoints. We exhibit a collection of algebraic transformations which reduce network cost and increase the set of objective functions that are neurally implementable. The transformations include simplification of products of expressions, functions of one or two expressions, and sparse matrix products (all of which may be interpreted as Legendre transformations); also the minimum and maximum of a set of expressions. These transformations introduce new interneurons which force the network to seek a saddle point rather than a minimum. Other transformations allow control of the network dynamics, by reconciling the Lagrangian formalism with the need for fixpoints. We apply the transformations to simplify a number of structured neural networks, beginning with the standard reduction of...
Parallel Incremental Graph Partitioning
"... Partitioning graphs into equally large groups of nodes while minimizing the number of edges between different groups is an extremely important problem in parallel computing. For instance, efficiently parallelizing several scientific and engineering applications requires the partitioning of data or t ..."
Abstract

Cited by 17 (0 self)
 Add to MetaCart
Partitioning graphs into equally large groups of nodes while minimizing the number of edges between different groups is an extremely important problem in parallel computing. For instance, efficiently parallelizing several scientific and engineering applications requires the partitioning of data or tasks among processors such that the computational load on each node is roughly the same, while communication is minimized. Obtaining exact solutions is computationally intractable, since graphpartitioning is an NPcomplete. For a large class of irregular and adaptive data parallel applications (such as adaptive meshes), the computational structure changes from one phase to another in an incremental fashion. In incremental graphpartitioning problems the partitioning of the graph needs to be updated as the graph changes over time; a small number of nodes or edges may be added or deleted at any given instant. In this paper we use a linear programmingbased method to solve the incremental grap...
Fast and Parallel Mapping Algorithms for Irregular Problems
 Journal of Supercomputing
, 1993
"... In this paper we develop simple indexbased graph partitioning techniques. We show our methods to be very fast, easily parallelizable and that they produce good quality mappings. These properties make them useful for parallelization of a number of irregular and adaptive applications. Index Terms: M ..."
Abstract

Cited by 16 (0 self)
 Add to MetaCart
In this paper we develop simple indexbased graph partitioning techniques. We show our methods to be very fast, easily parallelizable and that they produce good quality mappings. These properties make them useful for parallelization of a number of irregular and adaptive applications. Index Terms: Mapping, Remapping, Parallel, Merging, Sorting 1 Introduction Parallelization of dataparallel programs on distributedmemory parallel computers requires careful attention to load balancing and reduction of communication to achieve a good performance. For most regular and synchronous problems [13], mapping can be performed at the time of compilation by giving directives to decompose the data and its corresponding computations [8]. For irregular applications, achieving a good mapping is considerably more difficult; the nature of the irregularities may not be known at the time of compilation and can be derived only at runtime [7]. These applications can be represented as computational graphs...
Parallel Incremental Graph Partitioning Using Linear Programming
 Proceedings Supercomputing '94
, 1994
"... Partitioning graphs into equally large groups of nodes while minimizing the number of edges between different groups is an extremely important problem in parallel computing. For instance, efficiently parallelizing several scientific and engineering applications requires the partitioning of data or t ..."
Abstract

Cited by 8 (2 self)
 Add to MetaCart
Partitioning graphs into equally large groups of nodes while minimizing the number of edges between different groups is an extremely important problem in parallel computing. For instance, efficiently parallelizing several scientific and engineering applications requires the partitioning of data or tasks among processors such that the computational load on each node is roughly the same, while communication is minimized. Obtaining exact solutions is computationally intractable, since graphpartitioning is an NPcomplete. For a large class of irregular and adaptive data parallel applications (such as adaptive meshes), the computational structure changes from one phase to another in an incremental fashion. In incremental graphpartitioning problems the partitioning of the graph needs to be updated as the graph changes over time; a small number of nodes or edges may be added or deleted at any given instant. In this paper we use a linear programmingbased method to solve the incremental grap...
ArchitectureIndependent LocalityImproving Transformations of Computational Graphs Embedded in kDimensions
 In Proceedings of the 9th ACM International Conference on Supercomputing
, 1994
"... A large number of dataparallel applications can be represented as computational graphs from the perspective of parallel computing. The nodes of these graphs represent tasks that can be executed concurrently, while the edges represent the interactions between them. Further, the computational graphs ..."
Abstract

Cited by 7 (0 self)
 Add to MetaCart
A large number of dataparallel applications can be represented as computational graphs from the perspective of parallel computing. The nodes of these graphs represent tasks that can be executed concurrently, while the edges represent the interactions between them. Further, the computational graphs derived from many applications are such that the vertices correspond to two or threedimensional coordinates, and the interaction between computations is limited to vertices that are physically proximate. In this paper we show that graphs with these properties can be transformed into simple architectureindependent representations that encapsulate the locality in these graphs. This representation allows a fast mapping of the computational graph onto the underlying architecture at the time of execution. This is necessary for environments where available computational resources can be determined only at the time of execution or that change during execution. 1 Introduction From the perspecti...
Runtime Support for Parallelization of DataParallel Applications on Adaptive and Nonuniform Computational Environments,
 JOURNAL OF PARALLEL AND DISTRIBUTED COMPUTING
, 1995
"... In this paper we discuss the runtime support required for the parallelization of unstructured dataparallel applications on nonuniform and adaptive environments. The approach presented is reasonably general and is applicable to a wide variety of regular as well as irregular applications. We present p ..."
Abstract

Cited by 5 (1 self)
 Add to MetaCart
In this paper we discuss the runtime support required for the parallelization of unstructured dataparallel applications on nonuniform and adaptive environments. The approach presented is reasonably general and is applicable to a wide variety of regular as well as irregular applications. We present performance results for the solution of an unstructured mesh on a cluster of heterogeneous workstations.
A Comparison of Optimization Heuristics for the Data Mapping Problem
 Center For
, 1995
"... In this paper we compare the performance of six heuristics with suboptimal solutions for the data distribution of two dimensional meshes that are used for the numerical solution of Partial Di erential Equations (PDEs) on multicomputers. The data mapping heuristics are evaluated with respect to seven ..."
Abstract

Cited by 4 (2 self)
 Add to MetaCart
In this paper we compare the performance of six heuristics with suboptimal solutions for the data distribution of two dimensional meshes that are used for the numerical solution of Partial Di erential Equations (PDEs) on multicomputers. The data mapping heuristics are evaluated with respect to seven criteria covering load balancing, interprocessor communication, exibility and ease of use for a class of singlephase iterative PDE solvers. Our evaluation suggests that the simple and fast block distribution heuristic can be as e ective as the other ve complex and computational expensive algorithms. This work was supported by the Alex Nason Prize Award and in part by the Cornell Theory Center. 1.
Parallel computers and complex systems
 In Bossomaier, T.R.J. & Green, D.G. (eds) Complex Systems. Cambridge
, 2000
"... SCCS370b We present an overview of the state of the art and future trends in high performance parallel and distributed computing, and discuss techniques for using such computers in the simulation of complex problems in computational science. The use of high performance parallel computers can help i ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
SCCS370b We present an overview of the state of the art and future trends in high performance parallel and distributed computing, and discuss techniques for using such computers in the simulation of complex problems in computational science. The use of high performance parallel computers can help improve our understanding of complex systems, and the converse is also true — we can apply techniques used for the study of complex systems to improve our understanding of parallel computing. We consider parallel computing as the mapping of one complex system — typically a model of the world — into another complex system — the parallel computer. We study static, dynamic, spatial and temporal properties of both the complex systems and the map between them. The result is a better understanding of which computer architectures are good for which problems, and of software structure, automatic partitioning of data, and the performance of parallel machines. 1
Problem Solving with Optimization Networks
, 1993
"... previously seemed, since they can be successfully applied to only a limited number of problems exhibiting special, amenable properties. Combinatorial optimization, neural networks, mean eld annealing. i optimization networks Key words: Summary I am greatly indebted to my supervisor, Richard Prager, ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
previously seemed, since they can be successfully applied to only a limited number of problems exhibiting special, amenable properties. Combinatorial optimization, neural networks, mean eld annealing. i optimization networks Key words: Summary I am greatly indebted to my supervisor, Richard Prager, for initially allowing me the freedom to explore various research areas, and subsequently providing invaluable support as my work progressed. Members of the Speech, Vision and Robotics Group at the Cambridge University Department of Engineering have provided a stimulating and friendly environment to work in: special thanks must go to Patrick Gosling and Tony Robinson for maintaining a superb computing service, and to Sree Aiyer for both setting me on the right course and for numerous helpful discussions since then. I would like to thank the Science and Engineering Research Council of Great Britain, the Cambridge University Department of Engineering and Queen