An analogy with the way ant colonies function has suggested the definition of a new computational paradigm, which we call Ant System. We propose it as a viable new approach to stochastic combinatorial optimization. The main characteristics of this model are positive feedback, distributed computation, and the use of a constructive greedy heuristic. Positive feedback accounts for rapid discovery of good solutions, distributed computation avoids premature convergence, and the greedy heuristic helps find acceptable solutions in the early stages of the search process. We apply the proposed methodology to the classical Traveling Salesman Problem (TSP), and report simulation results. We also discuss parameter selection and the early setups of the model, and compare it with tabu search and simulated annealing using TSP. To demonstrate the robustness of the approach, we show how the Ant System (AS) can be applied to other optimization problems like the asymmetric traveling salesman, the quadrat...

: A novel modified method for obtaining approximate solutions to difficult optimization problems within the neural network paradigm is presented. We consider the graph partition and the travelling salesman problems. The key new ingredient is a reduction of solution space by one dimension by using graded neurons, thereby avoiding the destructive redundancy that has plagued these problems when using straightforward neural network techniques. This approach maps the problems onto Potts glass rather than spin glass theories. A systematic prescription is given for estimating the phase transition temperatures in advance, which facilitates the choice of optimal parameters. This analysis, which is performed for both serial and synchronous updating of the mean field theory equations, makes it possible to consistently avoid chaotic bahaviour. When exploring this new technique numerically we find the results very encouraging

: We present and summarize the results from 50-, 100- and 200-city TSP benchmarks presented at the 1989 NIPS post-conference workshop using neural network, elastic net, genetic algorithm and simulated annealing approaches. These results are also compared with a state-of-the-art hybrid approach consisting of greedy solution, simulated annealing, and exhaustive search. 1 carsten@thep.lu.se Background Using neural networks to find approximate solutions to difficult optimization problems is a very attractive prospect. In the original paper [1] 10- and 30-city traveling salesman problems (TSP) were studied with very good results for the N=10 case. For N=30 the authors report on difficulties in finding optimal parameters. In ref. [2] further studies of the Tank-Hopfield approach were made with respect to refinements and extension to larger problem sizes. The authors of ref. [2] find the results discouraging. These and other similar findings have created a negative opinion about the entire...

: Rotor neurons are introduced to encode states living on the surface of a sphere in D dimensions. Such rotors can be regarded as continuous generalizations of binary (Ising) neurons. The corresponding mean field equations are derived, and phase transition properties based on linearized dynamics are given. The power of this approach is illustrated with an optimization problem -- placing N identical charges on a sphere such that the overall repulsive energy is minimized. The rotor approach appears superior to other methods for this problem both with respect to solution quality and computational effort needed. 1 larsg@thep.lu.se 2 carsten@thep.lu.se 3 bs@tf1.lu.se Background Standard McCulloch-Pitts neurons are characterized by sigmoidal updating equations v i = g(u i ) = tanh u i (1) where the local field u i is given by u i = X j ! ij v j =T (2) and the inverse temperature 1/T sets the gain. The neurons are binary in the high gain (T! 0) limit. In feed-back networks [1] wit...

. A hybrid genetic algorithm (HGA) for the optimization of the ground-state structure of a metallic atomic cluster has been implemented on a MIMDSIMD parallel platform. The concept of building block (BB) is generalized to cover this real-coded optimization problem. On the basis of some reasonings on the dependence of the convergence of Genetic Algorithms (GAs) from BBs, an hybrid genetic algorithm (HGA) is proposed to solve the minimization problem. All the elements of each new population are optimized through a Molecular Dynamics algorithm: the aim of MD is to create ever better BBs and, consequently, to improve the convergence of GAs. HGA has been implemented on a MIMDSIMD platform based on the massively parallel processing supercomputer Quadrics/APE100 which offers a peak performance of 25.6 Gflops; we obtained a sustained computational power greater than 10 Gflops. 1. Introduction A genetic algorithm (GA) [Raw 91] has been recently successfully applied to determine the lowest ener...

This report contains some remarks about the backpropagation method for neural net learning. We concentrate in particular in the study of local minima of error functions and the growth of weights during learning. Rutgers Center for Systems and Control, 1 Introduction Backpropagation is probably the most popular method currently being used for neural net learning. It was introduced in the neural literature by Rumelhart and Hinton in the seminal work [PDP]. See for instance [Hi87] for an introduction and references to current work. It can be understood as an iterative gradient technique for nonlinear least squares fitting, the cost function corresponding to a clustering problem to be solved. Its study gives rise to many open mathematical problems. This note makes some remarks about factors affecting the ultimate performance of backprop and its variants. A basic issue that must be understood when applying a gradient technique is that of the structure of the set of local minima of the err...

This paper provides an application oriented analysis of local search procedures for Operation Scheduling and Shop Floor Management of a major German manufacturer of cigarette machines. The heuristics applied are the Threshold- and Simulated Annealing-Algorithm considering Job Shop as well as embedded FMS production features. In this context a new neighbourhood search technique is developed, which is based on a small set of local neighbourhoods and is flexible with respect to the performance measurements of production control. By this approach the scheduling, loading and workload allocation problems of a production facility consisting of an embedded FMS and a conventional Job Shop can be solved simultaneously.

In this paper we present a global approach for programming distributed multiprocessor systems. In this approach, applications are developed as a global parallel program that is independent of the particular hardware architecture, and is represented through an extended Petri net model. The building blocks for the global program are tasks that are implemented using standard programming languages. A highly automated tool is used to allocate the different tasks to processing nodes in a near-optimum way, minimizing message traffic in the interconnection network and balancing the execution workload in the different nodes. The combined use of this tool with analysis and simulation tools for Petri nets allows us to obtain information about the performance and behavior of the global program. The tool divides the original extended Petri net into several subnets that are distributed among the different nodes, and provides for the installation, execution, and monitoring of the program. An example is presented in which our programming strategy is compared to PVM, which is a widely extended software tool for the distribution of programs in a network of computers.