Abstract

This paper highlights the role of mathematical programming, particularly linear programming, in training neural networks. A neural network description is given in terms of separating planes in the input space that suggests the use of linear programming for determining these planes. A more standard description in terms of a mean square error in the output space is also given, which leads to the use of unconstrained minimization techniques for training a neural network. The linear programming approach is demonstrated by a brief description of a system for breast cancer diagnosis that has been in use for the last four years at a major medical facility. 1 What is a Neural Network? A neural network is a representation of a map between an input space and an output space. A principal aim of such a map is to discriminate between the elements of a finite number of disjoint sets in the input space. Typically one wishes to discriminate between the elements of two disjoint point sets in the n-dim...

Citations

1530	Introduction to the Theory of Neural Computation - Hertz, Krogh, et al. - 1991
976	Multilayer feedforward networks are universal approximators - Hornik, Stinchcombe, et al. - 1989
763	Numerical Methods for Unconstrained Optimization and Nonlinear Equations - Dennis - 1983
549	A logical calculus of the ideas immanent in nervous activity - McCulloch, Pitts - 1943
439	Beyond Regression: New Tools for Prediction and Analysis - Werbos
436	Perceptrons; An introduction to computational geometry - Minsky, Papert - 1969
375	Optimum brain damage - Cun, Denker, et al. - 1990
186	Training a 3-node neural networks is np-complete - Blum, Rivest - 1992
184	Robust linear programming discrimination of two linearly inseparable sets - Bennett, Mangasarian - 1992
127	Learning Machines - Nilsson
113	Multisurface method of pattern separation for medical diagnosis applied to breast cytology - Wolberg, Mangasarian - 1990
108	Second-Order Methods for Learning: Between Steepest Descent and Newton's Method - Battiti, First- - 1992
83	Improving the convergence of back-propagation learning with second order methods - Becker, LeCun - 1989
81	MINOS 5.0 User’s Guide - Murtagh, Saunders - 1983
72	Principles of Neurodynamics (Spartan - Rosenblatt - 1962
64	Pattern recognition via linear programming: Theory and Applications to medical diagnosis - Mangasarian, Setiono, et al. - 1990
45	Linear and nonlinear separation of patterns by linear programming - Mangasarian - 1965
42	The perceptron: A perceiving and recognizing automaton - Rosenblatt - 1957
38	On the complexity of polyhedral separability - Megiddo - 1988
36	Neural network training via linear programming - Bennett, Mangasarian - 1992
35	Decision Tree Construction via Linear Programming - Bennett - 1992
35	I.J.: The relaxation method for linear inequalities - Motzkin, Schoenberg - 1954
34	Multi-surface method of pattern separation - Mangasarian - 1968
32	Bilinear separation of two sets in n-space - Bennett, Mangasarian - 1993
31	Simulated Annealing and Boltzman Machines - Aarts, Korst - 1989
27	Nuclear feature extraction for breast tumor diagnosis - Street, Wolberg, et al. - 1905
23	Improved linear programming models for discriminant analysis - Glover - 1990
22	Artificial Neural Systems - Simpson - 1990
14	Pattern classifier design by linear programming - Smith - 1968
13	Multicategory separation via linear programming - Bennett, Mangasarian - 1993
12	Decomposition into functions in the minimization problem,” Automat - Kibardin - 1980
8	Global convergence and stabilization of unconstrained minimization methods without derivatives - Grippo, Lampariello, et al. - 1988
6	Some fundamental theorems of perceptron theory and their geometry - Charnes - 1964
5	On the Boundedness of an Iterative Procedure for Solving a System of Linear Inequalities - Block, Levin - 1970
5	Mathematical methods for pattern classification - Grinold - 1972
4	Predicting salinity in Chesapeake Bay using backpropagation - DeSilets, Golden, et al. - 1992
3	Comments on hidden nodes in neural nets - Georgiou - 1991
3	Neural Networks for Selecting Vehicle Routing Heuristics - Nygard, Juell, et al. - 1990
3	Pattern classification using linear programming - Roy, Mukhopadhyay - 1990
2	Symbolic and neural network learning algorithms: An experimental comparison - Shavlik, Mooney, et al. - 1991
2	Design of multicategory pattern classifiers with two-category classifier design procedures - Smith - 1969
1	Neural nets for massively parallel optimisation - Dixon, Mills - 1992
1	Stochastic gradient methods for neural networks - Gaivoronski - 1994
1	Private communication - Grippo - 1992
1	Neural network optimization methods - Noyes - 1991