## Bilinear Separation of Two Sets in n-Space (1993)

Venue: | COMPUTATIONAL OPTIMIZATION AND APPLICATIONS |

Citations: | 36 - 17 self |

### BibTeX

@ARTICLE{Bennett93bilinearseparation,

author = {Kristin P. Bennett and O. L. Mangasarian},

title = {Bilinear Separation of Two Sets in n-Space},

journal = {COMPUTATIONAL OPTIMIZATION AND APPLICATIONS},

year = {1993},

volume = {2}

}

### Years of Citing Articles

### OpenURL

### Abstract

The NP-complete problem of determining whether two disjoint point sets in the n-dimensional real space R n can be separated by two planes is cast as a bilinear program, that is minimizing the scalar product of two linear functions on a polyhedral set. The bilinear program, which has a vertex solution, is processed by an iterative linear programming algorithm that terminates in a finite number of steps at a point satisfying a necessary optimality condition or at a global minimum. Encouraging computational experience on a number of test problems is reported.

### Citations

3696 | Convex analysis - Rockafellar - 1970 |

1910 |
Introduction to the theory of neural computation
- Hertz, Krogh, et al.
- 1991
(Show Context)
Citation Context ...o be equivalent to the polynomial-time solution of a single linear program [9, 13, 26, 5]. Linear separation is also equivalent to separation by Rosenblatt's perceptron or linear threshold unit (LTU) =-=[24, 25, 11]-=- (see Figure 2). However most problems are not linearly separable. For example the simple Minsky-Papert exclusive-or classical problem [20], is not linearly separable, but is bilinearly separable. It ... |

557 |
Perceptrons: an introduction to computational geometry
- MINSKY, PAPERT
- 1969
(Show Context)
Citation Context ...senblatt's perceptron or linear threshold unit (LTU) [24, 25, 11] (see Figure 2). However most problems are not linearly separable. For example the simple Minsky-Papert exclusive-or classical problem =-=[20]-=-, is not linearly separable, but is bilinearly separable. It can be solved by a neural network with 2 layers of linear threshold units [25, 18](see Figure 3). Other methods of separation by more than ... |

320 |
What size net gives valid generalization
- Baum, Haussler
- 1989
(Show Context)
Citation Context ...ses. This is indicated by lower curves in Figure 8 for the 1000-point training set compared to the curves for the 500-point training set. These trends agree with computational learning theory results =-=[3]-=- that provide necessary and sufficient conditions for valid generalization (that is correctness on testing sets) which are dependent on problem dimension and number of points in the training set. To m... |

223 | Robust linear programming discrimination of two linearly inseparable sets
- Bennett, Mangasarian
- 1992
(Show Context)
Citation Context ...th 2 layers of linear threshold units [25, 18](see Figure 3). Other methods of separation by more than one plane, for example multisurface methods (MSM) of pattern separation, have also been proposed =-=[14, 5, 6]-=- and extensively used for medical diagnosis [29, 17, 5]. MSM which has been shown to be equivalent to a feed-forward neural network with a single hidden layer [5], can be trained by a greedy algorithm... |

218 |
Nonlinear Programming
- Mangasarian
- 1994
(Show Context)
Citation Context ... (1) be feasible. Then UBPA 2.1 terminates in a finite number of steps at a global solution or a point (x i+1 ; y i ; r i+1 ; s i ) that satisfies the minimum principle necessary optimality condition =-=[15]-=- y i (x \Gamma x i+1 ) + x i+1 (y \Gamma y i )s0 8 (x; r) 2 X ; 8 (y; s) 2 Y : (3) Proof. If for some i; x i+1 y i+1 6! x i y i , then each of the linear programs of UBPA must have been solved to opti... |

202 | Training a 3-node neural network is NP-complete
- Blum, Rivest
- 1990
(Show Context)
Citation Context ...sh to consider is the following: Given two disjoint points sets A and B in the n-dimensional real space R n , can they be (strictly) separated by two planes? This is a fundamental NP-complete problem =-=[19, 8]-=- that is depicted in Figure 1 for the 2-dimensional real space R 2 . The configurations (a) and (b) of Figure 1 are equivalent as can easily be seen if the roles of A and B are interchanged. Bilinear ... |

181 |
An Algorithm for Quadratic Programming
- Frank, Wolfe
- 1956
(Show Context)
Citation Context ...programming-based finite algorithms for their solution. Because of the special property of a zero minimum for bilinearly separable problems, we have opted for the simpler Franke-Wolfe type algorithms =-=[10]-=-, rather than the more complex algorithms that have been given for bilinear programs [1, 30, 27, 28]. Our computational experience, summarized in Section 4, indicates that the proposed algorithms are ... |

157 |
Exploration in Parallel Distributed Processing. A handbook of Models, Programs, and Exercises
- McLelland, Rumelhart
- 1988
(Show Context)
Citation Context ...mple the simple Minsky-Papert exclusive-or classical problem [20], is not linearly separable, but is bilinearly separable. It can be solved by a neural network with 2 layers of linear threshold units =-=[25, 18]-=-(see Figure 3). Other methods of separation by more than one plane, for example multisurface methods (MSM) of pattern separation, have also been proposed [14, 5, 6] and extensively used for medical di... |

138 |
Multisurface method of pattern separation for medical diagnosis applied to breast cytology
- Wolberg, Mangasarian
- 1990
(Show Context)
Citation Context ...ure 3). Other methods of separation by more than one plane, for example multisurface methods (MSM) of pattern separation, have also been proposed [14, 5, 6] and extensively used for medical diagnosis =-=[29, 17, 5]-=-. MSM which has been shown to be equivalent to a feed-forward neural network with a single hidden layer [5], can be trained by a greedy algorithm using linear programming [14, 5, 6]. Thus bilinear and... |

103 |
MINOS 5.0 user’s guide
- Murtagh, Saunders
- 1983
(Show Context)
Citation Context ... 2 ; i 1 ; i 2 )s0. 4 Computational Results Both the uncoupled and coupled bilinear programming algorithms were implemented and tested on a suite of problems. The linear programming package MINOS 5.4 =-=[21]-=- was used to solve the linear subproblems. Initial experimentation with UBPA 2.1, UBPA1 2.2, and BPA 2.3 showed that they were prone to halting at solutions with w 1 = 0, w 2 = 0, or w 1 = w 2 which s... |

89 |
J.E.: Jointly constrained biconvex programming
- Al-Khayyal, Falk
- 1983
(Show Context)
Citation Context ...f a zero minimum for bilinearly separable problems, we have opted for the simpler Franke-Wolfe type algorithms [10], rather than the more complex algorithms that have been given for bilinear programs =-=[1, 30, 27, 28]-=-. Our computational experience, summarized in Section 4, indicates that the proposed algorithms are effective ones, especially in view of the fact that the underlying problem is an NP-complete problem... |

88 |
Principles of Neurodynamics." Spartan
- Rosenblatt
- 1962
(Show Context)
Citation Context ...o be equivalent to the polynomial-time solution of a single linear program [9, 13, 26, 5]. Linear separation is also equivalent to separation by Rosenblatt's perceptron or linear threshold unit (LTU) =-=[24, 25, 11]-=- (see Figure 2). However most problems are not linearly separable. For example the simple Minsky-Papert exclusive-or classical problem [20], is not linearly separable, but is bilinearly separable. It ... |

74 | Pattern recognition via linear programming: Theory and application to medical diagnosis’, Large-scale numerical optimization
- Mangasarian, Setiono, et al.
- 1990
(Show Context)
Citation Context ...ure 3). Other methods of separation by more than one plane, for example multisurface methods (MSM) of pattern separation, have also been proposed [14, 5, 6] and extensively used for medical diagnosis =-=[29, 17, 5]-=-. MSM which has been shown to be equivalent to a feed-forward neural network with a single hidden layer [5], can be trained by a greedy algorithm using linear programming [14, 5, 6]. Thus bilinear and... |

40 |
Decision tree construction via linear programming
- Bennett
- 1992
(Show Context)
Citation Context ...n practice. The w 1 = w 2 case for the uncoupled problem was practically avoided by starting with a good initial solution found by using the first two planes of the MultisurfaceMethod -Tree algorithm =-=[4, 6]-=-. The problem of w 1 = 0 and w 2 = 0 was tackled by detecting when this occurred and then adding a linear constraint to the problem which made the zero solution infeasible. This did not affect the con... |

39 |
Multi-surface method of pattern separation
- Mangasarian
- 1968
(Show Context)
Citation Context ...th 2 layers of linear threshold units [25, 18](see Figure 3). Other methods of separation by more than one plane, for example multisurface methods (MSM) of pattern separation, have also been proposed =-=[14, 5, 6]-=- and extensively used for medical diagnosis [29, 17, 5]. MSM which has been shown to be equivalent to a feed-forward neural network with a single hidden layer [5], can be trained by a greedy algorithm... |

37 |
Neural network training via linear programming
- Bennett, Mangasarian
- 1992
(Show Context)
Citation Context ... B are interchanged. Bilinear separation is a natural extension of linear separation which, for a long time, has been known to be equivalent to the polynomial-time solution of a single linear program =-=[9, 13, 26, 5]-=-. Linear separation is also equivalent to separation by Rosenblatt's perceptron or linear threshold unit (LTU) [24, 25, 11] (see Figure 2). However most problems are not linearly separable. For exampl... |

37 | On the complexity of polyhedral separability
- Megiddo
- 1988
(Show Context)
Citation Context ...sh to consider is the following: Given two disjoint points sets A and B in the n-dimensional real space R n , can they be (strictly) separated by two planes? This is a fundamental NP-complete problem =-=[19, 8]-=- that is depicted in Figure 1 for the 2-dimensional real space R 2 . The configurations (a) and (b) of Figure 1 are equivalent as can easily be seen if the roles of A and B are interchanged. Bilinear ... |

28 | Solution of general linear complementarity problems via nondifferentiable concave minimization
- Mangasarian
- 1997
(Show Context)
Citation Context ...ction on a polyhedral set not containing lines going to infinity in both directions, it must have a vertex solution [23, Corollary 32.3.4]. The above proof is based on the proofs of Lemmas 1 and 2 of =-=[16]-=- which show that every solvable linear complementarity problem, monotonic or not, has a vertex solution. With the above theorems we can formulate finite algorithms for each of the uncoupled and couple... |

22 |
Pattern classifier design by linear programming
- Smith
- 1968
(Show Context)
Citation Context ... B are interchanged. Bilinear separation is a natural extension of linear separation which, for a long time, has been known to be equivalent to the polynomial-time solution of a single linear program =-=[9, 13, 26, 5]-=-. Linear separation is also equivalent to separation by Rosenblatt's perceptron or linear threshold unit (LTU) [24, 25, 11] (see Figure 2). However most problems are not linearly separable. For exampl... |

17 |
and nonlinear separation of patterns by linear programming
- “Linear
- 1965
(Show Context)
Citation Context ... B are interchanged. Bilinear separation is a natural extension of linear separation which, for a long time, has been known to be equivalent to the polynomial-time solution of a single linear program =-=[9, 13, 26, 5]-=-. Linear separation is also equivalent to separation by Rosenblatt's perceptron or linear threshold unit (LTU) [24, 25, 11] (see Figure 2). However most problems are not linearly separable. For exampl... |

14 |
Programming, Games and Transportation Networks [M
- Berge, Ghouila-Houri
- 1965
(Show Context)
Citation Context ... algorithm tend to a zero minimum. These results, which do not seem to be readily available in the stated form, are needed for our bilinear program algorithm BPA 2.3. Our proofs are based on those of =-=[7]-=- for the convex case. We consider the following problem and assumptions. Problem A.1 min x2X f(x) where f : R n ! R; X is a polyhedral set in R n that does not contain straight lines that go to infini... |

6 |
Some fundamental theorems of perceptron theory and their geometry
- Charnes
- 1964
(Show Context)
Citation Context |

5 |
Learning internal representations
- Rumelhart, Hinton, et al.
(Show Context)
Citation Context ...o be equivalent to the polynomial-time solution of a single linear program [9, 13, 26, 5]. Linear separation is also equivalent to separation by Rosenblatt's perceptron or linear threshold unit (LTU) =-=[24, 25, 11]-=- (see Figure 2). However most problems are not linearly separable. For example the simple Minsky-Papert exclusive-or classical problem [20], is not linearly separable, but is bilinearly separable. It ... |

4 |
A linear programming approach to solving bilinear programmes
- White
- 1992
(Show Context)
Citation Context ...f a zero minimum for bilinearly separable problems, we have opted for the simpler Franke-Wolfe type algorithms [10], rather than the more complex algorithms that have been given for bilinear programs =-=[1, 30, 27, 28]-=-. Our computational experience, summarized in Section 4, indicates that the proposed algorithms are effective ones, especially in view of the fact that the underlying problem is an NP-complete problem... |

2 |
A note on linear separation
- Highleyman
- 1961
(Show Context)
Citation Context ...arable. By using the duality theory of linear programming, this can be shown to be equivalent to the existence of a plane wx = fl strictly separating A from B (see Figure 2(a)) which is equivalent to =-=[12, 13, 26, 5]-=- \Gamma Aw + efl + es0; Bw \Gamma efl + es0; for some w 2 R n ; fl 2 R: (5) Based on the above definition of linear separability, we now define bilinear separability as follows: 7 Definition 3.1 (Bili... |

2 |
A note on the solution of bilinear programming problems by reduction to concave minimization
- Thieu
- 1988
(Show Context)
Citation Context ...f a zero minimum for bilinearly separable problems, we have opted for the simpler Franke-Wolfe type algorithms [10], rather than the more complex algorithms that have been given for bilinear programs =-=[1, 30, 27, 28]-=-. Our computational experience, summarized in Section 4, indicates that the proposed algorithms are effective ones, especially in view of the fact that the underlying problem is an NP-complete problem... |

2 |
Efficient algorithms for solving rank two and rank three bilinear programming problems
- Yajima, Konno
- 1991
(Show Context)
Citation Context |