## Multicategory Discrimination via Linear Programming (1992)

Venue: | OPTIMIZATION METHODS AND SOFTWARE |

Citations: | 24 - 2 self |

### BibTeX

@ARTICLE{Bennett92multicategorydiscrimination,

author = {Kristin P. Bennett and O. L. Mangasarian},

title = {Multicategory Discrimination via Linear Programming},

journal = {OPTIMIZATION METHODS AND SOFTWARE},

year = {1992},

volume = {3},

pages = {27--39}

}

### OpenURL

### Abstract

A single linear program is proposed for discriminating between the elements of k disjoint point sets in the n-dimensional real space R n : When the conical hulls of the k sets are (k \Gamma 1)-point disjoint in R n+1 , a k-piece piecewise-linear surface generated by the linear program completely separates the k sets. This improves on a previous linear programming approach which required that each set be linearly separable from the remaining k \Gamma 1 sets. When the conical hulls of the k sets are not (k \Gamma 1)-point disjoint, the proposed linear program generates an error-minimizing piecewise-linear separator for the k sets. For this case it is shown that the null solution is never a unique solver of the linear program and occurs only under the rather rare condition when the mean of each point set equals the mean of the means of the other k \Gamma 1 sets. This makes the proposed linear computational programming formulation useful for approximately discriminating between k sets...

### Citations

4120 |
Pattern classification and scene analysis
- Duda, Hart
- 1973
(Show Context)
Citation Context ...ce projected on R 2 are depicted in Figure 1 together with the four sets in R 2 that are separated from each other. Many authors have considered this problem. Nilsson [16], Duda-Fossum [5], Duda-Hart =-=[6]-=-, and Fukunaga [8], considered iterative methods which are extensions of the perceptron algorithm or the Motzkin-Schoenberg algorithm [14] for determining a piecewise-linear separator when it exists. ... |

2862 |
Introduction to Statistical Pattern Recognition (2nd ed
- Fukunaga
- 1990
(Show Context)
Citation Context ...2 are depicted in Figure 1 together with the four sets in R 2 that are separated from each other. Many authors have considered this problem. Nilsson [16], Duda-Fossum [5], Duda-Hart [6], and Fukunaga =-=[8]-=-, considered iterative methods which are extensions of the perceptron algorithm or the Motzkin-Schoenberg algorithm [14] for determining a piecewise-linear separator when it exists. Convergence of the... |

1063 |
The Use of Multiple Measurements in Taxonomic Problems
- Fisher
- 1936
(Show Context)
Citation Context ...on (8) for discriminating between k sets for both the piecewise-linear separable and inseparable cases. Three different problems were considered: wine cultivar discrimination [1], iris classification =-=[7]-=-, and breast 8 Classes Dimension Size Percent Correct Problem k n Training Testing Wine Cultivars 3 13 178 100.0 91.0 Iris Plants 3 4 150 98.7 96.7 Breast Cancer 3 11 122 66.3 56.6 Table 1: Performanc... |

770 |
UCI Repository of Machine Learning Databases Available at: http:// www. ics.uci.edu/âˆ¼mlearn/MLRepository.html
- Murphy, Aha
- 1992
(Show Context)
Citation Context ...lidation. cancer prognosis [18]. The first two databases are available via anonymous file transfer protocol (ftp) from the University of California Irvine UCI Repository Of Machine Learning Databases =-=[15]-=-. The breast cancer prognosis data is available by request from the authors. Table 1 gives the number of classes, dimension, and size of each database as well as the results of using linear program (8... |

223 | Robust linear programming discrimination of two linearly inseparable sets
- Bennett, Mangasarian
- 1992
(Show Context)
Citation Context ...ation of two disjoint point sets A 1 and A 2 in R n with nonintersecting convex hulls by using the piecewise-linear maximum of two linear functions. This is equivalent to separation by a single plane =-=[9, 11, 4]-=-. See Figure 2. Computer Sciences Department, University of Wisconsin, 1210 West Dayton Street, Madison, WI 53706, email: bennett@cs.wisc.edu, olvi@cs.wisc.edu. This material is based on research supp... |

211 |
Nonlinear Programming
- Mangasarian
- 1969
(Show Context)
Citation Context ... A j + k X j=1 j 6=i u ij A i = 0; k X j=1 j 6=i u ji e \Gamma k X j=1 j 6=i u ij e = 0 k X i=1 k X j=1 j 6=i u ij e ? 0; have no solution u ijs0; u ij 2 R m i (By Motzkin's Theorem of the Alternative=-=[13]-=-) , (b): Remark 2.2 It is evident from Definition 2.1 that piecewise-linear separability of the sets A i ; i = 1; : : : ; k implies pairwise linear separability of the same sets. However the converse ... |

155 | Learning machines
- Nilsson
- 1965
(Show Context)
Citation Context ...ch typical piecewise-linear surface projected on R 2 are depicted in Figure 1 together with the four sets in R 2 that are separated from each other. Many authors have considered this problem. Nilsson =-=[16]-=-, Duda-Fossum [5], Duda-Hart [6], and Fukunaga [8], considered iterative methods which are extensions of the perceptron algorithm or the Motzkin-Schoenberg algorithm [14] for determining a piecewise-l... |

101 |
Estimation of Error Rates in Discriminant Analysis
- Lachenbruch, Mickey
- 1968
(Show Context)
Citation Context ...asses, dimension, and size of each database as well as the results of using linear program (8) to discriminate between the classes. Cross validation testing was done by using the leave-one-out method =-=[10, 8]-=- to estimate the training set error and the testing set error (error on unseen examples). A brief discussion of the numerical results follows. The Italian wine cultivar database is piecewise-linear se... |

60 |
Linear and Nonlinear Separation of Patterns by Linear Programming
- Mangasarian
- 1965
(Show Context)
Citation Context ...ation of two disjoint point sets A 1 and A 2 in R n with nonintersecting convex hulls by using the piecewise-linear maximum of two linear functions. This is equivalent to separation by a single plane =-=[9, 11, 4]-=-. See Figure 2. Computer Sciences Department, University of Wisconsin, 1210 West Dayton Street, Madison, WI 53706, email: bennett@cs.wisc.edu, olvi@cs.wisc.edu. This material is based on research supp... |

48 |
The relaxation method for linear inequalities
- Motzkin, Schoenberg
- 1954
(Show Context)
Citation Context ...sidered this problem. Nilsson [16], Duda-Fossum [5], Duda-Hart [6], and Fukunaga [8], considered iterative methods which are extensions of the perceptron algorithm or the Motzkin-Schoenberg algorithm =-=[14]-=- for determining a piecewise-linear separator when it exists. Convergence of these methods is not known if such a piecewise-linear surface does not exist [8, page 374]. Smith [17] on the other hand co... |

40 |
Decision tree construction via linear programming
- Bennett
- 1992
(Show Context)
Citation Context ...curacy shown in Table 1. A single linear program is insufficient for solving such problems. However, the results of several multicategory linear programs can be combined by using multisurface methods =-=[12, 3, 2]-=-. To demonstrate this, we used the multicategory linear program to create the multivariate splits in the multisurface method tree algorithm (MSMT) [2, 18]. MSMT-multicategory works by applying the lin... |

39 |
Multisurface method of pattern separation
- Mangasarian
- 1968
(Show Context)
Citation Context ...curacy shown in Table 1. A single linear program is insufficient for solving such problems. However, the results of several multicategory linear programs can be combined by using multisurface methods =-=[12, 3, 2]-=-. To demonstrate this, we used the multicategory linear program to create the multivariate splits in the multisurface method tree algorithm (MSMT) [2, 18]. MSMT-multicategory works by applying the lin... |

37 |
Neural network training via linear programming
- Bennett, Mangasarian
- 1992
(Show Context)
Citation Context ...ase any solution (w i ; fl i ; y ij ); i; j = 1; : : : ; k; j 6= k; provides a piecewise-linear separation as defined by (3). 6 As was the case for linear separation of two sets by linear programming =-=[3, 4]-=-, it is important here also to rule out the null solution in case the sets A i ; i = 1; : : : ; k are not piecewise-linear separable. Since the piecewise-linear separation (3) is in effect achieved by... |

12 |
Coomans and O. de Vel, Comparison of classifiers in high dimensional settings
- Aeberhard, D
(Show Context)
Citation Context ...ear programming formulation (8) for discriminating between k sets for both the piecewise-linear separable and inseparable cases. Three different problems were considered: wine cultivar discrimination =-=[1]-=-, iris classification [7], and breast 8 Classes Dimension Size Percent Correct Problem k n Training Testing Wine Cultivars 3 13 178 100.0 91.0 Iris Plants 3 4 150 98.7 96.7 Breast Cancer 3 11 122 66.3... |

11 |
Pattern classification by iteratively determined linear and piecewise linear discriminant functions
- DIJDA, FOSSUM
- 1966
(Show Context)
Citation Context ...se-linear surface projected on R 2 are depicted in Figure 1 together with the four sets in R 2 that are separated from each other. Many authors have considered this problem. Nilsson [16], Duda-Fossum =-=[5]-=-, Duda-Hart [6], and Fukunaga [8], considered iterative methods which are extensions of the perceptron algorithm or the Motzkin-Schoenberg algorithm [14] for determining a piecewise-linear separator w... |

4 |
Breast cancer diagnosis and prognostic determination from cell analysis
- Wolberg, Bennett, et al.
- 1992
(Show Context)
Citation Context ...4 150 98.7 96.7 Breast Cancer 3 11 122 66.3 56.6 Table 1: Performance of multicategory linear program on three problems. Correctness estimated using "leave-one-out" cross validation. cancer =-=prognosis [18]-=-. The first two databases are available via anonymous file transfer protocol (ftp) from the University of California Irvine UCI Repository Of Machine Learning Databases [15]. The breast cancer prognos... |

3 |
Design of multicategory pattern classifiers with two-category classifier design procedures
- Smith
- 1969
(Show Context)
Citation Context ...Schoenberg algorithm [14] for determining a piecewise-linear separator when it exists. Convergence of these methods is not known if such a piecewise-linear surface does not exist [8, page 374]. Smith =-=[17]-=- on the other hand considered solving k systems of linear inequalities by solving k linear programs to obtain a piecewise-linear separator. Unfortunately, this may not be possible for many simple piec... |

2 |
A note on linear separation
- Highleyman
- 1961
(Show Context)
Citation Context ...ation of two disjoint point sets A 1 and A 2 in R n with nonintersecting convex hulls by using the piecewise-linear maximum of two linear functions. This is equivalent to separation by a single plane =-=[9, 11, 4]-=-. See Figure 2. Computer Sciences Department, University of Wisconsin, 1210 West Dayton Street, Madison, WI 53706, email: bennett@cs.wisc.edu, olvi@cs.wisc.edu. This material is based on research supp... |