A training algorithm that maximizes the margin between the training patterns and the decision boundary is presented. The technique is applicable to a wide variety of classifiaction functions, including Perceptrons, polynomials, and Radial Basis Functions. The effective number of parameters is adjusted automatically to match the complexity of the problem. The solution is expressed as a linear combination of supporting patterns. These are the subset of training patterns that are closest to the decision boundary. Bounds on the generalization performance based on the leave-one-out method and the VC-dimension are given. Experimental results on optical character recognition problems demonstrate the good generalization obtained when compared with other learning algorithms.

We consider the problem of minimizing the largest generalized eigenvalue of a pair of symmetric matrices, each of which depends affinely on the decision variables. Although this problem may appear specialized, it is in fact quite general, and includes for example all linear, quadratic, and linear fractional programs. Many problems arising in control theory can be cast in this form. The problem is nondifferentiable but quasiconvex, so methods such as Kelley's cuttingplane algorithm or the ellipsoid algorithm of Shor, Nemirovksy, and Yudin are guaranteed to minimize it. In this paper we describe relevant background material and a simple interior point method that solves such problems more efficiently. The algorithm is a variation on Huard's method of centers, using a self-concordant barrier for matrix inequalities developed by Nesterov and Nemirovsky. (Nesterov and Nemirovsky have also extended their potential reduction methods to handle the same problem [NN91b].) Since the problem is quasiconvex but not convex, devising a non-heuristic stopping criterion (i.e., one that guarantees a given accuracy) is more difficult than in the convex case. We describe several non-heuristic stopping criteria that are based on the dual of a related convex problem and a new ellipsoidal approximation that is slightly sharper, in some cases, than a more general result due to Nesterov and Nemirovsky. The algorithm is demonstrated on an example: determining the quadratic Lyapunov function that optimizes a decay rate estimate for a differential inclusion.

Many problems in control system analysis and design can be posed in a setting where a system with a fixed model structure and nominal parameter values is affected by parameter variations. An example is parametric robustness analysis, where the parameters might represent physical quantities that are known only to within a certain accuracy, or vary depending on operating conditions etc. Frequently asked questions here deal with performance issues: "How bad can a certain performance measure of the system be over all possible values of the parameters?" Another example is parametric controller design, where the parameters represent degrees of freedom available to the control system designer. A typical question here would be: "What is the best choice of parameters, one that optimizes a certain design objective?" Many of the questions above may be directly restated as optimization problems: If q denotes the vector of parameters, Q