This paper brings together two strands of machine learning of increasing importance: kernel methods and highly structured data. We propose a general method for constructing a kernel following the syntactic structure of the data, as defined by its type signature in a higher-order logic. Our main theoretical result is the positive definiteness of any kernel thus defined. We report encouraging experimental results on a range of real-world datasets. By converting our kernel to a distance pseudo-metric for 1-nearest neighbour, we were able to improve the best accuracy from the literature on the Diterpene dataset by more than 10%.

At the intersection of statistical physics and probability theory, Markov random elds and Gibbs distributions have emerged in the early eighties as powerful tools for modeling images and coping with high-dimensional inverse problems from lowlevel vision. Since then, they have been used in many studies from the image processing and computer vision community. Abrief and simple introduction to the basics of the domain is proposed. 1. Introduction and

The constrained least-squares regularization of nonlinear ill-posed problems is a nonlinear programming problem for which trust-region methods have been developed. In this paper the convergence theory of one of those methods is addressed. It will be proved that, under suitable hypotheses, local (superlinear or quadratic) convergence holds and every accumulation point is second-order stationary. Key words. Trust-region methods, Regularization, Ill Conditioning, Ill-Posed Problems, Constrained Minimization, Fixed-Point QuasiNewton methods. 1 Introduction Many practical problems in applied sciences and engineering give rise to ill-conditioned (linear or nonlinear) systems F (x) = y (1) where F : IR n ! IR m . Neither "exact solutions" of (1) (when they exist), nor global minimizers of kF (x) \Gamma yk have physical meaning since they are, to a great extent, contaminated by the influence of measuring and rounding errors and, perhaps, uncertainty in the model formulation. From the ...

Ridge regression is a classical statistical technique that attempts to address the bias-variance trade-off in the design of linear regression models. A reformulation of ridge regression in dual variables permits a non-linear form of ridge regression via the well-known “kernel trick”. Unfortunately, unlike support vector regression models, the resulting kernel expansion is typically fully dense. In this paper, we introduce a reduced rank kernel ridge regression (RRKRR) algorithm, capable of generating an optimally sparse kernel expansion that is functionally identical to that resulting from conventional kernel ridge regression (KRR). The proposed method is demonstrated to out-perform an alternative sparse kernel ridge regression algorithm on the Motorcycle and Boston Housing benchmarks.

A variational inequality problem (VIP) satisfying a constraint qualification can be reduced to a mixed complementarity problem (MCP). Monotonicity of the VIP implies that the MCP is also monotone. Introducing regularizing perturbations, a sequence of strictly monotone mixed complementarity problems is generated. It is shown that, if the original problem is solvable, the sequence of computable inexact solutions of the strictly monotone MCP's is bounded and every accumulation point is a solution. Under an additional condition on the precision used for solving each subproblem, the sequence converges to the minimum norm solution of the MCP. Keywords. Variational inequalities, complementarity, perturbations, inexact solutions, minimization algorithms, reformulation. AMS: 90C33, 90C30 1 Introduction The variational inequality problem was introduced as a tool in the study of partial differential equations [21]. Modern applications of the VIP include Department of Computer Scienc...