Results 1  10
of
60
A tutorial on support vector machines for pattern recognition
 Data Mining and Knowledge Discovery
, 1998
"... The tutorial starts with an overview of the concepts of VC dimension and structural risk minimization. We then describe linear Support Vector Machines (SVMs) for separable and nonseparable data, working through a nontrivial example in detail. We describe a mechanical analogy, and discuss when SV ..."
Abstract

Cited by 2272 (11 self)
 Add to MetaCart
The tutorial starts with an overview of the concepts of VC dimension and structural risk minimization. We then describe linear Support Vector Machines (SVMs) for separable and nonseparable data, working through a nontrivial example in detail. We describe a mechanical analogy, and discuss when SVM solutions are unique and when they are global. We describe how support vector training can be practically implemented, and discuss in detail the kernel mapping technique which is used to construct SVM solutions which are nonlinear in the data. We show how Support Vector machines can have very large (even infinite) VC dimension by computing the VC dimension for homogeneous polynomial and Gaussian radial basis function kernels. While very high VC dimension would normally bode ill for generalization performance, and while at present there exists no theory which shows that good generalization performance is guaranteed for SVMs, there are several arguments which support the observed high accuracy of SVMs, which we review. Results of some experiments which were inspired by these arguments are also presented. We give numerous examples and proofs of most of the key theorems. There is new material, and I hope that the reader will find that even old material is cast in a fresh light.
An introduction to kernelbased learning algorithms
 IEEE TRANSACTIONS ON NEURAL NETWORKS
, 2001
"... This paper provides an introduction to support vector machines (SVMs), kernel Fisher discriminant analysis, and ..."
Abstract

Cited by 373 (48 self)
 Add to MetaCart
This paper provides an introduction to support vector machines (SVMs), kernel Fisher discriminant analysis, and
Support Vector Machines for Classification and Regression
 UNIVERSITY OF SOUTHAMPTON, TECHNICAL REPORT
, 1998
"... The problem of empirical data modelling is germane to many engineering applications.
In empirical data modelling a process of induction is used to build up a model of the
system, from which it is hoped to deduce responses of the system that have yet to be observed.
Ultimately the quantity and qualit ..."
Abstract

Cited by 193 (5 self)
 Add to MetaCart
The problem of empirical data modelling is germane to many engineering applications.
In empirical data modelling a process of induction is used to build up a model of the
system, from which it is hoped to deduce responses of the system that have yet to be observed.
Ultimately the quantity and quality of the observations govern the performance
of this empirical model. By its observational nature data obtained is finite and sampled;
typically this sampling is nonuniform and due to the high dimensional nature of the
problem the data will form only a sparse distribution in the input space. Consequently
the problem is nearly always ill posed (Poggio et al., 1985) in the sense of Hadamard
(Hadamard, 1923). Traditional neural network approaches have suffered difficulties with
generalisation, producing models that can overfit the data. This is a consequence of the
optimisation algorithms used for parameter selection and the statistical measures used
to select the ’best’ model. The foundations of Support Vector Machines (SVM) have
been developed by Vapnik (1995) and are gaining popularity due to many attractive
features, and promising empirical performance. The formulation embodies the Structural
Risk Minimisation (SRM) principle, which has been shown to be superior, (Gunn
et al., 1997), to traditional Empirical Risk Minimisation (ERM) principle, employed by
conventional neural networks. SRM minimises an upper bound on the expected risk,
as opposed to ERM that minimises the error on the training data. It is this difference
which equips SVM with a greater ability to generalise, which is the goal in statistical
learning. SVMs were developed to solve the classification problem, but recently they
have been extended to the domain of regression problems (Vapnik et al., 1997). In the
literature the terminology for SVMs can be slightly confusing. The term SVM is typically
used to describe classification with support vector methods and support vector
regression is used to describe regression with support vector methods. In this report
the term SVM will refer to both classification and regression methods, and the terms
Support Vector Classification (SVC) and Support Vector Regression (SVR) will be used
for specification. This section continues with a brief introduction to the structural risk
Noisy Time Series Prediction using a Recurrent Neural Network and Grammatical Inference
 Machine Learning
, 2001
"... Financial forecasting is an example of a signal processing problem which is challenging due to small sample sizes, high noise, nonstationarity, and nonlinearity. Neural networks have been very successful in a number of signal processing applications. We discuss fundamental limitations and inherent ..."
Abstract

Cited by 47 (0 self)
 Add to MetaCart
Financial forecasting is an example of a signal processing problem which is challenging due to small sample sizes, high noise, nonstationarity, and nonlinearity. Neural networks have been very successful in a number of signal processing applications. We discuss fundamental limitations and inherent difficulties when using neural networks for the processing of high noise, small sample size signals. We introduce a new intelligent signal processing method which addresses the difficulties. The method proposed uses conversion into a symbolic representation with a selforganizing map, and grammatical inference with recurrent neural networks. We apply the method to the prediction of daily foreign exchange rates, addressing difficulties with nonstationarity, overfitting, and unequal a priori class probabilities, and we find significant predictability in comprehensive experiments covering 5 different foreign exchange rates. The method correctly predicts the direction of change for th...
A Sparse Representation for Function Approximation
 NEURAL COMPUTATION
, 1998
"... We derive a new general representation for a function as a linear combination of local correlation kernels at optimal sparse locations (and scales) and characterize its relation to PCA, regularization, sparsity principles and Support Vector Machines. ..."
Abstract

Cited by 40 (7 self)
 Add to MetaCart
We derive a new general representation for a function as a linear combination of local correlation kernels at optimal sparse locations (and scales) and characterize its relation to PCA, regularization, sparsity principles and Support Vector Machines.
Support vector machine with adaptive parameters in financial time series forecasting
 IEEE Transactions on Neural Networks
, 2003
"... Abstract—A novel type of learning machine called support vector machine (SVM) has been receiving increasing interest in areas ranging from its original application in pattern recognition to other applications such as regression estimation due to its remarkable generalization performance. This paper ..."
Abstract

Cited by 25 (1 self)
 Add to MetaCart
Abstract—A novel type of learning machine called support vector machine (SVM) has been receiving increasing interest in areas ranging from its original application in pattern recognition to other applications such as regression estimation due to its remarkable generalization performance. This paper deals with the application of SVM in financial time series forecasting. The feasibility of applying SVM in financial forecasting is first examined by comparing it with the multilayer backpropagation (BP) neural network and the regularized radial basis function (RBF) neural network. The variability in performance of SVM with respect to the free parameters is investigated experimentally. Adaptive parameters are then proposed by incorporating the nonstationarity of financial time series into SVM. Five real futures contracts collated from the Chicago Mercantile Market are used as the data sets. The simulation shows that among the three methods, SVM outperforms the BP neural network in financial forecasting, and there are comparable generalization performance between SVM and the regularized RBF neural network. Furthermore, the free parameters of SVM have a great effect on the generalization performance. SVM with adaptive parameters can both achieve higher generalization performance and use fewer support vectors than the standard SVM in financial forecasting. Index Terms—Backpropagation (BP) neural network, nonstationarity, regularized radial basis function (RBF) neural network, support vector machine (SVM). I.
Dynamical modeling with kernels for nonlinear time series prediction
"... We consider the question of predicting nonlinear time series. Kernel Dynamical Modeling, a new method based on kernels, is proposed as an extension to linear dynamical models. The kernel trick is used twice: first, to learn the parameter of the model, and second, to compute preimages of the time ser ..."
Abstract

Cited by 18 (1 self)
 Add to MetaCart
We consider the question of predicting nonlinear time series. Kernel Dynamical Modeling, a new method based on kernels, is proposed as an extension to linear dynamical models. The kernel trick is used twice: first, to learn the parameter of the model, and second, to compute preimages of the time series predicted in the feature space by means of Support Vector Regression. Our model shows strong connection with the classic Kalman Filter model, with the kernel feature space as hidden state space. Kernel Dynamical Modeling is tested against two benchmark time series and achieves high quality predictions. 1
Training Recurrent Networks by Evolino
, 2007
"... In recent years, gradientbased LSTM recurrent neural networks (RNNs) solved many previously RNNunlearnable tasks. Sometimes, however, gradient information is of little use for training RNNs, due to numerous local minima. For such cases, we present a novel method: EVOlution of systems with LINear O ..."
Abstract

Cited by 17 (4 self)
 Add to MetaCart
In recent years, gradientbased LSTM recurrent neural networks (RNNs) solved many previously RNNunlearnable tasks. Sometimes, however, gradient information is of little use for training RNNs, due to numerous local minima. For such cases, we present a novel method: EVOlution of systems with LINear Outputs (Evolino). Evolino evolves weights to the nonlinear, hidden nodes of RNNs while computing optimal linear mappings from hidden state to output, using methods such as pseudoinversebased linear regression. If we instead use quadratic programming to maximize the margin, we obtain the first evolutionary recurrent support vector machines. We show that Evolinobased LSTM can solve tasks that Echo State nets (Jaeger, 2004a) cannot and achieves higher accuracy in certain continuous function generation tasks than conventional gradient descent RNNs, including gradientbased LSTM.
Model induction with support vector machines
 Introduction and PWASET VOLUME 26 DECEMBER 2007 ISSN 13076884 797 © 2007 WASET.ORG OF WORLD ACADEMY OF SCIENCE, ENGINEERING AND TECHNOLOGY VOLUME 26 DECEMBER 2007 ISSN 13076884 applications.”Journal of Computing in Civil Engineering, ASCE
, 2001
"... ..."
Efficient SVM Regression Training with SMO
, 2001
"... The sequential minimal optimization algorithm (SMO) has been shown to be an effective method for training support vector machines ..."
Abstract

Cited by 14 (2 self)
 Add to MetaCart
The sequential minimal optimization algorithm (SMO) has been shown to be an effective method for training support vector machines