Results 1  10
of
42
Theory of classification: A survey of some recent advances
, 2005
"... The last few years have witnessed important new developments in the theory and practice of pattern classification. We intend to survey some of the main new ideas that have led to these recent results. ..."
Abstract

Cited by 56 (3 self)
 Add to MetaCart
The last few years have witnessed important new developments in the theory and practice of pattern classification. We intend to survey some of the main new ideas that have led to these recent results.
Polynomial Bounds for VC Dimension of Sigmoidal and General Pfaffian Neural Networks
 JOURNAL OF COMPUTER AND SYSTEM SCIENCES
, 1995
"... We introduce a new method for proving explicit upper bounds on the VC Dimension of general functional basis networks, and prove as an application, for the first time, that the VC Dimension of analog neural networks with the sigmoidal activation function oe(y) = 1=1+e \Gammay is bounded by a q ..."
Abstract

Cited by 50 (0 self)
 Add to MetaCart
We introduce a new method for proving explicit upper bounds on the VC Dimension of general functional basis networks, and prove as an application, for the first time, that the VC Dimension of analog neural networks with the sigmoidal activation function oe(y) = 1=1+e \Gammay is bounded by a quadratic polynomial O((lm) 2 ) in both the number l of programmable parameters, and the number m of nodes. The proof method of this paper generalizes to much wider class of Pfaffian activation functions and formulas, and gives also for the first time polynomial bounds on their VC Dimension. We present also some other applications of our method.
Neural networks for control
 in Essays on Control: Perspectives in the Theory and its Applications (H.L. Trentelman and
, 1993
"... ..."
(Show Context)
On the Complexity of Training Neural Networks with Continuous Activation Functions
, 1993
"... We deal with computational issues of loading a fixedarchitecture neural network with a set of positive and negative examples. This is the first result on the hardness of loading networks which do not consist of the binarythreshold neurons, but rather utilize a particular continuous activation func ..."
Abstract

Cited by 22 (2 self)
 Add to MetaCart
(Show Context)
We deal with computational issues of loading a fixedarchitecture neural network with a set of positive and negative examples. This is the first result on the hardness of loading networks which do not consist of the binarythreshold neurons, but rather utilize a particular continuous activation function, commonly used in the neural network literature. We observe that the loading problem is polynomialtime if the input dimension is constant. Otherwise, however, any possible learning algorithm based on particular fixed architectures faces severe computational barriers. Similar theorems have already been proved by Megiddo and by Blum and Rivest, to the case of binarythreshold networks only. Our theoretical results lend further justification to the use of incremental (architecturechanging) techniques for training networks rather than fixed architectures. Furthermore, they imply hardness of learnability in the probablyapproximatelycorrect sense as well.
Probabilistic Analysis of Learning in Artificial Neural Networks: The PAC Model and its Variants
, 1997
"... There are a number of mathematical approaches to the study of learning and generalization in artificial neural networks. Here we survey the `probably approximately correct' (PAC) model of learning and some of its variants. These models provide a probabilistic framework for the discussion of gen ..."
Abstract

Cited by 20 (4 self)
 Add to MetaCart
There are a number of mathematical approaches to the study of learning and generalization in artificial neural networks. Here we survey the `probably approximately correct' (PAC) model of learning and some of its variants. These models provide a probabilistic framework for the discussion of generalization and learning. This survey concentrates on the sample complexity questions in these models; that is, the emphasis is on how many examples should be used for training. Computational complexity considerations are briefly discussed for the basic PAC model. Throughout, the importance of the VapnikChervonenkis dimension is highlighted. Particular attention is devoted to describing how the probabilistic models apply in the context of neural network learning, both for networks with binaryvalued output and for networks with realvalued output.
VC Dimension of Neural Networks
 Neural Networks and Machine Learning
, 1998
"... . This paper presents a brief introduction to VapnikChervonenkis (VC) dimension, a quantity which characterizes the difficulty of distributionindependent learning. The paper establishes various elementary results, and discusses how to estimate the VC dimension in several examples of interest in ne ..."
Abstract

Cited by 20 (3 self)
 Add to MetaCart
. This paper presents a brief introduction to VapnikChervonenkis (VC) dimension, a quantity which characterizes the difficulty of distributionindependent learning. The paper establishes various elementary results, and discusses how to estimate the VC dimension in several examples of interest in neural network theory. 1 Introduction In this expository paper, we present a brief introduction to the subject of computing and estimating the VC dimension of neural network architectures. We provide precise definitions and prove several basic results, discussing also how one estimates VC dimension in several examples of interest in neural network theory. We do not address the learning and estimationtheoretic applications of VC dimension. (Roughly, the VC dimension is a number which helps to quantify the difficulty when learning from examples. The sample complexity, that is, the number of "learning instances" that one must be exposed to, in order to be reasonably certain to derive accurate p...
Perspectives of Current Research about the Complexity of Learning on Neural Nets
, 1994
"... This paper discusses within the framework of computational learning theory the current state of knowledge and some open problems in three areas of research about learning on feedforward neural nets:  Neural nets that learn from mistakes  Bounds for the VapnikChervonenkis dimension of neural net ..."
Abstract

Cited by 17 (1 self)
 Add to MetaCart
This paper discusses within the framework of computational learning theory the current state of knowledge and some open problems in three areas of research about learning on feedforward neural nets:  Neural nets that learn from mistakes  Bounds for the VapnikChervonenkis dimension of neural nets  Agnostic PAClearning of functions on neural nets. All relevant definitions are given in this paper, and no previous knowledge about computational learning theory or neural nets is required. We refer to [RSO] for further introductory material and survey papers about the complexity of learning on neural nets. Throughout this paper we consider the following rather general notion of a (feedforward) neural net.
Learning pattern classification  A survey
 IEEE TRANS. INFORM. THEORY
, 1998
"... Classical and recent results in statistical pattern recognition and learning theory are reviewed in a twoclass pattern classification setting. This basic model best illustrates intuition and analysis techniques while still containing the essential features and serving as a prototype for many applic ..."
Abstract

Cited by 16 (4 self)
 Add to MetaCart
Classical and recent results in statistical pattern recognition and learning theory are reviewed in a twoclass pattern classification setting. This basic model best illustrates intuition and analysis techniques while still containing the essential features and serving as a prototype for many applications. Topics discussed include nearest neighbor, kernel, and histogram methods, Vapnik–Chervonenkis theory, and neural networks. The presentation and the large (thogh nonexhaustive) list of references is geared to provide a useful overview of this field for both specialists and nonspecialists.
Analog versus Discrete Neural Networks
 Neural Computation
, 1996
"... We show that neural networks with threetimes continuously differentiable activation functions are capable of computing a certain family of nbit Boolean functions with two gates, whereas networks composed of binary threshold functions require at least \Omega\Gammaast n) gates. Thus, for a large cla ..."
Abstract

Cited by 16 (1 self)
 Add to MetaCart
We show that neural networks with threetimes continuously differentiable activation functions are capable of computing a certain family of nbit Boolean functions with two gates, whereas networks composed of binary threshold functions require at least \Omega\Gammaast n) gates. Thus, for a large class of activation functions, analog neural networks can be more powerful than discrete neural networks, even when computing Boolean functions. 1 Introduction. Artificial neural networks have become a popular model for machine learning and many results have been obtained regarding their application to practical problems. Typically, the network is trained to encode complex associations between inputs and outputs during supervised training cycles, where the associations are encoded by the weights of the network. Once trained, the network will compute an input/output mapping which (hopefully) is a good approximation of the original mapping. 1 Partially supported by NSF Grant CCR9114545 In thi...