Results 1  10
of
27
Training a 3node neural network is NPComplete
 In Proceedings of the 1988 Workshop on Computational Learning Theory
, 1988
"... rivest~theory.lcs.mit.edu ..."
An experimental and theoretical comparison of model selection methods. Machine Learning 27
, 1997
"... In the model selection problem, we must balance the complexity of a statistical model with its goodness of fit to the training data. This problem arises repeatedly in statistical estimation, machine learning, and scientific inquiry in general. ..."
Abstract

Cited by 110 (5 self)
 Add to MetaCart
In the model selection problem, we must balance the complexity of a statistical model with its goodness of fit to the training data. This problem arises repeatedly in statistical estimation, machine learning, and scientific inquiry in general.
Overfitting and Undercomputing in Machine Learning
 Computing Surveys
, 1995
"... suggests a reasonable line of research: find algorithms that can search the hypothesis class better. Hence, there is been extensive research in applying secondorder methods to fit neural networks and in conducting much more thorough searches in learning decision trees and rule sets. Ironically, wh ..."
Abstract

Cited by 26 (0 self)
 Add to MetaCart
suggests a reasonable line of research: find algorithms that can search the hypothesis class better. Hence, there is been extensive research in applying secondorder methods to fit neural networks and in conducting much more thorough searches in learning decision trees and rule sets. Ironically, when these algorithms were tested on real datasets, it was found that their performance was often worse than simple gradient descent or greedy search [3, 5]. In short: it appears to be better not to optimize! One of the other important trends in machine learning research has been the establishment and nurturing of connections between various previouslydisparate fields including computational learning theory, connectionist learning, symbolic learning, and statistics. The connection to statistics was crucial in resolving this paradox. The key problem arises from the structure of the machine learning task. A learning algorithm is trained on a set of training data, but then it is applied to make
PAC Learning Intersections of Halfspaces with Membership Queries
 ALGORITHMICA
, 1998
"... A randomized learning algorithm Polly is presented that efficiently learns intersections of s halfspaces in n dimensions, in time polynomial in both s and n. The learning protocol is the "PAC" (probably approximately correct) model of Valiant, augmented with membership queries. In particular, Polly ..."
Abstract

Cited by 21 (1 self)
 Add to MetaCart
A randomized learning algorithm Polly is presented that efficiently learns intersections of s halfspaces in n dimensions, in time polynomial in both s and n. The learning protocol is the "PAC" (probably approximately correct) model of Valiant, augmented with membership queries. In particular, Polly receives a set S of m = poly(n; s; 1=ffl; 1=ffi) randomly generated points from an arbitrary distribution over the unit hypercube, and is told exactly which points are contained in, and which points are not contained in, the convex polyhedron P defined by the halfspaces. Polly may also obtain the same information about points of its own choosing. It is shown that after poly(n, s, 1=ffl, 1=ffi, log(1=d)) time, the probability that Polly fails to output a collection of s halfspaces with classification error at most ffl, is at most ffi . Here, d is the minimum distance between the boundary of the target and those examples in S that are not lying on the boundary. The parameter log(1=d) can be ...
Complexity Theoretic Hardness Results for Query Learning
 COMPUTATIONAL COMPLEXITY
, 1998
"... We investigate the complexity of learning for the wellstudied model in which the learning algorithm may ask membership and equivalence queries. While complexity theoretic techniques have previously been used to prove hardness results in various learning models, these techniques typically are no ..."
Abstract

Cited by 19 (5 self)
 Add to MetaCart
We investigate the complexity of learning for the wellstudied model in which the learning algorithm may ask membership and equivalence queries. While complexity theoretic techniques have previously been used to prove hardness results in various learning models, these techniques typically are not strong enough to use when a learning algorithm may make membership queries. We develop a general technique for proving hardness results for learning with membership and equivalence queries (and for more general query models). We apply the technique to show that, assuming NP != coNP, no polynomialtime membership and (proper) equivalence query algorithms exist for exactly learning readthrice DNF formulas, unions of k 3 halfspaces over the Boolean domain, or some other related classes. Our hardness results are representation dependent, and do not preclude the existence of representation independent algorithms. The general
Probabilistic Analysis of Learning in Artificial Neural Networks: The PAC Model and its Variants
, 1997
"... There are a number of mathematical approaches to the study of learning and generalization in artificial neural networks. Here we survey the `probably approximately correct' (PAC) model of learning and some of its variants. These models provide a probabilistic framework for the discussion of generali ..."
Abstract

Cited by 18 (4 self)
 Add to MetaCart
There are a number of mathematical approaches to the study of learning and generalization in artificial neural networks. Here we survey the `probably approximately correct' (PAC) model of learning and some of its variants. These models provide a probabilistic framework for the discussion of generalization and learning. This survey concentrates on the sample complexity questions in these models; that is, the emphasis is on how many examples should be used for training. Computational complexity considerations are briefly discussed for the basic PAC model. Throughout, the importance of the VapnikChervonenkis dimension is highlighted. Particular attention is devoted to describing how the probabilistic models apply in the context of neural network learning, both for networks with binaryvalued output and for networks with realvalued output.
NoiseTolerant DistributionFree Learning of General Geometric Concepts
, 1996
"... this paper. First, we give an algorithm to learn C ..."
Abstract

Cited by 16 (3 self)
 Add to MetaCart
this paper. First, we give an algorithm to learn C
Recurrent Neural Networks and Prior Knowledge for Sequence Processing: A Constrained Nondeterministic Approach
, 1995
"... this paper we focus on processing sequential streams of data by recurrent neural networks ..."
Abstract

Cited by 14 (5 self)
 Add to MetaCart
this paper we focus on processing sequential streams of data by recurrent neural networks
On the Complexity of Optimization Problems for 3Dimensional Convex Polyhedra and Decision Trees
 Comput. Geom. Theory Appl
, 1995
"... We show that several wellknown optimization problems involving 3dimensional convex polyhedra and decision trees are NPhard or NPcomplete. One of the techniques we employ is a lineartime method for realizing a planar 3connected triangulation as a convex polyhedron, which may be of independent i ..."
Abstract

Cited by 14 (0 self)
 Add to MetaCart
We show that several wellknown optimization problems involving 3dimensional convex polyhedra and decision trees are NPhard or NPcomplete. One of the techniques we employ is a lineartime method for realizing a planar 3connected triangulation as a convex polyhedron, which may be of independent interest. Key words: Convex polyhedra, approximation, Steinitz's theorem, planar graphs, art gallery theorems, decision trees. 1 Introduction Convex polyhedra are fundamental geometric structures (e.g., see [20]). They are the product of convex hull algorithms, and are key components for problems in robot motion planning and computeraided geometric design. Moreover, due to a beautiful theorem of Steinitz [20, 38], they provide a strong link between computational geometry and graph theory, for Steinitz shows that a graph forms the edge structure of a convex polyhedra if and only if it is planar and 3connected. Unfortunately, algorithmic problems dealing with 3dimensional convex polyhedra ...