Results 1  10
of
161
A solution to Plato’s problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge
 Psychological review
, 1997
"... How do people know as much as they do with as little information as they get? The problem takes many forms; learning vocabulary from text is an especially dramatic and convenient case for research. A new general theory of acquired similarity and knowledge representation, latent semantic analysis (LS ..."
Abstract

Cited by 1093 (9 self)
 Add to MetaCart
How do people know as much as they do with as little information as they get? The problem takes many forms; learning vocabulary from text is an especially dramatic and convenient case for research. A new general theory of acquired similarity and knowledge representation, latent semantic analysis (LSA), is presented and used to successfully simulate such learning and several other psycholinguistic phenomena. By inducing global knowledge indirectly from local cooccurrence data in a large body of representative text, LSA acquired knowledge about the full vocabulary of English at a comparable rate to schoolchildren. LSA uses no prior linguistic or perceptual similarity knowledge; it is based solely on a general mathematical learning method that achieves powerful inductive effects by extracting the right number of dimensions (e.g., 300) to represent objects and contexts. Relations to other theories, phenomena, and problems are sketched. Prologue "How much do we know at any time? Much more, or so I believe, than we know we know!" —Agatha Christie, The Moving Finger A typical American seventh grader knows the meaning of
Learning and development in neural networks: The importance of starting small
 Cognition
, 1993
"... It is a striking fact that in humans the greatest learnmg occurs precisely at that point in time childhood when the most dramatic maturational changes also occur. This report describes possible synergistic interactions between maturational change and the ability to learn a complex domain (language ..."
Abstract

Cited by 368 (15 self)
 Add to MetaCart
It is a striking fact that in humans the greatest learnmg occurs precisely at that point in time childhood when the most dramatic maturational changes also occur. This report describes possible synergistic interactions between maturational change and the ability to learn a complex domain (language), as investigated in connectionist networks. The networks are trained to process complex sentences involving relative clauses, number agreement, and several types of verb argument structure. Training fails in the case of networks which are fully formed and ‘adultlike ’ in their capacity. Training succeeds only when networks begin with limited working memory and gradually ‘mature ’ to the adult state. This result suggests that rather than being a limitation, developmental restrictions on resources may constitute a necessary prerequisite for mastering certain complex domains. Specifically, successful learning may depend on starting small.
Biometric identification
 Communications of the ACM
, 2000
"... Identification of grammars (r. e. indices) for recursively enumerable languages from positive data by algorithmic devices is a well studied problem in learning theory. The present paper considers identification of r. e. languages by machines that have access to membership oracles for noncomputable s ..."
Abstract

Cited by 60 (4 self)
 Add to MetaCart
Identification of grammars (r. e. indices) for recursively enumerable languages from positive data by algorithmic devices is a well studied problem in learning theory. The present paper considers identification of r. e. languages by machines that have access to membership oracles for noncomputable sets. It is shown that for any set A there exists another set B such that the collections of r. e. languages that can be identified by machines with access to a membership oracle for B is strictly larger than the collections of r. e. languages that can be identified by machines with access to a membership oracle for A. In other words, there is no maximal inference degree for language identification.
The Power of Vacillation in Language Learning
, 1992
"... Some extensions are considered of Gold's influential model of language learning by machine from positive data. Studied are criteria of successful learning featuring convergence in the limit to vacillation between several alternative correct grammars. The main theorem of this paper is that there are ..."
Abstract

Cited by 44 (11 self)
 Add to MetaCart
Some extensions are considered of Gold's influential model of language learning by machine from positive data. Studied are criteria of successful learning featuring convergence in the limit to vacillation between several alternative correct grammars. The main theorem of this paper is that there are classes of languages that can be learned if convergence in the limit to up to (n+1) exactly correct grammars is allowed but which cannot be learned if convergence in the limit is to no more than n grammars, where the no more than n grammars can each make finitely many mistakes. This contrasts sharply with results of Barzdin and Podnieks and, later, Case and Smith, for learnability from both positive and negative data. A subset principle from a 1980 paper of Angluin is extended to the vacillatory and other criteria of this paper. This principle, provides a necessary condition for circumventing overgeneralization in learning from positive data. It is applied to prove another theorem to the eff...
Incremental concept learning for bounded data mining
 Information and Computation
, 1999
"... Important re nements of concept learning in the limit from positive data considerably restricting the accessibility of input data are studied. Let c be any concept; every in nite sequence of elements exhausting c is called positive presentation of c. In all learning models considered the learning ma ..."
Abstract

Cited by 39 (29 self)
 Add to MetaCart
Important re nements of concept learning in the limit from positive data considerably restricting the accessibility of input data are studied. Let c be any concept; every in nite sequence of elements exhausting c is called positive presentation of c. In all learning models considered the learning machine computes a sequence of hypotheses about the target concept from a positive presentation of it. With iterative learning, the learning machine, in making a conjecture, has access to its previous conjecture and the latest data item coming in. In kbounded examplememory inference (k is a priori xed) the learner is allowed to access, in making a conjecture, its previous hypothesis, its memory of up to k data items it has already seen, and the next element coming in. In the case of kfeedback identi cation, the learning machine, in making a conjecture, has access to its previous conjecture, the latest data item coming in, and, on the basis of this information, it can compute k items and query the database of previous data to nd out, for each of the k items, whether or not it is in the database (k is again a priori xed). In all cases, the sequence of conjectures has to converge to a hypothesis
On the computational content of the axiom of choice
 The Journal of Symbolic Logic
, 1998
"... We present a possible computational content of the negative translation of classical analysis with the Axiom of Choice. Our interpretation seems computationally more direct than the one based on Godel's Dialectica interpretation [10, 18]. Interestingly, thisinterpretation uses a re nement of the rea ..."
Abstract

Cited by 34 (1 self)
 Add to MetaCart
We present a possible computational content of the negative translation of classical analysis with the Axiom of Choice. Our interpretation seems computationally more direct than the one based on Godel's Dialectica interpretation [10, 18]. Interestingly, thisinterpretation uses a re nement of the realizibility semantics of the absurdity proposition, which is not interpreted as the empty type here. We alsoshowhow to compute witnesses from proofs in classical analysis, and how to interpret the axiom of Dependent Choice and Spector's Double Negation Shift.
Types of Monotonic Language Learning and Their Characterization
 in "Proceedings 5th Annual ACM Workshop on Computational Learning Theory," July 27  29, Pittsburgh
, 1992
"... The present paper deals with strongmonotonic, monotonic and weakmonotonic language learning from positive data as well as from positive and negative examples. The three notions of monotonicity reflect different formalizations of the requirement that the learner has to produce always better and b ..."
Abstract

Cited by 32 (26 self)
 Add to MetaCart
The present paper deals with strongmonotonic, monotonic and weakmonotonic language learning from positive data as well as from positive and negative examples. The three notions of monotonicity reflect different formalizations of the requirement that the learner has to produce always better and better generalizations when fed more and more data on the concept to be learnt. We characterize strong monotonic, monotonic, weakmonotonic and finite language learning from positive data in terms of recursively generable finite sets, thereby solving a problem of Angluin (1980). Moreover, we study monotonic inference with iteratively working learning devices which are of special interest in applications. In particular, it is proved that strongmonotonic inference can be performed with iteratively learning devices without limiting the inference capabilities, while monotonic and weakmonotonic inference cannot. 1 Introduction The process of hypothesizing a general rule from eventually inc...
Language Learning from Texts: Mind Changes, Limited Memory and Monotonicity (Extended Abstract)
 INFORMATION AND COMPUTATION
, 1995
"... The paper explores language learning in the limit under various constraints on the number of mindchanges, memory, and monotonicity. We define language learning with limited (long term) memory and prove that learning with limited memory is exactly the same as learning via set driven machines (when t ..."
Abstract

Cited by 26 (9 self)
 Add to MetaCart
The paper explores language learning in the limit under various constraints on the number of mindchanges, memory, and monotonicity. We define language learning with limited (long term) memory and prove that learning with limited memory is exactly the same as learning via set driven machines (when the order of the input string is not taken into account). Further we show that every language learnable via a set driven machine is learnable via a conservative machine (making only justifiable mindchanges). We get a variety of separation results for learning with bounded number of mindchanges or limited memory under restrictions on monotonicity. Many separation results have a variant: If a criterion A can be separated from B, then often it is possible to find a family L of languages such that L is A and B learnable, but while it is possible to restrict the number of mindchanges or long term memory...
Monotonic Versus Nonmonotonic Language Learning
, 1993
"... In the present paper strongmonotonic, monotonic and weakmonotonic reasoning is studied in the context of algorithmic language learning theory from positive as well as from positive and negative data. Strongmonotonicity describes the requirement to only produce better and better generalization ..."
Abstract

Cited by 21 (13 self)
 Add to MetaCart
In the present paper strongmonotonic, monotonic and weakmonotonic reasoning is studied in the context of algorithmic language learning theory from positive as well as from positive and negative data. Strongmonotonicity describes the requirement to only produce better and better generalizations when more and more data are fed to the inference device. Monotonic learning reflects the eventual interplay between generalization and restriction during the process of inferring a language. However, it is demanded that for any two hypotheses the one output later has to be at least as good as the previously produced one with respect to the language to be learnt. Weakmonotonicity is the analogue of cumulativity in learning theory. We relate all these notions one to the other as well as to previously studied modes of identification, thereby in particular obtaining a strong hierarchy.
A Bayesian Framework for Concept Learning
 DEPARTMENT OF ARTIFICIAL INTELLIGENCE, EDINBURGH UNIVERSITY
, 1999
"... Human concept learning presents a version of the classic problem of induction, which is made particularly difficult by the combination of two requirements: the need to learn from a rich (i.e. nested and overlapping) vocabulary of possible concepts and the need to be able to generalize concepts reaso ..."
Abstract

Cited by 21 (3 self)
 Add to MetaCart
Human concept learning presents a version of the classic problem of induction, which is made particularly difficult by the combination of two requirements: the need to learn from a rich (i.e. nested and overlapping) vocabulary of possible concepts and the need to be able to generalize concepts reasonably from only a few positive examples. I begin this thesis by considering a simple number concept game as a concrete illustration of this ability. On this task, human learners can with reasonable confidence lock in on one out of a billion billion billion logically possible concepts, after seeing only four positive examples of the concept, and can generalize informatively after seeing just a single example. Neither of the two classic approaches to inductive inference  hypothesis testing in a constrained space of possible rules and computing similarity to the observed examples  can provide a complete picture of how people generalize concepts in even this simple setting. This thesis prop...