this article we outline the foundations of such a theory, working in the general framework of Bayesian inference. Much of our proposal for extending Shepard's theory to the cases of multiple examples and arbitrary stimulus structures has already been introduced in other papers (Griffiths & Tenenbaum, 2000; Tenenbaum, 1997, 1999a, 1999b; Tenenbaum & Xu, 2000). Our goal here is to make explicit the link to Shepard's work and to use our framework to make connections between his work and other models of learning (Feldman, 1997; Gluck & Shanks, 1994; Haussler, Kearns & Schapire, 1994; Kruschke, 1992; Mitchell, 1997), generalization (Nosofsky, 1986; Heit, 1998), and similarity (Chater & Hahn, 1997; Medin, Goldstone & Gentner, 1993; Tversky, 1977). In particular, we will have a lot to say about how our generalization of Shepard's theory relates to Tversky's (1977) well-known set-theoretic models of similarity. Tversky's set-theoretic approach and Shepard's metric space approach are often considered the two classic -- and classically opposed -- theories of similarity and generalization. By demonstrating close parallels between Tversky's approach and our Bayesian generalization of Shepard's approach, we hope to go some way towards unifying these two theoretical approaches and advancing the explanatory power of each. The plan of our article is as follows. In Section 2, we recast Shepard's analysis of generalization in a more general Bayesian framework, preserving the basic principles of his approach in a form that allows us to apply the theory to situations with multiple examples and arbitrary (non-spatially represented) stimulus structures. Sections 3 and 4 describe those extensions, and Section 5 concludes by discussing some implications of our theory for the internalization of...

Abstract not found

We explore the connections between machine learning and human learning in one form of semi-supervised classification. 22 human subjects completed a novel 2class categorization task in which they were first taught to categorize a single labeled example from each category, and subsequently were asked to categorize, without feedback, a large set of additional items. Stimuli were visually complex and unrecognizable shapes. The unlabeled examples were sampled from a bimodal distribution with modes appearing either to the left (leftshift condition) or right (right-shift condition) of the two labeled examples. Results showed that, although initial decision boundaries were near the middle of the two labeled examples, after exposure to the unlabeled examples, they shifted in different directions in the two groups. In this respect, the human behavior conformed well to the predictions of a Gaussian mixture model for semi-supervised learning. The human behavior differed from model predictions in other interesting respects, suggesting some fruitful avenues for future inquiry.

This paper presents a previously unnoticed universal property of stress patterns in the world’s languages: they are, for small neighborhoods, neighborhood-distinct. Neighborhood-distinctness is a locality condition defined in automata-theoretic terms. This universal is established by examining stress patterns contained in two typological studies, Bailey (1995) and Gordon (2002). Strikingly, many logically possible— but unattested—patterns do not have this property. Not only does neighborhood-distinctness unite the attested patterns in a non-trivial way, it also naturally provides an inductive principle allowing learners to generalise from limited data. A learning algorithm is presented which generalises by failing to distinguish same-neighborhood environments perceived in the learner’s linguistic input—hence learning neighborhood-distinct patterns—as well as almost every stress pattern in the typology. In this way, this work lends support to the idea that properties of the learner can explain certain properties of the attested typology, an idea not straightforwardly available in Optimality-theoretic and Principle and Parameter frameworks.

This thesis investigates language acquisition and evolution, using the methodologies of Bayesian inference and expression-induction modelling, making specific reference to colour term typology, and syntactic acquisition. In order to test Berlin and Kay's (1969) hypothesis that the typological patterns observed in basic colour term systems are produced by a process of cultural evolution under the influence of universal aspects of human neurophysiology, an expression-induction model was created. Ten artificial people were simulated, each of which was a computational agent. These people could learn colour term denotations by generalizing from examples using Bayesian inference, and the resulting denotations had the prototype properties characteristic of basic colour terms.

Abstract — Serious efforts to develop computerized systems for natural language understanding and machine translation have taken place for more than half a century. Some successful systems that translate texts in limited domains such as weather forecasts have been implemented. However, the more general the domain or complex the style of the text the more difficult it is to reach high quality translation. The same applies to natural language understanding. All systems need to deal with problems like ambiguity, lack of semantic coverage and pragmatic insight. In this article, some philosophical questions that underlie the difficulty of natural language understanding and good quality translation are first studied. These two areas of dealing with languages are actually closely related. Namely, for instance Quine’s notion of indeterminacy of translation have shown that the problem of translation does not only hold for translation between different languages but similar problems are encountered when communication between users of same language is considered. The term intralingual translation has been used e.g. by Roman Jakobson. Intralingual translation relates to translation between languages and to the problem of sameness of meaning. In this article, arguments and methods of considering translation and meaning within the framework of continuous-valued multidimensional representations, probability theory, fuzzy sets and neural adaptive systems are considered. I.

We present a novel experimental method for identifying the inductive biases of human learners. The key idea behind this method is simple: we use participants ’ responses on one trial to generate the stimuli they see on the next. A theoretical analysis of this “iterated learning” procedure, based on the assumption that learners are Bayesian agents, predicts that it should reveal the inductive biases of the learners, as expressed in a prior probability distribution. We test this prediction through two experiments in iterated category learning. Many of the cognitive challenges faced by human beings can be framed as inductive problems, in which observed data are used to evaluate underdetermined hypotheses. To take two common examples, in language acquisition the hypotheses are languages and the data are the utterances to which the learner is exposed, while

A popular view holds that learning and generalizing concepts depends on two fundamentally distinct modes of representation: rules and similarityto -exemplars. Through a combination of experiments and formal analysis, I show how a Bayesian framework offers a unifying account of both rule-based and similarity-based generalization. Bayes explains the specific workings of these two modes -- which rules are abstracted, how similarity is measured -- as well as why generalization appears rule-based or similarity-based in different situations. I conclude that the distinction between rules and similarity in concept learning may be useful at the level of heuristic algorithms, but is not computationally fundamental. 1 Introduction In domains ranging from reasoning to language acquisition, a broad view is emerging of cognition as a hybrid of two distinct modes of computation, one based on applying abstract rules and the other based on assessing similarity to stored exemplars [6]. Much s...

People are active experimenters, constantly seeking new information relevant to their goals. A reasonable approach to active information gathering is to ask questions and conduct experiments that minimize the expected state of uncertainty, or maximize the expected information gain, given current beliefs (Fedorov, 1972; MacKay, 1992; Oaksford & Chater, 1994). In this paper we present results on an exploratory experiment designed to study people's active information gathering behavior on a concept learning task. The results of the experiment suggest subjects' behavior may be explained well from the point of view of Bayesian information maximization.

Towards a rational theory of human information acquisition