~ ROA Version, 8/2002. Essentially identical to the Tech Report, with new pagination (but the same footnote and example numbering); correction of typos, oversights & outright errors; improved typography; and occasional small-scale clarificatory rewordings. Citation should include reference to this version.

Computational models in cognitive neuroscience should ideally use biological properties and powerful computational principles to produce behavior consistent with psychological findings. Error-driven backpropagation is computationally powerful, and has proven useful for modeling a range of psychological data, but is not biologically plausible. Several approaches to implementing backpropagation in a biologically plausible fashion converge on the idea of using bidirectional activation propagation in interactive networks to convey error signals. This paper demonstrates two main points about these error-driven interactive networks: (a) they generalize poorly due to attractor dynamics that interfere with the network's ability to systematically produce novel combinatorial representations in response to novel inputs; and (b) this generalization problem can be remedied by adding two widely used mechanistic principles, inhibitory competition and Hebbian learning, that can be independent...

Introduction Connectionist networks have earned recognition in many domains that can be characterised as hard or impossible to explicitly formalise, e.g., driving cars (Pomerleau, 1993), emotion recognition (Cottrell and Metcalfe, 1990) and pronunciation (Sejnowski and Rosenberg, 1987). Connectionists have also claimed that their networks can exhibit behaviours that can be described by a set of formal rules, without actually implementing explicit rule following (McClelland & Rumelhart 1985). The radical implication of this claim is that connectionism does not appear to neatly line up with the classical view of the cognitive architecture, i.e., the computational, the representational and the implementational levels (cf., Marr 1982; Pylyshyn 1984; Newell 1986; Andersson 1983). The intention of this chapter is to investigate a number of different aspects of this claim. We will compare two computationally equivalent systems. The behaviour of both these systems can be described by

Standard feedforward and recurrent networks cannot support strong systematicity when constituents are presented as local input/output vectors (Phillips, 1998). To explain systematicity connectionists must either: (1) develop alternative models; or (2) justify the assumption of similar (non-local) constituent representations prior to the learning task. I show that the second commonly presumed option cannot account for systematicity, in general. This option, termed first-order connectionism, relies upon established spatial relationships between common-class constituents to account for systematic generalization: Inferences (functions) learned over, e.g., cats extend systematically to dogs by virtue of both being nouns with similar internal representations so that the function learned to make inferences employing one simultaneously has the capacity to make inferences employing the other. But, humans generalize beyond common-class constituents. Cross-category generalization (e.g., inferences that require treating mango as a colour, rather than a fruit) makes having had the necessary common context to learn similar constituent representations highly unlikely. At best, the constituent similarity proposal encodes for one binary relationship between any two constituents, at any one time. It cannot account for inferences, such as transverse patterning that require identifying and applying one of many possible binary constituent relationships that is contingent on a third constituent (i.e., ternary relationship). Connectionists are, therefore, left with the first option which amounts to developing models with the symbol-like capacity to explicitly represent constituent relations independent of constituent contents, such as in tensor-related models. However, rather just simply impl...

The authors would like to thank Prof. Paul Smolensky for his comments and suggestions. This work