ed on visual and speech data. The ability of the network to automatically generate wavelet codes from natural images is demonstrated. These bear a close resemblance to 2-D Gabor functions, which have previously been used to describe physiological receptive fields, and as a means of producing compact image representations. Keywords: neural networks, unsupervised learning, self-organisation, feature extraction, information theory, redundancy reduction, sparse coding, imaging models, occlusion, image coding, speech coding. Declaration This dissertation is the result of my own original work, except where reference is made to the work of others. No part of it has been submitted for any other university degree or diploma. Its length, including captions, footnotes, appendix and bibliography, is approximately 58000 words. Acknowledgements I would like first and foremost to thank Richard Prager, my supervisor, fo

A large and influential class of neural network architectures use post-integration lateral inhibition as a mechanism for competition. We argue that these algorithms are computationally deficient in that they fail to generate, or learn, appropriate perceptual representations under certain circumstances. An alternative neural network architecture is presented in which nodes compete for the right to receive inputs rather than for the right to generate outputs. This form of competition, implemented through pre-integration lateral inhibition, does provide appropriate coding properties and can be used to efficiently learn such representations. Furthermore, this architecture is consistent with both neuro-anatomical and neuro-physiological data. We thus argue that pre-integration lateral inhibition has computational advantages over conventional neural network architectures while remaining equally biologically plausible.

A new way of measuring generalization in unsupervised learning is presented. The measure is based on an exclusive allocation, or credit assignment , criterion. In a classifier that satisfies the criterion, input patterns are parsed so that the credit for each input feature is assigned exclusively to one of multiple, possibly overlapping, output categories. Such a classifier achieves context-sensitive, global representations of pattern data. Two additional constraints, sequence masking and uncertainty multiplexing, are described; these can be used to refine the measure of generalization. The generalization performance of EXIN networks, winner-take-all competitive learning networks, linear decorrelator networks, and Nigrin's SONNET--2 network is compared. Keywords Generalization, Exclusive allocation, Credit assignment, Binding, Unsupervised learning, Pattern classification, Distributed coding, EXIN (excitatory+inhibitory) learning, Sparse coding, Rule extraction, Regularization, Blind ...

The position, size, and shape of the visual receptive field (RF) of some primary visual cortical neurons change dynamically, in response to artificial scotoma conditioning in cats (Pettet & Gilbert, 1992) and to retinal lesions in cats and monkeys (DarianSmith & Gilbert, 1995). The "EXIN" learning rules (Marshall, 1995) are used to model dynamic RF changes. The EXIN model is compared with an adaptation model (Xing & Gerstein, 1994) and the LISSOM model (Sirosh & Miikkulainen, 1994; Sirosh et al., 1996). To emphasize the role of the lateral inhibitory learning rules, the EXIN and the LISSOM simulations were done with only lateral inhibitory learning. During scotoma conditioning, the EXIN model without feedforward learning produces centrifugal expansion of RFs initially inside the scotoma region, accompanied by increased responsiveness, without changes in spontaneous activation. The EXIN model without feedforward learning is more consistent with the neurophysiological data than are the a...

When interpreting the results of experiments that investigate biological development, one is faced with a wealth of data. Producing a model of such development must always involve some degree of abstraction. The appropriate level of abstraction and the importance of particular experimental evidence is determined by one's modelling objective. Models may potentially be motivated by one of two complementary aims: 1. To understand how biological neurons achieve their mature interconnectivity...

Self-organizing computational models with specific intracortical connections can explain many functional features of visual cortex, such as topographic orientation and ocular dominance maps. However, due to their computational requirements, it is difficult to use such detailed models to study large-scale phenomena like object segmentation and binding, object recognition, tilt illusions, optic flow, and fovea periphery interaction. This paper introduces two techniques that make large simulations practical. First, a set of general linear scaling equations for the RF-LISSOM self-organizing model is derived and shown to result in quantitatively equivalent maps over a wide range of simulation sizes. This capability makes it possible to debug small simulations and then scale them up to larger simulations only when needed. The scaling equations also facilitate the comparison of biological maps and parameters between individuals and species with different brain region sizes. Second, the equations are combined into a new growing map method called GLISSOM, which dramatically reduces the memory and computational requirements of large self-organizing networks. With GLISSOM it should be possible to simulate all of human V1 at the single-column level using existing supercomputers, making detailed computational study of large-scale phenomena possible.

Intracortical microstimulation (ICMS) of a localized site in the somatosensory cortex of rats and monkeys for 2--6 hours produces a large increase in the cortical representation of the skin region represented by the ICMS-site neurons before ICMS, with very little effect on the ICMS-site neuron's RF location, RF size, and responsiveness (Recanzone et al., 1992). The "EXIN" (afferent excitatory and lateral inhibitory) learning rules (Marshall, 1995) are used to model RF changes during ICMS. The EXIN model produces reorganization of RF topography similar to that observed experimentally. The possible role of inhibitory learning in producing the effects of ICMS is studied by simulating the EXIN model with only lateral inhibitory learning. The model also produces an increase in the cortical representation of the skin region represented by the ICMS-site RF. ICMS is compared to artificial scotoma conditioning (Pettet & Gilbert, 1992) and retinal lesions (Darian-Smith & Gilbert, 1995), and it i...

Inhibitory synapses contacting the soma and axon initial segment are commonly presumed to participate in shaping the response properties of cortical pyramidal cells. Such an inhibitory mechanism has been explored in numerous computational models. However, the majority of inhibitory synapses target the dendrites of pyramidal cells, and recent physiological data suggests that this dendritic inhibition affects tuning properties. We describe a model that can be used to investigate the role of dendritic inhibition in the competition between neurons. With this model we demonstrate that dendritic inhibition significantly enhances the computational and representational properties of neural networks.

We study a model of the cortical macrocolumn consisting of a collection of inhibitorily coupled minicolumns. The proposed system overcomes several severe deficits of systems based on single neurons as cerebral functional units, notably limited robustness to damage and unrealistically large computation time. Motivated by neuroanatomical and neurophysiological findings the utilized dynamics is based on a simple model of a spiking neuron with refractory period, fixed random excitatory interconnection within minicolumns, and instantaneous inhibition within one macrocolumn. A stability analysis of the system’s dynamical equations shows that minicolumns can act as monolithic functional units for purposes of critical, fast decisions and learning. Oscillating inhibition (in the gamma frequency range) leads to a phase-coupled population rate code and high sensitivity to small imbalances in minicolumn inputs. Minicolumns are shown to be able to organize their collective inputs without supervision by Hebbian plasticity into selective receptive field shapes, thereby becoming classifiers for input patterns. Using the bars test, we critically compare our system’s performance with that of others and demonstrate its ability for distributed neural coding.

A long running debate has concerned the question of whether neural representations are encoded using a distributed or a local coding scheme. In both schemes individual neurons respond to certain specific patterns of pre-synaptic activity. Hence, rather than being dichotomous, both coding schemes are based on the same representational mechanism. We argue that a population of neurons needs to be capable of learning both local and distributed representations, as appropriate to the task, and should be capable of generating both local and distributed codes in response to different stimuli. Many neural network algorithms, which are often employed as models of cognitive processes, fail to meet all these requirements. In contrast, we present a neural network architecture which enables a single algorithm to efficiently learn, and respond using, both types of coding scheme.