This paper investigates fault tolerance in feedforward neural networks, for a realistic fault model based on analog hardware. In our previous work with synaptic weight noise [26] we showed significant fault tolerance enhancement over standard training algorithms. We proposed that when introduced into training, weight noise distributes the network computationmore evenly across the weights and thus enhances fault tolerance. Here we compare those results with an approximation to the mechanisms induced by stochastic weight noise, incorporated into training deterministicallyvia penalty terms. The penalty terms are an approximation to weight saliency and therefore, in addition, we assess a number of other weight saliency measures and perform comparison experiments. The results show that the first term approximation is an incomplete model of weight noise in terms of fault tolerance. Also the error hessian is shown to be the most accurate measure of weight saliency. 1 Introduction The presenc...

This paper details the experiments carried out with the EPSILON processor card, configured as a MultiLayer Perceptron, and networks optimised for fault tolerance performance using the weight-noise training technique. The aim of the experiments is to show that networks can be trained in advance to be fault tolerant; to be able to compensate for hardware errors after being downloaded onto a hardware platform. The results show that while there is the potential for obtaining such fault tolerant weights sets, robust noise-trained networks require an increased dynamic weight range. This case study with the EPSILON processor card highlights what we believe to be a number of common inadequacies with custom designed hardware. In particular the limitations of EPSILON in terms of its dynamic range performance has been shown to be a problem. In summary, we show that networks trained with weight-noise are fault tolerant but also require an increased dynamic range to exploit the benefits. I. INTRODU...

This work argues that it should be possible to combine pulse-based VLSI techniques with the relatively simple training rules of the Helmholtz Machine stochastic neural architecture, in order to build an analogue probabilistic hardware model of the latter. An overview of the necessary components is presented, as well as a design for a pulse-width modulation oscillator, capable of transforming a current input (which represents the squashed, post-synaptic signal processed by a particular neuron) into the probability associated with the binary state of that neuron. A CMOS hardware prototype has been designed and fabricated, and precautions were taken during the design and simulation stages in order to prevent the oscillators on the same chip from locking together. Apart from testing the hardware prototype, future plans involve the hardware implementation of other modules, such as the synapse, the squashing function and weight changing circuitry.

In this thesis a new signal processing technique is presented. This technique exploits the use of pulses as the signalling mechanism. This Palmo 1 signalling method applied to signal processing is novel, combining the advantages of both digital and analogue techniques. Pulsed signals are robust, inherently low-power, easily regenerated, and easily distributed across and between chips. The Palmo cells used to perform analogue operations on the pulsed signals are compact, fast, simple and programmable.