A new algorithm, called Hamming Clustering (HC), for the solution of classification problems with binary inputs is proposed. It builds a logical network containing only and, or and not ports, which, besides satisfying all the input-output pairs included in a given finite consistent training set, is able to reconstruct the underlying Boolean function. The basic

We analyse the effects of analog noise on the synaptic arithmetic during MultiLayer Perceptron training, by expanding the cost function to include noise-mediated terms. Predictions are made in the light of these calculations which suggest that fault tolerance, training quality and training trajectory should be improved by such noise-injection. Extensive simulation experiments on two distinct classification problems substantiate the claims. The results appear to be perfectly general for all training schemes where weights are adjusted incrementally, and have wide-ranging implications for all applications, particularly those involving "inaccurate" analog neural VLSI. 1 Introduction and Background Arithmetic inaccuracy at the synapse and neuron level is widely held to be tolerable during neural computation, but not during training. In arriving at this conclusion, parallels are drawn between analog noise-induced "uncertainty", and digital inaccuracy, limited by bit-length. This has lead ...

In this paper we analyse the training dynamics of limited precision feedforward multilayer perceptrons implemented in digital hardware. We show that special techniques have to be employed to train such networks where each variable is quantised to a limited number of bits. Based on the analysis, we propose a Combined Search (CS) training algorithm which consists of partial random search and weight perturbation and can easily be implemented in hardware. Computer simulations were conducted on IntraCardiac ElectroGrams and sonar reflection pattern classification problems. The results show that using CS, the training performance of limited precision feedforward MLPs with 8 to 10 bit resolution can be as good as that of unlimited precision networks. The results also show that CS is insensitive to training parameter variations. 1 Introduction When neural networks are to be used on limited precision digital hardware, problems may arise in their training because all network parameter...

Artificial neural networks (ANNs) involve a large amount of inter-node communications. To reduce the communication cost as well as the time of learning process in ANNs, we earlier proposed an incremental inter-node communication method. In the incremental communication method, instead of communicating the full magnitude of the output value of a node, only the increment or decrement to its previous value is sent on a communication link. In this paper, the effects of the limited precision incremental communication method on the convergence behavior and performance of multilayer neural networks are investigated. The nonlinear aspects of representing the incremental values with reduced (limited) precision for the commonly used error backpropagation training algorithm are analyzed. It is shown that the nonlinear effect of small perturbations in the input(s)/output of a node does not enforce instability. The analysis is supported by simulation studies of two problems. The simulation results ...

Quantization of the parameters of a Perceptron is a central problem in hardware implementation of neural networks using a numerical technology. A neural model with each weight limited to a small integer range will require little surface of silicon. Moreover, according to Ockham's razor principle, better generalization abilities can be expected from a simpler computational model. The price to pay for these benefits lies in the difficulty to train these kind of networks. This paper proposes essentially two new ideas for constructive training algorithms, and demonstrates their efficiency for the generation of feedforward networks composed of Boolean threshold gates with discrete weights. A proof of the convergence of these algorithms is given. Some numerical experiments have been carried out and the results are presented in terms of the size of the generated networks and of their generalization abilities. 1 Introduction Artificial neural networks (ANN) are proposed today as alternative...

Current neurocomputers are more than 50 million times slower than the brain. Although chip speeds exceed the switching speed of biological neurons with several orders of magnitude, artificial neural networks are of a much smaller scale than real brains. The primary aim of most neurocomputer designs is speeding up neural paradigms rather than implementing large-scale neural networks. In order to simulate neural networks of brain-size, neurocomputers need to be scaled up. We here present MINDSHAPE which is a design concept for a very large-scale neurocomputer based on a hierarchical-modular or Fractal Architecture. A Fractal Architecture can be built up from two types of elements: neural processing elements (NPEs) and communication elements (CEs). Massive usage of these elements allows for both distributed calculation and distributed control. A detailed description of this machine is presented, with reference to a realized feasibility study (the BSP400, see [1][2]). Through performance a...

: The analog VLSI implementation looks an attractive way for implementing Artificial Neural Networks; in fact, it gives small area, low power consumption and compact design of neural computational primitive circuits. On the other hand, major drawbacks result to be the low computational accuracy and the non-linear behaviour of analog circuits. In this paper, we present the design and the detailed behavioural models of CMOS multipliers for the analog VLSI implementation of neural algorithms. The circuits implement the feedforward operations of the Multi Layer Perceptron architecture and of the Back Propagation (on-chip learning) algorithm; they operate in the subthreshold regime to obtain a low power consumption and high dynamic range of weights. The circuit behavioural models take into account: i) non-linearity effects; ii) environmental effects (variations of temperature and of signal reference voltage). The models that we present in this paper, are used in the behavioural validation o...

We examine the efficient implementation of back prop type algorithms on T0 [4], a vector processor with a fixed point engine, designed for neural network simulation. A matrix formulation of back prop, Matrix Back Prop [1], has been shown to be very efficient on some RISCs [2]. Using Matrix Back Prop, we achieve an asymptotically optimal performance on T0 (about 0.8 GOPS) for both forward and backward phases, which is not possible with the standard on-line method. Since high efficiency is futile if convergence is poor (due to the use of fixed point arithmetic), we use a mixture of fixed and floating point operations. The key observation is that the precision of fixed point is sufficient for good convergence, if the range is appropriately chosen. Though the most expensive computations are implemented in fixed point, we achieve a rate of convergence that is comparable to the floating point version. The time taken for conversion between fixed and floating point is also shown to be reasonab...

In this paper, we workout a detailed mathematical analysis for a new learning algorithm termed Cascade Error Projection (CEP) and a general learning fvame work. This frame work can be used to obtain the cascade correlation learning algorithm by choosing a particular set of parameters. Furthermore, CEP learning algorithm is operated only on one layer, whereas the other set of weights can be calculated deterministically. In association with the dynamical stepsize change concept to convert the weight update from infinite space into a finite space, the relation between the current stepsize and the previous enera level is also given and the estimation procedure for optimal stepsize is used for validation of our proposed technique. The weight values of zero are used for starting the learning for every layer, and a single hidden unit is applied instead of using a pool of candidate hidden units similar to cascade correlation scheme. Therefore, simplicity in hardware implementation is also obtained Furthermore, this analysis allows us to select from other methods (such as the conjugate gradient descent or the Newton’s second order) one of which will be a good candidate for the learning technique. The choice of learning technique depends on the constraints of the problem (e.g., speed, performance, and hardware implementation); one technique may be more suitable than others. Moreover, for a discrete weight space, the theoretical

Determination of atmospheric and surface elemental and molecular composition of various solar system bodies is essential to the development of a firm understanding of the origin and evolution of the solar system. Furthermore, such data is needed to address the intriguing question of whether or not life exists or once existed elsewhere in the Solar System. As such, these measurements are among the primary scientific goals of NASA’s current and future planetary missions. In recent years, significant progress toward both miniaturization and field portability of in situ analytical separation and detection devices have been made with future planetary explorations in mind. However, despite all these advances, accurate