### Table 1: Average performances for neural networks trained using traditional back-propagation learning parameters.

"... In PAGE 4: ...02, and no weight decay or sigmoid prime offsets. The average performances of these standard networks are shown, as percentages, in Table1 . Each row shows the classification performance of the networks on the current data-set, and the performance on the same dataset after training on subsequent datasets.... In PAGE 5: ... Table 3 shows the performance averages over ten runs of the fittest ten evolved networks. As one would expect, all aspects show an improvement over the standard network results of Table1 . More importantly, we also see an improvement over the Learn++ results of Table 2.... ..."

### Table 1: Weight discretization in multilayer neural networks: o -chip learning.

"... In PAGE 4: ... neural network paradigms. A compact overview of a large variety of results on the e ects of limited precision in neural networks can be found in Table1 to 4. These tables list the number of bits that are required for satisfactory (learning) performance and brie y describe the core idea of the algorithms.... In PAGE 4: ... Only the forward propagation pass in the recall phase is performed on-chip whichmakes these quantization e ects amenable for mathematical analysis using a statistical model. Some of the results have been summarized in Table1 which indicate that the accuracy needed in the on-chip forward pass is around 8 bits. In [Pich e-95] a comparison between Heaviside and sigmoidal multilayer networks is given, showing that the weight precision required inaHeaviside network is much higher and even doubles when a layer is added to the network.... In PAGE 6: ...lgorithms with the entropy(number of bits) upper bounds of the data set [Beiu-96.2]. Finally,wewould like to point out that a comparativebenchmarking study of quantization e ects on di erent neural network models and the improvements that can be obtained byweight discretization algorithms has not yet been done. The accuracies listed in Table1 to 4 are therefore highly biased by... ..."

### Table 14: Learn schedule for back-propagation neural network Learn count 10000 30000 50000

"... In PAGE 19: ... We reproduced this experiment with 14 hidden units, without a pruning. We used the learn schedule displayed in Table14 to train a network with 14 units in the hidden layer. Epoch size 1 was selected for this experiment.... ..."

### Table 3: Weight discretization in multilayer neural networks: on-chip learning. by allowing a dynamic rescaling of the weights (and hence the weight range) by adapting the gain of the activation function. The calculation of an activation value aj in a multilayer network is namely done as follows:

"... In PAGE 5: ... This means in speci c that at least the weight values are represented with only a limited precision. Simulations have shown that the popular backpropagation algorithm (see for example [Rumelhart-86]) is highly sensitive to the use of limited precision weights and that training fails when the weight accuracy is lower than 16 bits ( rst two references in Table3 ). This is mainly because the weight updates are often smaller than the quantization step which prevents the weights from changing.... In PAGE 5: ... In order to reduce the chip area needed for weight storage and to overcome system noise, a further reduction of the number of allowed weight values is desirable. Several weight discretization algorithms have therefore been designed and an extensive list of them and the attainable reduction in required precision is given in Table3 . Some of these weight discretization algorithms have already proven their usefulness in hardware implementations.... ..."

### Table 1: The logistic activation function used by backpropagation.

"... In PAGE 3: ... Units1 X and Y in Figure 2c are introduced into the KNN to represent the disjunction in the 1Units are the basic processing elements of neural networks. They receive real-numbered signals from other units over weighted connections (referred to as \links quot;) and mathematically transform these signals into a real-numbered output which is sent on to other units (see Table1 ). Generally, units are divided into three categories: input, that receive signals from the environment, output, that send signals to the... ..."

### Table 1: Backpropagation artificial neural network versus regression.

"... In PAGE 11: ... 4. Results and Discussion Results for the baseline performance of the best performing two hidden layer backpropagation neural network using all eight input variable values and the rhythmicity regression equation are shown in Table1 . Accuracy is the total percentage of correct predictions.... In PAGE 12: ...3.6 percent classification accuracy is established by two of the seven variable models. Because the sharpness variable produced the largest decrease in classification performance when it was left out, a regression model using just the sharpness variable is constructed to determine the correlation between sharpness and epileptiform seizures and another regression model using both the sharpness and physiologic state (the second largest performance decrease) is also constructed. Both of these new regression models have a smaller accuracy than the original rhythmicity regression model (shown in Table1 ), with the sharpness regression model... ..."

### Table 4. Fuzzy control rule

"... In PAGE 13: ...In Table4 , NL means #5CNegatively Large quot;, NM means #5CNegatively Medium quot;, NS means #5CNega- tively Small quot;, ZE means #5CZero Equivalence quot;, PS means #5CPositively Small quot;, PM means #5CPositively Medium quot;, PL means #5CPositively Large quot;, S means #5CSmall quot;, M means #5CMedium quot;, and L means #5CLarge quot; #28Kosko, 1992, and Zurada, 1992#29. Note that this rule could be generic for all neural networks using back-propagation learning algorithms.... ..."

### Table 2. Simulation of the Osgood Transfer Surface, using a three-layer network with backpropagation learning.

"... In PAGE 9: ...andom weights (uniform in [-0.3,0.3]) and with new lists. Scoring was identical to Simulation 1. Insert Table2 about here. Discussion of Simulation 2.... In PAGE 9: ... Discussion of Simulation 2. The results of this simulation are shown in Table2 . For identical stimuli and dissimilar responses there is relatively more negative transfer than in Simulation 1 (-16.... In PAGE 9: ... This hypertransfer is not observed in human or animal learning and represents another major difficulty of backpropagation as a model for human learning and memory. Because the transfer for identical responses and dissimilar stimuli in Table2 is only marginally above zero, we decided to run a number of very small scale simulations to investigate the characteristics of this effect. In these simulations, lists of only three item- pairs were used.... ..."

### Table 2. The learning process results for the recognition neural network

"... In PAGE 7: ... By using the three implemented algorithms: Rprop, Batch Backpropagation and On-Line Backpropagation; the topology 4800X10X3 showed to be a suitable topology to perform this recognition task. This topology means: 4800 neurons in the input layer (80X60 binary pixels), 10 neurons in the hidden layer and 3 neurons in the output layer (X, Y and P) Table2 present the results of the application of the cross-validation technique (10-folds) with the set neural network (4800X10X3) using the three implemented algorithms. In the Table we can observed that the result of both algorithms was almost the same with advantages to the Rprop algorithm that was faster than the others.... ..."

### TABLE 3. Performance of neural network implementations on workstations, parallel MIMD/ SIMD computers and dedicated neural network hardware (Adaptive Solution CNAPS).

1995

"... In PAGE 5: ... Cray T3E performance, communication overhead and scaleup measured with a TDNN network (4 layer, 4680 weights) used for promoter site detection, Backpropagation limit off-line learning algorithm and 3157 training patterns. System Software Performance [MCUPS] Comments TABLE3 . Performance of neural network implementations on workstations, parallel MIMD/... ..."

Cited by 1