Artificial neural networks (ANNs) involve a large amount of inter-node communications. To reduce the communication cost as well as the time of learning process in ANNs, we earlier proposed an incremental inter-node communication method. In the incremental communication method, instead of communicating the full magnitude of the output value of a node, only the increment or decrement to its previous value is sent on a communication link. In this paper, the effects of the limited precision incremental communication method on the convergence behavior and performance of multilayer neural networks are investigated. The nonlinear aspects of representing the incremental values with reduced (limited) precision for the commonly used error backpropagation training algorithm are analyzed. It is shown that the nonlinear effect of small perturbations in the input(s)/output of a node does not enforce instability. The analysis is supported by simulation studies of two problems. The simulation results ...

We have earlier proposed incremental inter-node communication to reduce the communication cost as well as time of the learning process in artificial neural networks. In the incremental communication, instead of communicating the full magnitude of an input (output) variable of a neuron, only the increment/decrement to the previous value of the variable, using reduced precision, is sent on a communication link. In this paper, a variable precision incremental communication scheme is proposed. Variable precision, which can be implemented in either hardware or software, can further reduce the complexity of intercommunication and speed up the computations in massively parallel computers. This scheme is applied to the multilayer feedforward networks and simulation studies are carried out. The results of our simulations reveal that, regardless of the degree of the complexity of the problems used, variable precision scheme has stable convergence behavior and shows considerable degree of saving in terms of the number of bits used for communications.