Abstract

The parity function is one of the most used Boolean function for testing learning algorithms because both of its simple definition and its great complexity. Being one of the hardest problems, many different architectures have been constructed to compute parity, essentially by adding neurons in the hidden layer in order to reduce the number of local minima where gradient-descent learning algorithms could get stuck. We construct a family of modular architectures that implement the parity function in which, every member of the family can be characterized by the fan-in max of the network, i.e., the maximum number of connections that a neuron can receive. We analyze the generalization ability of the modular networks first by computing analytically the minimum number of examples needed for perfect generalization and second by numerical simulations. Both results show that the generalization ability of these networks is systematically improved by the degree of modularity of the network. We also analyze the influence of the selection of examples in the emergence of generalization ability, by comparing the learning curves obtained through a random selection of examples to those obtained through examples selected accordingly to a general algorithm we recently proposed.

Citations