Won Yong Gwan, Min Byung Eui

The Transactions of the Korea Information Processing Society (1994 ~ 2000), Vol. 3, No. 2, pp. 381-389, Mar. 1996
10.3745/KIPSTE.1996.3.2.381,   PDF Download:

Abstract

A simple modification of the standard back-propagation algorithm to eliminate redundant connections (weights and biases) is described. It was motivated by speculations from the distribution of the magnitudes of the weights and the biases, analysis of the classification boundary, and the nonlinearity of the sigmoid function. After initial training, this algorithm eliminates all connections of which magnitude is below a threshould by setting them to zero. The algorithm then conducts retraining in which all weights and biases are adjusted to allow important ones to recover. In studies with Bollean functions, the algorithm reconstructed the theoretical minimum architecture and eliminated the connections which are not necessary to solve the functions. For simulated random signal classification problems, the algorithm produced the results which is consistent with the idea that easier problems require simpler networks and yield lower misclassification rates. Furthermore, in comparison, our algorithm produced better generalization than the standard algorithm by reducing overfitting and pattern memorization problems.

Statistics
Cite this article