Fault Tolerant Multi-Layer

Perceptron Networks

George Bolt1

James Austin, Gary Morgan

Technical Report: YCS 180

July 1992

Advanced Computer Architecture Group
Department of Computer Science
University of York
Heslington, York, YO1 5DD, U.K.
Tel: +44-904-432771 Fax:+44-904-432767

Abstract

This report examines the fault tolerance of multi-layer perceptron networks. First, the operation of a single perceptron unit is analysed, and it is found that they are highly fault tolerant. This suggests that neural networks composed from these units could in theory be extremely reliable. The multi-layer perceptron network was then examined, but surprisingly was found to be non-fault tolerant. This result lead to further research into techniques to embed fault tolerance into such a neural network. It was found that injecting a few weight faults during training produced a MLP network which was fault tolerant. Further, it would tolerate more faults than the number injected during training. The trained network was then extensively analysed to locate the source of this fault tolerance. It was found that the magnitude of weight vectors was greatly increased in such networks, and from this it was discovered that the loss of potential fault tolerance in a MLP is due to training with back-error propagation algorithm. Finally, it is shown that the lengthy and computationally expensive training sessions in which faults are injected are not needed, since either binary thresholded units can be used, or else the trained networks' weight vectors can be scalar multiplied to produce a fault tolerant classification system.

1 E-mail: [email protected]
This work was supported by SERC and also by a CASE sponsorship with British Aerospace Brough, MAL.