@MISC{Chorowski_extractingrules, author = {Jan Chorowski and Jacek M. Zurada}, title = {Extracting Rules from Neural Networks as Decision Diagrams}, year = {} }

Share

OpenURL

Abstract

Abstract—Rule extraction from neural networks solves two fundamental problems: it gives insight into the logic behind the network and, in many cases, it improves the network’s ability to generalize the acquired knowledge. This article presents a novel, eclectic approach to rule extraction from neural networks, named LORE, suited for multilayer perceptron networks with discrete (logical or categorical) inputs. The extracted rules mimic network behavior on training set and relax this condition on the remaining input space. First, a multilayer perceptron network is trained under standard regime. It is then transformed into an equivalent form, returning the same numerical result as the original network, yet being able to produce rules generalizing the network output for cases similar to a given input. The partial rules extracted for every training set sample are then merged to form a decision diagram from which logic rules can be extracted. A rule format explicitly separating subsets of inputs for which an answer is known from those with an undetermined answer is presented. A special data structure, the decision diagram, allowing efficient partial rule merging is introduced. With regard to rules ’ complexity and generalization abilities, LORE gives results comparable with those reported previously. An algorithm transforming decision diagrams into interpretable boolean expressions is described. Experimental running times of rule extraction are proportional to the network’s training time. Index Terms—Feedforward neural networks, rule extraction, logic rules, true false unknown logic, decision diagrams. I.