A Neural Network Methodology

for Process Fault Diagnosis

The ability of knowledge-based expert systems to facilitate the auto-
mation of difficult problems in process engineering that require symbolic
reasoning and an efficient manipulation of diverse knowledge has gen-
erated considerable interest recently. Rapid deployment of these sys-
tems, however, has been difficult because of the tedious nature of
knowledge acquisition, the inability of the system to learn or dynami-
cally improve its performance, and the unpredictability of the system Venkat Venkatasubramanian
outside its domain of expertise. King Chan
This paper proposes a neural-network-based methodology for pra- Laboratory for Intelligent Process Systems
viding a potential solution to the preceding problems in the area of pro- School of Chemical Engineering
cess fault diagnosis. The potential of this approach is demonstrated Purdue University
with the aid of an oil refinery case study of the fluidized catalytic crack- West Lafayette, IN 47907
ing process. The neural-network-based system successfully diagnoses
the faults it is trained upon. It is able to generalize its knowledge to suc-
cessfully diagnose novel fault combinations it is not explicitly trained
upon. Furthermore, the network can also handle incomplete and uncer-
tain data. In addition, this approach is compared with the knowledge-
based approach.

Introduction diagnosis case study of a fluidized catalytic cracking unit

The ability of knowledge-based expert systems (KBES) to (FCCU). Previously, the FCCU case study has been used to
facilitate the automation of difficult problems in process engi- develop a knowledge-based diagnostic expert system imple-
neering that require symbolic reasoning and an efficient manip- mented in OPS5 (Brownston et al., 1985). Thus, it serves as a
ulation of diverse knowledge has generated considerable interest useful test bed for evaluating the proposed neural network diag-
over the recent past. Much attention has been paid recently to nostic methodology, and it also facilitates the comparison of
the problem of process fault detection and diagnosis by artifi- both approaches.
cial-intelligence-based techniques (Kramer and Palowitch, In the current rescarch, an analysis of the recall capability to
1985; Shum et al., 1988; Venkatasubramanian and Rich, 1988). trained faults and the generalization capability to symptoms
However, rapid deployment of these systems has been difficult resulting from novel fault combinations is performed. The gen-
to achieve because of certain inherent limitations associated eralization proficiency vs. network topology (i.e., number of hid-
with current KBES. These limitations include the tedious nature den layers and hidden units) is examined. A comparison
of knowledge acquisition, the inability of the system to learn or between the neural-network-based and the knowledge-based
dynamically improve its performance, and the unpredictability FCCU diagnostic versions are also investigated.
of the system outside its domain of expertise. The remainder of the paper is organized in the following man-
The present study proposes a neural-network-based approach ner. First, a synopsis of neural networks along with motivational
for providing a potential solution to the preceding problems in issues for their application is addressed. Second, the learning
the area of process fault diagnosis. The feasibility of this algorithm used in the present study is discussed. Following this,
approach is explored through a neural-network-based fault an overview of CATDEX, a rule-based diagnostic expert system
for a fluidized catalytic cracking unit, is given. Next, the details
of thc neural-network-based fault diagnosis system are de-
Correspondence concerning this paper should be addressed 10 V. Venkatasubramanian scribed. Subsequently, the fault diagnosis experiments and sim-

AIChE Journal December 1989 Vol. 35, No. 12 1993

ulation results are presented. Lastly, from these results, the process engineering problems requiring pattern recognition and
facility of a neural-network-based approach for overcoming the pattern classification. These areas include fault diagnosis and
KBES bottlenecks is explicated. detection, process design, process control, and process simula-
tion. Recently, researchers have begun to look at applications
Neural Networks: Synopsis and such as interpreting biosensor data (McAvoy et al., 1989), fault
Motivational Issues diagnosis of a chemical process system (Venkatasubramanian,
Rapid advances in neuroscience and in computer science are 1985), and for single fault diagnosis of three continuous-stirred-
arousing renewed interests in neural networks as potentially new tank reactors connected in series (Hoskins and Himmelblau,
problem-solving architectures. Neural networks are dynamic 1988).
systems composed of highly interconnected layers of simple neu- Back-Propagation Neural Network
ron-like processing elements (Amari, 1977; Feldman and Bal-
The back-propagation learning algorithm (Jones and Hos-
lard, 1982; Kohoncn, 1984; Rumelhart et al., 19863). These
kins, 1987; Rumelhart et al., 1986b, Werbos, 1974) has been
layers arc categorized as input layers where patterns are pre-
widely explored in recent years and has produced successful
sented to the network and as output layers which contain the learning and generalization results in various task domains. For
response to a given input. Furthermore, they may also contain
instance, some systems include those for text-to-speech conver-
intermediate layers or hidden layers. Figure 1 presents an exam-
sion (Sejnowski and Rosenberg, 1986), handwritten character
ple of a typical neural network architecture.
recognition (Burr, 1986), speech recognition (Ellman and
Neural network operations consist of a learning or training
Zipser, 1987), and sonar target identification (Gorman and
phase, a recall phase, and a generalization phase. During the
Scjnowski, 1988). It is an error-correcting learning procedure
learning phase, the network is repeatedly presented with a set of
which generalizes the Widrow-Hoff algorithm (or delta rule) to
input-output patterns. Learning is accomplished by a general
multilayer networks. Back-propagation is intended for networks
rule which dynamically modifies the weights of all interconnec-
with an input layer, any number of hidden layers, and an output
tions in an attempt to generate the desired output pattern for
layer. Each layer is fully connected to the succeeding layer, and
each presented input pattern. After the learning is complete, the
connections within a layer or from higher to lower layers are
network operates in a recall phase where it generates responses
prohibited.
to input patterns used in training. It also operates in a general-
The networks employed in the present study are feedforward
ization phase where it generates responses to similar or novel
and possess one or two hidden layers of processing elements
input patterns.
(PE). Upon presentation of an input-output pattern, the total
Neural network computations are collectively performed by
input, net,, to the j t h processing element is obtained by calcu-
the entire network with knowledge represented in the connection
lating the weighted sum of all PE i outputs that are connected to
weights between processing elements. Consequently, the collec-
PE j ,
tive operations result in a high degree of parallel computation
which enables the network to solve complex problems rapidly. Tn
addition, the distributed representations lead to greater fault net, =
i
wji x i + b, (1)
tolerance and to graceful degradation when problems are
encountered beyond its range of experience. Furthermore, other where w,, is the connection weight from the ith PE to thejth PE,
beneficial attributes include the ability to adjust dynamically to x iis the output of PE i, and 6, is the bias of thejth PE. However,
environmental changes, to perform useful generalizations from the bias, b,, is eliminated by giving every PE an extra input con-
specific examples, and to recognize invariances from complex nection with an activity level fixed at one (Figure 2a). Conse-
high-dimensional data. quently, the weights on this special connection are just the nega-
Thus, neural networks appear to be applicable to chemical tive of the threshold and must be learned in the same manner as
other weights. A sigmoid transfer function (Figure 2b) is then
applied to net, to obtain the output of thejth PE,
t
. ..... output
Layer
where @ is a constant gain term which determines the slope of
the sigmoid function at net, = 0. The output serves as input to
succeeding layers which is continued until the output layer is
Hidden reached and is referred to as forward-activation flow. The subse-
quent weight adaptation or learning process is accomplished by
the back-propagation learning algorithm.
The goal of the back-propagation learning algorithm is to
iteratively minimize the average squared error between values
of the output PE and the correct patterns provided by a teaching
. ..... input (i.e., gradient descent in error space). This is accomplished
by first computing the error gradient (6,) for each PE on the out-
t t t put layer,

Figure 1. Typical neural network architecture.

1994 December 1989 Vol. 35, No. 12 AlChE Journal

 possibility of equipment failure and operator error. Current
efforts in solving this problem employ KBES for automating the
detection and diagnosis of process abnormalities. This is accom-
plished by the application of artificial intelligence techniques for
capturing the reasoning processes of a diagnostic specialist
(Firebaugh, 1988; Harmon and King, 1985; Nilsson, 1980).
The catalytic cracking unit diagnostic expert (CATDEX) is
one such prototypical expert system for diagnosing a fluidized
catalytic cracking unit (Figure 3) (Venkatasubramanian, 1988)
which is a major conversion apparatus of the petroleum refining
industry (consult Shreve and Brink, 1977, for the details of cata-
lytic cracking). CATDEX was developed as an off-line interac-
tive decision support expert system for a novice petroleum engi-
Xi
neer or a refinery operator. Implemented in OPS5, it utilizes a
backward-chaining or goal-driven inference mechanism and a
generalized depth-first search control strategy.
CATDEX focuses on three categories of operational prob-
lems which may eventually lead to plant shutdown and/or unit
damage. These problems are excessive catalyst loss, poor cata-
lyst circulation, and yield loss. Furthermore, each of these faults
net, exhibits symptoms which in turn may be faults that exhibit
-m 0.0 +- other symptoms. This proceeds until one reaches rudimentary
Figure 2. Fundamental unit of neural networks: a. pro- faults, called elemental symptoms, which cause no other process
cessing element or artificial neuron; b. sigmoid malfunctions. As a result, an exhaustive set of potential FCCU
transfer function. faults and their associated symptoms may be fully enumerated
into a fault-tree-like structure (Himmelblau, 1978), called an
inference network (Rich and Venkatasubramanian, 1987). This
where tpj is the correct teaching value for output unit j and for
network is then encoded as rules in the knowledge base of the
input pattern p. The error gradient is then recursively deter-
expert system. Consequently, fault diagnosis is performed by
mined for hidden layers (i.e., internal PE's) by computing the
systematically searching for links and relationships within the
weighted sum of the errors at the previous layer,
inference network, which produce the causal source of the symp-
toms resulting from operational malfunctions.
6, = x,(l - Xi) SkWY (4)
k

Neural Network Methodology for

where k is over all P E s in the layers above j . Thus, the errors are Diagnosing a FCCU
propagated backwards one layer, and the same procedure is
recursively applied until the input layer is reached. This process CATDEX is thus a fault diagnostic expert system which uti-
of back-propagating the errors is referred to as backward-error lizes an interactive guided inference on a knowledge base con-
flow. The error gradients are then used to update the network sisting of production rules that represent the inference network.
weights, The resulting system shows promise in efficient diagnosis. How-
ever, it lacks the flexibility to process changes, is incapable of
diagnosing novel symptom combinations, and requires a consid-
erable amount of implementation effort. In addition, as is typi-
wji(n + 1) = w,j(n) + Awji(n) (5b) cal of fault trees and inference networks, it requires considerable
effort in getting them to perform multiple fault diagnosis or
where n indexes the presentation number during training and 7 diagnosis based on incomplete or partial symptoms.
is the learning rate which provides the step size during gradient The proposed neural-network-based methodology (Neural
descent. Generally, to assure rapid convergence, large step sizes CATDEX) treats fault diagnosis as one of classification. For in-
which do not lead to oscillations are used. Convergence, how- stance, in the initial training process the neural network is pre-
ever, may be improved by including a momentum term, a,which sented with N observation vectors (the symptoms resulting from
determines the effect of previous weight changes on present a fault) and its associated class (the fault or malfunction). Simi-
changes in weight space. The weight change after the nth data lar to classical decision theory, neural networks perform the
presentation is, classification by creating decision boundaries to separate the N
different pattern classes. Unlike traditional classifiers, however,
Awjj(n) = Sj X , + eAwji(n - 1 ) (6)
neural networks, using the back-propagation learning algo-
whereO<a<l. rithm, develop complex input-output mappings which represent
salient features and regularities of the task domain (Rumelhart
Overview of CATDEX: An Expert System for et al., 1 9 8 6 ~ ) .Moreover, it is these internal representations
Diagnosing a FCCU which lead to novel generalizations in future classification activ-
The complexity of modern chemical plants is causing in- ities involving new or slightly modified fault symptoms.
creased safety and reliability problems because of the greater The representation of C A T D E X s inference network consid-

AIChE Journal December 1989 Vol. 35,No. 12 1995

 To Fmctionator To Atmospkre

-Dilute Phase Sprays

-BedLevel
overflow well

Bed Level -- Standpipe Level +H 1

-TOrChOil

Grid

oil Feed

t
Blast Steam

Figure 3. Typical fluidized catalytic cracking unit.

ers each fault and symptom node as a neural-network-process- rcsults of this study were obtained by the Neuralworks Profes-
ing element. Most importantly, hidden layers are added between sional 11 neural network simulation package.
symptoms and their immediate faults to enable the development
of internal representations. The simulation issues for Neural Accuracy calculations
CATDEX are presented in the following sections. It is evident from Table 1 that the training data are binary
values, in which 1 represents a fault and 0 a nonfault. The out-
Network training puts, however, are real values since infinite weights are required
In this initial study, training was performed only on the bot- to drive the results to 1 or 0.
tom-most inference network layer consisting of the elemental In this study, the accuracy of a single output node is defined
symptoms and its immediate faults (Figure 4). In addition, to as one minus the absolute difference between the desired fault
simplify the ensuing analysis, the excessive catalyst loss and and the predicted fault determined during the recall or general-
yield loss malfunction categories were chosen because they con- ization phase. The network accuracy in fault detection for a sin-
tain the most intriguing Boolean combinations for learning and gle training pattern is computed as the average of the 13 output
subsequent generalization. During network training, the input node accuracies. Consequently, the overall accuracy is the aver-
patterns consist of symptoms resulting from a fault, and the out- age network accuracy for all training patterns over all training
put patterns contain the specific fault. Thus, as numerically repetitions.
depicted in Figure 4, the neural network contains 18 input nodes
and 13 output nodes which represent the symptoms and its Number of hidden layers and hidden units
immediate faults, respectively. I n this study, the network was As previously mentioned, neural networks partition the prob-
trained on all single fault occurrences in the partial FCCU lem space into numerous decision regions when performing its
inference network. Table 1 lists the training data for the 13 classifications. The partitioning capability is a function of both
faults involved. The training data were presented a total of 500, the number of hidden layers and hidden units. For example, net-
1,000, 2,000, 4,000, 6,000, and 8,000 iterations or time steps. works utilizing sigmoid nonlinearities form fuzzy hyperplanes
Lastly, all network training was repeated three times with dif- with no hidden layers, convex polygonal boundaries for one hid-
ferent initial random weight values to average over variations in den layer, and arbitrarily complex decision regions for two hid-
performance. den layers (Lippmann, 1987; also see Kolmogorov, 1956, for a
mathematical treatment on the capabilities of multilayer net-
Back -propagation learning parameters works). Likewise, the number of hidden units must also be
In all simulation experiments, the learning rate parameter, 1, determined such that the complexity of the decision regions
was set at 0.9, the momentum term, a, was set at 0.6, and the formed corresponds to the problem complexity needed for cor-
gain term, p, was set at 1 .O. Furthermore, the initial weights of rect classification.
the network were assigned to small uniformly-distributed ran- Presently, neural networks with one hidden layer were used
dom values between -0.1 and +0.1 to prevent the hidden units primarily in simulations. Nevertheless, experiments on networks
from acquiring identical weights during training. Finally, the with two hidden layers were performed to examine fault diag-

I996 December 1989 Vol. 35, No. 12 AlChE Journal

 Losses in
Reactor

5 6

9 I1 13
Vanadium Sodium Nickel Hydrc-
Thermal
Poisoning Poisoning Poisoning Thermal
Deactivation
off catalyst
o of catalyst off Catalyst
o Deactivation

Vanadium

1L
High
Sodium
on Catalyst
13
Coke
Make has
Increased
14

Figure 4. Presently employed FCCU inference network for Neural CATDEX.

Hydrogen to
Methane
Ratio High
15
Catalyst
Pore Size
has Increased
16
11 Catalyst
Surface Area
has Increased
17
Catalyst Pore
Size Remained
Constant
18

The top layer represents the operational problem categories. The next layer (1-13) represent the faults of the elemental symptom layer (1-18)

Legend: /4 AND Node OR Node.

nosis on the FCCU with more complex decision regions. The by presenting symptoms resulting from two and three process
number of hidden units required to perform accurate diagnosis malfunctions. Following this, the fault diagnosis behavior is
was determined empirically. All series of experiments were analyzed for cases where faults share common symptoms and
repeated using networks of 5 , 7, 1 1, 15, 2 1 and 27 hidden units. for cases where partial symptoms of a fault are presented to the
Due to overcrowding of the plots, only the results of the net- network. Lastly, the recall and generalization results are exam-
works that contain 5, 7, 15 and 27 hidden units are shown. The ined for networks trained on a subset of single fault input-output
other results were similar to these. patterns. The preceding experiments are duplicated for two hid-
den-layer neural network architectures.
A Performance Evaluation of Neural CATDEX
In the ensuing sections, the fault diagnosis simulation results Recall results of trained single fault patterns
are presented in their entirety. The facility of neural networks To test the learning proficiency and thus the network conver-
for recalling trained faults is explored. Next, the trained net- gence, the symptoms resulting from the 13 faults were presented
work generalization capacity to multiple faults is investigated to all of the trained networks. During this stage, all output val-

AIChE Journal December 1989 Vol. 35, No. 12 1997

 Table 1. Single-FaultTraining Data
~

i = Input Pattern Symptoms ( 18 PE's)

d = Output Pattern Single Fault Occurrence (13PEs)
- - ____-___ ______-__--
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
i 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
d 0 l 0 0 0 0 0 0 0 0 0 0 0
-- __ __ _____ __-__ _____________
i 0 I 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
d O 1 0 0 0 0 0 0 0 0 0 0 0 ____
I 0 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
d O O 1 0 0 0 0 0 0 0 0 0 0
__________
I 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0
d O O O l O O O O O O O O O ________-_-
--__ _ _ _ _ _
I 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0
d 0 . 0 0 0 1 0 0 0 0 0 0 0 0
- - _ _ _ _ _ _ _ ___---- __ _____I______

I 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0
d O O O O O l O O O O O O O
~

i 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0
d O O O O O O l O O O O O O
~~ ~ ~

I 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0
d O O O O O O O l O O O O O
~~~~ ~ ~

I 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0
d O O O O O O O O l O O O O
~~ ~~

I 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0
d O O O O O O O O O l O O O
i 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0
d O O O O O O O O O O l O O
i O O O O O O O O O O O O O O O l 1 0
d O O O O O O O O O O O l O
i O O O O O O O O O O O O O O O O l 1
d O O O O O O O O O O O O l
Consult Figure 4 for the input-output pattern legend I = fault; 0 - nonfault

the correct fault. Figure 5 shows the overall average learning

Number of curves for networks trained on single faults.
Hidden Units The networks, therefore, were able to learn the correct asso-
ciation between the process symptoms and process malfunc-
______*-.-.
-
5
_._.7
& tions. The best average performance was achieved by a network
*....... " 15 with 27 hidden units which attained 98.18% accuracy in fault
27 classification at 8,000 time steps (Table 2). Table 2 suggests an
insignificant increase in performance with the number of hidden
. units since the observed recall deviation is less than 4% between
:?
networks of five and 27 hidden units. However, as nearly exact
40- ::j
;
. :i

Table 2. Network Performance During Recall of

20 - i
Single-Fault Patterns
.A
No. of Overall Accuracy (96)
0 Hidden Units @ 8,000 Time Steps
. . I . - I . . I . . I = ' I . ' I . .
_ _ _ _ _______ __

Figure 5. Network performance curves for recalling 21 98.02

27 98. I8
trained single faults.

1998 December 1989 Vol. 35, No. 12 AIChE Journal

 ,-
recall is usually expected, the number of hidden units is more
critical for developing desirable generalizations to novel symp- 1 Number of
tom combinations. Hidden Units
50-
5
Multiple fault generalization results
In the next series of simulations, a set of symptoms excluded
40 -
from the initial training patterns was presented to the network
to determine its ability to perform useful generalizations. This 30-
capacity was examined by introducing input symptoms resulting
from two and three process malfunctions. This was deemed
interesting because multiple fault diagnosis is difficult to
20 -
accomplish by traditionzl fault tree analysis. Since there was an
inordinate number of two- and three-fault combinations, a sub- 10 -
set of symptom patterns which were considered difficult to gen-
eralize was chosen. Thus, 33 symptoms for two faults and 30
symptoms for three faults were presented to the network for gen-
eralization. Figure 6 shows the overall average two-fault gener-
0 t
0 2000 4000 6000 8000 1MXx) 12000 14000

alization curves for networks trained on single faults. Similarly, Time Steps
Figure 7 shows the overall average three-fault generalization
Figure 7. Network performance curves for three-fault
curves.
generalization.
The networks are thus able to perform novel generalizations
to multiple fault conditions. As in recall, the asymptotic limits
exhibited by these curves suggest a sufficient number of hidden performance tends to increase with the number of hidden units.
units to perform the task. However, unlike recall where learning Lastly, Figures 8 and 9 display bar charts of selected two- and
performances are not significantly affected by the number of three-fault generalizations for networks trained on single faults,
hidden units, useful generalizations seem to require a certain respectively. All the bar charts correspond to a network with 27
minimum number of hidden units which depend on the complex- hidden units in a single hidden layer. Figure 8 displays the
ity of the particular task. If too few are present, all the essential detection accuracy of the detection of two fault occurrence by
attributes of the input-output patterns cannot be completely the neural network. One of the two faults in Figure 8 is always
represented. fault 1, namely, “Hole in reactor plenum” as seen in Figure 4.
The best average performance was achieved by a network The other member of the two fault pair varies from fault 2
with 27 hidden units which attained 77.54% accuracy for two- through fault 13. So, the x-axis displays the fault combina-
fault generalization and 53.60% accuracy for three-fault gener- tions-( 1, 2), ( I , 3), and so on. The dark-colored bars corre-
alization at 8,000 time steps. It appears from Figures 6 and 7 spond to the accuracy of the detection of fault 1 , while the
that the generalization accuracy can further be improved with hatched bars correspond to the accuracy of the other faults in
more hidden units as saturation levels have definitely not been the set 2 through 13. Figure 9 has similar results, except that
attained. Thus, for the task of generalizing to multiple faults, they now correspond to three fault combinations.

Number of
Hidden Units
80
60-

60

40

40

20 -
20
i
1 0
2 3 4 5 6 7 8 9 1011 1213

Time Steps Output Node Number “ A

Figure 6. Network performance curves for two-fault gen- Figure 8. Bar chart for two-fault generalization.
eralization. Predicted faults: H, I ; 0 ,“A,”

AlChE Journal December 1989 Vol. 35, No. 12 1999

 tom nodes 5 , 9 and 7 , 8 , 9 , 10, respectively, result in a 29%)accu-
racy for fault 4 and a 80% accuracy for fault 6. Since neural
networks perform cooperative computation, that is the output
from any node depends on the contributions from all other nodes
2. in the network it is connected to, the faults that have more symp-
E toms (or evidences) supporting them than other faults tend to

-$
4
Y
1
get weighted more in the corresponding outputs generated by
the neural network. This is reflected in the accuracy shown in
Figure 10, such as in the case of fault 4 and 6. Fault 4 has two
i4“
symptoms and fault 6 has four symptoms; as a result, when all
these symptoms are observed, fault 6 is recommended with a
higher accuracy than fault 4. The accuracy may be interpreted
in a limitcd sense as the probability of the recommendations for
the different faults predicted by the network.

Generalization results for partial symptom patterns

From the preceding sections, it appears that the neural net-

.,
Output Node Number “A,B” work gives greater emphasis to faults displaying more symp-
toms. This behavior is a result of the cooperative nature of the
Figure 9. Bar chart for three-fault generalization. computations performed by the neural networks. In this section,
Predicted faults: I ; 0 ,“A”; cl, “B.” we present the network’s response to input patterns with incom-
plete set of symptoms for fault occurrence. This is examined for
Generalization results for faults sharing the “high catalyst losses in reactor” operational problem which
CommOn symptoms contains four symptom nodes and three fault nodes (Figure 4).
From the prcceding, it is evident that Neural CATDEX, The results are presented in Figure 1 1 in which the abscissa con-
unlike CATDEX which uncovers only single causal sources for a tains selected symptom patterns represented as a vertical binary
given symptom pattern, performs multiple fault diagnosis. This string with symptom node 1 located at the base. From this fig-
section presents the generalization results for faults which share ure, it is evident that the network generalizes to a “no fault”
common symptoms. It examines the two-fault generahations condition for an input of no faulty symptoms (‘‘0000”). How-
involving fault node 4 with fault node 5-8 for the “high catalyst ever, this is not the case for other partial symptom patterns.
losses i n regenerator” operational problem. The common symp- Even though the symptom patterns are incomplete and thus do
tom of faults 4-8, node number 9 (“losses are high and steady”), not warrant the existence of a fault, the network generali~esto
is explicit in Figure 4. predict the occurrence of an appropriate fault with some non-
The generalization results are shown in Figure 10. From this zero accuracy.
ligure, the outputs of fault nodes 4 and 6, which exhibit symp- For example, the appearance of the “rate of loss increasing”
symptom (“OIOO”), which is shared by fault nodes 1 , 2, and 3

5 6 7 8

Output Node Number “ A

Figure 10. Probabilistic multiple-fault generalization for
Symptom Patterns

., .,
faults which share common symptoms for the Figure 1 1. Probabilistic generalization for partial symp-
“high catalyst losses in regenerator” opera- toms for the “high catalyst losses in reactor”
tional problem. operational problem.
Predicted faults: 4,ffl “A.” Predicted faults: I ; 0 ,2; 0,3.

2000 December 1989 Vol. 35,No. 12 AIChE Journal

suggests the possible occurrence of any one of the three faults or Table 3. Partial Listing of Two-Fault Generalization for Two
any other combination among these three. Since fault nodes 1 Hidden Layer Networks @ 15,000 Time Steps
and 2 exhibit one of two symptoms resulting from a fault while ~

fault node 3 exhibits only one of three symptoms, the probability Network Actual Predicted
of fault 1 or 2 occurring is higher and is reflected in the pre- Architecture Faults Faults
dicted accuracies (i.e., 28%, 17% and 0.3% for faults 1 , 2 and 3, 18-14-1 3-1 3 1,6 6
respectively). Compare these accuracies with the case when all I , 12 9, 12
the evidences are present for these faults from Table 2 (the aver- 1.8 3 , 7, 8
age overall accuracy of recall for these faults is 98.18%). Thus, 18-27-27-13 1, 3 2. 3
for incomplete symptom patterns, the neural network gives 1.9 1,279
greater emphasis to a fault displaying a higher percentage of 1,4 I , 4, 12
symptoms which may result in its occurrence. Therefore, the
network is able to generalize its diagnostic knowledge even for
cases with incomplete evidence and come up with a probabilistic performing multiple fault diagnosis for all initially-trained
recommendation which are quite accurate. This is a very useful faults.
diagnostic characteristic in practical situations.
Performance results for two hidden layers
The final series of simulations analyze the decision regions
Recall and generalization results for incomplete obtained for four-layer neural networks (i.e., two hidden layers).
training of single fault patterns The topology of the networks in consideration contains 14-1 3
This experiment investigates generalizations produced by and 27-27 hidden units in each hidden layer, respectively. As in
networks trained on a subset of the single-fault input-output previous cases, the trained networks are tested on both recall
patterns. Networks in this experiment are trained with one of and multiple fault generalization. These results are subse-
the 13 faults withheld. However, during the recall phase, symp- quently correlated with those obtained for the 27 hidden-unit
toms resulting from all the 13 faults are presented to the net- single-layer network.
work. Similarly, during generalization, symptoms resulting As anticipated, the increased connections of the four-layered
from trained and untrained faults are present. networks require significantly more training iterations in attain-
The recall on trained faults were very good, similar to the ing comparable single hidden layer recall accuracies. For exam-
accuracies shown in Figure 5 and Table 2. However, the recall ple, recall accuracies of 98.1 8% were achieved by an 18-27-13
accuracy for untrained faults were close to zero indicating that network in 8,000 time steps, while an 18-14-13-13 (96.12%)
the network exhibited no knowledge of the untrained faults. Fig- and an 18-27-27-1 3 (97.66%) networks require 15,000 time
ure 12 displays the two-fault generalization results for cases steps. Therefore, using an additional hidden layer by dividing
where one single fault pattern is withheld during training (i.e., the single hidden layer units into two layers (i.e., 14-13) or by
faults 5 , 6 , 9 , 10, and 1 1 withheld one at a time). In all instances, incorporating another hidden layer of equal size (i.e., 27-27)
the network is unable to generalize to the untrained faults. This does not significantly aid the recall process.
is illustrated in Figure 12 as all untrained fault output nodes dis- While the preceding results are expected, generalization re-
play near zero accuracies (the hatched bars are indistinguish- sults produced by an additional hidden layer are unpredictable.
able from the x-axis). As expected, the network is capable of A partial listing of two- and three-fault generalizations for the
four-layer networks trained on 15,000 time steps are in Tables 3
and 4. respectively. Thus the fault discriminatory capability
with two hidden layers leads to erroneous fault predictions,
while single hidden layer networks offer useful generalizations.

c Conclusions and Future Work

8
Y
In this paper, we have proposed a neural-network-based
methodology for developing automated systems for process fault
diagnosis. As demonstrated, neural networks are able to acquire
diagnostic knowledge from examples of fault scenarios. This
a
g!
u Table 4. Partial Listing of Three-Fault Generalization for
3a Two Hidden Layer Networks @ 15,000 Time Steps

Network Actual Predicted

Architecture Faults Faults
~~

5 6 9 10 11 1 8- 14- 1 3- I 3 1, 4, 13 13
1,6, 1 1 9, 12
I , 5, 13 5, 13
Output Node Number “ A
18-27-27- 13 1,5,9 None
Figure 12. Generalization results for incomplete single-
1.7, 1 1 6
fault training. I , 8, 12 4, 8, 12
Predicted faults: m, I ; B,“A” (untrained single fault).

AIChE Journal December 1989 Vol. 35, No. 12 2001

knowledge acquisition is a n a u t o m a t i c process driven by a learn- Fircbaugh, M. W., Art$cial Intelligence: A Knowledge-Based Ap-
ing algorithm called t h e back-propagation algorithm. T h e neu- proach. Boyd & Fraser, Boston (1988).
Gorman. R. P., and T. J. Sejnowski, “Analysis of Hidden Units in a
ral network’s rccall t o trained faults is nearly perfect a t 98%. Layered Network Trained to Classify Sonar Targets,” Neural Net-
Furthermorc, unlike traditional fault tree analysis a n d inference works. 1,75 (1988).
networks which furnish a single causal source for a set of symp- Harmon, P., and D. King, Expert Systems, Wiley, New York (1985).
toms, this methodology performs multiple fault diagnosis while Himmelblau, D. M., Fault Detection and Diagnosis in Chemical and
Petrochemical Proeesses. Elsevier, Amsterdam ( 1978).
being trained on knowledge of single faults. T h i s generalization
Hoskins, J. C., and D. M. Himmelblau, “Artificial Neural Network
to two a n d three faults yields accuracy measures of 78% a n d Models of Knowledge Representation in Chemical Engineering,”
54%, respectively, on t h e test cases w e studied. I t is also shown Comp. Chem. Eng.. 12(9/IO), 881 (1988).
t h a t N e u r a l C A T D E X ’ s o u t p u t c a n be interpreted a s a likeli- Jones, W. P., and J. C. Hoskins, “Back-Propagation, A Generalized
hood for process malfunction. T h u s t h e network has t h e useful Delta Learning Rule.” BYTE Mag.. 155, (Oct., 1987).
Kohonen, T., Sel/Organization and Associative Memory. Springer-
characteristic of being a b l e t o deal with incomplete a n d uncer- Verlag, Berlin (1984).
tain evidences. W h i l e t h e network is able t o generalize its knowl- Kolmogorov, A. N., “On the Representation of Continuous Functions of
edge for diagnosing multiple fault combinations, it w a s not Several Variables by Superposition of Continuous Functions of a
explicitly trained upon, it is unable t o diagnose single f a u l t cases Smaller Number of Variables,” Doklady Akademii Nauk SSSR,
108, I79 (1956).
if t h a t fault w a s not a m e m b e r of t h e initial training set. Thus, i t
Kramer, M., and B. Palowitch, “Expert System and Knowledge-Based
a p p e a r s t h a t these networks seem t o perform well for novel con- Approaches to Process Malfunction Diagnosis,” AlChE Meeting,
ditions which a r c somewhat similar t o w h a t t h e y h a d seen Chicago (Nov., 1985).
before, whereas they fail for other novel scenarios which a r e Lippmann, R. P., “An Introduction to Computing with Neural Nets,”
totally difycrcnt from w h a t they h a d seen before. I t also a p p e a r s f E E E ASSP Mag.. 4 , 4 (Apr., 1987).
McAvoy, T. J., N. S. Wang, S Naidu, N. Bhat, J. Gunter, and M. Sim-
t h a t singlc hidden layer networks seem to outperform t w o hid- mons, “Interpreting Biosensor Data via Back-propagation,” IJCNN
den layer networks both in efficiency a n d accuracy. In:. Joint Conf. on Neural Networks. 1,227, Washington DC (June,
T h i s research demonstrates t h e feasibility for f u t u r e develop- 1989).
m e n t s in t h e application of neural networks t o chemical process Nilsson. N . J., Principles of Artificial Intelligence. Morgan Kaufmann
fault detection a n d diagnosis. It is also believed t h a t a n integra- Publishers, Los Altos, CA (1980).
Rich, S. II., and V. Venkatasubramanian, “Model-Based Reasoning in
tion of neural networks a n d knowledge-based expert systems is Diagnostic Expert Systems for Chemical Process Plants,” Comp.
essential t o resolving t h e problems inherent in both problem- Chem. Eng., 11(2), I l l (1987).
solving paradigms. Thus, expert systems endowed with a dy- Rumelhart, D. E., G. E. Hinton and J. L. McClelland, “A General
n a m i c adaptive ability c a n greatly simplify t h e knowledge Framework for Parallel Dstributed Processing,” Parallel Distrib-
uted Processing: Explorations in the Microstructure of Cognition:
acquisition proccss. As a result, instead of supplying t h e knowl- I . Foundations, D. E. Rumelhart and J. L. McClelland, eds., MIT
edge, t h e domain expert need only refer t o relevant d a t a for sub- Press, Cambridgc, MA (l986a).
sequent network training. T h u s t h e proposed neural network Rumelhart, D. E., G. E. Hinton, and R. J. Williams, “Learning Internal
component c a n a c t a s a preprocessor whose d u t i e s m a y encom- Representations by Error Propagation,” Parallel Disiributed P r e
pass pattern recognition, sensor fault analysis, a n d / o r signal cessing: Explorations in the Microstructure of Cognitiotr I . Founda-
tions, D. E. Rumelhart and J. L. McClelland, eds., M I T Press, Cam-
recognition. It c a n also b e a component of t h e rule-based system bridge, MA ( 1 986b).
itself which performs learning, generalization, a n d / o r classifi- Rumelhart, D. E., G. E. Hinton, and R. J. Williams, “Learning Repre-
c a t ion. sentations by Back-Propagating Errors,” Nat.. 323(9), 533 (Oct.,
T h e ability to learn from example, extract salient features 1986~).
Sejnowski, T. J., and C. R. Roscnberg, “NETtalk: A Parallel Network
from d a t a , reason in t h e presence of novel, imprecise or incom- that Learns to Read Aloud,” Technical Report 86/01, Department of
plete information, tolerate noisy a n d r a n d o m d a t a , a n d d e g r a d e Electrical Engineering and Computer Science, Johns Hopkins Uni-
gracefully in performance m a k e s neural networks ideal for versity, Baltimore (1986).
chemical process engineering fault detection a n d diagnosis. T h e Shreve, R. N., and J. A. Brink, Chemical Process Industries. McCraw-
combination of neural-based a n d knowledge-based systems m a y Hill, New York (1977).
Shum, S. K., J. F. Davis, W. F. Punch I l l , and B. Chandrasekaran, “An
improve t h e performance speed a n d reduce t h e complexity a n d Expert System Approach to Malfunction Diagnosis in Chemical
t i m e required for building intelligent systems. Plants,” Comp. Cfzem.Eng~,12(1), 27 (1988).
Venkatasubramanian, V., “CATDEX: An Expert System for Diagnos-
ing a Fluidized Catalytic Cracking Unit,” CACHE Case-Studies
Literature Cited Series: Knowledge-Based Sysrems in Process Engineering. G . Ste-
Amari, S. A., “A Mathematical Approach to Neural Systems,” Sys- phanopoulos, ed., 1,41, Austin, T X (1988).
tems Neuroscience, J. Metzler, ed., 67, Academic Press, New York , “Inexact Reasoning in Expert Systems: A Stochastic Parallel
( I 977). Network Approach,” Conf on Artificial Intelligence Applications,
Brownston, L., R. Farrel, E. Kant, and N. Martin, Programming 13, Miami Beach, FL (Dec., 1985).
Expert Systems in OPSS: An Introduction to Rule-Based Program- Venkatasubramanian, V., and S. H. Rich, “An Object-Oriented Two-
ming, Addison-Wesley, Reading, MA (1985). Tier Architecture for Integrating Compiled and Deep-Level Knowl-
Burr, D. J., “A Neural Network Digit Recognizer,” Proc. o f l E E E In:. edge for Process Diagnosis,” Comp. Chem. Eng.. 12(9/10), 903
Conf: on Systems. Man, and Cybernetics, Atlanta, G A (Oct., 1986). (1988).
Ellman, J. L., and D. Zipser, “Learning the Hidden Structure of Werbos, P. J., “Beyond Regression: New Tools for Prediction and Anal-
Speech,” Tech. Rep. ICS Report No. 8701, University of California ysis in the Behavioral Sciences,” PhD Thesis in Applied Mathemat-
at San Diego, Institute for Cognitive Science, La Jolla, CA (1987). ics, Harvard University (1974).
Feldman, J . A., and D. H. Ballard, “Connectionist Models and their
Properties,” Cognitive Sci., 6, 205 (1982). Manuscript received May 22. 1989. and revision received Oct. 3. 1989.

2002 December 1989 Vol. 35,No. 12 AIChE Journal