See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/242073375

Neuro-Fuzzy Systems: A Survey

Article  in  WSEAS Transactions on Systems · January 2004

CITATIONS READS
55 4,220

3 authors:

José Vieira F. Morgado-Dias

Polytechnic Institute of Castelo Branco Universidade da Madeira
31 PUBLICATIONS   365 CITATIONS    163 PUBLICATIONS   962 CITATIONS   

SEE PROFILE SEE PROFILE

Alexandre Mota
University of Aveiro
70 PUBLICATIONS   639 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Natural Computing and Computational Intelligence for Numerical and Optimization Problems View project

Artificial Neural Networks Fault Tolerance View project

All content following this page was uploaded by José Vieira on 07 January 2015.

The user has requested enhancement of the downloaded file.

 Neuro-Fuzzy Systems: A Survey

JOSÉ VIEIRA * , ALEXANDRE MOTA **

* Department of Eng. Electronics and Telecommunications
Polytechnic Superior School of Technology of Castelo Branco
Avenida do Empresario 6000 Castelo Branco, PORTUGAL
zevieira@est.ipcb.pt
** Department of Eng. Electronics and Telecommunications
University of Aveiro,
Campus Universitário de Santiago, 3810 Aveiro, PORTUGAL
alex@det.ua.pt

Abstract: – The techniques of artificial intelligence based in fuzzy logic and neural networks are many times
applied together. The reasons to combine these two paradigms come out of the difficulties and inherent
limitations of each isolated paradigm. Generically, when they are used in a combined way, they are called
Neuro-Fuzzy Systems. This term, however, is many times used to assign a specific type of system that integrates
both techniques. This type of system is characterised for a fuzzy system where fuzzy sets and fuzzy rules are
adjusted using input output patterns. There are several different implementations of neuro-suzzy systems,
therefore each author defined its own model. This article summarizes a general vision of the area describing the
most known hybrid neuro-fuzzy techniques, its advantages and disadvantages.

Key Words – Hybrid Systems, Cooperative Systems, Concurrent Systems, Neuro-Fuzzy Architectures, Non-
Linear Modelling.

together, theirs cooperage providing better results

than the ones achieved with the use of any one
1 Introduction isolated technique.
The modern techniques of artificial intelligence have
found application in almost all the fields of the
human knowledge. However, a great emphasis is 1.1 Fuzzy Systems
given to the accurate sciences areas, perhaps the Fuzzy systems propose a mathematic calculus to
biggest expression of the success of these techniques translate the subjective human knowledge of the real
is in engineering field. These two techniques neural processes. This is a way to manipulate practical
networks and fuzzy logic are many times applied knowledge with some level of uncertainty. The
together for solving engineering problems where the fuzzy sets theory was initiated by Lofti Zadeh [16],
classic techniques do not supply an easy and in 1965. The behaviour of such systems is described
accurate solution. The neuro-fuzzy term was born of through a set of fuzzy rules, like:
course by the fusing of these two techniques. As
each investigator combines these two tools in IF <premise> THEN <consequent>
different way, then, some confusion was created on that uses linguistics variables with symbolic terms.
the exact meaning of this term. Still there is no Each term represents a fuzzy set. The terms of the
absolute consense but in general, the neuro-fuzzy input space (typically 5-7 for each linguistic
term means a type of system characterized for a variable) compose the fuzzy partition.
similar structure of a fuzzy controller where the The fuzzy inference mechanism consists of three
fuzzy sets and rules are adjusted using neural stages: in the first stage, the values of the numerical
networks tuning techniques in an iterative way with inputs are mapped by a function according to a
data vectors (input and output systems data). degree of compatibility of the respective fuzzy sets,
Such systems show two distinct ways of behaviour. this operation can be called fuzzyfication. In the
In a first phase, called learning phase, it behaves like second stage, the fuzzy system processes the rules in
neural networks learning its internal structure accordance with the firing strengths of the inputs. In
parameters off-line. Later, in the execution phase, it the third stage, the resultant fuzzy values are
behaves like a fuzzy logic system. transformed again into numerical values, this
Separately, each one of these techniques possesses operation can be called defuzzyfication. Essentially,
advantages and disadvantages that, when mixed this procedure makes possible the use fuzzy
categories in representation of words and abstracts • difficulty in determining the number of layers
ideas of the human beings in the description of the and number of neurons.
decision taking procedure.

The advantages of the fuzzy systems are:

2 Neuro Fuzzy Systems
Since the moment that fuzzy systems become
• capacity to represent inherent uncertainties of
popular in industrial application, the engineers
the human knowledge with linguistic variables;
designer perceived that the development of a fuzzy
• simple interaction of the expert of the domain
system with good performance is not an easy task.
with the engineer designer of the system;
The problem of finding membership functions and
• easy interpretation of the results, because of the appropriate rules are frequently a tiring process of
natural rules representation; attempt and error. Thus, appears the idea of applying
• easy extension of the base of knowledge learning algorithms to the fuzzy systems. The neural
through the addition of new rules; networks, that have efficient learning algorithms,
• robustness in relation of the possible had been presented as an alternative to automates or
disturbances in the system. to support the development of tuning fuzzy systems.
The first studies of the neuro-fuzzy systems date of
And its disadvantages are: the beginning of the 90’s decade, with Jang, Lin and
• incapable to generalize, or either, it only Lee in 1991, Berenji in 1992 and Nauck from 1993,
answers to what is written in its rule base; etc. The majority of the first applications were in
• not robust in relation the topological changes process control. Gradually, its application spread for
of the system, such changes would demand all the areas of the knowledge like, data analysis,
alterations in the rule base; data classification, imperfections detention and
• depends on the existence of a expert to support to decision-making, etc.
determine the inference logical rules; The neural networks and the fuzzy systems can be
combined to join its advantages and to cure its
1.2 Neural Networks individual illness. The neural networks introduce its
The neural networks try to shape the biological computational characteristics of learning in the
functions of the human brain. This leads to the fuzzy systems and receive from them the
idealisation of the neurones as discrete units of interpretation and clearity of systems representation.
distributed processing. Its local or global Thus, the disadvantages of the fuzzy systems are
connections inside of a net also are idealized, thus compensated by the capacities of the neural
leading to the capacity of the nervous system in networks. These techniques are complementary,
assimilating, learning or to foresee reactions or which justifies its.
decisions to be taken. W. S. McCulloch, W. Pits,
described the first Neural Network model and F. 3 Types of Neuro-Fuzzy Systems
Rosenblatt (Perceptron) and B. Widrow (Adaline) In a general way, all the combinations of techniques
develop the first training algorithm. The main based on neural networks and fuzzy logic can be
characteristic of the neural networks is the fact of called neuro-fuzzy systems. The different
these structures could learn with examples (training combinations of these techniques can be divided, in
vectors, input and output samples of the system). accordance with [10], in the following classes:
The neural networks modify’s its internal structure
and the weights of the connections between its Cooperative Neuro-Fuzzy System: In the cooperative
artificial neurones to make the mapping, with a level systems there is a pre-processing phase where the
of acceptable error for the application, of the relation neural networks mechanisms of learning determine
input/output that represent the behaviour of the some sub-blocks of the fuzzy system with the
modelled system. patterns . For instance, the fuzzy sets and/or fuzzy
rules (fuzzy associative memories [8] or the use of
The advantages of the neural networks are: clustering algorithms to determine the rules and
• learning capacity; fuzzy sets position [3]). After the fuzzy sub-blocks
• generalization capacity; are calculated the neural network is taken away
• robustness in relation to disturbances. executing only the fuzzy system.

And its disadvantages are: Concurrent Neuro-Fuzzy System: In the concurrent

• impossible interpretation of the functionality; system the neural network and the fuzzy system
work continuously together. In general, the neural neural networks theory (heuristical learning
networks pre-processes the inputs (or pos-processes strategies) to determine its parameters (fuzzy sets
the outputs) of the fuzzy system. and fuzzy rules) through the patterns processing
(input and output)”.
Hybrid Neuro-Fuzzy System: In this category, a A neuro-fuzzy system can be interpreted as a set of
neural network is used to learn some parameters of fuzzy rules. This system can be total created from
the fuzzy system (parameters of the fuzzy sets, fuzzy input output data or initialised with the à priori
rules and weights of the rules) of a fuzzy system in knowledge in the way of fuzzy rules. The resultant
an iterative way. The majority of the investigators system by fusing fuzzy systems and neural networks
uses the neuro-fuzzy term to assign only hybrid has as advantages of learning through patterns and
neuro-fuzzy system. the easy interpretation of its functionality.
There are several different ways to develop hybrid
neuro-fuzzy systems, therefore, for being a recent
4 Cooperative Neuro-Fuzzy Systems research subject, each investigator has defined its
In a cooperative system the neural networks are only
own particular models. These models are similar in
used in an initial phase. In this case, the neural
its essence, but they present basic differences.
networks determine sub-blocks of the fuzzy system Many types of neuro-fuzzy systems are represented
using training data, after this, the neural networks
by neural networks that implement logical functions.
are forget and only the fuzzy system is executed. In
This is not necessary for the application of an
the cooperative neuro-fuzzy systems, the structure is
learning algorithm in to a fuzzy system, however,
not total interpretable what can be considered a
the representation trouth a neural networks is more
disadvantage.
convenient because it allows to visualise the flow of
data through the system and the error signals that are
F UZZY F UZZY
R ULES SE T S used to update its parameters. The aditional benefit
is to allow the comparison of the different models
NEURAL FUZZY
NETWORK SYSTEM and visualise its structural differences. There are
several neuro-fuzzy architectures like:

Fuzzy Adaptive Learning Control Network

(FALCON) C. T. Lin and C. S. Lee [9];
Figure 1. Cooperative Systems Adaptive Network based Fuzzy Inference System
(ANFIS) R. R. Jang [5];
5 Concurrent Neuro-Fuzzy Systems Generalized Approximate Reasoning based
A concurrent system is not a neuro-fuzzy system in Intelligence Control (GARIC) H. Berenji [2];
the strict sense, because the neural network works Neuronal Fuzzy Controller (NEFCON) D. Nauck &
together with the fuzzy system. This means that the Kruse [11];
inputs enters in the fuzzy system, are pre-processed Fuzzy Inference and Neural Network in Fuzzy
and then the neural network processes the outputs of Inference Software (FINEST) Tano, Oyama and
the concurrent system or in the reverse way. In the Arnould [15];
concurrent neuro-fuzzy systems, the results are not Fuzzy Net (FUN) S. Sulzberger, N. Tschichold and
completely interpretable, what can be considered a S. Vestli [14];
disadvantage. Self Constructing Neural Fuzzy Inference Network
(SONFIN) Juang and Lin [6].
Fuzzy Neural Network (NFN) Figueiredo and
FUZZY
NEURAL Gomide [4];
SYSTEM NEURAL
NETWORK Dynamic/Evolving Fuzzy Neural Network (EFuNN
FUZZY
NETWORK SYSTEM and dmEFuNN) Kasabov and Song [7];

A summarise description of the five most popular

Figure 2. Concurrent Systems neuro-fuzzy architectures is made in next section.

6 Hybrid Neuro-Fuzzy Systems

In Nauck [10] definition: “A hybrid neuro-fuzzy 6.1 FALCON Architecture
system is a fuzzy system that uses a learning The Fuzzy Adaptive Learning Control Network
algorithm based on gradients or inspired by the FALCON [9] is an architecture of five layers as it is
shown in figure 3. There are two linguistics nod es
for each output. One is for the patterns and the other
is for the real output of the FALCON. The first
hidden layer is responsible for the mapping of the
input variables relatively to each membership
functions. The second hidden layer defines the
antecedents of the rules followed of the consequents
in the third hidden layer. FALCON uses an hybrid
learning algorithm composed by a unsupervised
learning to define the initial membership functions
and initial rule base and it uses a learning algorithm
based on the gradient descent to optimise/adjust the
final parameters of the membership functions to
Figure 4. ANFIS achitecture.
produce the desired output.
6.3 GARIC Achitecture
The Generalized Approximate Reasoning based
Intelligence Control GARIC [2] implements a
neuro-fuzzy system using two neral netwoks
modules, ASN (Action Selection Network) and AEN
(Action State Evaluation Network). The AEN is an
adaptative evaluator of ASN actions. The ASN of
the GARIC is an advanced network of five layers.
Figure 5 illustrates GARIC-ASN structure. The
connections between the layers are not weighted.
The first hidden layer stores the linguistics values of
Figure 3. FALCON achitecture. all input variables. Each input can only connect to
the first layer, which represent its associates
linguistics values. The second hidden layer
6.2 ANFIS Architecture represents the fuzzy rules nodes that determine the
The Adaptive Network based Fuzzy Inference compatibility degree of each rule using a softmin
System ANFIS [5] implements a Takagi Sugeno operator. The third hidden layer represents the
fuzzy inference system and it has five layers as linguistics values of the output variables. The
shows in figure 4. The first hidden layer is conclusions of each rule are calculated depending on
responsible for the mapping of the input variable the strength of the rules antecedents calculated in the
relatively to each membership functions. The rules nodes. GARIC uses the mean of local mean of
operator T-norm is applied in the second hidden maximum method to calculate the output of the
layer to calculate the antecedents of the rules. The rules. This method needs for a numerical value in
third hidden layer normalizes the rules strengths the exit of each rule. Thus, the conclusions should be
followed by the fourth hidden layer where the transformed from fuzzy values for numerical values
consequents of the rules are determined. The output before being accumulated in the final output value of
layer calculates the global output as the summation the system. GARIC uses a mixture of the gradient
of all the signals that arrive to this layer. descending with a reinforcement learning for a fine
ANFIS uses backpropagation learning to determine adjustment of its internal parameters.
the the input membership functions parameters and
the least mean square method to determine the
consequents parameters. Each step of the iterative
learning algorithm has two parts. In the first part, the
input patterns are propagated and the parameters of
the consequents are calculated using the iterative
minimum squared method algorithm, while the
parameters of the premises are considered fixed. In
the second part, the input patterns are propagated
again and in each iteration, the learning algorithm
backpropagation are used to modify the parameters Figura 5. GARIC achitecture.
of the premises, while the consequents remain fixed.
6.4 NEFCON Architecture Neural Fuzzy Network (dmEFuNN) [10] is a
The Neural Fuzzy Controller NEFCON [11] was modified version of the EFuNN with the idea of not
drawn to implement a Mamdani type inference fuzzy only the winning rule node’s activation is
system as illustrated in figure 6. The connections in propagated but a group of rule nodes that is dynamic
this architecture are weighted with fuzzy sets and selected for every new input vector and their
rules using the same antecedents (called shared activation values are used to calculate the dynamical
weights), which are represented by the drawn parameters of the output function. While EFuNN
ellipses. They assure the integrity of the base of implements Mamdani type fuzzy rules, dmEFuNN
rules. The input units assume the function of implements Takagi Sugeno fuzzy rules.
fuzzyfication interface, the logical interface is
represented by the propagation function and the
output unit is responsible for the defuzzyfication
interface. The process of learning in architecture
NEFCON is based on the mixture of reinforcement
learning with the backpropagation algorithm. This
architecture can be used to learn the rule base from
the beginning, if there is no à priori knowledge of
the system, or to optimise an initial manually
defined rule base. NEFCON has two variants
NEFPROX (for function approximation) and
NEFCLASS (for classification tasks) [14].
Figure 7. EfuNN architecture.
Defuzzyfi-
cation Layer To get a more detail description of this architectures,
beyond the specific pointed references made in this
paper, a detailed survey was made by Abraham [1]
in 2000 where it can be found a detailed description
of several well known neuro-fuzzy architectures
theirs respective learning algorithms.

7 Discussion and Application

The hybrid neuro-fuzzy systems present a
intepreteble model and they have learning capacities
Fuzzyfica- in a supervised way. In FALCON, GARIC, ANFIS,
tion Layer NEFCON, SONFIN and FINEST the learning
process only worried with the adaptation of internal
Figure 6. NEFCON architecture. parameters of a fixed structure of the system. For
complex problems, it will be computational
demanding to determine all the parameters (of
6.5 EFuNN Architecture premises parameters, consequents parameters,
In Evolving Neural Fuzzy Network EFuNN [10] all number of rules, etc) because the parameters will
nodes are created during the learning phase. The first grow exponentially.
layer passes data to the second layer that calculates An important characteristic of the architecture
the degrees of compatibility relate to the predefined dmEFuNN and EFuNN is to make the training only
membership functions. The third layer contains in one iteration. This characteristic will allowed the
fuzzy rule nodes representing prototypes of input- implementation of on-line adaptation in a simple
output data as an association of hyper-spheres from way.
the fuzzy input and fuzzy output spaces. Each rule Abraham proposed [1] a evolutionary approach
node is defined by two vectors of connection based on genetic algorithms for the optimisation of
weights, which are adjusted through a hybrid all parameters of the structure of a neuro-fuzzy
learning technique. The fourth layer calculates the system (type of fuzzy system, number of rules,
degree to which output membership functions are parameters, inference operators, rules and
matched the input data and the fifth layer carries out membership functions).
the defuzzyfication and calculates the numerical
value for the output variable. Dynamic Evolving
In the industrial field, initially these architectures Soft Computing”, Springer Verlag, Germany,
were applied in modelling non-linear systems and 2000.
control engineering. Actually, however these [4] M. Figueiredo and F. Gomide; "Design of Fuzzy
architectures are used in almost all knowledge areas Systems Using Neuro-Fuzzy Networks", IEEE
where a non-linear function should be approximated. Transactions on Neural Networks, 1999, Vol.
The actual neuro fuzzy systems application areas are 10, no. 4, pp.815-827.
medicine, economy, control, mechanics, physics, [5] R. Jang, “Neuro-Fuzzy Modelling:
chemistry, etc. Architectures, Analysis and Applications”, PhD
Thesis, University of California, Berkley, July
1992.
8 Conclusions [6] F. C. Juang, T. Chin Lin, “An On-Line Self
In this article it is presented, in a summarize way,
Constructing Neural Fuzzy Inference Network
the last decade of investigation in the area of the
and its applications”, IEEE Transactions on
modelling non-linear functions through neuro-fuzzy
Fuzzy Systems, 1998, Vol. 6, pp. 12-32.
systems. Duo to the vast number of common tools it
[7] N. Kasabov e Qun Song, “Dynamic Evolving
continues to be difficult to compare conceptual the
Fuzzy Neural Networks with ‘m-out-of-n’
different architectures and to evaluate comparatively Activation Nodes for On-Line Adaptive
its performances. In generic terms the bibliography
Systems”, Technical Report TR99/04,
points that neuro fuzzy systems that implement
Departement of Information Science, University
Takagi-Sugeno type fuzzy inference systems get
of Otago, 1999.
accurate results than the approaches that implement
[8] B. Kosko, “Neural Networks and Fuzzy
neuro fuzzy inference systems of Mamdani type,
Systems: A Dynamical System Approach to
although its bigger computational demanding. As a
Machine Intelligence”, Prentice Hall,
guide line for implementing highly efficient neuro- Englewood Cliffs, New Jersey, 1992.
fuzzy systems they should have the following [9] T. C. Lin, C. S. Lee, “Neural Network Based
characteristics; fast learning; adaptability on-line; to
Fuzzy Logic Control and Decision System”,
adjust itself with the aim of obtaining the small
IEEE Transactions on Computers, 1991, Vol.
global error possible; small computational
40, no. 12, pp. 1320-1336.
complexity.
[10] D. Nauck, F. Klawon; R. Kruse,
The data acquisition and the pre-processing of input
“Foundations of Neuro-Fuzzy Systems”, J.
training data are also very important for the success Wiley & Sons, 1997.
of the application of the neuro-fuzzy architectures. [11] D. Nauck, R, Kurse, “Neuro-FuzzySystems
All the neuro-fuzzy architectures use the gradient
for Function Approximation”, 4th International
descent techniques of for the learning its internal
Workshop Fuzzy-Neuro Systems, 1997.
parameters. For a faster convergence of the
[12] D. Nauck, “Beyond Neuro-Fuzzy Systems:
calculation of these parameters it would be
Perspectives and Directions”. Proc. of the Third
interesting to explore other efficient algorithms of
European Congress on Intelligent Techniques
neural networks learning as the conjugated gradient
and Soft Computing (EUFIT’95), Aachen, 1995.
search in spite of the backpropagation algorithm. [13] D. Nauck; “A Fuzzy Perceptron as a
Generic Model for Neuro-Fuzzy Approaches”.
References
Proc. Fuzzy-Systems, 2nd GI-Workshop,
[1] A. Abraham and Baikunth Nath, “ Hybrid
Munich, 1994.
Intelligent Systems: A Review of a decade of
[14] S. Sulzberger, N. Tschichold e S. Vestli,
Research”, School of Computing and
“FUN: Optimization of Fuzzy Rule Based
Information Technology, Faculty of Information
Systems Using Neural Networks”, Proceedings
Technology , Monash University, Autralia, of IEEE Conference on Neural Networks, San
Technical Report Series, 5/2000, 2000, pp. 1-55. Francisco, March 1993, pp. 312-316.
[2] H. R. Berenji and P. Khedkar, “Learning and
[15] S. Tano, T. Oyama, T. Arnould, “ Deep
Tuning Fuzzy Logic Controllers through
Combination of Fuzzy Inference and Neural
Reinforcements”, IEEE Transactions on Neural
Network in Fuzzy Inference”, Fuzzy Sets and
Networks, 1992, Vol. 3, pp. 724-740.
Systems, 1996, Vol. 82(2), pp. 151-160.
[3] E. Czogala and J. Leski, “Neuro-Fuzzy
[16] L. A. Zadeh; “Fuzzy Sets”, Information and
Intelligent Systems, Studies in Fuzziness and
Control, 1965, Vol. 8, pp. 338-353.

View publication stats