zyx zyxwvutsrq zyxwvutsrqpo Learning Fuzzy Cognitive Maps using Evolution Strategies: a Novel Schema for Modeling and Simulating High-Level Behavior D. E. Koulouriotis I. E. Diakoulakis D. M. Emiris Dept. of Production Eng. & Management, Dept. of Production Eng. & Management, Athens University of Technical University of Crete, Technical University of Crete, Economics & Business, Chania, Greece, Athens, Greece, Chania, Greece, e-mail:jimk@dpem.tuc.gr e-mail: dgiannis@dpem.tuc.gr e-mail: emirisadpem. tuc.gr Abstract - FCM is recognized as a flexible and powerful modeling and simulating technique; however, it is a relatively new methodology, which exhibits weaknesses mainly in the algorithmic background. Such weaknesses become evident during heuristic evaluation of the causeeffect relationships describing FCM-based systems. The external intervention (typically from experts) for the determination and fine-tuning of FCM parameters cannot be regarded as an accurate and efficient way to design and manage FCMs, especially in the case of highly complicated structures, where even experts meet difficulties in their attempts for an holistic interpretation. The introduction and implementation of a training procedure based on a robust and flexible optimization tool constitutes a promising alternative. The present study focuses on Evolutionary Computation, since this domain encompasses optimization techniques possessing the needed features for this type of problems. Evolution Strategies appear as the most appropriate methodology, and as such, they are tested herein for a potential implementation in FCM-based systems. The proposed approach combines FCM & ES concepts and sets the basis for establishment and deployment of structural evolution, which will broaden the applicability of FCMs. 1 Introduction The FCM theory was first introduced by Kosko [Kos86] and was followed by the presentation of a significant number of research works extending the initial principles [TS87,ZC88, ZC89, Tab91,ZCW92, PK95,OH97, TM97, KL98, Mar99, KDEOl]. Subsequent research has focused on the analysis of the operation mode of Fuzzy Cognitive Maps (FCMs) and the development of potential applications in various scientific fields, e.g. analysis of electrical circuits, socialeconomic-political systems, organizational behavior, systems control, policy, etc. [SM88, Si195a-c, CGW96, PB96, LK97, SSK98, JSK99, GSOO]. The main incentive which led to further research and development in this relatively new domain was the wide recognition of FCM as a promising modeling and simulation methodology, with remarkable characteristics such as abstraction, flexibility, adaptability and fuzzy reasoning; yet, FCM methodology cannot be considered as a well-defined technique because the underlying theoretical framework presents certain deficiencies, such as the strict dependence of FCM design and inference mechanism on experts’ knowledge, which involves subjective reasoning and therefore restricts accuracy and reliability in high-level structures. It is thus necessary to overcome such deficiencies in order to improve efficiency and robustness of FCM. A promising approach for reducing or even eliminating experts’ intervention is the development of a learning (training) algorithm. The term training is common in the domain of leaming systems and describes the procedure through which the FCM cause-effect relationships (and even the whole structure) may be estimated using historical data on concept states and total effects; thus, past information can be profited of and used as a means to design and finetune FCM-based systems. Once a learning procedure has been set, one should select the most convenient optimization technique that will be associated to the training algorithm. Needless to say, the choice of an optimization tool is not easy, as numerous parameters are involved, such as flexibility, adaptability, convergence velocity and stability, etc. One category of optimization tools that fits the standards and special characteristics of FCMs is Evolutionary Computation. Evolutionary Algorithms (EAs) have exhibited an impressive deployment in a variety of domains. In this study, a connection between FCM theory and EAs is attempted, since the most important research and application field of EAs is numerical optimization and, at the same time, their adaptability in many problems is widely recognized. Among EAs, Evolution Strategies (ES) are considered the most suitable algorithmic approach; thus, their applicability and effectiveness in FCM training is tested. This article is organized as follows: in Section 2, Cognitive Maps are introduced and the additional features of FCM that have been endowed from fuzzy theory and practice are presentcd. In Section 3, the basic principles of EAs and the way Evolution Strategies operate are highlighted. In Section 4, the training and structure evolution concepts are presented and, the incorporation of ES in FCM-based modeling is demonstrated using two systems as paradigms. In Section 5, the ensuing results are interpreted and finally, in Section 6 the core conclusions are discussed and future research directions are outlined. zyxw zyxwvutsr zyxwvutsrqpo 0-7803-6657-3/01/$10.00 02001 IEEE 364 z zyxwvu zyxwvutsrqpo 2 Basic Principles of Fuzzy Cognitive Maps Thc basis of FCM theory is Cognitive Maps (CMs), which were firstly presented by Tolman in 1948 [To148]. The function of CMs concerns the analysis of systems described by diverse factors (concepts) that are perceived connected with cause-effect relationships (causality). The underlying inference mechanism estimates the final condition of given systems whcn changes in the state of some (or all) of their concepts occur. A solid theoretical base is not inherent in this mechanism, as evident by the numerous differences between the published papers analyzing or applying CMs. A cognitive map is a network where the nodes represent the concepts and the links represent the cause-effect relationships between the concepts of a given system. An example of a CM that has been presented in various published studies [ZC88, PK95, LK971 is depicted in Figure 1. The links between the nodes may take the value +1 or -1 while the nodes the value - 1 , 0 or + 1 . For instance, the value +1 in the link between the nodes “Modernization” and “Migration into city” in Figure 1 means that an increase of concept “Modernization” state causes an increase of concept “Migration into city” state and, in parallel, a decrease of concept “Modernization” state causes a decrease of concept “Migration into city” state. A value +I in a node implies an increase or improvement of the concept state that the node represents while a value -1 expresses a decrease of the corresponding concept state. simple CMs. High (absolute) values in the links between concepts (e.g. 0.8, -0.9) signify strong cause-effect relationships between the concepts while high (absolute) values in nodes indicate significant changes in the corresponding concept states. A noticeable characteristic of FCMs is that the cause-effect relationships represent many fuzzy rules and not just a premise as in typical structures using fuzzy rule bases. The inference mechanism underlying FCM-based systems, in its simple form, stimulates a portion of the constituent concept nodes and applies matrix manipulations between the concept vector and the adjacency matrix in order to specify the final (stable) condition of the whole system. The number of elements in the concept vector is equal to the number of concepts, and their values signify the corresponding node values; the elements of the adjacency matrix signify the cause-effect relationships. Despite the positive aspect about FCM modeling capabilities, their major weakness is still the intervention of experts’ for the determination of the structure and the estimation of link values. Despite the simplicity and flexibility of this practice, it is not an efficient procedure, as in complicated systems, nobody is able even to approximate the basic structure (what cause-effect relationships exist) and moreover the link values of the system. This paper proposes the exploitation of evolution strategies, as they constitute a quite flexible and effective tool that could cover the existing deficiency. zyxwvutsrq zyxwvutsrq + Number of people in a city + Migration zyxwvutsr zyxwvutsrqponmlkjihgfe ‘1 Garbage per area I+ \?[ sgp; \xModernization Number of diseases per 1000 residents Bacteria per area Figure 1 : A CMRepresenting System of Public Health. Number of people in a city +”$ 3 Evolution Strategies yinto city Evolutionary Computation is the research area concerning Garbage per area I s \ 1 +0.9 - 4 +o. I +yinto Migration city Modemization Number of diseases per 1000 residents ..lito.’ Bacteria $er area Figure 2: The FCMof Public Health System Fuzzy Cognitive Maps (FCMs) have been developed through the combination of cognitive map principles and hzzy logic reasoning. Nodes and links in an FCM (Figure 2) obtain values in the interval [-1,1], while their positive and negative values have the same meaning as in 365 the development of algorithms that imitate the principles of natural evolution mainly for the solution of complicated optimization problems. In fact, since they firstly emerged, Evolutionary Algorithms (EAs) have extensively covered many research fields such as artificial intelligence, numerical optimization and decision support systems. The benefits of evolutionary algorithms that account for their progress are adaptability and flexibility in problem modeling, use of problem-specific knowledge, rapid implementation of EA processes, robust performance (convergence reliability-exploitation of the available information in the search space) and global search characteristics (controllable premature convergence exploration of the search space) [Go189, Dav91, FFA91, Fog93, Bac96, BFM97, BHS971. Considering an optimization problem, EAs assume the existence of apopulation of individuals - P(t) at generation t - each of which not only represents a search point in the space of potential solutions but also encompasses problemsolution information. Individuals are vectors of length proportionate to the number of object variables constituting the optimization problem. Conceming individuals representation, diverse approaches may be adopted; Evolution Strategies allow the manipulation and use of the real-world values of the applied object variables contrary to zy zyx zyxwvutsrqpon zyxwvutsrq other EAs like Genetic Algorithms that call for encodingldecoding procedures to manage binary strings. In most cases in ES the genotype space coincides with the phenotype space. Individuals take their initial values in a mostly random manner and afterwards evolve successively to better regions of the search space (according to their fitness value, which is estimated applying a problemdependent evaluation procedure). The evolution processes are recombination, mutation and selection. During recombination, pairs of (or more than two) individuals combine their characteristics through a random exchange of genetic information while, during mutation, a portion of the individuals undergoes random changes in some of their characteristics. Mutation is extremely important for an effective evolution operation as it is a means of obtaining new information not included in the ancestors and therefore new subspaces of the search space may be examined. Finally, selection aims to improve the average quality of the population in order to reinforce the search in promising areas in the space of potential solutions. This becomes feasible through the indirect increase of the reproduction probability of the favorable individuals. A typical EA structure is as follows: parameter CJ, (oI€ R : , l<i<n) that also undergoes variations and determines the vector element as follows: Parameter t is used in order to impose different variation in each vector element separately, while parameter z' is used in order to differentiate the whole population from generation to generation. Admittedly, it is the most appropriate and powerful mutation schema for the conduction of applications with evolution strategies, and that's the reason we eventually use it herein for the FCM training procedure. However, apart of this schema, three other modes involving adaptation of strategy parameters may be used (each one having its own advantages and disadvantages). The first one described in [Sch95, Bac961 considers the existence of only one strategy parameter per individual and the mutation procedure is the following: zyxwvutsrq zyxwvutsrq zyxwvutsrqp zyxw zyxwvutsr zyxw zyxwvutsr zyxwvuts Evolutionarv AlPorithm t=O strategies. According to primary approaches, mutation in evolution strategies is performed independently 011 each vector element by adding a normally distributed random value with mean value 0 and standard deviation CJ: x ,=x, +CJ Nl(O,l), where N,(O,1) is a normally distributed random number with mean value equal to 0 and standard deviation equal to I. Although this general form of mutation is sufficient to assure the existence of variation in the population, further modifications have been proposed. The most extended form [Sch95, BHS97, Bac961 requires that each vector element corresponds to a strategy Initialize: P(t) Evaluate: P(t) While (STOP-CRITERION not satisfied) Recombine: P'(t) = r (P(t)) Mutate: P"(t) = m (P'(t)) Evaluate : P "(t ) Select: P(t+l) = s (P"(t)) t = t+l End While The idea to use principles of organic evolution processes as rules for optimum seeking procedures was developed independently by Holland [Ho175], who introduced theory of Genetic Algorithms, and by Rechenberg and Schwefel who described the Evolution Strategies (ES) [Rud92, Mic94, Sch95, SR951. Three new branches of evolutionary computation have appeared recently: Genetic Programming, Evolutionary Programming, and Classifier Systems. In order to describe the structure and operation mode of evolution strategies, a general optimization problem must be defined. An optimization (minimization) problem requires finding the parameters XEMGR" that minimize the value of the objective function f (x), where j M+R. If the*global minimum of f is achieved with the vector x then VXEM=.f(x*)<flx). An evolution strategy requires the existence of a population with p individuals that are realvalued (float numbers) vectors with n elements. The vector elements (x,ER, l<i<n) are the object variables. The representation of the variables as float numbers (which are usually their real-world values) and not as complicated coded strings constitutes a great advantage of evolution g' = o . eN(o,Aa), x: = x, + 0'.N,(OJ) This form is obviously restrictive as it applies just one strategy parameter. The second one presented in [Mic94] considers the existence of many strategy parameters per individual. The adaptation of strategy parameters at each generation is conducted,in a global mode; that is, the same variation applies to all C J ~This . mutation schema proceeds as follows: c' = B . eN(o,Au), x]= x, + 0;. N,(OJ) This form is inadequate as it restricts variability between of, which is an important characteristic of ES. The third one presented in [BHS91, HB901 considers the existence of many strategy parameters per individual and adaptation proceeds separately to each 0,. This mutation schema proceeds as follows: 0: = 0, .eN(o,Au), X: + = X, 0: . N I(0,l) Later, Schwefel [Sch95] and Back [Bac96] mentioned that this form has a disadvantage, as in optimization problems with n>>l, overall step sizes between generations are not substantially different. In general, the selection of specific sub-forms is case sensitive. This procedure has been firther modified and a sophisticated mutation scheme that exploits information zyxwvuts zyxwvutsr zyxwvutsrqpo zyxwvut zyxwvutsrqp zyxwvuts about the correlation between the strategy parameters has been proposed. In detail, this procedure is: o'= , zyxwvutsrqpo , ,(f"(O,l)+.rN,(O,l), I existing weakness, as with such a procedure the valuable knowledge of experts' can be extracted just by analyzing past information about the given systems. The present work focuses on the development of an ES-based procedure that determines the values of the cause-effect relationships (causality), that is, it operates as a learning algorithm. Although there are various other points about FCM operation, which require fine-tuning, an improvement of the process that specifies the link values is of major significance as it contributes towards the establishment of FCM as a robust technique. The main idea behind the proposed approach concems the design and development of algorithms that support structure evolution. This term involves whole FCM design and optimum assessment of the cause-effect relationships that are perceived to exist between the concepts of given systems; in addition, the present scheme combines adaptive mechanisms and leaming procedures for the flexible redesign and finally effective operation of cognitive maps. This approach certainly fits to the standards of the majority of engineering and social-economic models, which are dynamic and continuously structurally transformed systems. In this study, the first step towards the establishment of a structure evolution philosophy is attempted. In other words, the development of a training algorithm for the estimation of FCM cause-effect relationships, having fixed struotures, is examined. The conclusion of this part will undoubtedly affect future research about FCM structure evolution algorithms. Once the general theoretical approach conceming FCM design and fine-tuning is described, a detailed demonstration and analysis of the developed training algorithm follows. The process of causality (cause-effect relationship) estimation, which could be called training, calls for data sets that accurately (and as completely as possible) represent the structure and operation of given systems. These data sets will guide the applied evolution strategy to determine the appropriate FCM link values. The training phase of FCMs is to some extent similar to that of neural networks, that is, it is based on inpudoutput pairs that are called examples. The distinction of FCM concepts as inputs or outputs depends on the focus of the developer. In general, all the concepts of a given system may constitute the inputs and outputs at the same time. However, in its simple version, training phase is conducted selecting a limited number of concepts as outputs (usually the concepts that are in the center of interest - those that we want to estimate their value) and regarding the rest ones as input nodes in the system. In FCMs, the inputs are considered initial stimulators of the systems and the outputs indicate the final states of the corresponding concepts after the applied stimulations. Practically, the implementation of evolution strategies in FCMs entails the random formation and hrther evolution of a population of individuals that are vectors with n elements, where n equals to the number of the cause-effect relationships that is to be estimated. The specific application constitutes a constrained optimization (minimization) a; = a , + P . N , ( O , ~ ) (, j = 1 , ..., n * ( n - I ) / 2 ) x' = x + N ( 0 , C(a, a)) where N(O,C(o,a))is the generalized n-dimensional normal distribution with expectation value zero and probability density function 1 exp(--z'C-'z) P(z>= ( 2 ~ )detC +- where C-' is the covariance matrix with diagonal elements c,,=d?, . As far as the recombination operator is concemed, this is incorporated in the algorithm before the mutation and generates a new intermediate population of A individuals. The recombination process is not specifically defined, therefore, diverse types have been presented. The prominent techniques are the discrete recombination (random choice for each offspring element between those of the two parents) and the intermediary recombination (each offspring element is the arithmetic average of the corresponding parental elements). The formula that is used for the selection of the pairs of individuals in order to produce the offspring must sustain a random combination in each generation. Like mutation and recombination, selection has been applied using diverse approaches. The two basic strategies that have been followed are the @,A) and @+A). Both strategies indicate that p parents create A2p offspring but the difference between them appears when the individuals for the next generation must be determined. The @,A) strategy imposes the elimination of the parental population and its complete replacement with the best p offspring. On the other hand, with the @+A) strategy, the best p individuals of the whole pool of parents and offspring are selected. The quick overview of ES methodological framework clearly reveals the flexibility of ES and, at the same time, their capability of instant adaptation to diverse structures and incorporation of problem-specific information. Therefore, the formula of ES and generally all the approaches of evolutionary computation encompass a list of characteristics that are indisputably advantageous in modeling and in general managing highly complicated systems like Fuzzy Cognitive Maps are. zyxwvuts 4 FCM optimization using ES: Two Examples As already mentioned, the main questionable operation in FCM theoretical framework and practical implementation is the determination of their structure and of the value of the cause-effect relationships. At present, experts' intervention has been the only way to address this problem. The design of a leaming (training) algorithm could eliminate the 367 problem as the possible values of each vector element, that is causality, lie in the interval [-1, t-13. In this study, a barrier method is used; in other words, the fitness of individuals outside the feasible region is drastically deteriorated and therefore these individuals cannot reproduce any more, so they are indirectly eliminated. The fitness of each individual is computed taking into account the degree to which the outputs that are produced by the given initial stimulations, are close to the outputs indicated by the predetermined inputloutput pairs. The sum of the absolute deviations between real and estimated outputs is an appropriate measure of fitness: k=l zzyx yxwvutsrqpo Money Flow- " +0.5 I -0.6 Situation zyxwvuts zyxwvutsrqponmlkjihgfedcb zyxw i=l The role of the examples for conducting the training is crucial, since in the case of inappropriately selected inputJoutput pairs, the applied evolution strategy may produce individuals with zero fitness (almost perfect approximation of the examples) but with significant divergence of the actual values of the cause-effect relationships that they are supposed to be estimated. The purpose of the present study is thus the demonstration of ES ability to eliminate the existing weakness and not to analytically estimate the optimum ES parameters that minimize the error during FCM training. Specifically, two simple but distinctive examples were selected and then the performance of some evolution strategies was tested using hypothetical structures, that is, predetermined link values. The goal of these tests was the extraction of the possibility with which ES-based training can approximate the causality between the system nodes. The two FCMs that form the test set are: (a) a frequently mentioned FCM in many published papers, which is based on an article by Henry Kissinger and illustrates the Middle East peace policy (Figure 3), and (b) the FCM depicted in Figure 4, which represents a plain way through "Economic Situation" is determined. The cause-effect relationships in the FCM of Figures 3 and 4 are completely hypothetical. In1 rest Rates economic +o,, Events Figure 4: Economy System -0.2 Stability The procedure 'that was followed for approximating the FCM link values was simple. The input and output nodes for each FCM (Figures 3 and 4) were first defined. In the case of the FCM depicted in Figure 3 the output is the node "Strength of Lebanese Government" while in case of the. FCM depicted in Figure 4 the output is the node "Economic Situation". The input nodes for both FCMs are the remaining ones. A modified FCM inference mechanism working in multistimulus and ambiguous conditions is described in [KDEOl]. Given the structure and the link values of each FCM, a list of examples was formulated. In order to test the effect of the examples over the ES outcome, for each FCM, two different sets of examples were formed. In the first set, each example was based on a single node initial stimulation, while in the second set each example was based on parallel stimulations of pairs of nodes. For both FCMs, the number of examples in the first set is 25 (each of the 5 input nodes is stimulated for the values 0.2, 0.4, 0.6, 0.8 and l), while in the second set is 10 (parallel stimulated nodes: 1-2, 1-3, 1-4, 1-5,2-3,2-4,2-5,3-1, ...,4-5). Finally, two evolution strategies applied: @,A) and @+A), with A=4p for both strategies. For each strategy, the following cases were tested: zy zyxwvutsrqp zyxwvutsrq zyxwvutsr zyxwvutsrq Population=20: The termination criterion of the algorithm was: (Best Individual Fitness<l 0-3)OR (Generation> 100) The results about the system depict the Middle East policy are given in Tables 1 and 2, while those about the economic system are shown in Tables 3 and 4. The information given in each table concems: (a) the number of generations and time (in seconds on a PC Pentium I1 Celeron-350 MHz) that was needed for fulfilling the termination criterion, and (b) the minimum mean absolute error between the actual and the estimated FCM link values (n: number of FCM links to be estimated, examples: number of inputloutput pairs): Arab Radicalism . fSyrian Control o -0.4 PLO Terrorism Lebanon / Strength of Lebanese Government L(-0.5 Figure 3: FCM representing Middle East peace policy 368 zyxw zyxwvu zyxwvutsrqponmlkjihg I I zyxwvutsrqponmlk zyxwvutsrqponml I EXAMPLES SET 1 (PA) (P+W time generation (in time sec) error generation r=0.3r'=0.3 r=0.3r'=0.2 r=O3r'=O1 r=0.4r'=0.2 T=0.4T'=0.1 I iI (in sec) error 21 I 392 I 0.36 I 64 I 855 I 0.29 72 I 1360 I 0.34 , I 100 I 1241 I 0.21 028 88 1172 036 I 617 33 I I I 23 I 431 I 0.17 I 43 I 546 I 0.18 I 26 I 100 I 1222 I 0.24 I 0.19 I (izc,l error lgenerationl error1 - I 518 generation ltim:cyl zyxwvutsrqponm zyxwvutsrq Table 1: Middle East policy FCM(Examp1e Set I ) Table 4: FCMofEconomy (Example Set 2) 5 Analysis of the Results (Ph) (P+V Analyzing the results presented in Tables 1 to 4,significant conclusions can be drawn about the special characteristics of the underlying problem (and particularly those of the examined systems), the most suitable strategy for searching the space of solutions and the role of the examples in the leaming procedure. Specifically: Large populations do not guarantee a better performance. On average, strategies with population of 50 individuals needed 30-40% more computational time (and 10-30% less generations) than strategies with population of 20 individuals. Moreover, strategies with 50 individuals exhibited a performance that was worse about 10% (except from the case of example set 2 in economy system) than their counterparts. The interpretation of this outcome is that most generations in case of strategies with population of 20 individuals proved quite advantageous, as a better exploitation of genetic information took place. In other words, the recombination operation that allows the interchange of information about search space characteristics proved to be a valuable tool according to the standards of the specific problem. Despite the similarity of the produced errors, bJ)-ES proved better than the @+l)-ES as it needed 50% less computational time than the corresponding gl+L)-ES. This conclusion corroborates the results of other researchers as well, who mentioned that the (pJ) strategy is more efficient than its counterpart. As far as the strategy parameters r and r' are concerned, the combinations (t,r'):(0.3,0.2) and (r,r'):(0.4,0.2) seem to offer, on average, a better level of performance contrary to the other settings. The aforementioned combinations support the use of large values for the strategy parameters comparatively to the feasible region generation time (in generation time sec) error (insec) error Table 2: Middle East policy FCM (Example Set 2) (PA generation time (in sec, zyxwvutsrqpo error generation time (in error sec) Table 3: FCMof Economy (Example Set I) 369 zyxwvutsrq zyxwvutsrq zyxwvuts zyxwvuts of the object variables, which is [-1,I], and also indicate the necessity to sustain a mediocre level of deviation between t and t‘ (O.l<t-t’<0.2). The high problem complexity and the limited volume of example sets impede an accurate* estimation of convergence rates (rate of progress q ), in relation to step length and @,+A) strategies. Consequently, only the ratio th‘ between the default settings r and r’, where I= q1*.(2n)-I’~ and t‘= q*~(2n”’)-”2as proposed by Schwefel [Sch95], may be computed and compared with our findings. According to defaults settings for n=7, tlr’=1.6 and for n=10, rlt‘=l.8, while in this study we found that the best ratio rlr‘ lies into the interval [ 1.5,2]. The set 1 of examples, despite its simplicity and limited amount of information, produced 5- 10% smaller error than the second set but with 100% more computational time. These remarks plainly reveal the significance of the type of examples that will be used in the training procedure. Although a combination of the two sets of examples may produce the optimum data set for the specific FCM-based systems, systematic analysis and research of the FCM structures is needed in order to construct a methodological approach for setting properly the examples on which evolution strategies will be based. The performance of evolution strategies in the case of the Middle-East policy system was better than that of the economy system, because the first FCM not only had fewer cause-effect relationships but also had a simpler (fewer circles etc.) structure. On the other hand, it’s interesting to point out that, despite the difference in the structure of the two FCMs, the deviation of the results between them was not significant. This fact leads to the conclusion that most of the produced error when trying to approximate FCM structures is due to factors that are irrelevant to the specific tested FCMs but are dependent on exogenous parameters like the number and type of examples orland the number and type of outputs. Regarding the fact that both tested FCMs had only one output node, the quality of the ensuing results can be considered to be at an acceptable level. An increase of the output nodes would undoubtedly provide additional information that makes the examples more simple and accurate (and therefore more useful) and in parallel it can limit and even resolve ambiguous situations due to the existence of circles in FCM structures. those selected for training the given systems; in other words, there are many local minima that prevent the accurate estimation of the appropriate structures. The mediocre performance of ES in the tested examples should not discourage research; rather it should prompt for further research in the field of Fuzzy Cognitive Maps. Special attention must be given to the analysis and determination of the best evolution strategies for estimating the cause-effect relationships. This task must be conducted considering the diverse characteristics of FCM structures, such as the number of nodes, the links and the potential circles (graphical and network analysis), the settings about the inputs and the outputs etc. In addition, the production (or selection) of the example sets has to be studied in a more systematic way, as it is a crucial factor for the performance of evolution strategies. The endogenous anomalies of the formulated problem (example-based leaming of FCM links), such as large dimensionality, non-differentiability, strong non-linearity and noisy objective function, restrict applicability of conventional optimization techniques, while ES prove to be an effectual methodology because they surpass the existing drawbacks; nevertheless, a major future research direction is the investigation of innovative optimization techniques and hybrid training tools to conduct FCM leaming. The synthesis of FCM with ES in this article sets the basis for the development of structure evolution strategies. The estimation of the values in the cause-effect relationships constitutes only a part of the entire theoretical scheme, which involves construction of the structure of FCMs from zero ground and further modification and adaptability to dynamic environments. The combinatory application of FCMs and ES creates an effective knowledge representation technique and consists a novel research and application field to cover needs and deficiencies in a variety of modeling and simulating problems. zyxwv zyxwvuts 6 Conclusions - Future Research The ensuing results and conclusions of the tested systems revealed the complexity of the problem concerning the FCM cause-effect relationships estimation, but on the other hand indicated that ES can be fine-tuned and eventually applied with success. The difficulty arises because there are many structures (potential combinations of cause-effect relationships) that produce almost identical results with References [Bac96] Back, T. (1996). Evolutionary Algorithms in Theory and Practice, Oxford University Press, NY. [BHS91] Back, T., Hoffmeister, F. and Schwefel, H.P.( 199l), “A Survey of Evolution Strategies”, Fourth International Conference on Genetic Algorithms, pp.2-9. [BFM97] Back, T., Fogel, D.B. and Michalewicz, Z. (1997), Handbook of Evolutionary Computation. Oxford University Press and Institute of Physics, New York. [BHS97] Back, T., Hammel, U. and Schwefel, H.P. (1997), “Evolutionary Computation: Comments on the History and Current State”, rEEE Transactions on Evolutionary Computation, vol. 1, no. 1, pp.3-17. [CGW96] Craiger, J.P., Goodman, D.F., Weiss, R.J. and Butler, A.B. (1 996), “Modeling organizational behavior with fuzzy cognitive maps”, Int. Journal of Computational Intelligence and Organizations, vol. 1, pp. 120-123. zyxwvuts 370 zyxwvutsrqp zyxwvutsrq zyxwvutsrq zyxwvut zyxwvutsr [Dav91J Davis, L. (1991), Handbook of Genetic Algorithms. International Thomson Computer Press, NY. [FFA91] Fogel, D.B., Fogel, L.J. and Atmar, W. (1991), “Meta-evolutionary Programming”, 25lh Asiloniar Conference on Signal, Systems & Computers, pp.540-545. [Fog931 Fogel, D.B. (1993), “On the Philosophical Differences between Evolutionary Algorithms and Genetic Algorithms”, 2”d Annual Conference on Evolutionary Programming, pp.23-29. [Go1891 Goldberg, D.E. (1989), Genetic Algorithms in Scarch, Optimization and Machine Learning. Adn-Wesley. [GSOO] Groumpos P.P. and Stylios C.D. (2000), “Modeling supervisory control systems using fuzzy cognitive maps”, Chaos, Solitons andFractals, vol. 11, pp. 329-336. [HB90] Hoffmeister, F. and Back, T. (1990), “Genetic Algorithms and Evolution Strategies: Similarities and Differences”, PPSN I Parallel Problem Solving from Nature, pp .455-469. [Ho175] Holland, J.H. (1975), Adaptation in Natural and Artificial Systems, Univ. of Michigan Press, Ann Arbor. [KDEOl] Koulouriotis D.E., Diakoulakis I.E. and Emiris D.M. (2001), “Anamorphosis of Fuzzy Cognitive Maps for Operation in Ambiguous and Multistimulus Real-World Environments”, IO‘” IEEE International Conference on Fuzzy Systems, submitted. [KK99] Kardaras D. and Karakostas, B. (1999), “The use of fuzzy cognitive maps to simulate the information systems strategic planning process”, Information and Software Technology, vol. 41, 197-2 10. [KL98] Kim, H.S. and Lee, K.C. (1998), “Fuzzy implications of fuzzy cognitive map with emphasis on fuzzy causal relationship and fuzzy partially causal relationship”, Fuzzy Sets and Svstems, vo1.97, pp.303-3 13. [Kos86] Kosko, B. (1986), “Fuzzy Cognitive Maps”, Intern. Journal of Man-Machine Studies, ~01.24,pp.65-75. [LK97] Lee, K.C. and Kim, H.S. (1997), “A Fuzzy Cognitive Map-Based Bi-Directional Inference Mechanism: An Application to Stock Investment Analysis”, Intelligent Systems in Accounting, Finance & Mngmt, vol. 6, pp. 41-57. [Mar991 Marchant T. (1999), “Cognitive maps and fuzzy implications”, European Journal of Operational Research, vol. 114, pp. 626-637. [Mic94] Michalewicz, Z. (1994), Genetic Algorithms + Data Structures = Evolution Programs. Spring. Verlag, NY. [OH971 Obata, T. and Hagiwara, M. (1997), “Neural cognitive maps (NCMs)”, IEEE International Conference on Systems, Man, and Cybernetics, pp.3337-42. [PB96] Pelaez, E. and Bowles, J. (1996), “Using fuzzy cognitive maps as a system model for failure models and effects analysis”, Information Sciences, ~01.88,pp. 177- 199. [PIC951 Park, K.S. and Kim, S.H. (1995), “Fuzzy cognitive maps considering time relationships”, International Journal Human-Computer Studies, pp. 157-168. [Rud92] Rudolph, G. (1992), “On Correlated Mutations in Evolution Strategies”, Parallel Problem Solving from Nature 2, pp.105-114. [Sch95] Schwefel, H.P. (1999, Evolution and Optimum Seeking. John Wiley & Sons, New York. [Si195a] Silva, P.C. .(1995), “New forms of combined matrices in fuzzy cognitive maps”, IEEE International Conjerence on Neural Networks, pp. 77 1-776. [Si195b] Silva, P.C. (1995), “Fuzzy cognitive maps over possible worlds”, IEEE International Conference on Fuzzy Systems, v01.2, pp. 555-560. [Si195c] Silva, P.C. (1995), “Fuzzy cognitive maps in multiagent environments”, 4‘” Congress of the Italian Association for Artificial Intelligence, pp. 37-43. [SM88] Styblinski, M.A. and Meyer, B.D. (1988), “Fuzzy cognitive maps, signal flow graphs and qualitative circuit analysis”, 2”d IEEE International Conference on Neural Network, vol. 2, pp. 549-56. [SR95] Schwefel, H.P. and Rudolph, G. (1995), “Contemporary Evolution Strategies”, 3rd International Conference on Artijkial Life, pp.893-907. [SSK98] Schneider, M., Shnaider, E., Kandel, A. and Chew, G. (1998), “Automatic Construction of FCMs”, Fuzzy Sets andsystems, vo1.93, pp.161-172. [Tab911 Taber, W.R. (199 l), “Knowledge Processing with Fuzzy Cognitive Maps”, Expert Systems with Applications, ~01.2,pp.83-87. [TM97] Tsadiras, A.K. and Margaritis, K.G. (1997), “Cognitive mapping and certainty neuron fuzzy cognitive maps”, Information Sciences, vol. 10 1, pp. 109-130. [To1481 Tolman, E.C. (1948), “Cognitive maps in rats and men”, Psychological Review, vo1.55, pp. 189-208. [TS87] Taber, W. and Siegel, M. (1987) “Estimation of experts’ weights using fuzzy cognitive maps”, IEEE Intern. Conference on Neural Networks, v01.2, pp.3 19-326. [ZCSS] Zhang, W. and Chen, S. (1988), “A logical architecture for cognitive maps”, 2”d IEEE International Conference on Neural Networks, vol. 1, pp. 231-238. [ZC89] Zhang, W.R. and Chen, S.S. (1989), “A Generic System for Cognitive Map Development and Decision analysis”, IEEE Transactions on Systems, Man and Cybernetics, vol. 19, pp.31-9 [ZCW92] Zhang, W.R., Chen, S.S., Wang, W. and King, R.S. (1992), “A cognitive map based approach to the coordination of distributed cooperative agents”, IEEE Transactions on Systems, Man and Cybernetics, vol. 22, no.1, pp.103-114. zyxwvutsrq 371