Sonification of time dependent data

VDM@ECML/PKDD2002 Proceedings International Workshop on Visual Data Mining 19th August, 2002, Helsinki, Finland Edited by Simeon J. Simoff, Monique Noirhomme-Fraiture and Michael H. Böhlen in conjunction with 13th European Conference on Machine Learning (ECML'02) and 6th European Conference on Principles and Practice of Knowledge Discovery in Databases (PKDD'02), 19-23 August 2002, Helsinki, Finland © Copyright 2002. The copyright of these papers belongs to the paper's authors. Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage. Proceedings of the 2nd International Workshop on Visual Data Mining VDM@ECML/PKDD'2002, in conjunction with 13th European Conference on Machine Learning (ECML'02) and 6th European Conference on Principles and Practice of Knowledge Discovery in Databases (PKDD'02), 19-23 August 2002, Helsinki, Finland S. J. Simoff, M. Noirhomme-Fraiture and M. H. Böhlen (eds) Workshop Web Site: http://www-staff.it.uts.edu.au/~simeon/vdm_pkdd2002/ Foreword Visual data mining is a collection of interactive methods for knowledge discovery from data, which integrates human perceptual capabilities to spot patterns, trends, relationships and exceptions with the capabilities of the modern digital computing to characterise data structures and display data. The underlying technology builds on visual and analytical processes developed in various disciplines including data mining, information visualisation, and statistical learning from data with algorithmic extensions that handle very large, multidimensional, multivariate data sets. The growing research and development in visual data mining offers machine learning and data mining communities complementary means of analysis that can assist in uncovering patterns and trends that are likely to be missed with other non-visual methods. Consequently, the machine learning and data mining communities have recognised the significance of this area. The first edition of the workshop took place at the ECML/PKDD conference in Freiburg, following a similar workshop at the ACM KDD conference in San Francisco. The first edition of the workshop offered to the ECML/PKDD2001 participants a mixture of presentations on state-of-art methods and techniques, with controversial research issues and applications. A report about this workshop has been published in SIGKDD Explorations 3 (2), pp. 78-81. The presentations were grouped in three streams: Methodologies for Visual Data Mining; Applications of Visual Data Mining; and Support for Visual Data Mining. The workshop included also two invited presentations – one from Erik Granum, the head of the Laboratory of Computer Vision and Media Technology, Aalborg University (on the behalf of the 3DVDM group), who presented an overview of the unique interdisciplinary 3DVDM group, its current projects and research opportunities there; and one from Monique Noirhomme-Fraiture, who demonstrated 2D and 3D visualisation support for visual data mining of symbolic data. The workshop brought together a number of cross-disciplinary researchers, who were pleased with the event and there was consensus about the necessity of turning it into an annual meeting, where researchers, both from the academia and industry can exchange and compare both relatively mature and green house theories, methodologies, algorithms and frameworks in the emerging field of visual data mining. Meantime a discussion list on visual data mining (vdm@it.uts.edu.au) was setup to facilitate this exchange. This workshop has been initiated and organised in response to this interest. Being a second edition, the workshop this year is aiming to create a stimulating atmosphere for open discussions of the crossdisciplinary theoretical foundations, frameworks, interactive methods, algorithms and environments for visual data processing and analysis; novel applications and utilisation of other perceptual channels. Consequently, the papers selected for presentation at the workshop are grouped in the following sessions: Visual Data Pre-processing; Visual Environments for Data Mining and Analysis; Interactive Visual Data Mining Algorithms and Applications; and ‘Perceptual’ Data Mining. The works selected for presentation at this workshop form more cohesive body of work, which indicates that the field has made a step forward towards achieving some level of maturity. We would like to thank all those who submitted their work to the workshop. As part of the ECML/PKDD joint conference series, the workshop follows a rigid blind peer-review process. All papers were extensively reviewed by at least three referees drawn from the program committee. Special thanks go to them. Once again, we would like to thank all those, who supported this year’s efforts on all stages – from the development and submission of the workshop proposal to the preparation of the final program and proceedings. Simeon J. Simoff Monique Noirhomme-Fraiture Michael H. Böhlen Workshop co-Chairs July 2002 ii Workshop Chairs Simeon J. Simoff Monique Noirhomme-Fraiture Michael H. Böhlen Program Committee Michael Ankerst Boeing, USA James L. Alty Loughborough University, UK Katy Börner Indiana University, USA Alberto Del Bimbo Universitá degli Studi di Firenze, Italy Maria F. Costabile Universita' di Bari, Italy Alex Duffy University of Strathclyde, UK Erik Granum Aalborg University, Denmark Markus Hegland Australian National University, Australia Maolin Huang University of Technology Sydney, Australia Alfred Inselberg Multidimensional Graph Ltd, Israel Carlo Lauro University of Naples, Italy Donato Malerba Università degli Studi, Italy Carl H. Smith University of Maryland, USA Michael Schroeder City University, UK Bruce Thomas University of South Australia, Australia iii Program for VDM@ECML/PKDD2002 Workshop Monday, 19 August 2002, Helsinki, Finland 9:00 - 9:10 Opening and Welcome 9:10 - 10:30 Session 1 – Visual Data Pre-processing • 09:10 - 09:50 CLUSTERING BY ORDERING DENSITY-BASED SUBSPACES Kan Liu, Dongru Zhou, Xiaozheng Zhou • 09:50 - 10:30 CAN HIERARCHICAL CLUSTERING IMPROVE THE EFFICIENCY OF NON-LINEAR DIMENSION REDUCTION WITH SPRING EMBEDDING Michael Schroeder and George Katopodis 10:30 - 11:00 Coffee break 11:00 - 13:00 Session 2 - Visual Environments for Data Mining and Analysis • 11:00 - 11:40 A VISUAL DATA MINING ENVIRONMENT Stephen Kimani, Tiziana Catarci and Giuseppe Santucci • 11:40 - 12:20 A POST-PROCESSING ENVIRONMENT FOR BROWSING LARGE SETS OF ASSOCIATION RULES Alipio Jorge, João Poças and Paulo Azevedo • 12:20 - 13:00 VISUAL POST-ANALYSIS OF ASSOCIATION RULES Dario Bruzzese and Cristina Davino 13:00 - 14:30 Lunch 14:30 - 16:00 Session 3 – Interactive Visual Data Mining Algorithms and Applications • 14:30 - 15:00 COOPERATION BETWEEN AUTOMATIC ALGORITHMS, INTERACTIVE ALGORITHMS AND VISUALIZATION TOOLS FOR VISUAL DATA MINING François Poulet • 15:00 - 15:30 DEFINING LIKE-MINDED AGENTS WITH THE AID OF VISUALIZATION Penny Noy and Michael Schroeder • 15:30 - 16:00 VISUAL DATA MINING OF CLINICAL DATABASES: AN APPLICATION TO THE HEMODIALYTIC TREATMENT BASED ON 3D INTERACTIVE BAR CHARTS Luca Chittaro, Carlo Combi and Giampaolo Trapasso 16:00 - 16:30 Coffee break iv 16:30 - 17:30 Session 4 – ‘Perceptual’ Data Mining • 16:30 - 17:00 SONIFICATION OF TIME DEPENDENT DATA Monique Noirhomme-Fraiture, Olivier Schöller, Christophe Demoulin, Simeon J. Simoff • 17:00 - 17:30 A SURPRISE FROM THE 3DVDM GROUP Michael H. Böhlen 17:30 - 18:00 Discussion “Where Visual Data Mining is Heading” and Closure v Table of Contents Clustering By Ordering Density-Based Subspaces K. Liu, D. Zhou, X. Zhou ……….…………………………….…………. 1 Can Hierarchical Clustering Improve The Efficiency Of Non-Linear Dimension Reduction With Spring Embedding M. Schroeder, G. Katopodis ……………………………………….……. 11 A Visual Data Mining Environment S. Kimani, T. Catarci, G. Santucci ……………………………………….. 27 A Post-Processing Environment For Browsing Large Sets Of Association Rules A. Jorge, J. Poças, P. Azevedo …………………………………………. 43 Visual Post-Analysis Of Association Rules D. Bruzzese, C. Davino ……………………………………..…………. 55 Cooperation Between Automatic Algorithms, Interactive Algorithms And Visualization Tools For Visual Data Mining F. Poulet ………………………………………………………….…. 67 Defining Like-Minded Agents With The Aid Of Visualization P. Noy, M. Schroeder ………………………………………………….. 81 Visual Data Mining Of Clinical Databases: An Application To The Hemodialytic Treatment Based On 3D Interactive Bar Charts L. Chittaro, C. Combi, G. Trapasso ………………………………..…… 97 Sonification Of Time Dependent Data M. Noirhomme-Fraiture, O. Schöller, C. Demoulin, S. J. Simoff Author Index vi ……………. 113 Clustering by Ordering Density-Based Subspaces Kan LIU*, Dongru ZHOU, Xiaozheng ZHOU School of Computer, Wuhan University 430072 Wuhan, China * lk2000@public.wh.hb.cn Abstract. Finding clusters on the basis of density distribution is a traditional approach to discover clusters with arbitrary shape. Some density-based clustering algorithms such as DBSCAN, OPTICS, DENCLUE, and CLIQUE etc have been explored in recent researches. This paper presents a new approach which is based on the ordered subspace to find clusters. The key idea is to sort the subspaces according to their density, and set a new cluster for the maximal subspace of the subspace list. Since the number of the subspaces is much less than that of the data, very large databases with high-dimensional data sets can be processed with high efficiency. We also present a new method to project high-dimensional data, and then some results of clustering with visualization are demonstrated in this paper. Keywords: Clustering, Density-based, Data visualization 1 Introduction The process of grouping the physical data or abstract data based on their similarity is called clustering. Clustering is an important analysis method in data mining, which could help people to better understand and observe the natural classification or structure of the data. Clustering algorithms are used to automatically classify data items into the relative, meaningful clusters. After clustering, the items within any cluster are highly relevant and the items across different clusters are lowly relevant. The factors listed below are always being considered when evaluating a clustering algorithm. • Scalability: A good clustering algorithm can deal with large datasets including up to millions data items. • Discovery of clusters with arbitrary shape: A cluster may have an arbitrary shape. A clustering algorithm should not only apply to the regular clusters. • Minimum parameters input: It is a heavy burden for the users to input those important parameters. In the meantime, this brings more trouble in getting good quality clustering. 2 Liu et al. • Insensitive to order of input records: Inputting data in different order should not lead to different results. • High-dimensionality: A dataset may include many attributes, clustering data in high-dimensional space is highly demanded in many applications. Current clustering algorithms can be broadly classified into two categories: hierarchical and partitional. Hierarchical algorithms, such as BIRCH [7], CURE [8] and CHAMELEON [9] etc, decompose a dataset into several levels of nested partitions. They start by placing each object in its own cluster and then merge these atomic clusters into larger and larger clusters until all objects are in a single cluster. Or they reverse the process by starting with all objects in a cluster and subdividing into smaller pieces. Partitional algorithms, such as CLARA [5] and CLARANS [6], partition the objects based on a clustering criterion. The popular methods, K-means and K-medoid, use the cluster average, or the closest object to the cluster center, to represent a cluster. New clustering algorithms have been proposed in recent researches [4]. For example, DBSCAN, OPTICS and CLIQUE are based on density; STING, WAVE CLUSTER are based on grid; and COBWEB, SOM are based on model. 2 Related Work The main idea of density-based approaches is to find regions of high-density and lowdensity, with high-density regions being separated from low-density regions. These approaches can make it easy to discover arbitrary clusters. A common way is to divide the high-dimensional space into density-based grid units. Units containing relatively high densities are the cluster centers and the boundaries between clusters fall in the regions of low-density units. For example, the CLIQUE [1] algorithm processes dense units level-by-level. It first determines 1-dimensional dense units by making a pass over the data. Having determined (k-1)-dimensional dense units, the candidates for k-dimensional units are determined. A cluster is a maximal set of connected dense units in k-dimensions. This algorithm automatically finds subspaces of the highest dimensionality such that high-density clusters exist in those subspaces, but the accuracy of the clustering result may be degraded at the expense of simplicity of the method. The alternative way is to calculate parameter ‘directly density-reachable’ or ‘reachability-distance’ of the object. For example, the DBSCAN [3] aims at discovering clusters of arbitrary shape based on the formal notion of densityreachability for k-dimensional points. OPTICS [2] solves the problem of DBSCAN, which only computes a single level clustering. OPTICS produces a special order of the database with respect to its density-based clustering structure, and according to the order of the reachability-distance of each object, it can quickly reach the high-density region. OPTICS is good for both automatic and interactive cluster analysis, including finding intrinsic clustering structure, and is not limited to one global parameter setting. But it is infeasible to apply it in its current form to a database containing several million high-dimensional objects. In this paper, we propose an integrative Clustering by Ordering Density-Based Subspaces 3 clustering method which is to partition the data space based on the density and gridbased techniques and realize the visualization of the clustered result. 3 Clustering by Ordering Dense Units 3.1 Basic statement The density-based technique is adopted in our approach, in which the data space is partitioned into non-overlapping rectangular units and the density distribution will be deduced by calculating the data volume of each rectangular unit. Suppose D is a n-dimensional dataset with m items: D={X1, X2, …, Xi, …, Xm}, where Xi=(Xi1, Xi2, …, Xij, …, Xin), (i≤m), and Xijis the value of the j th attribute of Xi. If each dimension of the dataset is equally divided into t parts, then all the items in the dataset fall into k units: U={U1, U2, …, Ui, …, Uk} (k≤tn), where Ui=(Ui1, Ui2, …, Uij, …, Uin) is the vector of each equally divided attribute. Two units U1 and U2 are defined to be adjacent only when any attribute of one unit is adjacent to that of the other unit: | U1j -U2j | = 1, U1s = U2s, (j, s ≤ k, j ≠ s). We define density peaks as those units whose densities are larger than those of the adjacent units; similarly, we define density valleys as the units whose densities are lower than those of the adjacent ones. 3.2 Algorithm When high-dimensional space is divided into k equal subspaces (units), the density peaks are regarded as the clusters centers. So the key process is how to find the density peaks. In our approach, CODBU (Clustering by Ordering Density-Based Units), the units with densities greater than threshold are ranked by the density value. The change from density peaks to density valleys is expressed by hierarchical level. The density peak is positioned in the first level of the cluster, the adjacent units are in the second level, and finally, the density valley is positioned in the last level of the cluster. First, we calculate the density values of each unit, and then rank the units whose densities are greater than threshold value. The largest-density unit will be analysed first. Each unit is compared with its adjacent units (neighbours) in density, if its density is greater than that of any other adjacent unit, it is considered as a density peak and then be set as the first level, a new cluster is emerging. If its density is less than one of adjacent units, then this unit will be grouped into the cluster of the adjacent unit; if its value is less than many of adjacent units, then this unit will be grouped into the cluster of the lowest-level unit. Fig.1 describes the process of cluster, in which only 2 basic parameters are required: the number of subdivisions for each dimension and the density threshold value. These two values are entered manually according to the size of the dataset and the required accuracy of the result. 4 Liu et al. CODBU (MinDen, t) BEGIN int cluster_no = 0; int k = 0; Divide each attribute into t equal parts, initialize U; Read data x from dataset If (x ∈ Uj) Uj.density++; For all Uj if (Uj.density >= MinDen){ U.addElement(Uj); Uj.cluster_no=0; Uj.layer_no=0; k++; } Quicksort ( U ); // sort in the descending order for (j=0; j<k; j++){ for all neighbors of Uj if (neighbor.density > Uj.density) // group into the high-density unit cluster if (Uj.layer_no = 0) { Uj.cluster_no = neighbor.cluster_no; Uj.layer_no = neighbor.layer_no+1;} else // group into the low-level unit cluster if (Uj.layer_no > neighbor.layer_no+1){ Uj.cluster_no=neighbor.cluster_no; Uj.layer_no=neighbor.layer_no+1;} // form a new cluster if (Uj.layer_no=0){ cluster_no++; Uj.cluster_no=cluster_no; Uj.layer_no=1;} } END. Fig.1 CODBU: Clustering by Ordering Density-Based Units Figure 2 shows a simple 2-dimension data set. The sequence number of each unit is shown in the unit, and ‘ * ’ stands for the spread points among them. Now sort all the squares whose density values are larger than 3, the result is (sorted by density): 4 (11), 5 (9), 8 (8), 10 (8), 1 (7), 9 (7), 11 (7), 3 (6), 6 (5), 12 (5), 7 (4), 2 (3). Based on the above result, we can find 2 clusters (sorted by level): C1={4 (1), 5 (2), 1 (2), 10 (2), 3 (2), 6 (3), 11 (3)}; C2={8 (1), 9 (2), 12 (2), 7 (2), 2 (2)}. Fig.2 Two clusters based on the density values of units Clustering by Ordering Density-Based Subspaces 5 We can see that, although the density of unit 5 is larger than that of unit 8, as it is adjacent to unit 4 which has higher density, unit 5 is still grouped into the first cluster. Since the density value of unit 8 is greater than that of any adjacent unit, it forms a new cluster. The density of unit 7 is lower than those of unit 6 and 8, but the unit 8 has a lower level than unit 6, so unit 7 is grouped into the second cluster. 3.3 Algorithm analysis The dataset is only scanned once in our approach. Suppose k is the number of subspaces whose densities are greater than threshold value, the time complexity of applying quick sort is O (k*log k). By building up a search tree, the time complexity of comparing the density value of each unit with those of its adjacent units is O(nk), and n is the number of dimensions. Since the total number of units is much less than that of the data items in the dataset, the time complexity is decreased dramatically. Furthermore, the analysis of the data space based on the density order can better reflect various clusters than the traditional approach in which the data items are simply grouped together if the densities are greater than a set threshold. Therefore our approach can cluster high-dimensional data space more quickly and accurately. In addition, the quality of the clustered result will not be influenced by the shape of the high-dimensional clusters or the order of the data input, and the parameters are easily set up and modified, so all the criteria mentioned in the Section 1 have been met. 4 Visualization Clustering high-dimensional datasets is used to help users to better understand the data structure and relationships. Visualization techniques play a very important role in displaying data and clustered results, making it more clear and reliable. The visualization techniques for high-dimensional dataset [10] [11] can be divided into two types. One type, such as “parallel coordinates” [12], is to divide the 2-d plane into several parts, each part representing an attribute. The other type is to reduce the dimensions, which is implemented through giving weights to the attributes of ndimensional data according to the relative importance and then combine them linearly. We present a new method to project high-dimensional data, which uses stimulation spectrum to project high-dimensional data on a 3-d space. The natural color is the summation of energy distribution in each wavelength in the range of visible spectrum. This energy distribution is called stimulation spectrum Φ(λ). Every stimulation spectrum can be transferred to a point in RGB color space. The quantitative relationship between stimulation spectrum Φ(λ) and RGB color coordinate are listed in the following formula: 6 Liu et al. R = k * å Φ (λ ) * r (λ ) * ∆λ λ G = k * å Φ(λ ) * g (λ ) * ∆λ (4.1) λ B = k * å Φ (λ ) * b(λ ) * ∆λ λ In this formula, k is the ratio. λ refers to wavelength of visible spectrum, ranging from 400nm to 700nm. r(λ), g(λ) and b(λ) stand for the spectrum tristimulus functions of red, green and blue, and the value of r(λ), g(λ) and b(λ) at every 5nm is measured by CIE, which is already known. Fig.3 is the spectrum tristimulus functions graph of CIE 1931 standard colorimetric system. Fig. 3 Spectrum tristimulus functions graph Each data item Xi =(Xi1, Xi2, …, Xij, …, Xin) in high-dimensional space can be viewed as a stimulation spectrum, and spreads equally in the range of visible spectrum with the wavelength from 400nm to 700nm. Here Xi can be regarded as the function of λ, and the change of the attribute values corresponds to that of the spectrum tristimulus. For example, Xi (400) = Xi1, …, Xi (700) = Xin. We can work out the 3-d coordinate of data Xi in projection space according to formula (4.2), and through adjusting the value of k can make projection space not only the RGB color space, but also any 3-d space. x = k * å Xi (λ ) * r (λ ) * ∆λ λ y = k * å Xi(λ ) * g (λ ) * ∆λ (4.2) λ z = k * å Xi(λ ) * b(λ ) * ∆λ λ We can see from the above that the process of projecting the data items as stimulation spectrums can also be viewed as a kind of weight linear combination of ndimensional data through spectrum tristimulus functions r(λ), g(λ) and b(λ). Taking advantage of spectrum tristimulus functions to convert high-dimensional data can completely project the data in projection space. This is because the fundamental Clustering by Ordering Density-Based Subspaces 7 function of r(λ), g(λ) and b(λ) is to project stimulation spectrums in color space, so it can well reflect the feature of the original data. From fig.3, we can see that the n attributes can be divided into 3 parts, and b(λ) corresponds to the attributes of the former part of the data while g(λ) and r(λ) mainly correspond to the middle and the last part of the data attributes. In this way, the projection result of a data item will be described by all of the 3 coordinate values. On the other hand, because the spectrum tristimulus functions cover the equal area, the ranges of the coordinate axes in projection space are equal, and the data will not be over-concentrated around some coordinate axes. 5 Experiments A 6-dimensional dataset containing 400 points was used in our CODBU testing experiment (it is a car dataset from http://stat.cmu.edu/datasets/). The attributes in the data set are: fuel economy in miles per U.S. gallon, number of cylinders in the engine, engine displacement in cubic inches, output of the engine in horsepower, 0 to 60 mph acceleration, and vehicle weight in U.S. pounds. Each attribute was divided into 5 parts, so there were 56=15,625 units and the 400 points scattered in 53 units. The threshold was set up as 1 which means all the points were processed. 7 clusters were obtained through linking the associated units. Two visualization techniques were explored in displaying the result: parallel coordinates and the spectrum tristimulus functions projection. Fig.4 shows the result using parallel coordinates, in which different clusters are represented by different colors but the characteristics of the clusters are not obvious. Fig. 4 Visualization of the clusters using parallel coordinates Fig.5 shows the result using the spectrum tristimulus functions projection. The side length of the projection space is 200, ∆λ=5nm, and r(λ), g(λ), b(λ) are given by CIE. In fig.5 (a), the values of the densities are reflected by color: the darker the color, the higher the density. 3 clusters are found. Fig.5 (b) displays the result of clustering using our algorithm; clusters are represented by different colors. 7 clusters are obtained based on the previous 3 big clusters. This is because that there are 7 density peaks being discovered. The clusters in white are removed due to only one data point 8 Liu et al. included. Fig.5 (c) uses symbols (such as *, +, o, ^, etc.) instead of colors to display the clusters of the data. (a) (b) (c) Fig. 5 Visualization of the clusters using the spectrum tristimulus functions projection. Shades in (a) reflect the different density, and different colors in (b) represent the different clusters, while in (c) the different clusters are represented by the symbols instead of the colors. 6 Conclusions The paper presents a new clustering approach by sorting density-based units. The basic idea is to rank the units in high-dimensional data space according to the values of the density, and start from the highest density unit to search for the density peaks in order to discover clusters. The experimental results are very promising. Clusters extend from the density peaks to density valleys and this will not be affected by the Clustering by Ordering Density-Based Subspaces 9 shape of data items. Arbitrary clusters can be obtained through our approach. We also propose the method of projecting high-dimensional data on the basis of the spectrum tristimulus functions, by which the data and its distribution can be displayed in the 3dimensional space. The combination of the data mining and the visualization techniques makes it easier for users to observe and understand data, and make better use of the data to do prediction and decision. References 1. R. Agrawal, J. Gehrke, D. Gunopulos, P. Raghavan: Automatic Subspace Clustering of High dimensional Data for Data Mining Applications. Proc. ACM SIGMOD’98 Int. Conf. on Management of Data, Seattle, WA (1998) .94~105 2. M. Ankerst, M.M. Breunig, H. Kriegel, J. Sander: OPTICS: Ordering Points To Identify the Clustreing Structure. Proc. ACM SIGMOD’99 Int. Conf. on Management of Data, Philadelphia, PA (1999) 3. M. Ester, H.P. Kriegel, J. Sander, X. Xu: A density-based algorithm for discovering clusters in large spatial databases. Proc. 1996 Int. Conf. Knowledge Discovery and Data Mining (KDD’96), Portland, OR, Aug (1996) 226-231 4. J.W. Han, M. Damber: Data Mining: Concepts and Techniques. Morgan Kaufmann Publishers (2001) 5. L. Kaufman, P.J. Rousseeuw: Finding Groups in Data: An Introduction to Cluster Analysis. New York, John Wiley & Sons (1990) 6. R. Ng, J. Han: Efficient and effective clustering method for spatial data mining. Proc. Int. Conf. Very Large Data Bases (VLDB’94) (1994) 144-155 7. T. Zhang, R. Ramakrishnan, M. Livny: BIRCH: An efficient data clustering method for very large databases. Proc. ACM-SIGMOD Int. Conf. Management of Data (SIGMOD’96), Montreal, Canada, June (1996) 103-114 8. S. Guha, R. Rastogi, K. Shim: Cure: An efficient clustering algorithm for large databases. Proc. ACM-SIGMOD Int. Conf. Management of Data (SIGMOD’98), Seattle, WA, June (1998) 73-84 9. G. Karypis, E. H. Han, V. Kumar: CHAMELEON: A hierarchical clustering algorithm using dynamic modeling. COMPUTER, 32 (1999) 68-75 10. I. Herman, G. MelancËon, M.S. Marshall: Graph Visualization and Navigation in Information Visualization: A Survey. IEEE Transactions on Visualization and Computer Graphics, Vol.6, No.1, January-March (2000) .24~43 11. A. Buja, D. Cook, D. Swayne: Interactive High-Dimensional Data Visualization. Journal of Computational and Graphical Statistics, No.5 (1996) 78-99. 12. H. Siirtola: Direct Manipulation of Parallel Coordinates. Proc. of the IEEE International Conference on Information Visualization (IV2000), London (2000) 373378 
 
 
      Æ    
           
          
     
 
 

      !"#    $!$

&
& &' ( ) *!!*
 
   
 


 
            
 
     
  
            
  
 
 
      

    
      
  
  
    
 
      
 
  
 !¾ "  
   
  
            
               
    
     
   
    
 
          
   

    
!¿ "                        
 
  
   
         
    

        


 
   
  
 
 
       

 
 #     

              
 

 
         
     
     
   
        
                                 -        &'$  (   7                          ''     +**℄2    5                %  !                            0                               +**℄  )                                   
&()*$      3
 4
      (           '' )(1'*        6 &'$               . "#$%       /              "#$%                        "( 89        :1'* (($* *% 12 Schroeder and Katopodis
    ℄              !   !    "  #  $ #     %# %  #  # %!   !           %    % &        !   ' %     &  &     &    #    (   !  %  # !              %  Æ  #   # &           &!) * ! %  %  + & #   !      &                &       !     +  ,-%
'./℄ '+ ! %  %%! %                    (   0          # #      +       #  (    ! #   +         '+ !      1   #     &  Æ   #      #  * # !    !        
    & !      # &             2  &          +
    %          # %         &     %  & '    # ,( 3℄ !    # #  % & + & %    &       +      " * /$ 4    + &   # &   #   # ,  5℄      5..   ! % & + 6    #      #        7 #   & +    2   %   %    +     /5  %           4      !  %  # !   + & 5.. ! " $   /5    "%$ '+ !     %   +#  8 #           
 

 ¾ Ê  
 
"
 $ 9  "

   
     "
 $ 9 
 
 
  

 
 $
 
  
 "   $ *  + ! !                %  &        + %
!  !  #  % :
              !      %             %# ; Can Hierarchical Clustering Improve the Efficiency
 
 
 
  «
«             «     
 
 
 
  
 

     13    
        
 
             
     
            

 
 !"#$$% ℄ '
       
   
( 
 
     

)  *   
 +                          
  -
  
  
 
 
 
.   ,      
  
    -
  
  


/ 

  0        1  
 
 
   0 
        
  
 

 

 
 
  
   
        

   


    

   )


   )  /    



 (    


) 
 (



(   
  2   
   
 
  
  
 !3  "4  544 6"57%  8%%℄ 9 

   /  
 

)   0  
      
 
 
  /     

      

 
    ) 0

 
  0   0       
  (  

) ) 
     
   0 :   
        
      
   ( ;
  /  
    )   0      0          #  
     
    

 
    
  *
      :'<37= * ( 
  
    
 
 
 + !#7, ℄ 
.  
) * !3  "4  8%%℄++ 
 
  
     
   )    
>

  *  !3  "4  544 8%%℄+ 1       
 
  
*¾ +  
     

 -   
 
.     
   *¾ +  
 !2%,℄ =
         
   

 ? 
    

 )  
 )  
   $   
   
  
   (
  
(
 
   
  
;   
 
  
 #
  
 !5@ A$4% 55 % % ==-%@ 6%  %% $"%%℄   )
    
 *
 @+ = 
  -.B     
  
       
 
       #
  
 
    
    )      ?  2 14 Schroeder Schroederand and Katopodis
     
 
 
℄     
 A A C B C B
  

 
  
 !
" 
  #
 $   "
 
 "       
 Can Hierarchical Clustering Improve the Efficiency
  


 

   
               * 
  
        
     
 
        

  

        
   
& ' (   
   
   
  
   
     
   
        
  


         
      
  
     
 
    )    
  
  
       
     
 
  


 ! "#  "$℄ 

   
 
   




   
      

   
   

    
 
     +   
   
 *       
  
  2           
  
         
  
     
      


  
   
 *  
   6-7 !04/℄ 


 
  
 
  
  
 



       
  
   
      

   
      
   

 
   

    

 
  
 
   +  
          
 
  99 
   
  
 8  23
   

  
  
        

     
  
 

   

 
  6-7 
  )  
   

   
  
  1 
      
     
  
      

      1
)   
      
         * 
             !99.4℄   
1
   

  )     
      !4/℄       99 
  
     23 -7

 
  
     9  9   

) 

 
  
        
  +    *
 6
          
  )  
      
  
  
 

   
     
 

  
    
 
 




 -  
       
          

 
1  
       1  
  

     
       
            
1 -  
 

  
             
  

 
     
 
          
    
     
       
               

     
   
  
     !-./℄  !2345℄      
    
   23 
    
   
  , -  
         
  0

 
         3 
  

 
  
 1)

 
  
   
  
  


 )              
   
 
     
 
      
  
    : 15 16 Schroeder and Katopodis

   
      

   
     


 
  
     

     
  

         
 
              
  
 


 
    
 
      
      
       
 

    !" #"$%&℄ (     
 
        
   
  )   *               +   * 
  
 ,
     *        
  
   
  -
   
  #"$%&℄ )  

   
  
  #%%℄ + 
    
     
    
   
 

    
   
.
 
   
   
  (  
 / /     
   
  
 

    
 
  
 
.
 
   
      
      
    
+   
 ,
      
    
    

  
-*
+   
    
     
    
      
  *
       
 
  
 
   
                0 01 2
  ,  2,  
  #2%&℄              
   
  +
    


  
 

 
         +



  
 ( +               +


     
 

   

    

 
       
 
  + 
    
     
  3  +
  
      
   
          !"  2,
  
          )    
  #%4℄
        
   2,   
 5 + +  
 
 
   !"     
         
    + 
 

    
   &0 
  #"$%&℄ ,
      +  
   
      6 
    +      - 
   6
                   " - 
    #(&%
71 ! 87 88 "98% : +%%℄  

       
     
   
+   +     

;
 :  
    
   
     < =
½     
> ¾ Ê
 
  
   < = > ¾ Ê      =
 
 > <     !     => +  
 
   9 
  

 

   

 
 
+    
  
    
    
 +
+ 
  

   ?
      +  .   !     $
     +
   3 9  A C D $       2    
   
  3      !   =
  
  
    @> 9 

       3 9 $   * +  
 3 9  $ 
2      B      
    (      
   *  
       & Can Hierarchical Clustering Improve the Efficiency
        
    
  
 
     
 
    ℄!
"
#$ #


 
      
  
 # 
   ! %  $       
&

 
! '  &
  
   &
 ! (


   #
 )(*+, --℄
     
  
$ 
  
 

   &
  
!  
./    
  
" 
   

#  $   

#       
0
    $   
   ! '
  
1  2
    
  
"  
     
  
   3''44℄$ 

 
    

#    & 

  
    #   
 
  $ 
 
 # 
&

 !  $  

  
   $     
 # 
 

 #! +             
 
  
 5! 

! ! + #
  
  
$ 

     
 
 
 $ .   
 
 
    
 ! !    




$
!!
6 )½       ,


    
    ! 5! 7              
    
   8 6  9  $              # 8   ¼ :!  ;  
 )   , 6
 )   , )   ,
6
 )   , 6 5  )  , ¼ ¼     6   9  8  $
!!

   # <   =$ 
 
 ! > 17 18 Schroeder and Katopodis
  
  
 
£ ¾  


     
     
      
      
     

  
    ¿ 
 Æ           
 
            
       
   
     
   
       
  !      
   
  
 
 "   
           
    
      #   
 $  
   
     
        !      ¯ % &           
   ¯ ' &        ( ¯ )   &             * 

     


    
       
     ¯ )   + 
      
   
          ) 
     , "    
   

 -    
  
                 
          
           
         ) 
 
    .
           
   
    
    
   
     

    
 -    
  $ 
        
    #
   
      
 
    /  
       

 
        
    
 -           0  
   
  1         2 ¾  &       3 ¯ 4   /          
 
      ( 
        
   

 


      
      
 *        


    2  ¯ 5)& &     5   )   & 5)& 67..℄   $     
 * 

      
          

   $    "       
          
       9

  
  5$   
 * 
 $    
      
    
       5  

        
  
  $  
    
 ¾  9 Can Hierarchical Clustering Improve the Efficiency
       
  
     

        
     

 
 
 
  
    
 
   

 
 
 
 
    
 
   
    


          
 ( #     
      


   

   #  

  ! &'  

  " #
  #  
 $
  %#  &' &' &¾ ' &¾ ' &¿ '   ) 
  ,  * +  
    19 20 Schroeder and Katopodis

 


 

 
 
  



 
 ¯ 
      

   
 
      

  
   



       

  

  
 

     

    
 ¯ 
   !"# 
    
#   



       
  
   
$ 
 # 


 
    
 
# 
 

       
 
       
  
   
#   


   
  %

  

 
 
  ¯ 
   !"#     #    
       
 


   
 
 

 
     


   

   
 
 

 
#

  
    



  
 
$  
  

   
   & 
   
$    
 
  
 '  
  
# 
 
     



   
(
   
 " ) 

#   # *& '#    
  # 
  
    #    
 
   

 
 
 
      
    
  
  


 #      
 
      +  # 
      
,
 #  

 

#     - 


      

 .  

# 

  
 


      
  
 


/        
    
 0 
1  %  
 & 
 !"# 
 2

   
 
  #      
  
/
 '  
  
#   
 
      
    

 
     3 2 # 
   
  
 
 



 *& ' # 


/
  
 
   
 

#     
# 

     
 0 
  

    
  
#  
  
   
 
   
  

    
 & 
 !"#  &

 
   
           4 


 
  
 '  
   # 
   
   
#  
 
$   
  

#   
 

#    
    
     
 
       '
$ 
  -#

 
  # 


 
     
 



 
  ( + 
 

$

# 

  /$
 



     %     
 
 & 
 !! +
   /$
 


#  
/
 



 


 

 
 
 
/
    



 
  56#
    7
  869# 

  
 



   /$
#   


   !!# # 
   
  

  
!" Can Hierarchical Clustering Improve the Efficiency Stress per iteration 6000 Stress Circle Spanning tree Zero 5000 Random Clustering PCoA 4000 3000 2000 1000 0 0 5 10 15 20 25 30 Iteration
    
  



   
  

 
     

  Stress per iteration Stress per iteration 2600 400 Spanning tree Zero 2400 Random Clustering PCoA 2200 350 300 Spanning tree Zero Random Clustering Stress Stress 250 2000 1800 200 150 1600 100 1400 50 1200 0 0 2 4 6 8 10 12 14 16 1 2 3 4 5 6 7 8 9 10 Iteration Iteration
    
  



      
       
 

  !
"  
    "
  "
   

   21 22 Schroeder and Katopodis
                             
      
  Stress per iteration 16000 14000 40 2% 4% 6% 8% 35 Oscilation threshold 18000 Threshold by Number of Nodes Stress 12000 10000 8000 6000 30 25 20 15 4000 10 2000 5 0 0 5 101520253035404550 0 Iteration 50 100 150 200 250 300 Number of Nodes 
          
              
      !"      #  $
        %     &        '      (  
     )
       
   
  * +   
  ,- 
    
               
 
     .
                     
   (          /   -    +      *%  0%    
     ( 
  /   /     /           /+  +     12
3 4567 22!7 #7 88973 #%7
77 5: 77℄    -  
 /     1<! :6! 2
66 =2567 >77℄   /          
 ?@8 1<+℄     )-   /  +  
 +  -    
 +       5
  ?@8          
 
  
       
   
      
       
   (  +  +     A       
 /      +     
 +   -   
     
      
    +         +   /    
   
     >
 '           
 B
      '   >  +  +   
  * Can Hierarchical Clustering Improve the Efficiency Stress per iteration 13000 12000 fixed temp temp as % 11000 10000 Stress 9000 8000 7000 6000 5000 4000 3000 2000 5 101520253035404550 Iteration
 
        
  
 
     ¯  


         
 
    
   
 ¯ 
        ¯   
        ¯ 
 
 
      


   
       
  
  
  ¯    

 
  


           
 ! 


    "  
              
        
  " 
#
      
# 

   
        
  !  
   

  
 $     %   
  " 
               
 
      $% &%    

   

   
  '(    
 
      )*&℄ "  #      
   
 

 
 #  
      

   
,         "       
 
    
 - 
  .   (  )-.(/℄    "
 "  
 
 
 0" )0/℄  

     "

  

   1   "
 
,  1  - 
     0" "  "
 
     " " "   "
    0"2      
     
 
 
 "
 
 
  0
 
 

 
 
 
   3 
  " 
, 
   
4         
,   0"
   
   
      3  " " 
       3
 $ 23 24 Schroeder and Katopodis
                                                



 ℄   ! "    #              $
  
 
    %&&'(% )   * +,--℄ . + , ,  / -   $ - 
 

 
     
  0   * 1,2#&℄ 3 1   ) ,  2     340 *&     . #        $+5℄ 6  / 7 / $    0 8 +   ,
          
          95&:;&(';&; *5 ,&℄ 0 ,      9:*'*;( *& ,)++&℄ # + ,  0 - )   0 8 +     1 +    3       <
     =>9=?:*&;%' *&;& *& , 5&℄ + ) ,  ,   +  #3;℄ 6+     
    /*℄ -  .  1 . &*℄     
  

    

        ! *5&  #    3/   
   < <       @       $
 
     5;'&5 )   *;      , /     

     "   .     <  ! 
  9*:**9'**; **   3    *&* . ;;℄ 6 3 .   )               <         
#  =%:%9='%& *;; 22&℄ - 2   ) 2       $ 
 
   %
 *:5'*= *& 2 ;℄ 6 2 ! #  <       2 55℄ 6 2 ! -    $  #2+5℄ 2 7 #  6 - 2    0  *5 #/3(9℄  #   9((9      
#        @   9:*'95 *;     A           0  *55 "    
     6 # + &
        . /    #  3  3             4   .  * Can Hierarchical Clustering Improve the Efficiency
℄ "#$℄ 
   % & *    " ' ( '   ) * ' #  5 6 )     '  '   ,801%4℄ (  ,   )** / 2  *  3  ' - )   **  '  /0' 8+     *  :* 4!  . ' 3  
 5 9 0 1' ' 3 0   +    $$!  $  

   
   ,  *  * 
     74!74 . % 3

  

       +  ' & /0'  ' / 1   ,.℄      !  
   ),-.. &/.℄       
      ''   

  
  !$ 44 ,(℄ & & ,  '   3 / (   * )    '  0-    

   ! "   74!7   ;7℄ / ;  ;   ) *    ** *    ' 2 '  '  *    (  <0 3 ,     * , -   7 ;℄ ;.4℄ ;. ℄ 64℄ / ;  6 ; ;  0: 

    4 6 ; ;       50   0  * 3  0 62 = 5 #%& * 7 *     * !7  = ,*     '  (  
 47! 4 .4 1 2  ' 2   *  '  (  


 $7 !$. . ) (0 ) ( 0  7 6+℄ 5     
 #
$ )' 2 6++   '    -/  ' 8 * / 2- 
 #
$ ) &   ' ' # *+++ (, * $! ,*   #  44 

 
    )  '   25 
   
        
    
 
    
    
  
               

 
 





  

 !   "  " #
$ %      
    ! " 
%& '
  " 
( 
#
$ %    "
  
   $ # " 
(  " 
    &   $ "

% "  
   (  

( "  $ #& (  

!  %% " (  ( ")" (

% ( 

   ! "! % 

% 
* %"
(


%   & +"  % 
%  
  "      "  (  

 
( 
% " ( " (%" ( " 
    & +" $ #   
"        ,   
%  %

%   -

%   $" (  


 "  (  " ( 
" 
    

% #
$ %&
   
  


            
 

       
   

 
 
 

  
 
 
    
 !    "
!   
  


       
 
 

 
   
 
       
 
#
    Æ 

 $

 
 
         % 
      

 

 
     &'℄     
  

 
       
 

 
 

     )

 

     )

   
 *
   

 

   
 
    
 * 
 

  


    
    
 
      + 
   ,
     
 
  # 

  
   

     
 
  
   
     

  

  
      
  $ 
   
    
   
       +" $ #  (  ! " - 

  
  ,  ./

       
 
 
 ! 28
 Kimani et  al.
 
      
                             
                      
        
   
  

             
                     
 
                               
       !       
           "        
               #   $    %    &                      
    ' 
 
       ( )

     
       *
  +      
           %      
       
 
           
    ,   -.℄   
  +  
  0 1+ 02 3
  + 0   !4 455. ,      
         
 
  6
  
 7 
    
 6
          
     
   
  
  
         )8   -!℄                                      "                
    )8              
 9       6
 "     %     :;<     -5℄  +9 "  = ,             

         %     
                  
    
 6
    &    
  +9      
      
 9 +        -*℄            >   +     
   
  &    +
        
           &   9        
                    
  >>  -44℄       
    8             
   &           ?>  -4@℄ A         
 
  >>     
 
       A Visual Data Mining Environment 29

 
      
 
  
   
 
 
 ℄ 
      

! #
!    

!      ! 

  
 
     
   # %  
  '      /%$#  )      #
   

! (



          
     
 /%$01  
%  "   
!   
&℄ 
    
,
(
 

 
$ %   
+++ 
  - ++ # .!  
   
'  
*   ,%# )      
 

!  %  
 
 
               '  
   
    (

       ,%#  $  

!  
     
     
!  


)2 "     !
!   3         
    )
!    
4
  ß  
 
  3    1  Æ  .   
 
 



 
 ß 

! 
 

 8 9   . 
.
 

)  $ 
  

 

$

 $ 

 !   
  !   )  
! 
   
 


  )  5 6  7 
   7 
  

!    
 
 
 : 3   
  )2    2    2          
 
 % 

!  )2    

! 
 

        )  
     

!
  #  
   
 .        

!    
    
!
  
  .      3
     
   
  

* !  ( 
2
  (      4  (
<(     
       
   )       

 !
;  
  <(  ) 


!  

! 7 3
1    
$ )  
  
   ! 
     
 )  
 8
 2
) ! 9 
       !  

 <(     

 
       
     4           .       
!    
       <( 

  !   

!       
    2
) ! 

  
 
!    

 ) 

  1
  
 *    1      

!  

1  
      1 30
 Kimani et  al. Parametric User Interface GUI Container Metaquery user interaction component Association Rule user interaction component Clustering user interaction component Abstract Data Mining Engine Metaquery Engine Association Rule Engine Metadata Clustering Engine Target Dataset
     

        
       
     

    
       
     
    

          
   
       

      
   
 
  
  
       
  
   

      ß 
       ! 
 
 ß  
    
     
  

 
   
   " 
 
 # ß $     ß    
    
   $       %   

       ß &     
% 
 '(  
 ß &       

   

 
)   "
 
% % 

 
  #  
 
    
 



        
      ! 
  

 

 



(     "
    
# (    
   
   !  
    
     

    
 "# *    % 
  !   

    
 (
  
  (    
    
 )  
       A Visual Data Mining Environment 31
       
                                     
                        ß !                          "        # $        %
ß !                        
&                                       
         '        $         (  " %
)                (                              
ß !                  
   '                  
          *         (   
ß !      (      (      
+'               (        *         (    
                   ,   *    ( -.        #
                  , (     ( -.       -.
/ #          '          '
     -.           '    
!  #                 ( ,     

                        ( -.
)            (          ½            
!                      (    "    
)  ( "     (          (    
               01℄
                  34 
   5      
         ½                                       
 Kimani et  al. 32
    
  

 

   

 
 
    

      
   
 
  
 
      
     
      

   
   
    
     
 
           $  
    
  

 


  "     !  
 %


     !   
   

   #  " 
   

    
             
 
       
  
 
 
  &
   
"      
         " 
" #     "   
       
      
 
 

  


 %
     
  
          %     (  "    " 
           
&   "     
  
        
    
& 
   
" 
!          & 
   
 
 
   
  
 "  
        
 #
  
  
          )

  
   #    
     
 
     
"   
    
  
  "    
'&  %
               %
        
 
    
Æ  
 #    %
 
     "  
  "   
" 
Æ  

            
  
         

  
  
        
%&  
 
 

 ( 
      
 "  
         " 
 "
    
           
          
  
 &   +  ,-    
 
       "                 
      
         
 
    

 
  

&
&   
 " "  " 
       % 
 

  
 
  *     % 
       
"           
  
      
 
    
  %       
  

  .
   "     "  
      
"  

 
       A Visual Data Mining Environment 33
            
 
  
          
      
  
       
  
     
  
 


            
     Æ  
  
 

    

   !    " CLUSTERING OUTPUT CLUSTERS All/Any Customer Information 6 Some 2 String Name of Relation DETAILS Number of Attributes Attribute Type 3 The customer is located 1.73 miles North East of the Main Office Numeric Any Relation Alphanumeric 4 Logical 1 Date/Time Service Customer      Specification Space Database            
   
      CLUSTERING INPUT Relation Type    
 


                   
     
 
          5 X Cluster quality Target Space Save/Export Only Export + Switch To X Export Facility Standard Tools CustID CustX CustY CustAmt SEPARATION OF THE CLUSTER FROM THE OTHERS Level of Similarity Dissimilarity Separation ServID 10% 10% Number of Clusters Attribute for Labels 2
    ß ß ß ß #  %
   &     
  $ 
   
 ( 
     &
 
     
              )
                     
   
        '
 (        
         
           )     
       5    
       & '
  4 3 Cluster quality Cluster density   $       

 
  
    
 34
 Kimani et  al.
 



        
  
      
      
          
 
  
    
        
       
           
  !
  
    
       
  
    
 
                    
         "  
            #   
  
      $    %        
    

       &  '      (   
           
 
   
        ) 
  
 "  
   
               *

   
 
      
 +  
     
    ) 
 
           
          "         
     
   
       +    
    '   ( 
   
   ,  
       


         
   
 


 
   
    
    
 & &      
  
  &       
  &        ) 
   
 "  
   
         
         
          
    
    ) 
 
           
       
    
      
    
     
 
     
       "  
 $     
 &           
         
 

           
    
           .          '              ( 
                   
       
  
   
    ' 
     
     (      
 
      
   
   ,  - 
 
   
     
   "    
 
     
           
  &            
       
 & &  
    
        
    
   
          
 /    
       
       A Visual Data Mining Environment 35
 
  
           
   
       Æ          

    
  
  
            
 
 
       Æ   
           
   
   
  
   
             
   
     
 

               
          
    
 
 ! 
  
   
         
 
          

    
  " 

  
       
    #  
$      
  # 
 $  
    
     
 
 
  
         
   
 
 
  
     
 
 
         
     %          
        
                 "  
           
       
   
 &  "     
 

 
 '

    
         
   
                  
 
 
     
        
  
  
         
   
  
    
   
 
      

       & 
            
 
 
    (    

      
  )
 
 
   

 
           

  
 
 
     

     
  
   
  
 
   
   "         
    
 
             * 
 + 
   
   #   
 

$   
    
   "  
  
    ,*%  +¾   # 
  $ 
          #     $ (    
 
 
   
        
    )
 
  

                                           ¾ 36
 Kimani et al. RULE VISUALIZATION DESIGN OF METAQUERIES Specification Space Relation Type SCATTER PLOT OF METARULES ServID All/Any 2 Some Support (%) 70 Numeric ServCent Alphanumeric Logical 60 50 40 30 20 A rule with the highest confidence 10 Date/Time ServCent CustAge CustGender CustID CentID 80 String Name of Relation VISUALIZATION OF TUPLES Number of Attributes Attribute Type Database 1 2 3 4 Number of items ServCent Target Space Male Female Confidence Save/Export Only Age Export + Switch To CustServ ServCent CustID Standard Tools Export Facility ServID ServID CentID DEDICATED VIEW OF METARULES ClustOut1 CustCent IF THEN CustID CentID Manual Joins Automatic Joins Add Pattern ServCent(ServID,CentID) CustServ(CustID,ServID) IF CustServ(ServID,CustID),CustCent(CustID,CentID) THEN ServCent(ServID,CentID) IF CustCent(CustID,CentID),ServCent(CentID,ServID) THEN CustServ(CustID,ServID) IF CustServ(CustID,ServID),ServCent(ServID,CentID) THEN CustCent(CustID,CentID) CustCent(CustID,CentID) 100 50 0 Sort Facility Minimum level of Confidence Support Rules Standard visualization tools 10% 10% Support Confidence
    
  
       

     
     
 
         
  
  
         
  
 
  
 
   
     
  
   
         
     
         !   
   
    "      
  
 #
  ! " 
     
 $        
 
 % %  &   
           
$                  
      
  
$    
'
   
 " (  )

   
 
 
 
* 
        #         


  
     
  
   
 
         
   
 



   "   
  
 
 
* 
      
  
 + 
 , (
      ß ß
  
      "

 
* 
 
           
   
       A Visual Data Mining Environment 37
    

      
 

      
   
     
        

 
    
     
        
                   
 
         
  
   
 
    !    
   
   ! ! "  
    #
 
       ! 

    
$ % + (
 
 
  
    

   & & 
 &   
 ' (((()  "    ' %**)  
   
    
 ' ,,,,)  "    ' -.)        

 

 ' .../)  "    ' /*)  0 1 ( 
 
  
  2     
 
 0   2   
 
 
    
       
!             
 
   
 

    
       
   
      
      !    
   
 
        
 
               
     
 
#    
 
   " # 
 
 
    
  
  
    3
 4
 
     
    
 

   
  
$         Æ   
 
     5  
   2          
  2  
 
 
 
  6 
   

  2 
  
 
       $      
     
 
            
        7       
   2      + 8
 
     8
 
    9
 

      8
 
           
     


       
 9       7
     
         

        
               
    
               

  
 :     
   
  

     38
 Kimani et al.
 
 
  
      
          
       
  
     
                  
     
           
   
         
      
                                
   
     
        
   
         !  
     
  
         
   
 
      

       
    "
    
    
     
      #$ %   
      
 DESIGN OF ASSOCIATION RULES Specification Space Relation Type Number of Attributes Attribute Type All/Any Database 2 Some String Name of Relation Numeric Service Alphanumeric Logical Date/Time Service Service Target Space Service ServID Prod_ID S001 S002 S003 S004 S005 ServName Product WWW-Access Printing Scanning Faxing Telephone Manual Automatic Define Joins Skip Joins IF Service = "WWW-Access" Minimum level of Confidence Support
  THEN Service = X 10% 10%                      
      !        
 
      
        
  
 
     
      
 "     

  & 
 
                    
        
       #      
     
       " 
           
       
  
       A Visual Data Mining Environment 39
              
  
         
             
       
           
   ! 


   
  

      
   
    
       
 
 

      
     
  
        
 
 

 
   
         
            
 
       
                 


    
  
 

  
    
            
 "

  
 
 

    
       
 
   
   $ 
    
  

 

      
   
 

    
 # 
 

  #     
 


      
%    
 
    "   
        &

 '(℄ *    
 
 
   + 
 
 ,      +      -&  
 
  . &   $ 

  #   
 
 

  

$   "  
 
 +.      
  1&2  #
  #  
&3   * &3  

 -

    
 
#   /    &

 4    
  
    $ 
   
        
  



       '℄
  1&2#   
 
 Æ 
    ,  5&&2 (6  
+
 
  
# 
 
   
     

1&2 ""    
    *        

 5&&2           
        
 #  
 
        +     &00   1&2   + +   
  " 

&
  * &
 
   
  
 
   
 & 
   + 

 & + 0   
0

 -&00.  /   
 

 
 &

  &   
 /   
 
  

 
     
     
 
   
 5&&2 (6 
     
      
          
  
   
     7     #  
            40
 Kimani et al.

           

   

      
   
  


       
  
     
   

          

     
   


 
    
      
    

   
    


    !      




 
     
 
 !   


  
     
   !     " 
      


     
    
!     
   #  $ 

 ß     
  
    $!     

   
  
%
  & 
    
    ' !         
%
       

      

    
 
            



    !
        
 (    !
  "
  
  "     
 
"
 
  

    
      

  "
 

 
 

  "
   "
    
 
 
     ' !  
      
%
     ß   
 
  
%
  "    !  


      


 )  #  $   
 
! 
 


 
   (    
 
 
 ! 
 
 
  $! 

 '
   
*
 
+    
           ! ! 
 
     &       (   !
 
    
*    
+    
 

    "  
"
       ß 
   


       
 
 ,

   
   
 

  
    
      

      
  

  
      

 ß 


  !   
 !    " )     

  #  $ 



 !    "
   
 

    
"
  
 
"
  
"

  - 
    ! ß 
   ! 


     
  
   #  $      ! 

  

 
     "  
 " 


 ß    
 
        
       
  !

 
                   !   



 * 
    +    !  !   


 )  #  $    '
 

 ! ! 
*
    
  
     
 
+  
 ! 

 
       A Visual Data Mining Environment 41

  
  
          
  
   


  
    

          
  

 
 
      !℄  #   $℄ %   

 
 
 
  
   
  
  
 


 
 
  
 
 


 & 
 
 
  




 
     
 

 '   
 (      
 
 
  
  

  
 
 
 )
 
     
  

 
  

#  

      
  * 
   
 
     
  


   
  

 -
 

   



 
 * 
  
 
  ,
 


  4  
 '   
    
 
     

     
 ( 

  +  

 
  
 
  
       
    
 
  
  

  

    
  
    
   ( 

   '  ,
 
 ' . 
 
 /)01 
  2℄  3℄   
%    ' ,
   

 
   )

  

   
  

 
 /
  
 + 
  

 
     5
  
            
        



 
  +       

 '

 ,
          ) /
  
      


   

       
  
 
  
                
!       " #       "  %&' ())( ( *  + *  % * 
 "  ,    - , %. - %  ,    , *
    ,  '  + '   /. 0/( 1 (     '1        2    ,   / %3 

   
  ()) 0 '4 * '4  '    - 4 5   /  + ,'  ,  6   1 6& %   " / '   ..  *    + '   /. *78)9 :     778 9 +'    . % - () 1 %    4. -  

  

    , 
 42

    / 3 4 5
  Kimani et al.    

  

    

   
  

      
 ! " # ! $! %!  &' (   ) *  + ,-   - & .*   -  $  !   *+  -       ,-        ! / ' " 0 1  

   2*,    +'-

 
  
   


   


   
 ! 6 !  !  ! ' !  # !  1!  #  #*  - " 72    " + *8 7, !   *+  -    ,  "' ,,    9(6 * 2)  222  !   ! 6 !  !   6- ,  ,  - " 72  !   *+  - /  :"       +  "     **!  ! -+!  (  !  )! ; ! , $!  6-  * "+ ;'  * -    < "+ ;' ½ = > ? / ✁✄✂✆☎✆✝✞☎✆✟✡✠☛✟✞☞✍✌✏✎✒✑✔✓✔✠✕✟✔✖✗✟✙✘✚✟✔✛✗✜✗✢☛✣✍✓✞✘✚✜✥✤✏✂✆✟✡✁✧✦✗✎✥★✏✎✗✩✔✟✫✪ ✬✫✭✯✮✱✰✳✲✴✲✴✵✷✶✞✸✺✹✴✻✽✼✳✾✫✿✷❀✥❁✕❂✽❃❄✿✷❅✫❆✕❅✫❁❇❅✫❈❉✹❊❈✫❂✱❋✷❈✒✻✽✹❊❈✥❂✱❋✷●✫❍✗❆☛■ ❆✕❏▲❑▼❈✫❂✱❍✗❁❖◆P■◗✿✷❆✕❘☛❘❖❙ ●✫❚✯❙ ❚✫❋ ❯ ✮✱✾✯❃◗❋◗✿✷❋✷●✫❋✷❈❉❱❲❆☛❘✕✿✷❈✫✾✫❆✕■❖❅✫❁❇✸❳❃❄❋✷❆☛❋✷❨▼❃❄❋✷✿◗❘✕❆❖✻❩✹❊❈✫❂✱❋✷●✥❍✥❆✕■ ❑▼❈✥❆☛❈✯❙ ❚✫❈✫❘✕❆❖❃❬◆P✿◗✾✥❁❖❙ ❚✥❋ ❭ ✼❲✾✫✿✷❀✗❁☛❂✽❃❄✿✷❅✫❆✕❅✫❁❇❅✫❈❉❪❫✿✷✾✥❴✫❈✯✻❩✹❊❈✥❂✱❋◗●✥❍✥❆✕■ ❚❩❑✍❆❵◆✡❅✫✿▼❙ ●✥❏❛✿✷✾✥❴✫❈✯❙ ❚✫❋ ❜✳❝✞❞✍❡✱❢☛❣✗❤☛❡✱✐ ✰❥❃✍❃❄❈✫❘✕✿◗❆✕❋✷✿◗❈✥✾❦❂✱●✫■✷❁❧❁✕✾✫❍✗✿✷✾✫❁❖❃✳❋✷♠♥❚✫✿✷❘✕❆☛■✷■✷♠❫❈✥●✫❋✷❚✫●✥❋❳❆❧❀✗❁☛❂✱♠♦■✷❆☛❂✱❍✗❁♣❃◗❁☛❋▲❈✫q❲❂✱●✥■✷❁❖❃❬❙ r ❖❁ ❃❄❚✥✿◗❋✷❁❧❋✷❴✥❁❧qs❆☛❘✕❋❳❋✷❴✥❆☛❋❇❆❖❃❬❃❄❈✥❘☛✿✷❆☛❋✷✿✷❈✫✾❫❂✱●✫■✷❁❖❃❇❆☛❂✱❁✙❂✱❁✕❍✥❆✕❂✱❅✫❁✕❅t❆❖❃❇❴✫✿✷❍✥❴✥■✷♠✉❘✕❈✫❏❉❚✫❂✱❁✕❴✫❁☛✾✒❃❄✿✷✈✫■✷❁ ❆✕✾✫❅❥●✒❃❄❁☛qs●✥■❊qs❈✥❂✇❅✥❆☛❋✷❆①❏❉✿◗✾✥✿✷✾✫❍✧❆✕✾✫❅✧❅✥❁☛❘☛✿▼❃❄✿✷❈✥✾♦❃❄●✫❚✥❚✫❈✥❂✱❋✔✿✷✾✧qs✿✷❁✕■◗❅✒❃✴❃❄●✫❘✕❴✧❆❖❃❊❏❉❆☛❂✱②✗❁☛❋✷✿✷✾✫❍❊✻☛❂✱❁☛③ ❋✷❆✕✿◗■▼✻✯❏❛❁✕❅✫✿✷❘☛✿✷✾✥❁❖✻✯❅✫❁☛❏❉❈✫❍✗❂✱❆☛❚✥❴✫✿✷❘❖❃✍✻✺❆✕❏❛❈✥✾✫❍❦❈✥❋◗❴✥❁☛❂✽❃❬✻✒■✷❁☛✾✥❍✥❋✷❴✥♠t❈✫●✥❋✷❚✫●✫❋▼❃❲❏❉❆☛♠t❅✥✿▼❃❄❘☛❈✥●✫❂✱❆☛❍✗❁ ●✒❃❄❁☛❂✽❃❉qs❂✱❈✫❏④●✯❃❄✿✷✾✥❍⑤❋✷❴✥❁❫❋✷❁☛❘✕❴✫✾✥✿◗⑥✥●✫❁❵❙✚✮✱✾⑤❋◗❴✥✿▼❃❉❚✫❆✕❚✫❁✕❂✧⑦✳❁✉❚✫❂✱❈✥❚✫❈✯❃◗❁❫❆✉❚✫❈✒❃❄❋✷③✱❚✫❂✱❈✫❘✕❁❖❃❬❃◗✿◗✾✥❍ ❏❉❁☛❋✷❴✥❈✫❅✫❈✥■✷❈✫❍✗♠P❆✕✾✫❅✡❋◗❈✥❈✫■✺qs❈✥❂✴✈✫❂✱❈✫⑦✧❃❄✿✷✾✫❍✗✵✺❀✥✿▼❃❄●✥❆☛■✷✿✷⑧☛✿✷✾✫❍P■✷❆☛❂✱❍✗❁⑨❃❄❁✕❋✍❃✚❈✫q⑩❆❵❃❬❃❄❈✫❘✕✿✷❆☛❋✷✿✷❈✫✾✙❂✱●✥■✷❁❖❃❬❙ ❶ ❴✥❁✄❏❉❁☛❋✷❴✫❈✥❅✡✿▼❃✺✈✫❆❖❃◗❁☛❅P❈✫✾P❆❧❃❄❁✕❋✯❈✫q✇❈✫❚✫❁✕❂✱❆☛❋✷❈✫❂✽❃✺❋✷❴✫❆☛❋✯❋✷❂✱❆✕✾✯❃❄qs❈✥❂✱❏❷❃❄❁☛❋▼❃✚❈✥q❸❂✱●✥■◗❁❵❃✚✿✷✾✫❋✷❈t❃❄❁☛❋▼❃ ❈✥q✇❂✱●✥■✷❁❖❃❬✻❹❆☛■✷■✷❈✫⑦❳✿◗✾✥❍❥qs❈✫❘✕●✯❃❄✿✷✾✥❍❥❈✫✾❺✿✷✾✫❋✷❁☛❂✱❁❵❃❄❋✷✿◗✾✥❍❥❂✱❁☛❍✗✿✷❈✫✾✯❃✯❈✫q✚❋✷❴✫❁❻❂✱●✫■✷❁✡❃❄❚✥❆☛❘☛❁❵❙♥✸✺❆✕❘☛❴❫❃❄❁☛❋✔❈✫q ❂✱●✥■✷❁❖❃❸❘☛❆✕✾❧✈✫❁❛❋✷❴✫❁☛✾✡❅✫❁✕❚✫✿✷❘☛❋✷❁☛❅✡⑦✳✿✷❋✷❴✙❅✥✿◗q✆qs❁☛❂✱❁✕✾✫❋✺❍✥❂✱❆✕❚✫❴✥✿◗❘✕❆☛■✺❂✱❁☛❚✥❂✱❁❖❃❄❁✕✾✫❋✷❆☛❋✷✿✷❈✫✾✒❃❬❙ ❶ ❴✥❁✄❋✷❈✫❈✥■✚✿▼❃ ⑦❳❁☛✈✥③✱✈✫❆❖❃◗❁☛❅❉❆☛✾✫❅❉●✯❃◗❁❖❃❸❼❹❽①❾❥❙ ❶ ❴✫❁❇✿✷✾✫❚✫●✥❋✚❃❄❁☛❋❖❈✥q❊❆❵❃❬❃❄❈✫❘✕✿✷❆☛❋✷✿✷❈✫✾❉❂✱●✫■✷❁❖❃✏✿✍❃✏❍✥✿✷❀✗❁☛✾❉✿✷✾❛✹✔❪❫❪✉✭❇❙ ❿✧➀☛➁✥➂❉➃ ❢☛➄✏❞❹➅ r ❆☛❋✷❆❳❏❉✿✷✾✫✿✷✾✫❍❊✻✽❆❵❃❬❃❄❈✥❘☛✿✷❆☛❋✷✿✷❈✫✾❉❂✱●✫■✷❁❖❃✍✻✽❚✫❈✒❃❄❋❖❚✫❂✱❈✥❘☛❁❵❃❬❃❄✿✷✾✫❍❊✻✽❅✥❁☛❘☛✿▼❃❄✿✷❈✥✾❧❃❄●✥❚✫❚✥❈✫❂✱❋▼✻✽❀✗✿▼❃❄●✫❆✕■◗✿✷⑧✕❆☛❋✷✿✷❈✫✾✯❙ ✁✧✢✕✢✽✟✞➆✥☎❄✜✥➇❄☎✆✟✞➈✉➉①✤✏✂❄✎✡➊▼✁✧➉①➋✳✩✞☎✆✢✽➆✥✟✞★✏✎✥☞▼➌⑨➊✍✁✧✌✏☞▼✜✥➍❛✜✥✂✇✎✥➇✇✜✥✂❬➎✫➏✔➐✞➋❳☎❄✢❻➑✙✜✗➈✞➌✙➇✆☎✆➑✡✎✥✢❻✤✏✢✽✎✥✩✞✓✯➒▼✟✞☞❇✩✔✎✗➆✥☎❄✢✕☎❄✟✔➈ ✢✕✤✏✝✞✝✔✟✞☞✍➇❄✓✧☎✆➈➓✩✞✜✗➇✆✜➔➑✙☎❄➈✏☎✆➈✏✌➓✜✗✝✔✝✞✂✆☎❄➆✥✜✗➇✆☎❄✟✔➈✏✢♣✂✆☎❄→✏✎➔➑✙✜✗☞✍→✏✎✥➇❥➣✞✜✗✢✕→✏✎✗➇❥✜✥➈✏✜✗✂✆➌❖✢✽☎✆✢✽✓✄➑✙✜✗☞✍→✏✎✗➇✆☎❄➈✞✌✏✓❛☞✍✎✗➇✆✜✗☎✆✂✆✓ ✢✕➇❄✤✞✩✞➌♦✟✔➒❳➆✥✎✗➈✏✢✕✤✏✢❛✩✔✜✗➇✆✜✗✓✇✜✗➈✏✜✥✂❄➌❖✢✕☎❄✢✄✟✞➒❳➑✡✎✥✩✞☎✆➆✗✜✗✂❸✩✞✜✥➇❄✜✥✓✺✜✗➑✙✟✞➈✞✌♦✟✞➇✆↔✏✎✗☞✍✢✫➎❖↕❇↔✏☎✆✢❉➇✆➌❖✝✞✎✡✟✞➒❇→✞➈✏✟✞➍✄✂❄✎✥✩✞✌✏✎ ✩✔☎❄✢✕➆✗✟✔★✏✎✗☞✍➌♣☎❄✢✡✜✗✩✔✎✗➙✞✤✞✜✗➇✆✎✉➍✄↔✏✎✗➈➛➇✆↔✏✎❫✩✞✜✗➇✆✜✉➑✙☎✆➈✏☎❄➈✞✌➛➇❄✜✥✢✽→♣↔✏✜✥✢❺➈✏✟➛✢✕☎❄➈✞✌✏✂❄✎✉➆✗✟✞➈✞➆✗☞✍✎✗➇✆✎♦✟✞➣♥➜➝✎✥➆✗➇✆☎❄★✞✎♦➇❄✟ ➒▼✤✏✂✆➒➞☎✆✂❛➊✍✢✽✤✏➆✥↔➟✜✗✢✉↔✏✟✞➍➠➇❄✟➟✩✔☎❄✢✕➆✗☞✍☎❄➑✙☎✆➈✏✜✗➇✆✎➛✌✏✟✔✟✞✩➟➆✥✂❄☎✆✎✗➈✏➇✆✢♦➒▼☞▼✟✔➑➡➣✞✜✥✩➟✟✞➈✞✎✗✢✽➋✍✓①➆✗✟✔➈✏➇✆☞▼✜✥☞▼☎✆✂❄➌➢➇✆✟➟➍✄↔✏✜✗➇ ↔✞✜✗✝✞✝✔✎✗➈✏✢t☎✆➈➤➆✗✂✆✜✗✢✕✢✽☎✆➒➞☎✆➆✗✜✥➇❄☎✆✟✞➈➥✟✞☞❧☞▼✎✥✌✏☞✍✎✗✢✽✢✕☎❄✟✔➈✇➎✴➦✍➈✏✢✕➇❄✎✥✜✗✩✞✓❛➇❄↔✞✎➧✤✏✢✕✎➧✟✔➒✡✁✧➉➨✜✥✂❄✂✆✟✞➍✄✢✉➇✆↔✏✎⑤✩✔✎✗➆✥☎❄✢✕☎❄✟✔➈ ➑✙✜✗→✞✎✗☞✍➩❇→✏➈✏✟✔➍❛✂✆✎✗✩✔✌✏✎♦✢✕✎✗✎✗→✞✎✗☞❉➇✆✟➛↔✏✜✥★✏✎♦➑✙✜✗➈✏➌❦✩✔☎❄➒▼➒➞✎✗☞✍✎✗➈✞➇❇★✏☎✆✎✗➍✄✢❺✟✞➈♣➇✆↔✏✎♦✩✔✜✗➇✆✜❊➎❊↕❳↔✏✎✗☞✍✎♦➑✙✜✗➌❦➣✞✎♦✜ ✢✕✎✗➇✇✟✞➒▲✌✏✎✥➈✏✎✗☞✍✜✗✂✚✌✏✟✞✜✥✂❄✢①✝✔✟✞✢✽✢✕☎❄➣✔✂❄➌✙➈✏✟✔➇✺➑✡✎✥✜✗✢✕✤✏☞▼✜✥➣✞✂✆✎❥➊▼✂✆☎✆→✏✎❥➫♥➍✄↔✏✜✗➇✺➆✥↔✏✜✗☞✍✜✗➆✥➇❄✎✥☞▼☎✆✦✗✎✥✢①✜❥✌✏✟✞✟✔✩⑨➆✗✂✆☎❄✎✥➈✏➇✆➭✥➯✥✓ ✬ ❶ ✫❴ ✿▼❃✴⑦✳❈✥❂✱②⑨✿✍❃❉❃❄●✥❚✫❚✫❈✥❂✱❋✷❁☛❅♦✈✫♠❧❋✷❴✥❁P✸✺●✥❂✱❈✫❚✫❁✕❆☛✾⑨✼❲✾✥✿✷❈✫✾⑨❍✗❂✱❆☛✾✫❋✇✮✽❼ ❶ ③✽➲❩➳✫➳✥➳✫③✽➲✫➲✥❙ ➵✫➳✒➸⑨❼♥❈✥■✷③✱✸✚●✥③✱❱❲❁☛❋✇❆☛✾✥❅⑨❋◗❴✥❁ ✹✔➺❺❼❹✮✱✵◗➻✥➼✫➼✒➲✽✵✷✲✴■✷❆❖❃❬❃✯✹❊❂✱❈❩❑✍❁✕❘☛❋▲❃◗❚✫❈✫✾✒❃❄❈✫❂✱❁✕❅❺✈✫♠✧✶✞●✫✾✥❅✫❆☛➽✕➾☛❈❥✲✴✿✷➚☛✾✥❘☛✿✷❆①❁ ❶ ❁☛❘✕✾✫❈✥■◗❈✥❍✥✿✷❆❵✻❹✶✞✸ r ✸✺➪➧❁①✹✔❂✱❈✫❍✗❂✱❆☛❏❉❆ ❅✥❁❳✶✞✿✷✾✫❆✕✾✫❘☛✿✷❆✕❏❛❁✕✾✫❋✷❈❛✹✔■✷●✫❂✱✿✷❆☛✾✥●✫❆☛■❖❅✥❁❳✼✳✾✫✿✷❅✫❆✕❅✫❁❖❃✏❅✫❁❇✮✒➶ r ❙ 44 Jorge et al. ➹✗➘❛➴✏➷✆➬✗➴➥➷❄➮✙➱✞✃✔❐▼❒✆❮✗❰✞❒✄Ï✞❐▼✃✔Ð✏➱✞Ñt✃✞Ò✙➬✗Ó✆➷❄Ô✥❰✏❒✆Ñ❫Õ✔✃➥Ö✙➴✏❮✗×✞Ô✗Ø✫Ù✗Ú❻➹♥➘✄➴✏➷✆➬✗➴➟➱✔❐▼✃✔Õ✞Ð✏➬✥❒❄Ñ❫Õ✞✃➥➘❛➴✏➷✆➬✗➴➤➬✗Ó✆➷❄Ô✥❰✏❒❄Ñ ❒✆Û❖➱✞➷✆➬✗❮✥Ó❄Ó✆Û♣Ü✞Ð✞Û❖Ø✥Ù✥ÝßÞ✒à➢✃✔❐▼Ô✥✃✞×✏Ô✥❐▼Ú▲❒✆➴✏Ô❫Õ✞Ô✗➬✥➷❄Ñ✕➷❄✃✔❰➛➮✙❮✗á✏Ô✥❐❛➮✙❮✗Û♣Ô✥×✏Ô✗❰➛Ò▼➷❄❰✞Õ⑤❐✍Ô✗Ó✆Ô✗×✏❮✥❰✏❒❳➱✔❮✗❒✆❒❄Ô✥❐▼❰✞ÑP❒✆➴✏❮✗❒ Õ✔✃♦❰✏✃✔❒✚➬✗✃✔❐▼❐✍Ô✗Ñ✕➱✞✃✔❰✏Õ⑨❒❄✃⑨❮✥❰✏Û✙â✞Ð✏Ô✥Ñ✽❒✆➷❄✃✔❰✙Ò▼✃✞❐✍➮✡Ð✞Ó❄❮✥❒❄Ô✥Õ⑨Ü✞Ô✗Ò▼✃✞❐✍Ô✗➴✏❮✥❰✏Õ❸Þ✥ã❳➴✏➷❄Ñ①Ñ✕❒❄Û❖Ó✆Ô❥✃✞Ò✴Õ✔❮✗❒✆❮❥➮✡➷✆❰✏➷✆❰✏Ï❧➷❄Ñ Ñ✕✃✞➮✙Ô✗❒✆➷❄➮✙Ô✗Ñ✳➬✗❮✥Ó❄Ó✆Ô✗Õ✙➹♥Ò▼➷❄Ñ✕➴✏➷❄❰✞Ï✏Ù❛ä✍Ò➞✃✔❐⑩á✏❰✏✃✞➘✄Ó✆Ô✗Õ✞Ï✞Ô✗Ý❹Þ å Ð✞Ô❫❒✆✃⑤❒✆➴✏Ô✉Õ✔❮✗❒✆❮✉➬✥➴✏❮✗❐✍❮✗➬✥❒❄Ô✥❐▼➷✆æ✗❮✥❒❄➷✆✃✞❰➛✃✔Ü❹ç❩Ô✗➬✥❒❄➷✆×✏Ô✥ÑP✃✞Ò❛❒❄➴✞Ô✉❮✗Ñ✕Ñ✽✃✔➬✗➷✆❮✗❒✆➷❄✃✔❰➛❐▼Ð✞Ó❄Ô❫Õ✞➷✆Ñ✽➬✗✃✔×✏Ô✗❐✍Û♣❒✆❮✗Ñ✽á✞Ú è✧éêÕ✔➷❄Ñ✕➬✗✃✞×✞Ô✗❐✍Û⑨❮✗Ó✆Ï✏✃✞❐✍➷❄❒✆➴✏➮✙Ñ❉➱✔❐▼✃✔Õ✞Ð✏➬✥ÔP❮P➬✥✃✞➮✙➱✞Ó✆Ô✗❒✆ÔPÑ✕Ô✗❒❸✃✞Ò▲❐✍Ð✏Ó✆Ô✗Ñ❻❮✥Ü✞✃✞×✞Ô❺Ð✏Ñ✽Ô✥❐▼ë▼➱✞❐✍✃✞×✏➷✆Õ✞Ô✥Õ♦❒✆➴✏❐▼Ô✥Ñ✽➴✏ë ✃✔Ó❄Õ✔ÑPä✍❒❄Û❖➱✔➷❄➬✥❮✗Ó✆Ó❄Û♣➮✙➷✆❰✏➷❄➮✙❮✗Ó✳Ñ✽Ð✞➱✞➱✞✃✔❐▼❒✳❮✗❰✞Õ⑤➮✙➷❄❰✏➷✆➮✙❮✗Ó✳➬✗✃✞❰✞Ò➞➷✆Õ✞Ô✗❰✞➬✗Ô✗Ú✴Õ✔Ô✗Ò▼➷❄❰✏Ô✥Õ➛➷✆❰♣ì✺Ô✗➬✥❒❄➷✆✃✞❰♣í✔ÝßÞ❊ã❳➴✏➷❄Ñ ➷✆➮✙➱✞Ó✆➷❄Ô✥Ñ⑨❒❄➴✏❮✥❒❻❒✆➴✏Ô♣✃✞Ð✏❒✆➱✞Ð✞❒❻✃✞Ò❥Ñ✕Ð✏➬✗➴➢❮✗❰➢❮✥Ó❄Ï✏✃✔❐▼➷✆❒✆➴✏➮î➷✆Ñ⑨❮t×✞Ô✗❐✍Û⑤Ó✆❮✗❐✍Ï✏Ô❦Ñ✽Ô✗❒①✃✔Ò✧❐▼Ð✏Ó✆Ô✗Ñ✕Ú❳➘✄➴✏➷✆➬✗➴ï➬✗❮✥❰ Ô✥❮✗Ñ✕➷❄Ó✆Û⑨Ï✏Ô✗❒❸❒✆✃✉❒✆➴✏ÔP❒✆➴✏✃✞Ð✞Ñ✽❮✗❰✞Õ✞Ñ✽Ú✚✃✞×✏Ô✥❐▼➘✄➴✏Ô✗Ó✆➮✙➷❄❰✏Ï✉❒❄➴✏Ô✡Ð✏Ñ✽Ô✥❐ßÞ✫ã❳✃♦➮✙❮✗á✏Ô❺❒✆➴✏➷✆❰✏Ï✏Ñ❻➘✄✃✞❐✍Ñ✽Ô✥Ú✯❒❄➴✞Ô❺❒❄Û❖➱✔➷❄➬✥❮✗Ó ❮✥Ñ✽Ñ✕✃✞➬✗➷✆❮✗❒✆➷✆✃✞❰♣❐✍Ð✏Ó❄Ô⑨❮✥Ó❄Ï✞✃✞❐✍➷❄❒✆➴✏➮ð✃✔Ð✏❒❄➱✔Ð✏❒✆Ñ❥❒❄➴✏Ô⑨Ó✆➷✆Ñ✽❒▲✃✔Ò①❐▼Ð✏Ó✆Ô✗Ñ❥❮✥Ñ❥❮⑨Ó❄✃✔❰✏Ï❦❒❄Ô✥ñ✏❒▲ä✍Ô✗×✏Ô✥❰❦➷❄❰❦❒✆➴✏Ô⑨➬✗❮✥Ñ✽Ô⑨✃✞Ò ➬✥✃✞➮✙➮✡Ô✥❐▼➬✥➷❄❮✥Ó❳❒✆✃✞✃✔Ó❄Ñ✡Ó❄➷✆á✏Ô❫ì✺ò✚ì✺ì➤ó①Ó❄Ô✥➮✡Ô✥❰✏❒❄➷✆❰✏Ô✥Ý▼Ú✴❮✥❰✏Õ➛Ó✆❮✗➬✗á✞Ñ❺➱✞✃✔Ñ✽❒❇➱✔❐▼✃✔➬✗Ô✥Ñ✽Ñ✽➷✆❰✏Ï♣ä✍Ñ✽✃✔➮✡Ô✥❒❄➷✆➮✡Ô✥Ñ❺❮✗Ó✆Ñ✽✃ ➬✥❮✗Ó✆Ó❄Ô✥Õ✡❐✍Ð✏Ó✆Ô❛➮✙➷✆❰✏➷❄❰✞Ï✏Ý⑩Ò➞❮✗➬✥➷❄Ó✆➷✆❒❄➷✆Ô✗Ñ✳Ò➞✃✔❐❸➷✆❰✏Ñ✕➱✞Ô✗➬✥❒❄➷✆❰✏Ï✡❒❄➴✏Ô✄Ñ✽Ô✥❒✒✃✞Ò❸➱✞❐✍✃✞Õ✞Ð✞➬✗Ô✗Õ✙❐✍Ð✏Ó❄Ô✥Ñ✫Þ Ö✍❰♣❒✆➴✏➷❄ÑP➱✞❮✗➱✔Ô✗❐❛➘❛Ô♦➱✔❐▼✃✔➱✞✃✔Ñ✽Ô♦❮♦➮✙Ô✗❒✆➴✏✃✔Õ➛❮✗❰✏Õ⑤❒❄✃✔✃✞Ó❇Ò▼✃✞❐❻❒✆➴✏Ô⑨Ü✔❐▼✃✔➘❛Ñ✕➷❄❰✏Ï❦❮✥❰✏Õ♣×✏➷✆Ñ✽Ð✏❮✥Ó❄➷✆æ✗❮✥❒❄➷✆✃✞❰❦✃✞Ò ❮✥Ñ✽Ñ✕✃✞➬✗➷✆❮✗❒✆➷✆✃✞❰⑨❐✍Ð✏Ó❄Ô✥Ñ✫Þ✫ã❳➴✏Ô❺❒✆✃✞✃✞Ó✇❐▼Ô✥❮✗Õ✔Ñ❻Ñ✽Ô✥❒❄Ñ❻✃✔Ò▲❐✍Ð✏Ó❄Ô✥Ñ①❐▼Ô✥➱✞❐✍Ô✗Ñ✕Ô✗❰✏❒✆Ô✗Õ⑨➷✆❰❧❒❄➴✏Ô❥➱✔❐▼✃✔➱✞✃✔Ñ✽Ô✗Õ⑨Ñ✕❒❄❮✥❰✏Õ✞❮✥❐▼Õ⑨Ò▼✃✞❐ ➱✔❐▼Ô✥Õ✞➷✆➬✗❒✆➷❄×✞Ô♦➮✡✃✔Õ✞Ô✥Ó❄Ñ✕Ú✴ò✚à➢à➢ôõä å ❮✗❒✆❮♦à➢➷✆❰✏➷❄❰✞Ï♣ö❥❐▼✃✔Ð✏➱✞Ý❹Þ✔ã❇➴✏Ô⑨➬✥✃✞➮✙➱✞Ó✆Ô✗❒✆Ô⑨Ñ✽Ô✗❒❇✃✞Ò①❐✍Ð✏Ó❄Ô✥Ñ❥➬✗❮✥❰t❒✆➴✏Ô✥❰ Ü✔Ô♣Ü✞❐✍✃✞➘✄Ñ✽Ô✥Õ➔Ü✞Û⑤❮✥➱✞➱✔Ó❄Û❖➷✆❰✏Ïï❐▼Ð✏Ó✆Ô❦Ñ✽Ô✥❒①✃✞➱✞Ô✥❐▼❮✥❒❄✃✔❐▼Ñ❧Ü✔❮✗Ñ✽Ô✥Õ➢✃✞❰ï❒❄➴✏Ô❦Ï✏Ô✥❰✏Ô✗❐✍❮✗Ó✆➷✆❒❄Û➧❐▼Ô✥Ó❄❮✥❒❄➷✆✃✞❰ïÜ✞Ô✥❒❄➘✄Ô✗Ô✥❰ ➷✆❒✆Ô✗➮✙Ñ✽Ô✗❒✆Ñ✫Þ❖ã❇➴✏Ô✡Ñ✽Ô✗❒❸✃✞Ò❳❐▼Ð✞Ó❄Ô✥Ñ❉❐✍Ô✗Ñ✽Ð✞Ó❄❒✆➷❄❰✞Ï♦Ò➞❐✍✃✞➮÷Ô✗❮✥➬✗➴♦✃✔➱✞Ô✗❐✍❮✗❒✆➷✆✃✞❰♦➬✥❮✗❰♦Ü✔ÔP×✏➷✆Ô✗➘✄Ô✗Õ❫❮✗Ñ❉❮✡Ó❄➷✆Ñ✽❒❸✃✞❐❳➬✗❮✥❰ Ü✔Ô❛Ï✞❐▼❮✥➱✞➴✏➷✆➬✗❮✥Ó❄Ó✆Û❺Ñ✽Ð✞➮✡➮✙❮✗❐✍➷❄æ✥Ô✗Õ✙❒❄➴✞❐▼✃✔Ð✏Ï✏➴P❮✄❰✏Ð✏➮✙Ü✞Ô✥❐⑩✃✞Ò❸❒❄Ô✥➬✗➴✏❰✏➷✆â✞Ð✞Ô✗Ñ✫Þ ã❳➴✏➷✆Ñ✳➱✔❮✗➱✔Ô✗❐✴➷✆Ñ✳✃✔❐▼Ï✞❮✗❰✏➷✆æ✗Ô✥Õ✙❮✥Ñ❲Ò➞✃✞Ó✆Ó✆✃✞➘✄Ñ✫ø✫➘✄Ô✄Ñ✕❒❄❮✥❐▼❒✯Ü✔ÛP➷✆❰✏❒✆❐▼✃✔Õ✞Ð✏➬✥➷❄❰✏Ï✙❒✆➴✏Ô✧Ü✞❮✗Ñ✕➷❄➬✧❰✏✃✔❒❄➷✆✃✞❰✞Ñ✳❐✍Ô✗Ó✆❮✗❒✆Ô✗Õ ❒✆✃ù❮✥Ñ✽Ñ✽✃✔➬✗➷✆❮✗❒✆➷❄✃✔❰✡❐✍Ð✏Ó✆Ô✧Õ✞➷✆Ñ✽➬✗✃✔×✏Ô✗❐✍Û❖Ú❊❮✗❰✏Õ❧❮✗Ñ✕Ñ✽✃✔➬✗➷✆❮✗❒✆➷❄✃✔❰✡❐✍Ð✏Ó✆Ô✄Ñ✕➱✞❮✥➬✗Ô❊Þ✗ú➓Ô❛❒✆➴✏Ô✥❰PÕ✞Ô✥Ñ✽➬✥❐▼➷✆Ü✞Ô✄ò✚û▲è✧é①Ú✔❒❄➴✏Ô ➱✔✃✞Ñ✕❒✯➱✞❐✍✃✞➬✗Ô✥Ñ✽Ñ✕➷❄❰✏Ï✙Ô✗❰✞×✏➷❄❐✍✃✞❰✞➮✡Ô✥❰✏❒✯Ò➞✃✔❐❸❮✥Ñ✽Ñ✕✃✞➬✗➷✆❮✗❒✆➷✆✃✞❰P❐✍Ð✏Ó✆Ô✗Ñ✳❮✗❰✏Õ✙➷✆❒❄Ñ✳➷❄➮✙➱✞Ó✆Ô✗➮✙Ô✗❰✞❒❄❮✥❒❄➷✆✃✞❰✇Þ♥ú➓Ô❛Õ✔Ô✗Ñ✽➬✥❐▼➷✆Ü✞Ô ❒✆➴✏Ô♣Ñ✕Ô✗❒❻✃✔Ò❥✃✞➱✞Ô✥❐▼❮✥❒❄✃✔❐▼Ñ⑨➷✆❰➢➮✡✃✔❐▼Ô♣Õ✔Ô✗❒✆❮✗➷✆Ó❄Ú✳Ñ✽➴✞✃✞➘➨✃✞❰✞ÔtÔ✥ñ✏❮✗➮✙➱✞Ó✆Ô❦✃✞Ò✧❒✆➴✏Ô❦❮✗➱✞➱✔Ó❄➷✆➬✗❮✥❒❄➷✆✃✞❰ï✃✞Ò✧ò✇û▲è✧é①Ú ➬✥✃✞➮✙➱✞❮✥❐▼Ô✄➘❛➷✆❒✆➴P❐✍Ô✗Ó✆❮✗❒✆Ô✗Õ✙➘❛✃✔❐▼á✡❮✗❰✏Õ✙➬✗✃✔❰✏➬✗Ó✆Ð✏Õ✞Ô✥Ú✞❮✥Ó❄Ñ✕✃✡Ñ✕Ð✏Ï✏Ï✏Ô✥Ñ✽❒✆➷❄❰✏Ï✡❒❄➴✞Ô❛❰✏Ô✥ñ✏❒✒Ñ✽❒✆Ô✗➱✔Ñ❳✃✔Ò✇✃✔Ð✏❐⑩➘❛✃✔❐▼á✇Þ è✧❰♦❮✥Ñ✽Ñ✕✃✞➬✗➷✆❮✗❒✆➷✆✃✞❰♦❐✍Ð✏Ó✆Ô❲ü →ý ❐▼Ô✥➱✞❐✍Ô✗Ñ✕Ô✗❰✏❒✆Ñ❉❮✡❐▼Ô✥Ó❄❮✥❒❄➷✆✃✞❰✏Ñ✕➴✏➷✆➱✉Ü✞Ô✥❒❄➘✄Ô✗Ô✥❰♦❒✆➴✏Ô❺Ñ✽Ô✥❒❄Ñ❉✃✞Ò▲➷✆❒❄Ô✥➮✡Ñ⑩üþ❮✗❰✏Õ ý Þ û❇❮✗➬✗➴⑨➷✆❒✆Ô✗➮➓ÿ❥➷❄Ñ❉❮✗❰⑨❮✗❒✆✃✞➮❷❐▼Ô✥➱✞❐✍Ô✗Ñ✕Ô✗❰✏❒✆➷❄❰✞Ï⑨❒❄➴✏Ô❥➱✔❐▼Ô✥Ñ✽Ô✥❰✏➬✗Ô❥✃✔Ò✴❮❥➱✞❮✗❐✍❒✆➷❄➬✥Ð✏Ó❄❮✥❐▲✃✞Ü♥ç❩Ô✗➬✗❒➝Þ✥ã❇➴✏Ô❥❐✍Ô✗Ó✆❮✗❒✆➷✆✃✞❰❧➷❄Ñ ➬✥➴✏❮✗❐✍❮✗➬✥❒❄Ô✥❐▼➷✆æ✗Ô✥Õ♣Ü✞Ût❒❄➘✄✃♣➮✙Ô✗❮✗Ñ✕Ð✏❐✍Ô✗Ñ✫ø✯Ñ✽Ð✏➱✔➱✞✃✞❐✍❒▲❮✥❰✏Õ♣➬✗✃✔❰✏Ò➞➷✆Õ✞Ô✥❰✏➬✗Ô⑨✃✔Ò①❒❄➴✏Ô⑨❐✍Ð✏Ó✆Ô❊Þ✔ã❇➴✞Ô⑨Ñ✽Ð✏➱✔➱✞✃✞❐✍❒▲✃✔Ò✳❮ ❐✍Ð✏Ó✆✁Ô ê➘✄➷❄❒✆➴✏➷✆❰✡❮✧Õ✔❮✗❒✆❮✗Ñ✽Ô✥✄❒ ✂❫Ú✔➘✄➴✏Ô✥❐▼✁Ô ✂ ➷✆❒✆Ñ✽Ô✗Ó✆Ò❸➷❄Ñ✳❮❛➬✥✃✞Ó✆Ó❄Ô✥➬✗❒✆➷❄✃✔❰P✃✔Ò✇Ñ✕Ô✗❒✆Ñ❳✃✔Ò✇➷✆❒❄Ô✥➮✡Ñ✳ä✍✃✞❐⑩➷❄❒✆Ô✗➮✙Ñ✽Ô✥❒❄Ñ✕Ý▼Ú✔➷❄Ñ ❒✆➴✏Ô✡❰✏Ð✏➮✙Ü✞Ô✗❐✳✃✞Ò❳❒❄❐✍❮✗❰✞Ñ✽❮✗➬✥❒❄➷✆✃✞❰✞Ñ❉➷✆☎❰ ✂➠❒❄➴✞❮✗❒❸➬✗✃✞❰✞❒❄❮✥➷❄❰♦❮✥Ó❄Ó❸❒✆➴✏ÔPÔ✥Ó❄Ô✥➮✡Ô✥❰✏❒❄Ñ❛➷❄❰✧ü ∪ý Þ❵ã❇➴✞ÔP➬✗✃✔❰✏Ò➞➷✆Õ✞Ô✥❰✏➬✗Ô ✃✔Ò▲❒✆➴✏Ô❺❐✍Ð✏Ó❄ÔP➷❄Ñ❻❒✆➴✏Ô❺➱✔❐▼✃✔➱✞✃✔❐▼❒✆➷❄✃✔❰⑨✃✞Ò▲❒✆❐✍❮✗❰✏Ñ✕❮✗➬✗❒✆➷✆✃✞❰✏Ñ❻❒✆➴✏❮✥❒✚➬✗✃✔❰✏❒✆❮✗➷✆❰❛ü ➘❛➷✆❒❄➴❧❐✍Ô✗Ñ✕➱✞Ô✗➬✥❒✺❒✆✃⑨❒❄➴✏Ô❥❰✞Ð✏➮✡ë Ü✔Ô✗❐❳✃✞Ò▲❒✆❐▼❮✥❰✏Ñ✽❮✥➬✗❒✆➷❄✃✔❰✏Ñ❻❒✆➴✏❮✗❒✇➬✗✃✔❰✏❒❄❮✥➷❄❰❛ü♦Þ✫û▲❮✗➬✥➴⑨❐▼Ð✏Ó✆Ô❺❐✍Ô✗➱✔❐▼Ô✥Ñ✽Ô✗❰✞❒❄Ñ①❮❥➱✔∪❮✗ý ❒✆❒❄Ô✥❐▼❰❧➬✗❮✥➱✞❒✆Ð✏❐✍Ô✗Õ⑨➷✆❰✙❮❥Õ✔❮✗❒✆❮✗Ñ✕Ô✗❒➝Þ ã❳➴✏Ô❺Ñ✕Ð✏➱✞➱✔✃✞❐✍❒✚✃✞Ò❇❒❄➴✏ÔP❐▼Ð✏Ó✆Ô❺➷✆Ñ❻❒✆➴✏Ô❺➬✗✃✔➮✡➮✙✃✞❰✞❰✏Ô✗Ñ✕Ñ①✃✞Ò✴❒✆➴✏❮✗❒✺➱✔❮✗❒✆❒❄Ô✥❐▼❰✞Ú❊➘✄➴✏➷✆Ó❄Ô❥❒✆➴✏Ô❥➬✥✃✞❰✏Ò▼➷❄Õ✔Ô✗❰✏➬✥Ô❥➮✡Ô✥❮✗Ñ✽ë Ð✞❐▼Ô✥Ñ❳➷✆❒❄Ñ✳➱✞❐✍Ô✗Õ✔➷❄➬✥❒❄➷✆×✏Ô✄❮✗Ü✔➷❄Ó✆➷❄❒✆Û✚Þ ã❳➴✏Ô⑤➮✙✃✞Ñ✕❒❛➬✥✃✞➮✙➮✡✃✔❰➟❮✗Ó✆Ï✏✃✔❐▼➷✆❒❄➴✞➮ Ò▼✃✞❐✙Õ✞➷✆Ñ✽➬✗✃✔×✏Ô✗❐✍➷✆❰✏Ï➟è✧é➨Ò➞❐✍✃✞➮ ❮➛Õ✞❮✥❒❄❮✥Ñ✽Ô✗✆❒ ✂ ➷❄Ñ♦è✧ò✇é①✞Ö ❥ ✝ é①Ö ä✍è✧Ï✏❐✍❮✗➘✄❮✗Ó✯Ô✗❒✯❮✥Ó❬✠Þ ✟☛✡✔ÝßÞ✗ã❇➴✞➷❄Ñ❲❮✗Ó✆Ï✏✃✔❐▼➷✆❒❄➴✞➮ ➱✞❐✍✃✞Õ✞Ð✞➬✗Ô✗Ñ❲❮✥Ó❄Ó✯❒✆➴✏Ô✄❮✗Ñ✽Ñ✕✃✞➬✥➷❄❮✥❒❄➷✆✃✞❰✡❐▼Ð✏Ó✆Ô✗Ñ✳❒✆➴✏❮✗❒✒➬✗❮✥❰PÜ✔Ô❛Ò▼✃✞Ð✏❰✏Õ Ò▼❐▼✃✔➮❷❮✡Õ✞❮✥❒❄❮✥Ñ✽Ô✗☞❒ ✂➠❮✗Ü✔✃✞×✏Ô✡Ï✏➷✆×✏Ô✗❰♦×✞❮✗Ó✆Ð✏Ô✗Ñ❻✃✔Ò▲Ñ✽Ð✞➱✞➱✞✃✔❐▼❒✇❮✗❰✏Õ✉➬✗✃✞❰✞Ò➞➷✆Õ✞Ô✗❰✞➬✗Ô✗Ú✯Ð✞Ñ✽Ð✏❮✥Ó❄Ó✆Û❧❐▼Ô✥Ò➞Ô✗❐✍❐✍Ô✗Õ♦❒✆✃♦❮✗Ñ ✌✎✍✑✏☛✒✔✓✖✕ ❮✥❰✏Õ ✌✎✍✑✏☛✗✄✘☛✏✖✙ Þ✫è✧ò✚é①Ö✖✝❥é①Ö❇➴✏❮✥Ñ①➮✡❮✥❰✏Û✙×✏❮✗❐✍➷❄❮✥❰✏❒✆Ñ①➘❛➷✆❒❄➴❧➮✙✃✞❐✍Ô❥❮✗➱✔➱✞Ô✗❮✥Ó❄➷✆❰✏Ï❧➬✗✃✔➮✡➱✔Ð✏❒✆❮✗❒✆➷❄✃✔❰✏❮✗Ó ➱✔❐▼✃✔➱✞Ô✥❐▼❒✆➷❄Ô✥Ñ✽Ú✳Ñ✽Ð✏➬✥➴➧❮✥Ñ✡ò✇è✧é①ã❇Ö✍ã❇✖Ö ✝✛✚✢✜▼ì✺❮✗×✞❮✗Ñ✽Ô✥❐▼ÔtÔ✗❒❲❮✥Ó❬Þ Ý✍Ú å Ö✍óù✖ä ✣❻❐✍➷❄❰➧Ô✗❒❲❮✗Ó➝Þ Ý✧✃✞❐✧ì✺è✧à➢ò✚ô✴✤Ö ✚✉ö ä✍ã❇✃✔➷❄×✞✃✞❰✏Ô✥❰✏Ý▼Ú❛Ü✞Ð✏❒✧❒✆➴✏❮✗❒✧Ñ✕➴✏✃✞Ð✏Ó✆Õõ➱✞❐✍✃✞Õ✞Ð✞➬✗Ô⑤Ô✥ñ✏❮✗➬✥❒❄Ó✆Û➔ä✍➷❄❰➟❒✆➴✏Ô➧➬✗❮✗Ñ✕Ô⑤✃✔ÒPì✺è✧à➢ò✚ô✴Ö✤✚✉ö ➷❄❒✄➬✗❮✥❰➟Ü✞Ô ❮✥➱✞➱✔❐▼✃✔ñ✏➷❄➮✙❮✗❒✆Ô✗Ó✆Û❖Ý❉❒✆➴✏Ô♦Ñ✕❮✗➮✙Ô♦Ñ✽Ô✥❒❇✃✔Ò❻❐▼Ð✞Ó❄Ô✥Ñ❺Ñ✽➷✆❰✏➬✗Ô✉❒❄➴✏Ô✉Ô✗ñ✏❮✥➬✗❒❇Ñ✕Ô✗❒❳✃✞Ò❻❐✍Ð✏Ó✆Ô✗Ñ❺❒✆✃➛➱✞❐✍✃✞Õ✔Ð✏➬✗Ô⑨➷✆Ñ❥Õ✞Ô✥❒❄Ô✥❐▼ë ➮✙➷✆❰✏Ô✗Õ✙Ü✞Û❺❒✆➴✏Ô✄➱✞❐✍✃✞Ü✔Ó❄Ô✥➮ Õ✔Ô✗Ò▼➷❄❰✏➷✆❒✆➷❄✃✔❰P❮✗❰✞Õ✡❒✆➴✏Ô✄Õ✞❮✗❒✆❮❊Þ A Post-Processing Environment 45 ✥✧✦✩★ ✪✬✫✮✭☎✯✎✰✱✰✔✲☛✳✄✴✶✵✧✷✞✴✑✲☛✸✺✹✎✻✮✼✶✭☎✰✱✽✾✵☛✳✠✭ ✿ ❀☞❂❄❃✔❅☛❆✄❇✠❂❄❈✧❉✁❊✑❋✶❂✄●❍❃✔❂✠❋✑❃❏■❑❇✠❆✄▲◆▼☛❂❑❃✔❋✶❖✖P☞❇✄❋✶P☞❖✖❂✠◗❘❊✶▲❙❆❑❚✑❆✠❋✑❋✑❊✶❇✄❂❑❯❱❊✑❋✑❀❙❋✶❀☞❂ ❖✖❂✠❚✑❆✠❋✑❊✑❈☛▲❙▼☛❂✄❋✶❯☎❂✠❂✄▲❙❃✱❂✄❋✶❃❳❲ ❁ ✿❁❀☞❂◆❂✄●❍❅✧❋✶❨❩❊✑❋✶❂✄●❍❃✔❂✠❋ ❊✑❃❑❆✠❋❁❋✶❀☛❂◆▼☛❈✧❋✶❋✑❈☛●❬❈☛❉❭❋✑❀☞❂◆❚✑❆✠❋✑❋✶❊✑❇✠❂❘❆✠▲☞◗❪❋✑❀☞❂◆❃✔⊆❂✠❋❁❈☛❉❭❆✄❚✶❚❫❊✑❋✑❂✠●❴❃✱❂✠❋✑❃✎❆✠❋✁❋✶❀☞❂ ❋✑❈☛❅❏❲❵✿❁❀☞❂ ❖✞❂✄❚✶❆✄❋✶❊✑❈☛▲❍❆✠∅❚✑❃✱❈❴❇✠❈☛❖✖❖✖❂✠❃✱❅✧❈☛▲☞◗✧❃❁❋✑❈❴❋✶❀☞❂☎❛☞❂✄▲☞❂✠❖✖❆✠❚✑❊✶❋✑❨❑❖✖❂✠❚✑❆✠❋✑❊✶❈✧▲❄▼✧❂✠❋✑❯❱❂✄❂✠▲❄❊✑❋✑❂✠●❴❃✱❂✠❋✑❃❜❲ ✿❁❈✺❃✔❋✶❖✖P☞⊆❇✄❋✶P☞❖✖❂❴❋✶❀☞❂❴❃✱❂✄❋❏❈☛❉✛❖✞P☞❚✑❂✠❃✔❝✮❯☎❂❄▲☞❂✄❂✠◗❘❆❍▲☞P☞●❴▼☛❂✄❖❁❈✧❉✁❚✑❆✠❋✑❋✶❊✑❇✠❂✄❃✱❝✮❇✄❈☛❖✖❖✞❂✄❃✱❅☛❈✧▲☞◗☛❊✑▲☞❛◆❂✄❆✠❇✄❀◆❚✶❆✄❋✶❞ ❋✑❊✑❇✠❂❡❋✑❈❢❈✧▲☞❂❪❅✧❆✠❖✖❋✶❊✑❇✠P☞❚✑❆✠❖❍❊✶❋✑❂✠●❴❃✱❂✄❋❣❋✑❀☞❆✠❋❱❆✠❅✧❅☛❂✠❆✄❖✞❃❘❆✠❃◆❋✑❀☞❂❪❆✄▲☞❋✶❂✄❇✠❂✄◗☛❂✠▲☛❋✶❝✬❈✧❖❄❋✑❈❤❈☛▲☞❂❡❊✶❋✑❂✠●❴❃✱❂✄❋❣❋✑❀☞❆✠❋ ❈✧❇✠❇✄P☞❖✞❃❱❆✠❃❭❆❄❇✠❈☛▲☛❃✱❂✠✐✧P☞❂✠▲☛❋❥❲❜❦✮❈✧❖✁❂✄❧☞❆✠●❴❅☛❚✑❂✠❝♠❋✑❀☞❂❑❖✖P☞❚✑❂✺♥✔♦✧❝q♣r❝qs✉t☞✈✇♥✔①✧❝q②♠t③❝♠▼☛❂✠❚✑❈☛▲☛❛☞❃❭❋✑❈◆❋✶❯☎❈◆❚✑❆✠❋✑❋✶❊✑❇✠❂✄❃❜④ ❋✑❀☞❂❍❈☛▲☞❂❍❈☛❉✁❋✑❀☞❂❍❖✞P☞❚✑❂✠❃❱❯❱❊✑❋✑❀◆❆✠▲☞❋✑❂✠❇✄❂✠◗✧❂✠▲☞❋✬♥✔♦✧❝q♣r❝qs✉t③❝♠❃✔❋✶❖✖P☞❇✄❋✶P☞❖✖❂✠◗❘▼☛❨⑤❋✶❀☛❂❑❛☞❂✠▲☛❂✠❖✖❆✠❚✑❊✶❋✑❨⑤❖✞❂✄❚✶❆✄❋✶❊✑❈☛▲❙❈✧⑥☞❂✠❖ ❋✑❀☞❂◆❇✄❈☛▲☞❃✔❂✠✐✧P☞❂✠▲☞❋✑❝❏❆✠▲☞◗⑦❋✑❀☞❂❙❚✑❆✠❋✑❋✶❊✑❇✠❂❙❈✧❉✬❖✞P☛❚✶❂✄❃✎❯❱❊✑❋✶❀❤♥✔①r❝q②✉t❄❆✠❃✎❆❙❇✄❈☛▲☞❃✔❂✠✐☛P☛❂✠▲☞❋✑❝❏❃✱❋✑❖✞P☞❇✄❋✶P☛❖✞❂✄◗⑦▼☛❨⑧❋✶❀☞❂ ❛☛❂✠▲☞❂✄❖✞❆✄❚✶❊✑❋✶❨❄❖✞❂✄❚✶❆✄❋✶❊✑❈☛▲❍❈☛⑥☞❂✄❖⑨❋✑❀☞❂☎❆✠▲☞❋✑❂✠❇✄❂✠◗✧❂✠▲☞❋✑❃❁❈✧❉⑩❋✑❀☞❂☎❖✞P☛❚✶❂✄❃❜❲ ❶❷❂❑❇✄❆✠▲⑤⑥☞❊✑❂✠❯✇❋✑❀☞❊✑❃✬❇✠❈✧❚✶❚✑❂✠❇✄❋✶❊✑❈☛▲⑤❈☛❉✆❚✑❆✠❋✑❋✑❊✶❇✄❂✠❃✬❆✠❃✬❆✎❛☛❖✞❊✑◗☛❝✉❯☎❀☞❂✠❖✖❂✎❂✠❆✄❇✠❀⑤❖✞P☛❚✶❂✎▼✧❂✠❚✑❈☛▲☞❛☞❃✬❋✑❈❙❈☛▲☛❂✎❊✶▲☞❞ ❋✑❂✠❖✖❃✱❂✄❇✠❋✑❊✶❈✧▲❩❈☛❉✬❋✑❯❱❈⑦❚✑❆✠❋✑❋✶❊✑❇✠❂✄❃❜❲✧✿✁❀☛❂❙❊✶◗✧❂✠❆❙▼☛❂✄❀☞❊✑▲☞◗⑦❋✶❀☛❂❙❖✞P☞❚✑❂❙▼☛❖✖❈☛❯☎❃✱❊✑▲☞❛❩❆✠❅✧❅☛❖✖❈☛❆✄❇✠❀❩❯❱❂⑤❅✧❖✞❂✄❃✱❂✠▲☛❋✶❝⑨❊✶❃ ❋✑❀☞❆✄❋❫❋✶❀☛❂❙P☞❃✱❂✄❖❭❇✠❆✄▲⑧⑥☛❊✶❃✔❊✶❋❫❈✧▲☞❂❙❈☛❉✬❋✑❀☞❂✄❃✱❂❙❚✑❆✠❋✑❋✶❊✑❇✠❂✄❃☎❸✖❈☛❖✬❅✧❆✠❖✖❋✆❈☛❉❹❊✑❋✶❺✬❆✄❋✆❆❴❋✑❊✑●❍❂✄❝⑩❆✄▲☞◗❩❋✶❆✄❻☞❂⑤❈☛▲☞❂⑤❅☛❆✄❖✞❞ ❋✑❊✑❇✠P☞❚✑❆✠❖❏❊✑▲☞❋✶❂✄❖✞❃✔❂✠❇✄❋✶❊✑❈☛▲❍❋✶❈❴●❍❈✧⑥☞❂☎❊✶▲☞❋✑❈❴❆✠▲☞❈✧❋✶❀☞❂✄❖❏❚✶❆✄❋✶❋✑❊✶❇✄❂❱❸✖❃✱❂✄❋✉❈☛❉⑨❖✞P☞❚✑❂✠❃✔❺❼❲ ★ ④✄➀✾➁✁➂☎➃❢❃✱❇✠❖✖❂✠❂✄▲❄❃✔❀☞❈☛❯☎❊✶▲☛❛❄❃✱❈✧●❍❂☎❖✖P☞❚✶❂✄❃❜❲ ❽ ✴✶❾✧✻✮❿✠✭ ✿❁❈⑤❀☞❂✠❚✑❅⑤❋✶❀☞❂➄P☞❃✔❂✠❖✆▼✧❖✞❈✧❯❱❃✔❊✶▲☞❛❍❆❱❚✑❆✠❖✖❛☞❂☎❃✱❂✄❋r❈✧❉⑩❖✖P☞❚✑❂✠❃✛❆✠▲☞◗❴P☞❚✑❋✶❊✑●❴❆✠❋✑❂✠❚✑❨❑❉➅❊✑▲☞◗❴❋✶❀☞❂☎❃✱P☛▼☛❃✱❂✄❋✉❈☛❉⑨❊✶▲☞❋✑❂✠❖✖❂✠❃✔❋✶❞ ❊✑▲☞❛❙❖✖P☞❚✑❂✠❃✱❝♠❯☎❂❑◗✧❂✠⑥☞❂✄❚✶❈✧❅☛❂✠◗❘➀✾➁❫➂➄➃➆❸✞➀⑩❈☛❃✱❋⑩❅☛❖✖❈☛❇✄❂✠❃✱❃✔❊✶▲☛❛❙➁❫▲☞⑥☞❊✑❖✞❈✧▲☞●❴❂✠▲☞❋⑩❉➅❈☛❖✁➂☎❃✔❃✱❈✧❇✠❊✑❆✠❋✑❊✶❈✧▲❴➃✬P☛❚✶❂✄❃✱❺❵❲ ➀⑩➁❫➂➄➃❷❊✶●❴❅☛❚✑❂✠●❴❂✠▲☞❋✑❃❭❋✑❀☞❂❑❃✔❂✠❋✾❈✧❉❫❈☛❅✧❂✠❖✖❆✠❋✑❈☛❖✖❃✬◗☛❂✄❃✱❇✠❖✖❊✑▼☛❂✠◗❙▼✧❂✠❚✑❈☛❯✇❋✑❀☞❆✄❋✮❋✶❖✖❆✠▲☛❃✱❉➅❈✧❖✞●➇❈☛▲☞❂✎❃✔❂✠❋✮❈✧❉✆❖✞P☞❚✑❂✠❃ ❊✑▲☞❋✑❈❡❆✄▲☞❈☛❋✑❀☞❂✠❖✖❝❫❆✄▲☞◗❡❆✄❚✶❚✑❈☛❯☎❃❄❆✺▲☞P☞●❴▼☛❂✠❖❱❈☛❉❭⑥☛❊✶❃✔P☞❆✠❚✑❊✶➈✄❆✠❋✑❊✶❈✧▲⑦❋✶❂✄❇✠❀☞▲☛❊✶✐✧P☞❂✠❃❳❲r➀✾➁❫➂➄➃❄➉q❃❄❃✱❂✠❖✖⑥☞❂✄❖❣❊✑❃❑❖✞P☛▲ 46 Jorge et al. ➊☛➋☞➌☛➍✄➎✛➏✄➋❘➐☞➑✑➑✑➒✺➓✔➍✠➎✖➔☞➍✠➎❵→↔➣➙↕⑩➛❫➣➄➜✇➝✠➞✑➟✶➍✄➋☞➑❏➟✶➓❱➎✞➊☛➋◆➠☛➋◆➏❍➡❱➍✄➢❘➢✧➎✞➠✧➡❱➓✔➍✠➎❵→❳➣➄➞✶➑✑➐☞➠☛➊☛➤☞➐◆➋☞➠✧➑⑩➝✄➊☞➎✞➎✖➍✠➋☛➑✶➞✑➥ ➟✑➦❴➒☛➞✑➍✠➦❴➍✠➋☞➑✑➍✠➌✧➧☛➦❴➊☞➞✶➑✑➟✑➒☛➞✑➍❱➝✄➞✶➟✑➍✠➋☞➑✑➓✛➝✠➏✠➋❍➒☛➠✧➑✶➍✄➋☞➑✶➟✑➏✠➞✑➞✑➥❑➎✞➊☛➋❄➝✠➠✧➋☞➝✠➊☛➎✞➎✖➍✠➋☞➑✑➞✑➥✾→ ↕⑩➛❫➣➄➜➨➠✧➒☛➍✄➎✞➏✄➑✶➍✄➓✬➢☛➥❴➞✶➠✧➏✠➌☛➟✑➋☞➤⑤➏✎↕✾➩➫➩➫➭❡➎✖➍✠➒✧➎✞➍✄➓✱➍✄➋☞➑✶➏✄➑✶➟✑➠☛➋❴➠☛➯❏➑✑➐☞➍➄➎✞➊☛➞✶➍➄➓✱➍✄➑❥→✠➲✁➐☛➟✶➓❹➟✑➋☞➟✶➑✑➟✑➏✠➞♠➓✱➍✄➑♠➟✶➓ ➌✧➟✶➓✔➒☛➞✑➏✠➥↔➍✄➌➳➏✄➓⑤➏⑧➡☎➍✠➢➫➒✧➏✠➤☞➍❩➵✖➸✮➟✶➤☛➊☞➎✞➍❤➺➼➻❼→✾➸✮➎✖➠☛➦✢➑✑➐☞➟✶➓⑤➒✧➏✠➤☞➍❩➑✑➐☞➍❩➊☞➓✱➍✄➎✎➝✠➏✠➋➳➤☞➠➫➑✑➠➳➠✧➑✶➐☛➍✠➎✎➒☛➏✄➤☞➍✠➓ ➝✄➠☛➋☞➑✑➏✠➟✑➋☞➟✑➋☞➤❄➠✧➎✞➌✧➍✠➎✖➍✠➌❴➞✶➟✑➓✱➑✑➓✛➠☛➯⑨➎✞➊☞➞✑➍✠➓✛➡☎➟✶➑✑➐❄➓✔➊☞➒☛➒✧➠☛➎✖➑r➏✄➋☞➌❴➝✠➠☛➋☛➯➅➟✑➌☛➍✠➋☛➝✠➍r→ ➲❁➠❪➦❴➠☛➔☞➍◆➯✞➎✞➠✧➦➽➒✧➏✠➤☞➍❙➵✖➓✱➍✄➑❫➠☛➯✬➎✖➊☞➞✑➍✠➓✱➻❣➑✶➠⑦➒✧➏✠➤☞➍✄➧⑨➑✑➐☞➍❙➊☞➓✔➍✠➎❭➏✄➒☛➒✧➞✶➟✑➍✠➓✎➎✖➍✠➓✔➑✶➎✖➟✶➝✄➑✶➟✑➠☛➋☛➓✎➏✠➋☞➌⑦➠✧➒☛➍✠➎✖➏✠➾ ➑✑➠☛➎✖➓❜→r➲❁➐☞➍❙➎✖➍✠➓✱➑✑➎✖➟✶➝✄➑✶➟✑➠☛➋☞➓✎➝✄➏✠➋❩➢☛➍❙➌✧➠☛➋☞➍❙➠✧➋❩➑✶➐☞➍❙➦❴➟✑➋☞➟✶➦❴➊☞➦➚➝✄➠☛➋☞➯✞➟✶➌✧➍✠➋☞➝✄➍✠➧❏➦❴➟✶➋☞➟✑➦❴➊☞➦➚➓✱➊☛➒☛➒☛➠✧➎✞➑✑➧❏➠☛➎ ➠✧➋❩➯➅➊☞➋☞➝✄➑✶➟✑➠☛➋☛➓✎➠☛➯✬➑✑➐☞➍❙➓✱➊☞➒✧➒☛➠✧➎✞➑❫➏✄➋☞➌⑦➝✠➠✧➋☞➯➅➟✑➌☛➍✄➋☞➝✠➍❙➠✧➯✬➑✶➐☛➍❙➟✶➑✑➍✠➦❴➓✱➍✄➑✶➓✎➟✑➋❩➑✶➐☞➍⑤➎✖➊☞➞✑➍r→☛➪✎➒☛➍✠➎✖➏✠➑✑➠☛➎✖➓➄➝✠➏✄➋ ➢✧➍❄➓✔➍✠➞✑➍✠➝✠➑✑➍✠➌❘➯➅➎✖➠☛➦➇➏❄➞✶➟✑➓✱➑➶→❜➹✞➯✁➟✶➑⑩➟✶➓❭➏⑧➘➷➴❱➬☛➮✑➱✉✃ → ➘✔❐☛➱✠❒✑❮❭❰❥Ï✛➴❱➬✧➮✶➱✄❮❳✃❱➠☛➒✧➍✠➎✖➏✠➑✑➠☛➎✖➧♠➑✶➐☞➍❄➟✶➋☞➒✧➊☞➑✾➎✖➊☞➞✑➍❑➦❴➊☞➓✱➑ ➏✄➞✶➓✔➠❍➢✧➍❱➓✔➍✠➞✑➍✠➝✄➑✶➍✄➌⑨→ Ð❫Ñ✶Ò✧Ó✮Ô✠Õ☎Ö⑨× ↕⑩➛❫➣➄➜❤➒☛➞✑➠☛➑✑➑✶➟✑➋☞➤❍➓✱➊☞➒✧➒☛➠☛➎✖➑✉Ø❄➝✄➠☛➋☞➯✞➟✶➌✧➍✠➋☞➝✄➍❱➒✧➠☛➟✑➋☞➑✶➓✛➯➅➠✧➎❏➏❱➓✔➊☞➢☛➓✔➍✠➑✉➠☛➯⑨➎✞➊☛➞✶➍✄➓✱➧☛➏✄➋☞➌❴➓✱➐☞➠✧➡❱➾ ➟✑➋☞➤❄➏☎➦❴➊☞➞✶➑✑➟✑➾➅➢☛➏✄➎❏➐☞➟✶➓✔➑✶➠✧➤☞➎✖➏✠➦⑦→ ➸✾➠☛➎✆➍✠➏✄➝✠➐❴➒☛➏✄➤☞➍✠➧r➑✶➐☛➍☎➊☛➓✱➍✠➎❏➝✄➏✠➋❄➏✄➞✶➓✔➠❍➓✔➍✠➞✑➍✠➝✄➑r➏☎➤☞➎✖➏✠➒✧➐☞➟✶➝✄➏✠➞✉➔☞➟✑➓✱➊☞➏✄➞✶➟✑Ù✠➏✄➑✶➟✑➠☛➋❍➑✶➐☛➏✠➑✉➓✱➊☞➦❴➦❴➏✠➎✖➟✶Ù✄➍✠➓✛➑✶➐☞➍ ➓✔➍✠➑⑨➠☛➯❁➎✞➊☞➞✑➍✠➓❱➠☛➋◆➑✑➐☞➍❍➒☛➏✠➤☛➍r→❳Ú✬➊☞➎✖➎✖➍✠➋☞➑✑➞✶➥↔➧✾➑✶➐☞➍❄➏✠➔☞➏✄➟✶➞✑➏✠➢✧➞✶➍❑➔☛➟✶➓✔➊☞➏✠➞✑➟✶Ù✄➏✠➑✑➟✶➠✧➋☞➓❭➏✄➎✞➍❑Ú✬➠✧➋☞➯➅➟✑➌☛➍✄➋☞➝✠➍ × Û ➊☞➒☛➾ ➒✧➠☛➎✖➑❣➒✧➞✶➠✧➑❣➏✠➋☛➌❤Ú✬➠☛➋☞➯✞➟✶➌✧➍✠➋☞➝✄➍❪Ü❣➓✔➊☞➒☛➒✧➠☛➎✖➑❣➐☞➟✑➓✱➑✑➠☛➤☛➎✞➏✄➦❍➓◆➵✖➸✮➟✑➤☞➊☞➎✖➍❪Ý☛➻❵→⑩➲✁➐☞➍⑦➒✧➎✞➠✧➌☛➊☞➝✄➍✠➌Þ➝✄➐☞➏✠➎✖➑✶➓❙➏✄➎✞➍ ➟✑➋☞➑✑➍✠➎✖➏✠➝✄➑✶➟✑➔☞➍☎➏✠➋☞➌❴➟✶➋☛➌☛➟✑➝✠➏✠➑✑➍☎➑✶➐☞➍☎➎✖➊☞➞✶➍☎➑✑➐☞➏✠➑✉➝✠➠✧➎✞➎✖➍✠➓✔➒☛➠✧➋☞➌☛➓✛➑✶➠❴➑✑➐☞➍❱➒✧➠☛➟✑➋☞➑✉➊☞➋☞➌✧➍✠➎❏➑✶➐☛➍❱➦❴➠☛➊☞➓✔➍r→ ➲❁➐☞➍⑤➏✠➓✔➓✱➠☛➝✄➟✶➏✄➑✶➟✑➠☛➋⑧➎✖➊☞➞✶➍⑤➢✧➎✞➠✧➡❱➓✔➍✠➎✬➐☞➍✄➞✶➒✧➓☎➑✑➐☞➍⑤➊☞➓✔➍✠➎✬➑✑➠⑧➋☛➏✠➔☞➟✑➤☞➏✠➑✑➍⑤➑✶➐☛➎✞➠✧➊☞➤☞➐❘➑✑➐☞➍❴➓✱➒✧➏✠➝✠➍❴➠☛➯✛➎✖➊☞➞✶➍✄➓❱➢✧➥ ➔☛➟✶➍✄➡❱➟✑➋☞➤❴➠☛➋☞➍➄➓✱➍✄➑✉➠☛➯⑨➎✞➊☞➞✑➍✠➓✛➏✠➑✉➏☎➑✶➟✑➦❴➍r→❵➛✁➏✠➝✠➐❍➓✱➍✄➑r➠✧➯⑩➎✖➊☞➞✑➍✠➓✛➝✠➠✧➎✞➎✖➍✠➓✔➒☛➠☛➋☛➌☛➓✛➑✶➠❴➠☛➋☞➍☎➒☛➏✄➤☞➍r→➼➸✮➎✖➠☛➦ß➠☛➋☞➍ ➤☛➟✶➔☞➍✄➋⑦➒☛➏✄➤☞➍◆➑✑➐☞➍◆➊☞➓✔➍✠➎❣➦❴➠☛➔☛➍✠➓❑➑✑➠❪➑✑➐☞➍◆➯➅➠✧➞✶➞✑➠☛➡☎➟✑➋☞➤⑦➢☛➥❩➏✠➒✧➒☛➞✑➥↔➟✶➋☛➤⑧➏❙➓✔➍✠➞✑➍✠➝✄➑✶➍✄➌⑦➠☛➒☛➍✄➎✞➏✄➑✶➠✧➎❭➑✑➠⑦➏✠➞✑➞❫➠☛➎ ➓✔➠☛➦❴➍⑤➠☛➯❹➑✶➐☛➍❍➎✖➊☞➞✑➍✠➓☎➔☞➟✶➍✄➡❱➍✄➌✺➠✧➋❘➑✑➐☞➍❴➝✠➊☞➎✖➎✞➍✄➋☞➑❏➒☛➏✄➤☞➍r→☞➹✞➋✺➑✶➐☞➟✑➓☎➓✱➍✠➝✄➑✶➟✑➠☛➋✺➡❱➍❴➌☛➍✄➯➅➟✑➋☞➍❴➑✶➐☞➍❴➓✱➍✄➑❏➠☛➯✛➠☛➒☛➾ ➍✄➎✞➏✄➑✶➠✧➎✞➓✛➑✑➠❍➏✄➒☛➒☛➞✑➥❑➑✑➠❍➓✔➍✠➑✑➓❁➠✧➯⑩➏✄➓✱➓✔➠☛➝✠➟✑➏✠➑✑➟✑➠☛➋❄➎✖➊☞➞✑➍✠➓❳→ ➲❁➐☞➍❙➠☛➒✧➍✠➎✖➏✠➑✑➠☛➎✖➓✎➡❱➍❙➌✧➍✠➓✔➝✠➎✖➟✶➢✧➍❙➐☞➍✠➎✖➍❙➑✶➎✖➏✠➋☛➓✱➯➅➠✧➎✞➦➽➠☛➋☞➍❙➓✔➟✶➋☞➤☛➞✶➍❙➎✖➊☞➞✑➍❱➴ ∈ ➘➷➴❱➬☛➮✑➱✠❮↔✃❑➟✑➋☞➑✑➠⑧➏⑤➓✔➍✠➑✆➠☛➯ ➎✖➊☞➞✑➍✠➓❑➴❱❐ ∈ ➘✔❐✧➱✠❒✑❮✺❰❥Ï❙➴☎➬☛➮✑➱✠❮❳✃③➏✄➋☞➌à➝✠➠☛➎✖➎✖➍✠➓✱➒✧➠☛➋☞➌à➑✑➠➨➑✑➐☞➍➳➝✠➊☞➎✖➎✞➍✄➋☞➑✑➞✶➥Þ➟✑➦❍➒✧➞✶➍✄➦❍➍✄➋☞➑✑➍✠➌❢➠✧➋☞➍✠➓❳→❏➪✎➑✑➐☞➍✠➎ ➟✑➋☞➑✑➍✠➎✖➍✠➓✔➑✶➟✑➋☞➤❘➠✧➒☛➍✄➎✞➏✄➑✶➠✧➎✞➓☎➦❍➏✄➥◆➑✑➎✞➏✄➋☞➓✱➯✞➠☛➎✖➦á➠✧➋☞➍❴➓✱➍✠➑❏➠✧➯❁➎✖➊☞➞✑➍✠➓☎➟✶➋☞➑✑➠⑧➏✠➋☞➠✧➑✶➐☞➍✄➎❼→☞➹✞➋❘➑✶➐☞➍❍➯➅➠✧➞✶➞✑➠☛➡☎➟✶➋☛➤◆➡❱➍ ➌✧➍✠➓✔➝✠➎✖➟✶➢✧➍❱➑✑➐☞➍☎➠☛➒☛➍✄➎✞➏✄➑✶➠✧➎✞➓✛➠☛➯⑨➑✑➐☞➍❱➯✞➠☛➎✖➦❍➍✄➎❏➝✠➞✑➏✠➓✱➓❳→ â✎ã✾ä Õ✄å✠Õ✠æ✾Õ ã✮ä Ò☛Õ ã Õ✠Ô✄ç☛è✑Ñ✶é✄ç ä Ñ✶ê ã❘ë✖â✎ã✮ä✖ì❴í A Post-Processing Environment 47 î ï✧ð❥ñ❹ò✩î ❱ î❙÷ î❙÷☞ú✑û✛ü☛ý☛þ✑ÿ✠ú✁✄✂✆☎❍ý✞✝✟☎✠✂✆✡☛✂✄þ✶ú✁✄☞❄ü✞✄✂☎ü✠✌✎✍❍ü✞✌✏☎ ✂ ÿ✠þ✑ü✠✍❴û✛ú☛✬î✒✑ →ó➄ô⑨õÞö →óùø ü ✖✠✂✆✌✞ÿ✄þ✶ü✞✌✗✖✠✌✖ü✠☎✠✘✠✙✚✂✠û✛✌✜✘✄✡✁✠✂ û➄û✱ú✁❍ ✍ ú✁✡✑ÿ✚✌✬þ✑ü✺þ ✔ ✂✢☞ ☞ ú☛✣✠✂✚❘ü✞✄✂❴✠ ý ✘☞þ✎✤☎ú✶þ ✔ ÿ❴✥û ✝✦☞þ✑ÿ✚✙✄þ✶ú✁✙✠ÿ✆✡☛✡✁✝◆û✔ú☛✍✢✖✠✡✁✂✚✌ ✓✕✔ ú✑û☎✞ ÿ✆☞þ✁✂✚✙✚✂✆☎✠✂✆☞þ★✧ ✓✩✔ ú✑û❣✆ÿ ✡☛✡✑✠ü ✤☎û❣þ ✔ ✂❄✁ú ☎✠✂✆☞þ✑☛ú ✪✞☛ú ✙✄ÿ✠þ✑ú✶✞ü ◆ü✠✪✕✌✏✂✆✡☛✂✆✣☞ÿ✚☞þ⑨ü✠✌✛ú✁✌✏✌✜✂✚✡✁✂✚✣☞ÿ✆☞þ⑨ú✶þ✁✂✚✍❴û❣ú✁❙þ ✔ ✂✟✙✚✘✄✌✜✌✜✂✚☞þ ✌✜✘✄✡✁✂✫✧ ✓✩✔ ✂⑦✥ û ✘✄✖✞✖☛✠ü ✌✖þ✬✚ÿ ✠☎✬✄✙ ✠ü ✄✪✞☛ú ☎✞✂✚✄✙✆✂✭✡✶✁ú ✄✂✠û⑤ü✞☎ ✪ þ ✔ ✂✭✌✜✂✠✥ û ✘✠✡✶þ✑☛ú ✠☞ ✮û ✂✠þ✬✠ü ✪✛✌✜✘✄✡✁✂✠û⑤✚ÿ ✡✁✡✶✞ü ✤ þ ✔ ✂✯✣☛ú✶✮û ✘☞ÿ✚✡ ✁ú ☎✠✂✆☞þ✑☛ú ✪✞☛ú ✙✄ÿ✠þ✑ú✶✞ ü ❘ü✠✪❁ú✶✁þ ✂✚✍❴û❣þ✑✰ ü ✖✞✌✏✘✠✄✂❄✁ú ◆þ ✔ ✂❄ÿ✆☞þ☛✂✆✙✚✂✆☎✠✂✚☛þ★✧✲✱✜◆þ✁✂✚✌✜✍❍û❱✠ü ✪❁þ ✔ ✂❍✚ÿ ☞þ✁✂✚✙✆✂✚☎✞✂✚☞þ✳✶✡ ÿ✄þ✶þ✑ú✁✙✚✂✚✴✾ú✶þ ☞☛☛ú ✣✄✂✄û❁✆ ÿ ✡☛✡✉þ ✔ ✂✵✌✏✘✄✡✁✠✂ û✛✠ý ✂✆✡✶✞ü ✤❷þ ✔ ✂✵✙✚✘✠✌✏✌✜✂✚☞þ✉✞ü ✄✂✶✤☎ú✑þ ✔ þ ✔ ✂☎û✱✆ÿ ✍✷✂✵✙✠ü✞☞û✥✂✆✸✠✘✄✂✆☞þ✹✧ ✺✼✻✾✽✏✿✆❀✚✿✚❁✾✿✚✻❂✽✎❃✁✿✚❄✠❅✮✽✎❆✠✿✚✻✾✿✚❇✚❄✞❃❈❆✠✿✚✻✾✿✚❇✚❄✞❃☛❉✁❊✚❄✞✽✏❉✁❋✠✻❍●✏✺✼✻✾✽✏■✗❏❑❏❑▲ î ï✧ð ▼➄ñ✎ñ❹ò✩î ❱ î❙÷ î❙÷☛ú✶û✛ü☛ý✧þ✶ÿ✄ú☛✄✂✆☎❍ý✞✝✟☎✠✂✆✡☛✂✄þ✶ú✁✄☞❍ü✠✄✂☎ÿ✠þ✑ü✠✍ß✁ú ❹î✒✑ →ó☎ô❏õ➫ö →ó➇ø ✙ þ✁✂✚✌◆✣✠✂✚✌✖û✱ú✑✠ ü ❩ü✠✪✬þ ✔ ✂❱î❱ï✧ð❥ñP❖◗✱✖þ❘☞☞ú✁✣✄✂✠û✎ü✞✄✡☛✝⑧þ ✔ ✂✒✌✏✘✠✡☛✂✄û✎✠ü ❩þ ✔ ✓✕✔ ú✑û✎ü✠✖✠✂✆✌✞ÿ✄þ✶ü✞✌❭ú✑û✎ÿ❙û✱✁þ ✌✞ú✁✠ þ ✔ ✂☎✚ÿ ☞þ✁✂✚✙✆✂✚☎✞✂✚☞þ❈✑✡ ÿ✠þ✑þ✶✁ú ✙✚☎ ✂ ☛ú ✍✢✍✢✂✚☎☛ú✑ÿ✠✁þ ✂✚✡✁❑ ✝ ✞ ý ✂✚✡✑✠ü ✤❷þ ✔ ✂✶✙✆✘✄✌✏✌✜✂✚☛þ✫✌✜✘✄✡✁✂✫✧ ✂✢✡✁✂✚✣✠✂✚✡ ✆ü✠✪ ❙✼❋✞✻❂❅✥✿✆❚❂❯❂✿✆✻❂✽✎❆✠✿✚✻✾✿✚❇✚❄✞❃☛❉✁❊✚❄✞✽✏❉✁❋✠✻❍●✏❙✼❋✞✻❂❅✥❏❑▲ ☛ï ❜ñ❹ò✩î ❱✳❲ ✞❳ î →ó➄ô⑨õÞö ÷ →ó ø✶ó ÷☞ú✑û✛ü☛ý☛þ✑ÿ✠ú✁✄✂✆☎❍ý✞✝✟☎✠✂✆✡☛✂✄þ✶ú✁✄☞❄ÿ✄þ✶ü✞❍ ✍ û✛ú✁ ó ✑ ❙✼❋✞✻❂❅✥✿✆❚❂❯❂✿✆✻❂✽✎❃☛✿✆❄✠❅✥✽✎❆✞✿✚✻❂✿✆❇✚❄✠❃❈❆✞✿✚✻❂✿✆❇✚❄✠❃✁❉✁❊✚❄✠✽✜❉✁❋✠✻✰●✜❙✼❋✠✻✾❅✥■✗❏❑❏❑▲ ☛ï ➄ñ✎ñ❹ò✩î î ÷ ÷☛ú✶û✛ü☛ý✧þ✶ÿ✄ú☛✄✂✆☎❍ý✞✝✟☎✠✂✆✡☛✄✂ þ✶ú✁✄☞❍ü✠✄✂☎ÿ✠þ✑✠ü ✍ßú✁ ✑ ó →ó☎ô❏õ➫ö →ó ø✤ó ❩✾ú☛✍❴☛ú ✡✑✚ ÿ ✌✁þ✶ü❱î❱ï✧ð❥ñ➨ÿ✚✠☎❱î❱ï☛ð ▼➄ñ✎ñ❬✌✏✄✂ û✥✖✠✂✆✙✠þ✑ú☛✣✠✂✚✡✁✝✦✴rý✞☞✘ þ♠þ ✔ ✂➄û✱ú✁✍✷✖✞✶✡ ú✁✪➅ú✁✙✠ÿ✄þ✶ú✑ü✠❴ú✶û❭☎✧✠ü ✄✂➄✠ü ❴þ ✔ ✂✛✙✠ü✠☛û✥✂✚❪ ✸✞✘✄✂✚☛þrú✁☞û✔☛þ ✂✄✚ ÿ ☎❴✠ü ✪⑨✠ü ❄þ ✔ ✂☎✚ÿ ☞þ✁✂✚✙✆✂✚☎✞✂✚☞þ✹✧ ❱✳❲ ✞❳❨▼ ✺✼✻✾✽✏✿✆❀✚✿✚❁✾✿✚✻❂✽✎❅✥❫✾✿✚❀✚❉✁❄✠❃✁❉✁❊✚❄✠✽✜❉✁❋✠✻✰●✜✺✼✻❂✽✜❴❂▲ ✓✕✔ î◆÷ î❙÷ î❛✑ î ï✧ð☛❵ròqî ❱ →ó☎ô❏õ➫ö →ó➇ø ⊇ ú✑✕ û ✖✞✌✞ü✞☎✠✘✄✙✆✂✠P û ✌✏✘✄✡✁✠✂ ûP✤☎ú✶þ ✔ ✡✑✠ü ✤✵✂✚✌❏û✥✘✠✖✠☛✖ ü✞✌✞þ✉✞ý ✘☞þ ✔ ☛ú ☞ ✠ü ➅ú ✔ ✂✚✌✎✙ ✞✄✪ ✁☎✠✂✆✄✙✚✂ ☎þ ✔ ÿ✚❍þ ✔ ✂✵✙✚✘✠✌✏✌✜✂✚ ☞þ✉ü✞✄✂✫✧ ✺✼✻✾✽✏✿✆❀✚✿✚❁✾✿✚✻❂✽✎❃✁✿✚❄✠❅✮✽✎❅✥❫❂✿✆❀✚❉✁❜✏❉✁❀✶❅✮❫❂✿✚❀✆❉☛❄✞❃☛❉✁❊✚❄✞✽✏❉✁❋✠✻❍●✏✺✼✻✾✽✏■✗❴❂❴✾▲ î❙÷ î❙÷☞ú✑û✛ü☛ý☛þ✑ÿ✠ú✁✄✂✆☎❍ý✞❑ ✝ ÿ✚☎✞☎☛ú✁✄☞❄ü✞✄✂❱ò✖ÿ✚✠✝ ÿ✄þ✶✞ü ✍ßþ✶ü❭î✒✑ î ï✧ð ▼✕❵✞❵✧ò✩î ❱ ô →ó➄ô⑨õÞö →óùø ❝➄û❱î❱ï✧ð☛❵✫✴❹ý✠✘☞þ✬ü✞✄✡☛✝❞✪➅ü✞✌✎þ ✔ ✂❩✁ú ✍✷✍✢✂✚☎✧ú✶ÿ✄þ☛✂✭✡✁✂✚✣✄✂✆✡✬ÿ✠ý✧✠ ü ✣✄✂❩þ ✔ ✂✭✙✚✘✄✌✜✌✏✂✆☞þ✗✌✜✘✄✡☛✂❩ü✞ þ ✔ ✂❩ÿ✚☞þ✁✂✚✙✆✂✚☎✞✂✚☞þ ✡✑ÿ✠þ✑þ✑☛ú ✙✆✂✫✧ ❙✼❋✞✻❂❅✥✿✆❚❂❯❂✿✆✻❂✽✎❅✥❫❂✿✆❀✚❉✁❄✠❃✁❉☛❊✆❄✠✽✜❉☛❋✞✻✰●✜❙✼❋✠✻❂❅✮❴❂▲ ☛ï ròqî ❱✳❲ ✞❳✥❵ →ó☎ô❏õ➫ö î →ó ÷ ÷ ø✤ó ⊇ó ✑ ❙✼❋✞✻❂❅✥✿✆❚❂❯❂✿✆✻❂✽✎❃☛✿✆❄✠❅✥✽✎❅✮❫❂✿✚❀✆❉☛❜✜❉☛❀✵❅✥❫✾✿✚❀✚❉✁❄✠❃✁❉✁❊✚❄✠✽✜❉✁❋✠✻✰●✜❙✼❋✠✻✾❅✥■✗❴❂❴❂▲ ☛ï ✧ò✩î î ÷ ÷☞ú✑û✛ü☛ý☛þ✑ÿ✠ú✁✄✂✆☎❍ý✞✝❑✚ÿ ☎✞☎☛ú✁✄☞❄ü✞✄✂❱ò✖ÿ✚✠✝ ÿ✄þ✶ü✞✍ßþ✶ü →ó➄ô⑨õÞö →ó ø✹❡ ô ó ❩✾ú☛✍❴ú☛✡✑✚ ÿ ❏✌ þ✑ü✬î☎ï☛ð✁❵❴ÿ✆✄☎✬î☎ï☛ð✁❵✠❵✞✴✧ý✠✘☞þ✉✞ü ❄þ ✔ ✂✶✙✄ü✠☞û✮✂✚✸✞✘✄✂✚☞þ✹✧ ❱✳❲ ✞❳❨▼✕❵✞❵ ❢ ❋✠❀✚❯✾❅✕❋✞✻✰❄✠✻✾✽✏✿✆❀✚✿✚❁✾✿✚✻❂✽✎● ❢ ✑ ✺✼✻❂✽✜▲ î❱ï✧ð✑ò✩î î ❱ ❱⑦ú✑û✛✚ÿ ✄✝✾✑ →ó☎ô❏õ➫ö → ø ❤✎ú✁✣✄✂✄û❁✆ ÿ ✡☛✡✉þ ✔ ✂✶✌✜✘✄✡✁✠✂ P û ✤❱ú✑þ ✔ ✂✚✐☛ÿ✚✠✙ þ✁✡✁✝❑þ ✔ ✂❱û✔ÿ✚✍✢✂❱ÿ✆☞þ☛✂✆✙✚✂✆☎✠✂✚☛þ★✧ ❣ î❱ï☛ð ò❦❥ ❣ ô❏õ î❱ï✧ð❥ñ❹ò❦❥ ô ∪ î❱ï✧ð☛❵rò❧❥ ô ✧ 48 Jorge et al. ♠✩♥✠♦✚♣✾q✕♥✞r✰♦✚♥✞r❂q✥s✆t❂♣❂s✆r❂✉✎✈✏♠❘✇✼♥✞r❂q✥① ②❑③✳④✞⑤✠⑥✮⑦❦⑧ →⑨✵⑩✎❶❸❷ ③ →⑨❺❹ ③❼❻✁❽P❾✚❿✄➀✾➁ ➂ ❻✁➃✄➄✆❽✕❾✆➅☛➅❈➆✁➇✄➄✶➈✜➉✄➅✁➄✚❽P➊✶❻✁➆✁➇➋➆✁➇✄➄✶❽✮❾✚➌✢➄✶➍✆➎✠❿✄❽✮➄✚➏✞➉✄➄✚❿✄➆✹➐✏②✒③✳④✠⑤✠⑥➑⑦❧➒ ➣ ➉✠➈✗➌✷➄✆➆☛➇✄➎✞↔✠➎✞➅☛➎✞↕✄➀✰❻✁❽✛➙✠❾✚❽✮➄✚↔✭➎✠❿✯➆✁➇✄➄❑➛✠➇✄❻✁➅☛➎✞❽✥➎✞➛✠➇✄➀❍➎✠➜❭➊✶➄✆➙✭➙✠➈✜➎✠➊✵❽✥❻✁❿✄↕✄➝✳➛✠❾✚↕✠➄✷➙✞➀❛➛✠❾✆↕✄➄✷➜✏➎✠➅✁➅☛➎✞➊✶➞ ⑩✳❶ ③✳④✠⑤✞⑥➔➓❭⑦❦➒ ⑩ ∪ ③✳④✠⑤✞⑥✥→✞⑦❦➒ ⑩ ➐ ❻✁❿✄↕➟➇✠➀✦➛✠➄✚➈✜➅✁❻☛❿✄➠✠❽➔➐❈➡✕➇✄➄❛➎✠➛✞➄✚➈✜❾✚➆✁➎✠➈✜❽✟❻✁➌✷➛✞➅☛➄✆➌✷➄✆❿✄➆✩➆✁➇✄➄❛➇✄➀✦➛✞➄✚➈✜➅☛❻✁❿✄➠✄❽➋➙✠➄✚➆✁➊✵➄✚➄✚❿❼➆✁➊✵➎➟➛✠❾✆↕✄➄✚❽✲➐✫➡✩➎➟❽✮➆☛❾✆➈✏➆ ➙✞➈✏➎✞➊✶❽✮❻☛❿✠↕✄➝❂➆✁➇✄➄➋➉✄❽✮➄✚➈P❿✄➄✚➄✆↔✠❽✶❾✚❿❛❻✁❿✄↔✠➄✆➢❛➛✠❾✆↕✄➄✫➐✦➡✩➇✄❻✁❽➤❽✮➇✄➎✠➉✄➅✁↔✰❻✁❿✄➍✆➅☛➉✄↔✞➄➋❾✷❽✥➉✄➙✞❽✥➄✚➆✳➎✠➜✕➆☛➇✠➄➋➈✜➉✄➅☛➄✆❽◆➆✁➇✄❾✚➆ ❽✮➉✄➌✢➌✷❾✆➈✏❻✁➥✚➄✒➆✁➇✄➄✒➊✵➇✄➎✠➅✁➄✒❽✥➄✆➆★➐✞➦✜❿✯➆✁➄✚➈✜➌✢❽✵➎✞➜❭➊✶➄✆➙✯➙✞➈✏➎✞➊✶❽✮❻☛❿✠↕✄➝➧❻✁➆◗❽✥➇✠➎✠➉✄➅✁↔✭➙✠➄❑❾❑❽✥➌✢❾✚➅✁➅◗❽✥➄✚➆◗➎✞➜❭➈✏➉✄➅✁➄✚❽ ➆✁➇✄❾✆➆❘❾✚➅✁➅☛➎✞➊✶❽✼↕✠➄✚➆✁➆☛❻✁❿✄↕✭➆☛➎❼❾✆❿✄➀✯➛✠❾✚↕✠➄✒❻☛❿✭❾✒➅✁❻☛➌✢❻☛➆✁➄✚↔❼❿✠➉✄➌✷➙✞➄✚➈✗➎✞➜P➍✆➅☛❻✁➍✚➠✠❽➔➐✠➨❺➍✚❾✆❿✄↔✠❻✁↔✠❾✆➆☛➄❑➜➩➎✞➈✗❽✥➉✄➍✆➇❍❾ ❽✮➄✚➆◆➍✆➎✠➉✄➅✁↔➫➙✠➄❼➆✁➇✄➄✆➝P➜✏➎✠➈✟➄✆➢✄❾✚➌✢➛✠➅✁➄✚➝❭➆✁➇✄➄❼❽✥➌✢❾✚➅✁➅✁➄✚❽✥➆➤➈✏➉✄➅✁➄❼➜➩➎✞➈✟➄✚❾✆➍✚➇✬➍✚➎✞❿✄❽✥➄✆➏✠➉✄➄✆❿✄➆★➐✾➭❘❾✆➍✚➇✬➎✠➜✛➆☛➇✠➄✚❽✥➄ ➈✜➉✄➅✁➄✚❽✛➊✵➎✠➉✄➅✁↔✭➈✏➄✆➛✠➈✜➄✚❽✥➄✆❿✄➆◗➆✁➇✄➄❑➅☛❾✆➆☛➆✁❻☛➍✆➄❑➎✠❿✯➆☛➇✄➄❑❾✆❿✄➆☛➄✆➍✚➄✆↔✠➄✚❿✠➆☛❽✛➎✠➜❭➆✁➇✄➄✢➈✏➉✄➅✁➄✚❽✵➊✵❻☛➆✁➇✰➆✁➇✄➄✢❽✥❾✚➌✢➄✢➍✚➎✠❿✠❽✥➄✚➞ ➏✞➉✄➄✚❿✠➆★➐✄➯❂❻☛❿✠➍✚➄✢➆☛➇✄➄✢➅✁❾✚➆✁➆☛❻✁➍✚➄✆❽✶❻✁❿✄➆✁➄✚➈✜❽✥➄✚➍✆➆☛➝➧➊✶➄✢➍✚❾✆❿✰➍✆➇✄❾✚❿✄↕✠➄✷➆✁➎❍❾✢➜➩➎✞➍✚➉✄❽✵➎✠❿❍➆☛➇✠➄✷❾✆❿✄➆☛➄✆➍✚➄✆↔✠➄✚❿✠➆✳➎✞❿✰❾✆❿✄➀ ➈✜➉✄➅✁➄✶➙✞➀✟❾✚➛✞➛✠➅✁➀✦❻☛❿✠↕➋❾✚❿✷❾✚➛✞➛✠➈✜➎✠➛✠➈✜❻✁❾✚➆✁➄✶➎✞➛✠➄✚➈✜❾✚➆✁➎✠➈➲➐ ➯✾❻☛➌✢❻☛➅✁❾✚➈✜➅✁➀✦➝✫➊✵➄✼➍✚➎✞➉✄➅✁↔✒❽✥➆✁❾✚➈✜➆❂➊✶❻✁➆✁➇✢➆✁➇✄➄✼❽✮➄✚➆❂➎✞➜◗❽✥➌✢❾✚➅✁➅☛➄✆❽✥➆❂➈✜➉✄➅✁➄✚❽✗➜➩➎✞➈❘➄✚❾✆➍✚➇❑❾✚❿✠➆☛➄✆➍✚➄✚↔✞➄✚❿✄➆✹➐✆➨✛➅☛➆✁➄✚➈✜❿✄❾✚➞ ➆✁❻✁➃✄➄✚➅✁➀✦➝✎❻☛❿✄❽✮➆☛➄✆❾✚↔❼➎✞➜✗➆☛➇✄➄✒❽✮❻☛➥✆➄✚➝✎➊✵➄✒➍✚➎✠➉✠➅☛↔❼➍✆➎✠❿✄❽✮❻☛↔✞➄✚➈◆➆✁➇✄➄✒❽✥➉✠➛✠➛✠➎✞➈✏➆✁➝✎➍✚➎✞❿✄➜➩❻✁↔✠➄✆❿✄➍✚➄✆➝✳➎✞➈◆➎✠➆✁➇✄➄✆➈✗➌✷➄✆❾✚❽✥➞ ➉✠➈✏➄❈➐✲➨✛➅☛➅✳➆☛➇✠➄✚❽✥➄✷➛✠➎✞❽✥❽✥❻✁➙✠❻✁➅✁❻☛➆✁❻☛➄✆❽➤➌✢➉✄❽✥➆✳➙✠➄➋❽✥➆✁➉✄↔✠❻✁➄✚↔❛❾✆❿✄↔❛❽✮➎✠➌✢➄✟➎✠➜❘➆✁➇✄➄✆➌➳❻✁➌✷➛✞➅☛➄✆➌✷➄✆❿✄➆☛➄✆↔❛❻✁❿✒➎✠➉✄➈✕❽✥➀✦❽✥➞ ➆✁➄✚➌✢➝✠➊✵➇✄❻✁➍✚➇✷➍✚➉✄➈✜➈✏➄✆❿✄➆✁➅☛➀✟❽✮➇✄➎✠➊✵❽✥➝✞❾✚❽P➆☛➇✠➄✶❻✁❿✄❻✁➆☛❻✁❾✚➅❈➛✠❾✆↕✄➄✚➝✞➆☛➇✠➄✶❽✮➄✚➆❈➎✠➜✳❾✚➅✁➅❈➈✏➉✄➅✁➄✚❽✲➐ ➵➸➄✟❿✠➎✠➊➺↔✞➄✚❽✮➍✚➈✜❻☛➙✞➄✟➇✄➎✠➊➻➆☛➇✠➄✟➌✷➄✆➆☛➇✠➎✠↔❛➙✞➄✚❻✁❿✄↕❑➛✠➈✜➎✠➛✠➎✞❽✥➄✆↔✒➍✚❾✚❿❑➙✞➄✼❾✚➛✠➛✞➅☛❻✁➄✚↔✒➆✁➎✒➙✠➈✜➎✠➊✵❽✥➄✼➆✁➇✄➈✜➎✠➉✄↕✠➇✢❾ ❽✮➄✚➆✵➎✠➜✷❾✚❽✥❽✮➎✠➍✆❻☛❾✆➆☛❻✁➎✠❿➼➈✜➉✄➅✁➄✚❽✲➐✎➡✩➇✄➄❞↔✠➎✞➌✷❾✆❻☛❿➼➍✆➎✠❿✄❽✮❻☛↔✞➄✚➈✜➄✚↔❬❻☛❽❍➆☛➇✄➄❞❾✚❿✄❾✆➅☛➀✦❽✮❻☛❽❍➎✠➜✷↔✠➎✠➊✵❿✄➅✁➎✠❾✆↔✠❽✰↔✞➎✠❿✄➄ ➜✏➈✏➎✞➌➽➆☛➇✠➄➾❽✮❻☛➆✁➄➚➎✠➜❸➆✁➇✄➄➚➪✾➎✠➈✜➆✁➉✄↕✄➉✄➄✆❽✥➄❬➶✰❾✚➆✁❻☛➎✞❿✄❾✚➅❼➦✜❿✄❽✮➆☛❻✁➆☛➉✠➆☛➄➚➎✠➜❸➯❂➆✁❾✚➆✁❻✁❽✥➆✁❻☛➍✆❽➹⑦✜➦➘➶✰➭ ⑦✜➊✵➊✶➊✒➐ ❻✁❿✄➄✫➐ ➛✞➆☛➴✁❻✁❿✄➜➩➎✞➅☛❻✁❿✄➄ ➐➋➡✩➇✠❻☛❽➹❽✥❻✁➆☛➄ ⑩ ➜✏➉✄❿✄➍✚➆✁❻✁➎✠❿✄❽✵➅☛❻✁➠✄➄✢❾✚❿❍➄✚➅✁➄✚➍✚➆✁➈✜➎✠❿✄❻✁➍✷❽✮➆☛➎✞➈✏➄✆➝✾➊✵➇✄➄✚➈✜➄✢➆☛➇✄➄✢➛✠➈✜➎✠↔✞➉✄➍✚➆✁❽✵❾✚➈✜➄➋➆✁❾✚➙✠➅✁➄✚❽ ⑩ ❻✁❿➋↔✞❻☛↕✠❻☛➆✁❾✚➅❈➜➩➎✞➈✏➌✢❾✚➆❈➊✵❻☛➆✁➇➋❽✮➆☛❾✆➆☛❻✁❽✥➆✁❻✁➍✚❽P❾✚➙✠➎✞➉✄➆❈➪✾➎✞➈✏➆✁➉✄↕✄❾✆➅★➐ ➷✾➈✏➎✞➌➬➆✁➇✄➄➋➊✵➄✚➙❍❾✚➍✆➍✚➄✚❽✮❽➤➅✁➎✠↕✄❽✶➎✠➜✕➆☛➇✄➄✷❽✥❻✁➆☛➄❈➮❧❽✶➇✄➆☛➆✁➛✰❽✮➄✚➈✜➃✄➄✚➈P➊✵➄➋➛✠➈✜➎✠↔✞➉✄➍✚➄✆↔❛❾✟❽✮➄✚➆✾➎✞➜❘❾✚❽✮❽✥➎✞➍✚❻✁❾✚➆✁❻☛➎✞❿ ➈✜➉✄➅✁➄✚❽✢➈✏➄✆➅☛❾✆➆☛❻✁❿✄↕❞➆☛➇✄➄✯➌✢❾✚❻✁❿➱➆✁➇✄➄✚➌✢❾✚➆✁❻☛➍✯➍✚❾✆➆☛➄✆↕✄➎✠➈✜❻✁➄✚❽✢➎✠➜✵➆☛➇✄➄✯↔✠➎✞➊✶❿✠➅☛➎✞❾✚↔✠➄✆↔❞➆✁❾✚➙✞➅☛➄✆❽➔➐❈➡✕➇✄❻✁❽➋❻✁❽➋❾✰➈✜➄✚➅✁❾✚➞ ➆✁❻✁➃✄➄✚➅✁➀❑❽✥➌✢❾✚➅✁➅✾❽✥➄✆➆✾❽✮➄✚➆❂➎✞➜◗➈✏➉✄➅✁➄✚❽✗⑦✜✃✳❐✞❐ ❻✁❿✄➃✄➎✞➅☛➃✠❻☛❿✄↕❑❒✒❻✁➆☛➄✆➌✷❽✗➆✁➇✄❾✆➆❂❽✥➄✆➈✏➃✄➄✆❽✗❾✚❽✗❾✆❿✢❻✁➅✁➅☛➉✄❽✮➆☛➈✜❾✚➆✁❻✁➃✄➄✼➄✚➢✄❾✆➌✷➞ ⑩ ➛✞➅☛➄❈➐➧➡✩➇✄➄❼❾✆❻☛➌✢❽✒➎✠➜✼➦➘➶✰➭➻❾✆➈✏➄✭➆✁➎❸❻☛➌✢➛✠➈✜➎✠➃✄➄✭➆✁➇✄➄✭➉✄❽✥❾✆➙✠❻✁➅☛❻✁➆✁➀➱➎✠➜✛➆✁➇✄➄✭❽✥❻✁➆☛➄✭➙✞➀➱↔✠❻✁❽✥➍✆➎✠➃✄➄✆➈✏❻✁❿✄↕✬➊✶➇✠❻☛➍✆➇ ❻✁➆✁➄✚➌✢❽✗❾✚➈✜➄✼➆☛➀✦➛✞❻☛➍✆❾✚➅✁➅☛➀✷➍✚➎✠➌✢➙✠❻✁❿✄➄✆↔✢➙✞➀➋➆✁➇✄➄✛❽✥❾✆➌✷➄✛➉✄❽✮➄✚➈➲➐❮➡✕➇✄➄✛➈✏➄✆❽✥➉✄➅✁➆☛❽❭➎✞➙✠➆✁❾✚❻✁❿✄➄✚↔❑➍✚❾✆❿✷➙✞➄✛➉✄❽✥➄✆↔✢❻✁❿✢➆☛➇✄➄ ➈✜➄✚❽✮➆☛➈✜➉✄➍✆➆☛➉✄➈✜❻✁❿✄↕➟➎✠➜✶➆☛➇✠➄✰❽✥❻✁➆✁➄✰➎✠➈✵❻✁❿➟➆☛➇✠➄✰❻✁❿✄➍✚➅✁➉✄❽✥❻✁➎✠❿❼➎✞➜◆➈✜➄✚➍✚➎✞➌✷➌✢➄✚❿✠↔✠❾✚➆✁❻✁➎✠❿❼➅✁❻☛❿✄➠✠❽✟➎✠❿❼❽✮➎✠➌✢➄❛➛✠❾✆↕✄➄✚❽✲➐ ➨✛➅✁➆☛➇✠➎✠➉✄↕✄➇❸➊✵➄❼❽✥➇✄➎✞➊❰➇✄➄✆➈✏➄❼➇✄➎✞➊❰➈✜➉✄➅☛➄✆❽✒❾✚➆✗➆✁➇✄➄✭➇✄❻✁↕✄➇✄➄✚❽✮➆✗➅☛➄✆➃✄➄✚➅✗➎✞➜✛➆☛➇✄➄✭➛✞➈✏➎✞↔✠➉✄➍✆➆☛❽❑➆✁❾✚➢✄➎✞❿✄➎✠➌✢➀✦➝✕❾ ❽✮❻☛➌✢❻✁➅☛❾✆➈✳❽✮➆☛➉✠↔✠➀✟➍✆➎✠➉✄➅✁↔✷➙✞➄✶➍✆❾✚➈✜➈✏❻✁➄✚↔✢➎✠➉✠➆✫➜✏➎✠➈✎➅✁➎✠➊✵➄✚➈✎➅☛➄✆➃✄➄✚➅✁❽➔➐ Rule Economics_and_Finance <= Population_and_Social_Conditions & Industry_and_Energy & External_Commerce Commerce_Tourism_and_Services <= Economics_and_Finance & Industry_and_Energy & General_Statistics Industry_and_Energy <= Economics_and_Finance & Commerce_Tourism_and_Services & General_Statistics Territory_and_Environment <= Population_and_Social_Conditions & Industry_and_Energy & General_Statistics General_Statistics <= Commerce_Tourism_and_Services & Industry_and_Energy & Territory_and_Environment External_Commerce <= Economics_and_Finance & Industry_and_Energy & General_Statistics Agriculture_and_Fishing <= Commerce_Tourism_and_Services & Territory_and_Environment & General_Statistics ♠✩Ï☛Ð✞♣❂Ñ✚s✵Ò✳Ó✠➷❂❻✁➈✏❽✮➆❈➛✠❾✚↕✠➄✶⑦✜❻☛❿✠↔✠➄✚➢ ⑩ Sup 0,038 0,036 0,043 0,043 0,040 0,036 0,043 Conf 0,94 0,93 0,77 0,77 0,73 0,62 0,51 A Post-Processing Environment 49 Ô✕Õ✄Ö❼×✜Ø✄Ù☛Ö✆Ú✒Û☛Ü❸Ý❂Û✁Þ✄Ø✄×✜Ö❼ß➫Ú✥Õ✠à✠á❰â✁Õ✄Ö❼ã✚à✞Ü✄â✁Ö✚Ü✄â✁Ú✒à✠ä✼à✞Ü✄Ö❼Û☛Ü✠å✠Ö✚æ❸ç✞è✚Þ✄Ö✆éPá✵Û✁â☛Õ❸à✠Ü✠Ö✯×✜Ø✄Ù✁Ö✭ä➩à✠×✼Ö✆è✚ã✆Õ ã✆à✠Ü✄Ú✮Ö✚ê✞Ø✄Ö✚Ü✄â◗ë✜ä➩×✜à✠ìíâ✁Õ✄Ö❑î✭Û☛â✁Ö✚ì✢Ú✥é✳à✠Ü✠Ù☛ï❍ð✯è✆ç✠ç✞Ö✚è✚×✜ñ➲ò✄Ô✕Õ✄Ö❑Ø✄Ú✥Ö✆×✗â☛Õ✄Ö✆Ü✰ä✏Û☛Ü✄å✞Ú✶â✁Õ✄Ö✢×✏Ø✠Ù☛Ö✢à✠Ü❍ó❮Ô✩Ö✆×✏×✜Û☛ô â✁à✠×✜ï✦õ✠è✆Ü✄õ✠ö✩Ü✄÷✄Û✁×✏à✞Ü✄ì✷Ö✆Ü✄â✁ø✷×✜Ö✚Ù✁Ö✚÷✄è✆Ü✄â✎ä➩à✞×PÚ✮â☛×✜Ø✄ã✆â☛Ø✄×✜Û✁Ü✄Þ✰â✁Õ✄Ö➋ã✆è✚â✁Ö✚Þ✄à✞×✏Û✁Ö✚Ú✶à✠Ü❛â✁Õ✄Ö✷Ú✥Û✁â☛Ö❈ò✲ù◆ï✒è✚ç✞ç✠Ù✁ï✦Û☛Ü✠Þ â✁Õ✄Ö❛ú✗à✞Ü✄Ú✥ûüà✞ç✠Ö✚×✜è✚â✁à✠×✜é◗Ú✥Õ✠Ö❛ã✚è✆Ü❼å✠×✜Û☛Ù✁Ù✩å✞à✠á✵Ü✯â✁Õ✄Ö✒Ù✁è✚â✁â☛Û✁ã✚Ö✒è✆×✏à✞Ø✄Ü✄å❼â✁Õ✄è✚â✩×✏Ø✄Ù✁Ö✚é✎à✞ý✠â✁è✚Û✁Ü✄Û☛Ü✠Þ✯è✆Ù☛Ù✩â☛Õ✄Ö ×✜Ø✄Ù✁Ö✚ÚPá✶Û✁â✁Õ➋è✵Þ✄Ö✚Ü✄Ö✆×✏è✆Ù☛Û✁þ✚Ö✆å✷è✆Ü✄â✁Ö✚ã✚Ö✆å✠Ö✆Ü✄â★ò Rule Sup Conf Territory_and_Environment <= Population_and_Social_Conditions & Industry_and_Energy & General_Statistics 0,043 0,77 Territory_and_Environment <= Population_and_Social_Conditions & Industry_and_Energy 0,130 0,41 Territory_and_Environment <= Population_and_Social_Conditions & General_Statistics 0,100 0,63 Territory_and_Environment <= Industry_and_Energy & General_Statistics 0,048 0,77 Territory_and_Environment <= General_Statistics 0,140 0,54 ✁✄✂✆☎✞✝✠✟☛✡✌☞✎✍✛ç✠ç✞Ù☛ï✦Û✁Ü✄Þ➋â✁Õ✄Ö✵à✠ç✞Ö✚×✜è✚â✁à✠×✎ú✗à✠Ü✄Ú✮û ë✜ã✚à✠Ü✠Ú✥Ö✚ê✞Ø✄Ö✚Ü✠â✫Þ✠Ö✚Ü✄Ö✆×✏è✆Ù☛Û✁þ✚è✆â☛Û✁à✠Ü✄ñ➲ò Ý✾×✏à✞ì Õ✄Ö✆×✏Ö✆é✟á✵Ö ã✚è✆Ü Ú✥Ö✆Ö â✁Õ✄è✚â✷✑ ó ✾ ✏ à✞ç✠Ø✄Ù✁è✚â✁Û✁à✠Ü✄õ✞è✚Ü✄å✞õ✓✒❂à✞ã✚Û✁è✚Ù✁õ✠ú✗à✠Ü✠å✠Û✁â☛Û✁à✠Ü✄Ú✮ø➼Û✁Ú➟Ü✄à✠â✷×✏Ö✆Ù☛Ö✆÷✄è✚Ü✠â☛Ù✁ï è✆Ú✥Ú✮à✠ã✚Û✁è✚â✁Ö✚å✒â✁à✒ó❮Ô✕Ö✚×✜×✏Û✁â☛à✞×✏ï✦õ✞è✚Ü✄å✞õ✠ö❘Ü✠÷✄Û☛×✜à✠Ü✠ì✷Ö✆Ü✄â☛ø❈ò✆Ô✩Õ✄Ö✼Ø✄Ú✮Ö✚×❘ã✆è✚Ü✢Ü✄à✠á✵é✫ä➩à✠×◗Ö✆æ✄è✚ì✢ç✠Ù✁Ö✚é✫Ù✁à✠à✓✢ ✔ Û✁Ü✄â☛à ×✜Ø✄Ù✁Ö✚Ú✵á✶Û✁â✁Õ✰✑ ó ➧ ✏ à✠ç✠Ø✠Ù☛è✆â☛Û✁à✠Ü✄õ✞è✚Ü✄å✞õ✓❂✒ à✞ã✚Û✁è✚Ù✁õ✠ú✗à✞Ü✄å✠Û✁â☛Û✁à✠Ü✠Ú✥ø✢ý✠ï✒è✚ç✞ç✠Ù✁ï✦Û☛Ü✠Þ❛â☛Õ✠Ö➋Ý✞✛ ✍ Ü✄â✳ë✏ä✏à✠ã✆Ø✄Ú➤à✞Ü❛è✚Ü✄â✁Ö✚ô ÿ ã✆Ö✚å✞Ö✚Ü✄â✁ñ➋à✞ç✠Ö✚×✜è✚â✁à✠×✷ë✏×✜Ö✚Ú✮Ø✄Ù✁â☛Ú❛Ü✠à✠â◆Ú✮Õ✄à✠á✵Ü❸Õ✄Ö✚×✜Ö✚ñ➲ò➧Ý❂×✏à✞ì â✁Õ✄Ö✚×✜Ö❼Ú✥Õ✄Ö❼ã✆à✠Ø✄Ù✁å➫Ú✥Ö✆Ö❼á✶Õ✠è✚â◆â✁Õ✄Ö❼ì✢è✚Û✁Ü è✆Ú✥Ú✮à✠ã✚Û✁è✚â✁Û✁à✠Ü✄ÚPâ☛à✢â✁Õ✄Û☛ÚPÛ✁â☛Ö✆ì➾è✆×✏Ö❈ò Ô✕Õ✄Ö❼ç✠×✜à✠ã✆Ö✚Ú✮Ú✒á✶à✞Ø✄Ù☛å➫â✁Õ✄Ö✆Ü❞Û✁â☛Ö✆×✏è✆â☛Ö✆é✕è✆Ù☛Ù✁à✠á✵Û✁Ü✄Þ❞â✁Õ✄Ö✭Ø✄Ú✮Ö✚×✼â✁à✬ä✏à✠Ù✁Ù☛à✞áíç✠è✆×✏â✁Û☛ã✆Ø✄Ù✁è✚×✼Û✁Ü✄â☛Ö✆×✏Ö✆Ú✥â✁Û☛Ü✠Þ ✕✏ â✁Õ✄×✜Ö✚è✆å✠Ú✢Û☛Ü❞â☛Õ✄Ö✯×✜Ø✄Ù☛Ö✯Ú✥ç✞è✚ã✆Ö✫ò ➧Ù☛à✞â☛Ú✷è✚Ü✄å❞ý✠è✚×✵ã✚Õ✠è✚×✜â☛Ú✷Ú✥Ø✄ì✢ì✷è✆×✏Û✁þ✚Ö❍â☛Õ✠Ö✰×✜Ø✄Ù☛Ö✆Ú➋Û✁Ü➟à✠Ü✠Ö✰ç✠è✆×✏â✁Û✁ã✚Ø✄Ù✁è✚× ✑✖ ç✞è✚Þ✄Ö❈ò✲Ô✩Õ✠Ö➋Ø✄Ú✮Ö✚×✩ã✆è✚Ü✒è✆Ù☛á✵è✚ï✦Ú◆×✜Ö✚â✁Ø✄×✜Ü✒â☛à✰è✚Ü✒Û✁Ü✄å✠Ö✆æ✒ç✠è✚Þ✠Ö✫ò➔Ô✩Õ✠Ö✟à✠ý ✹Ö✆ã✚â✁Û☛÷✠Ö✟Û☛Ú➤â☛à❛Þ✠è✚Û✁Ü✒Û☛Ü✄Ú✮Û☛Þ✠Õ✄â✾à✞Ü ✗✔ ✙✘➹Õ✠è✚â➤Û✁Ú❛è✆Ü ✚✔✄Ü✄à✞á✶Ù✁Ö✚å✞Þ✄Ö❑à✠äPâ☛Õ✄Ö â✁Õ✄Ö❞×✏Ø✄Ù✁Ö➱Ú✮Ö✚â✶ë✏è✆Ü✄å➼à✠Ü➫â✁Õ✄Ö➟å✞è✚â✁è✚ñ✷ý✠ï❸Ö✚æ✄è✆ì✷Û✁Ü✄Û✁Ü✄Þ➫å✠Û✁Þ✄Ö✆Ú✥â✁Û☛ý✞Ù☛Ö➟ã✆Õ✄Ø✄Ü ✠Ú❛à✠ä✟×✜Ø✄Ù✁Ö✚Ú✲ò Û✁Ü✄â✁Ö✚×✜Ö✚Ú✮â☛Û✁Ü✄Þ✯à✠×✗Ø✄Ü✄Û✁Ü✄â✁Ö✚×✜Ö✚Ú✮â☛Û✁Ü✄Þ✯×✏Ø✄Ù✁Ö❑å✠Ö✆ç✠Ö✚Ü✠å✠Ú✛à✠Ü✯â☛Õ✠Ö✢è✆ç✠ç✞Ù☛Û✁ã✚è✆â☛Û✁à✠Ü✯è✚Ü✄å✭â✁Õ✄Ö ✛✒ ☛✜ Ø✠Ú✥Ö✚×➲ò➲Ý✾à✠×✎ì✷à✞×✏Ö✵à✠Ü✷ì✷Ö✆è✚Ú✮Ø✄×✏Ö✆Ú✕à✞ä➧Û✁Ü✄â✁Ö✚×✜Ö✚Ú✮â☛Û✁Ü✄Þ✄Ü✄Ö✆Ú✥ÚPÚ✥Ö✆Ö✶ë ❂Û✁Ù☛ý✞Ö✚×✜Ú✥Õ✄è✆â☛þ ✣✢✬Û✁ã✚×✜à✠Ú✮à✠ä✏â✾ç✠Ù✁è✚â✁ä➩à✞×✏ì✢é Ô✕à✰å✞Ö✚÷✄Ö✆Ù☛à✞ç✰â✁Õ✄Û☛Ú✶á✶Ö✆ý✰Ö✆Ü✄÷✄Û✁×✏à✞Ü✄ì✷Ö✆Ü✄â✳á✶Ö✷ã✚Õ✄à✞Ú✥Ö➋è ✠✔ Ô✩Ø✄þ✆Õ✄Û✁Ù☛Û✁Ü✄ñ➲ò å✠Ø✠Ö✟â☛à✰â☛Õ✄Ö➋å✠Ö✚÷✠Ö✚Ù✁à✠ç✠ô ì✢Ö✚Ü✠â✾ý✠è✆ã ✄Þ✠×✏à✞Ø✄Ü✄å❛à✞ä❘â☛Õ✠Ö✟â☛Ö✆è✚ì✢é è✚Ü✄å✰è✚Ù✁Ú✥à❛ý✞Ö✚ã✚è✆Ø✄Ú✥Ö➋à✠ä❘â✁Õ✄Ö✟ç✞à✠Ú✥Ú✮Û☛ý✞Û☛Ù✁Û✁â☛Û✁Ö✚Ú◆à✞ä➩ä✏Ö✚×✜Ö✚å❛Û✁Ü✒â☛Ö✆×✏ì✢Ú✗à✠ä ✤✥✢✧✦➸å✠Ö✆÷✄Ö✚Ù✁à✠ç✞ì✷Ö✆Ü✄â✹ò✫Ô✩Õ✄Û✁Ú✟à✠ç✞â☛Û✁à✠Ü❼å✞à✠Ö✆Ú✟Ü✄à✠â✕ã✚à✞ì✷ç✞×✏à✞ì✷Û✁Ú✥Ö✒à✞Ø✄×◆Þ✄à✞è✚Ù❘à✞ä✗Õ✄è✚÷✄Û✁Ü✄Þ✭è✒ý✠×✜à✠á✵Ú✥Ö✆×✏ô ä✏×✏Ö✆Ö❑â☛à✞à✠Ù✹ò✄ú✗Ø✄×✜×✜Ö✚Ü✄â✁Ù☛ï✦é✳è✚Ù✁★ Ù ✾ ✏ ö✁✍☛✩✣❦✪ Ú✛ä➩Ö✚è✆â☛Ø✠×✏Ö✆Ú✵è✆×✏Ö❑Ú✮Ø✄ç✠ç✞à✠×✜â☛Ö✆å✯Û✁Ü✯ý✠à✠â✁Õ✣✰ ✫ Ö✚â✁Ú✥ã✚è✆ç✠Ö❑è✚Ü✠å✭✜✬ Ü✄â✁Ö✚×✜Ü✄Ö✚â ö✩æ✄ç✠Ù✁à✠×✜Ö✚×➲✮ ò ✜ ✬ Ü➹â☛Õ✄Ö❸ä✏à✠Ù✁Ù☛à✞á✶Û✁Ü✄Þ➹Ú✥Ö✚ã✆â☛Û✁à✠Ü✠Ú✯á✵Ö✬å✠Ö✚Ú✮ã✚×✜Û☛ý✞Ö❞â✁Õ✄Ö✬ì✷è✆Û☛Ü❬â☛Ö✆ã✚Õ✄Ü✠à✠Ù✁à✠Þ✄Û✁Ö✚Ú✯Û☛Ü✠÷✄à✠Ù✁÷✄Ö✚å➹Û✁Ü ✏➧ö✮✍✯➋ ✩ ò➲Ô✕Õ✄Ö✶Û✁Ü✄â✁Ö✚×✜è✚ã✆â☛Û✁à✠Ü✄ÚPè✚×✜Ö✵Ú✥Ø✄ì✢ì✷è✆×✏Û✁þ✚Ö✆å✷Û✁Ü➋Ý✾Û☛Þ✄Ø✠×✏☛ Ö ✳ ✰ ò ✱✆✲✴✳ ✷✶✠✝✎✸✓✹✺✸✆✻✽✼✿✾❁❀✞✼✽✟✎✝✠❀✞✟✎✼✿✾✺❀✞✻✛✸✓✝✎❂❄❃✓✼✛✷✸✓❀❄❅❆✟✠✝✠❇✆✟✠✝ ✘➸Ö✒Ø✄Ú✮Ö❑â☛Õ✄Ö✚✢❸Û✁ã✚×✜à✠Ú✥à✞ä➩★â ✜✬ Ü✄â✁Ö✚×✜Ü✄Ö✚★â ✜✬ Ü✄ä➩à✞×✏ì✢è✚â✁Û✁à✠❈ Ü ❂ ✒ Ö✚×✜÷✄Ö✚×✗ë✛✬✛✬✽✒❂ñ✗è✆✯ Ú ✾ ✏ ö✮✍✯✩✣✪❧Ú✛Õ✄â✁â✁ç✯Ú✮Ö✚×✜÷✄Ö✚×✗â✁à✭×✏Ø✠Ü â✁Õ✄Ö❉✍✛ã✚â✁Û☛÷✠Ö❈✾ ✒ Ö✚×✜÷✄Ö✚×✥➧✏ è✚Þ✄Ö✆Ú✢ë✛✍✯✒✞✾✏ ñ✼ä✏à✠×✼Ú✮Ö✚×✜÷✄Ö✚×✜ô➩Ú✮Û☛å✞Ö❍ç✞×✏à✞Þ✄×✏è✆ì✷ì✢Û✁Ü✄Þ✄é✩è✆Ù☛Ù✁à✠á✵Û✁Ü✄Þ➱å✞è✚â✁è✚ý✠è✆Ú✥Ö✯è✚Ü✄å ✤✥✢✧➫ ✦ ì✢è✚Ü✄Û✁ç✠Ø✠Ù☛è✆â☛Û✁à✠Ü❛è✆Ü✄å✰ä✏à✠×✜ì➬å✞è✚â✁è➋Ú✮Ø✄ý✠ì✢Û☛â✁â✁Ö✚å✰ý✞ï✒â☛Õ✠Ö➋Ø✄Ú✮Ö✚×➲ò❊✾✏ ö✮✍✯✩ è✆Ù☛Ú✮à❛×✏Ø✠Ü✄Ú◆à✠ä✏ä➩Ù✁Û☛Ü✠Ö✟á✶Û✁â✁Õ Ü✠à Ù☛Û✁ì✷Û✁â✁è✚â✁Û☛à✞Ü➾Ø✠Ü✄å✠Ö✆❋ × ✢ Û☛ã✆×✏à✞Ú✥à✠ä✏â●✾✏ Ö✆×✏Ú✮à✠Ü✄è✆Ù❍✘➸Ö✚ý✓✾✒ Ö✚×✜÷✄Ö✚×❼ë✛➸ ❸ ✘ Û☛Ü✄å✞à✠á✵Ú❸î✓✰✓✁■ ✓î ❏✆■✄❑✆▲✓▲✆▲✓■✷✬ ✢ Ö✆ñ❼à✠× Ø✠Ü✄å✠Ö✆×★✬ ✢ Û✁ã✚×✜à✠Ú✮à✠ä✏▼â ✾✏ Ö✚Ö✆×★➸ ✘ Ö✚ý✚❂✒ Ö✚×✜÷✄Û✁ã✚Ö✆ÚPë✛✘➸Û☛Ü✠å✠à✠á✵Ú✌✰ ✫ Ô✧➹ ✘ à✞×✽✔✠Ú✥â✁è✚â✁Û☛à✞Ü✄ñ ò✚Ô✩Õ✠Û☛Ú❭ì✢Ö✚è✚Ü✠ÚPÛ✁â❈ã✚è✆Ü➋ý✠Ö Û✁Ü✄Ú✮â☛è✆Ù☛Ù✁Ö✚å✢Û☛Ü✷è✚Ü✄ï✣✾ ✏ ú➼á✵Û☛â✁Õ➋è☛❸ ✢ Û☛ã✆×✏à✞Ú✥à✞ä➩â❈Ú✥ï✦Ú✮â☛Ö✆ì➾Û✁Ü➋Û✁â❈✽ë ➸ ✘ Û✁Ü✄å✠à✞á✶ÚPî✓❏✆■✄✢❸Ö✎■ ✰ ✫ Ô◆■✄❑✆▲✓▲✓▲✆■✄✤✥➧✏ ñ ò ✵ ✱✆✲✴❖ ✶✎✼✽✷❇✓✟☛❅✞✟✠✝✎❇✓✟✎✝❘◗✮❃✆✂✓✟✎✹◆❃✆❀✞❙❄❚✥❯✞❅❆✶✠✝✠✷❱✞✼ P ✍✛ã✆â☛Û✁÷✄❍ Ö ✒❂Ö✆×✏÷✄Ö✆❲ × ✏✾è✆Þ✄Ö✚Ú✼ë✛✍✯✒✞✏✾ñ➤ë✽❸ ✢ Û✁ã✚×✜à✠Ú✥à✞ä➩â✁ñ◆è✚×✜Ö❑å✠ï✦Ü✄è✆ì✷Û✁ã❑è✚Ü✄å✭Û✁Ü✄â✁Ö✚×✜è✚ã✚â✁Û✁÷✄Ö❑á✶Ö✆ý✯ç✞è✚Þ✄Ö✆Ú✛ç✠×✜à✠ã✚ô Ö✆Ú✥Ú✮Ö✚å➼à✞Ü✬â✁Õ✄Ö❼Ú✮Ö✚×✜÷✄Ö✚×✜ô➩Ú✮Û☛å✞Ö✚é❭â☛Õ✠Ø✄Ú✒Ø✄Ú✥Ö✆ä➩Ø✄Ù➤â☛à➫ì✢è✚Ü✄Û✁ç✠Ø✠Ù☛è✆â☛Ö❼å✞è✚â✁è❼Ú✥Ø✄ý✞ì✷Û✁â☛â✁Ö✚å➫ý✞ï❞Ø✠Ú✥Ö✚×✜Ú✒ë✜ä➩à✠×➋Û☛Ü✄ô 50 Jorge et al. ❳❁❨✄❩✎❬✗❭✠❪✎❫◆❳❁❪✠❴✷❪✠❭✎❨✄❵✷❬✗❛❜❩❈❳✺❪✎❨❞❝✓❡☛❩✠❳✺❳❁❝✓❭✎❵✄❩✎❨✄❵✷❝✓❬❣❢✽❤✗❴✷❪✠❳✐❛✗❵✷❥✗❪✠❬❣❭✠❪✎❢✽❨✷❩✠❵✷❬❜❢✛❪✠❳❁❨✄❢✛❵✄❭✎❨✄❵✷❝✓❬✓❳❧❦✆♠❋❨✄♥✓❪✭❤✗❳❁❪✠❢✛❳✺♦✯❩✠❳ ♣ ❪✠❴✷❴✕❩✠❳rq❧❩✎❬✗❵✷s✓❤✗❴✷❩✠❨✷❵✄❬✓❛✣t✓❩✎❨✄❩✎❦✓❩✎❳✺❪☛❢✽❪✎✉✓❤✗❪✎❳✺❨✷❳✇✈ ① ❬ ①✯②✞③ s✓❩✠❛✓❪☛❵✷❬✗❨✷❪✠❛✗❢✛❩✠❨✷❪✠❳❞④✥⑤◆⑥⑧⑦❣❨✄❩✎❛✗❳ ♣ ❵✷❨✷♥❧❳❁❭✠❢✛❵✄s✆❨▼❭✠❝✓q✐q✐❩✠❬✗t✆❳✇✈✠⑤✁♥✗❪✎❳✺❪☛❳✺❭✠❢✛❵✷s✓❨✷❳◆❭✎❩✠❬✣❦✆❪❘❪✎❵✄⑨ ❨✷♥✗❪✎❢★⑩✥❦ ② ❭✎❢✽❵✷s✓❨▼❝✆❢★❶❁❳❁❭✠❢✛❵✄s✆❨▼❷✽❶❸❩✠❥✗❩ ② ❭✠❢✛❵✷s✓❨▼❳✺❵✷q✐❵✄❴✷❩✠❢✛♦❹✈✠⑥✧❵✄❭✎❢✽❝✆❳✺❝✆❡❺❨▼❶ ② ❭✠❢✛❵✄s✆❨▼❵✄❳❞❩✎❬✣❝✓s✆❪✠❬❧❵✄q✐s✓❴✷❪✠q✐❪✠❬✗❨✷❩✠⑨ ❨✷❵✷❝✓❬❻❝✓❡✙❼❄❪✎❨✄❳❁❭✠❩✎s✓❪✠❽ ❳✥❶❸❩✠❥✗❩ ② ❭✠❢✛❵✷s✓❨ ♣ ♥✗❵✷❭✠♥❉❩✠❢✛❪❍❦✓❝✆❨✄♥❉❭✠❝✆q❧s✆❴✄❵✷❩✠❬✓❨ ♣ ❵✷❨✷♥❈❨✷♥✗❪❍❾✮❤✓❢✽❝✆s✓❪✠❩✎❬❉❿➀❝✓q✐s✓❤✗❨✷❪✠❢ ⑥✧❩✎❬✗❤✗❡❺❩✎❭✠❨✷❤✗❢✛❵✄❬✗❛ ① ❳✺❳❁❝✓❭✠❵✷❩✠❨✷❵✷❝✓❬✌➁✴❳☛❾✁❿➀⑥ ①✯② ❭✎❢✽❵✷s✓❨★⑦★❩✎❬✗❛✗❤✗❩✎❛✗❪ ② s✓❪✎❭✠❵✷❡❺❵✷❭✠❩✎❨✄❵✷❝✓❬✭❷✽❾✮❿➀⑥ ① ⑨❺➂✆➃✓➂❈❳✺❨✷❩✠❬✗⑨ t✆❩✠❢✛t➅➄❺♦❞➆➇♥✗❪✠❬✭❨✄♥✓❪❧s✆❩✠❛✗❪✐❵✄❳☛t✓❝ ♣ ❬✓❴✄❝✆❩✠t✓❪✎t✓❫✌❨✄♥✗❪✎❳✺❪✐❳✺❭✎❢✽❵✷s✓❨✷❳❘❩✎❢✽❪✐❪✠➈✓❪✠❭✠❤✓❨✄❪✎t✭❝✆❬❄❨✷♥✗❪ ① ❭✎❨✄❵✷❥✗❪ ② ❪✠❢✛❥✗❪✠❢ ③ ❩✠❛✗❪☛❪✠❬✓❥✗❵✄❢✛❝✓❬✓q❧❪✎❬✗❨✕❨✄♥✗❤✓❳◆s✆❢✽❝✆t✓❤✗❭✎❵✄❬✗❛❧❨✄♥✓❪❘❡✽❵✄❬✗❩✎❴✕④✥⑤✁⑥✧⑦❋❭✠❝✆t✓❪☛❨✄❝✐❨✷♥✗❪❘❢✛❪✠✉✆❤✗❪✠❳❁❨✄❵✷❬✗❛❧❦✓❢✛❝ ♣ ❳✺❪✠❢➉✈ Ú➉Û✽Ü✛Ý Þ ß✄àÐß Û✛á❮â ã ä➷å Ý æ✠ç✑è Ü✽è✕è✕í î➷éêá❮ï➷ð✎ë❮ñÐå ✚ Ú✜✛❆ð✠éêî➷ë✙✢ ✣ Þ ß❮àÐß Û✛á❮â ã ä➷å Ý æ✑ç✑è Ú✜✛✞æ ß å ß✙✤❁ß ì ë Ú➉Û✽Ü ✚ ✱Û ✰✠í✆â ë❮ì ï☎✢ å ✣ Ú➉Û✽Ü ✚ ✱Û ✠✰ í✆â ë★✲➷ï➷ë❮ì å ✣ Ü✛â éêá✄ë❮ì ì Ü✛è➛è✕í➛î➷éêá❮ï➷ð✎ë❮ñÐå ø Ú➉Û✛Ü✽Ý Þ ß❮àÐß Û✛á❮â ã ä➷å Ý æ✑ç✑è ò❹ó✄ô❒õêöê÷✷÷ ÷✄÷✷ô❒õêù ú❒û ù ô❒ü✕ý➉þ❒ÿ öê÷ Ú➉Û✛Ü ✂✁☎✄✝✆ ✞☎✁✠✟ ✡☛✟ ☞☎✌ ✍☎✎☎✞☎☞ ✏✠✑ ✑ ✍☎✒✝✆ ✁✠✟ ✆ ✍☎✓✕✔✖✎✠✗ ✡ ✑✠✑ ✘ ✁☎✒✙✡ Ú➉Û✛Ü✽Ý Þ ß❮àÐß Û✽á✄â ã ä➷å Ý æ✑ç✑è Ú➉Û✛Ü✛Ý Û✭✬✮✩✑Ý Þ ß✄à✷ß Û✽á✄â ã ä➷å ✚ æ✑ç✑è✯✣ ✩✑â ß✙✪ ãá ß ✢ Ú➉ñ ß ✢ ✫✄ì ã ì ✥✖✦ ➷ä éêâ å✛å é❊ñÐë★✧ Ü✛è➛è✕í î➷éêá✄ï➷ð✠ë❮ñÐå ➊✁➋✄➌✆➍✞➎✠➏☛➐✙➑✓➒ ❪✠❬✓❪✠❢✛❩✠❴✕❩✠❢✛❭✠♥✗❵✷❨✷❪✠❭✠❨✷❤✗❢✛❪❘❝✆❡ ③ ❾ ①✯➓ ✈ ③ ❾ ①✯➓ ❤✗❳❁❪✠❳r⑩✥❦ ② ❭✎❢✽❵✷s✓❨✕❷✛❵✄❬ ① ❭✠❨✷❵✄❥✗❪ ② ❪✎❢✽❥✗❪✎❢ ③ ❩✠❛✓❪✠❳✺♦✿❨✷❝❧s✆❢✽❝✆❭✠❪✎❳✺❳r❨✄♥✗❪☛❳✺❪✎❨✕❝✓❡✙❩✠❳✺❳❁❝✓❭✎❵✄❩✎❨✄❵✷❝✓❬❧❢✽❤✗❴✷❪✠❳ ❢✛❪✠s✆❢✽❪✎❳✺❪✎❬✗❨✄❪✎t❄❵✷❬●❩ ③ ⑥✧⑥⑧⑦➔t✆❝✓❭✎❤✗q❧❪✎❬✗❨✙❷✽❤✗❳❁❵✄❬✓❛●→✥❝✓❭✎❤✗q❧❪✎❬✗❨✙➣✥❦➉↔↕❪✠❭✎❨✌⑥✧❝✓t✆❪✠❴✷♦✽❫❆❨✄❝✭❩✠❴✷❴✄❝ ♣ ❨✷♥✗❪➙❤✗❳❁❪✠❢ ❨✷❝❍❦✓❢✛❝ ♣ ❳✺❪✥❨✷♥✗❢✛❝✓❤✗❛✓♥❧❵✷❨✄❫➛❩✠❬✓t✐❨✷❝✚❳✺❨✷❝✓❢✛❪✯❨✄♥✗❪✯❢✛❤✗❴✷❪✠❳❞❵✄❬✐❩✯❢✽❪✎❴✄❩✎❨✄❵✷❝✓❬✓❩✠❴▼t✓❩✎❨✄❩✎❦✓❩✎❳✺❪➛✈✠⑩✥❦ ② ❭✠❢✛❵✄s✆❨▼❵✄❳❞❤✗❳❁❪✠t✚❩✠❨ ❨✷♥✗❪☛❳✺❪✎❢✽❥✗❪✎❢✽⑨✽❳✺❵✷t✓❪☛❝✓❬✗❴✷♠❆✈ ➎✎➋✄➥❆➦ ➟✆➠✓➡✓➠✆➢✞➤ ❶❸❩✠❥✓❩ ② ❭✎❢✽❵✷s✓❨✷❫✌❷✛⑥✧❵✄❭✎❢✽❝✆❳✺❝✆❡❺❨★➆➇❪✠❦ ② ❵✷❨✄❪✎♦➀❵✄❳✯❤✗❳❁❪✠t❈❡❺❝✆❢rt✆❩✠❨✷❩❧q✐❩✠❬✓❵✄s✆❤✗❴✄❩✎❨✄❵✷❝✓❬✭❝✓❬✭❨✄♥✗❪✐❭✠❴✷❵✷❪✠❬✗❨✷⑨❺❳❁❵✄t✆❪✠❫❆❡✽❝✓❢ ❵✷❨✷❳❍s✓❝✓❢✛❨✷❩✠❦✓❵✷❴✷❵✄❨✷♠❆✈✌➆➧❪❻❨✷❢✛♠❣❨✷❝➔❤✗❳✺❪❻❝✆❬✗❴✷♠❣❭✎❝✓q✐q❧❩✎❬✗t✓❳✚❨✷♥✗❩✠❨➀❩✎❢✽❪❉❭✠❝✆q❧s✆❴✄❵✷❩✠❬✓❨ ♣ ❵✷❨✄♥⑧❾✮❿➀⑥ ①✯② ❭✠❢✛❵✷s✓❨➨✈ ⑤◆♥✗❵✷❳ ♣ ❩✎♠ ③ ❾ ①✯➓ q❧❩✎♠❉❢✽❤✗❦❜❤✗❬✗t✓❪✎❢★❼❄❪✎❨✄❳❁❭✠❩✠s✆❪●❩✠❬✓t❻➩✽❬✗❨✷❪✠❢✛❬✗❪✎❨✮❾✮➈✗s✆❴✄❝✆❢✽❪✎❢❹✈✆➆➇❵✄❨✷♥❉❶❁❩✎❥✗❩ ② ❭✠❢✛❵✄s✆❨ ♣ ❪ ❭✎❢✽❪✎❩✠❨✷❪➫❩✠❬✓t➭q❧❩✎❬✗❵✷s✓❤✗❴✷❩✠❨✷❪ ③ ⑥✧⑥✧⑦➯t✓❝✓❭✎❤✗q✐❪✠❬✗❨✷❳➔❝✓❢ ② ⑩ ➒ ❷✛❦✓❝✆❨✄♥➳➲✥⑥✧⑦➵t✆❝✓❭✎❤✗q❧❪✎❬✗❨✷❳✺♦❋❤✗❳❁❵✄❬✓❛ ➜✆➝✴➞ ➄❞➸❆➺★➻❄➼➾➽✛➚✮➪❸➶❄➽✴➶✇➹✴➘❸➴➷➶✎➪❸➹✴➽✴➬✇➶✇➪❁➮★➽✴➶✇➱✇✃✕➚✄➹✴➴➷❐❍➪➅➚❒➚❮➬✇❰❸➽✴➪❁➹❮➽✴➬✎➶✭ÏÐ➬✎✃✇➶✎➱✇➘❸➱❄➽✴➶✧Ñ↕Ò✇Ó✕Ñ➙➪❁➶✇➱❄➱✇➘❸➱✎➽✴❰❸➪❸➹✴➘❁➱✭➹✴➬❄➹❮Ô✎➘❉➚❮➹❮➪❁➶✇Õ ➱✎➪❸➴➷➱✎➽❮Ö❁➪❸➹✴➽✴➬✇➶×➬✇Ï✆➽❮➶✎ÏÐ➬✇➴➷Ø×➪❸➹✴➽✴➬✇➶×➪❸➶✎➱❘❰❁➬✇Ø×Ø❘✃✎➶✇➽✴❰❸➪❁➹✴➽❮➬✎➶✚➚❮❐➛➚❮➹❮➘❁Ø✚➚✛Ù A Post-Processing Environment 51 ✳✵✴✷✶✹✸✻✺✽✼✹✾☛✿❁❀✵❂✜❃✖✼❄✶✹✿❁❅❆✴✷❇☛✼✹❈❊❉●❋■❍✹❏✻❍❄❑▲✶✹▼✭◆✙❖✷✿❁◆◗P❘❍✹❈◗P✠✴❙◆✙✺✽❖☛✴✷▼✱✿◗❍✹✾✻✿❁❚❯✴☛▼❱❇☛❍✹✿◗❍❲❏✻❍❄❈✙◆◗❇☛❍❄✿✙◆◗✴☛✾❳❍✹✾☛❇❨❚❯✴☛▼❱◆✙✾✻❩ ✿◗✼✹▼✭❍✹✶❄✿✙◆◗✴☛✾❭❬❪◆◗✿◗❫❴✿◗❫✻✼❪✸☛P✠✼✹▼❛❵✭✼✹❏✻✼❄✾✻✿✯❫✻❍✹✾☛❇☛❈◗◆✙✾✻❜☛❝❞❉ ❡✷❢❤❣ ✐✵❥✷❦✹❧▲♠♦♥✹♣▲q❛r❲s✉t✜♥✹❦❄q✇✈①❥✷②▲♥✹③ ④⑤❫✻✼❙✳✵✴✷✶✹✸✻✺✽✼✹✾☛✿❪❀✵❂✮❃❊✼❄✶✹✿❪❅⑥✴✷❇☛✼❄❈❱❵✭✳✵❀✵❅⑥❝❭◆✙P⑦❍⑧✿✙▼✭✼✹✼❙P✠✿◗▼✱✸✻✶❄✿✙✸☛▼✱✼❄❩❯❂☛❍❄P✠✼❄❇⑨❍✹❖✷❖☛❈◗◆✙✶❄❍✹✿◗◆✙✴✷✾⑩❖☛▼✭✴☛❜✻▼✭❍✹✺ ◆◗✾✻✿◗✼✹▼✭❚❯❍❄✶✹✼✵❵✭❶❸❷❺❹✱❝❁❚❯✴☛▼❁❻✵④❁❅❆❼❽❍❄✾✻❇❲❾✵❅❆❼❽❇✷✴☛✶❄✸✻✺❭✼❄✾✻✿◗P❿◆✙P✂P✠✸✻✼❄❇✽❍❄P➀❍❸➁➃➂✷➄➆➅❿✼✹✶❄✴☛✺✽✺❭✼❄✾✻❇☛❍❄✿✙◆◗✴☛✾✽◆◗✾ ❀✵✶❄✿✙✴✷❂☛✼❄▼❱➇✜➈☛➈✷➉➊❵✱➁➃➂☛➄➋✳✵❀✵❅➌❼➍✼✹❏☛✼✹❈❁➇❪P✠❖✷✼✹✶✹◆◗❚❯◆◗✶✹❍❄✿✙◆◗✴☛✾☛❝❞❉✹❹✱✿➎◆◗P➀✸✻P✠✼❄❇➊✿✙✴➊❖☛▼✭✴☛✶❄✼✹P✂P⑤❍❄✾✻❇❭✺✽❍✹✾☛◆✙❖✷✸✻❈✙❍❄✿✙✼ ❍❄✾➏❾✵❅❆❼➐❇☛✴☛✶❄✸✻✺✽✼✹✾✻✿❲❂✷➑➒❍✹✶❄✶✹✼❄P✠P✠◆◗✾✻❜➓◆✙✿◗P❆◆✙✾☛✿✙✼❄▼✱✾✻❍❄❈❲P✠✿◗▼✱✸☛✶✹✿◗✸✻▼✱✼✯❉➔✳✵❀✵❅→▼✱✼❄❖☛▼✭✼✹P✠✼❄✾✻✿◗P❽❍❄✾➒❾✵❅❆❼ ❇✷✴☛✶❄✸✻✺❭✼❄✾✻✿➣❍❄P➔❍❘✿◗▼✱✼❄✼●❉✕❹✭✿✙P❱✾✻✴☛❇✷✼✹P➔❍❄▼✱✼❘✼❄❈✙✼❄✺❭✼❄✾✻✿◗P✠↔➎✿◗✼✹↕✻✿◗↔➎❍✹✾✻❇⑦P✠✴❲✴☛✾✇❉✹✳✵❀✵❅➙✺❭❍❄➛✻✼✹P❿◆◗✿▲✶❄✴☛✾✻❏✻✼❄✾✻◆◗✼✹✾✻✿ ❚✱✴☛▼❪❍✹❖☛❖✷❈✙◆◗✶✹❍❄✿✙◆◗✴☛✾❳❖✷▼✱✴✷❜✻▼✱❍❄✺❭P✵✿◗✴❨✿◗▼✱❍❄❏✻✼✹▼✭P✠✼❲✿◗❫✻✼❲✿◗▼✱✼❄✼❲❍✹✾✻❇❨❍❄✶✹✶✹✼❄P✠P✵✿◗❫✻✼❲✶✹✴✷✾✻✿◗✼✹✾✻✿◗P✵✴☛❚❿✿◗❫✻✼❲✿◗▼✱✼❄✼●❉✷④❁❫✻✼ ✳✵✴✷✶✹✸✻✺✽✼✹✾☛✿❿❀✵❂✜❃✖✼✹✶❄✿❿❅⑥✴✷❇☛✼❄❈❿❖☛▼✭✴☛❏✻◆◗❇☛✼❄P✽❍❳P✂✿✙❍❄✾✻❇☛❍❄▼✱❇❆❖☛▼✭✴☛❜☛▼✱❍❄✺❭✺✽◆✙✾☛❜❽✺✽✴☛❇✷✼✹❈❿❚❯✴✷▼✵❬❪✴✷▼✱➛☛◆✙✾✻❜❽❬❪◆◗✿◗❫ ❾✵❅❆❼❱❉ ❷❺➜➝❶➞➅➟✸✻P✂✼✹P✽✿✙❫✻✼➠❅❆◆✙✶❄▼✱✴✷P✠✴✷❚❯✿➀❾✵❅❆❼➡❖☛❍❄▼✱P✂✼✹▼❸❖☛▼✭✴☛❏✻◆◗❇☛✼❄❇❙◆◗✾⑧❅❆◆✙✶❄▼✱✴✷P✠✴☛❚✱✿⑤❹✭✾✻✿◗✼✹▼✭✾✻✼✹✿➢➜➝↕✻❖✷❈✙✴✷▼✱✼❄▼❪➤ ❵✭❍✹✾☛❇➥❍✹❂✷✴☛❏✻✼❄❝✱↔➎❬❸❫✻◆◗✶✹❫❲◆◗✺❭❖✷❈✙✼❄✺❭✼❄✾✻✿✙P❱✿✙❫✻✼❴➁➦➂✷➄➃✳✵❀✵❅➙P✂❖☛✼✹✶❄◆✙❚✱◆✙✶❄❍✹✿◗◆✙✴✷✾❺❉✕➁➃◆✙✿◗❫❲✿◗❫✻◆✙P❱❖☛❍✹▼✭P✠✼❄▼❁❬❸✼✵✶✹❍❄✾ ✼❄❍✹P✂◆✙❈◗➑⑦❍❄✶✹✶✹✼❄P✠P➞❍✹✾☛❇♦✺✽❍✹✾✻◆◗❖☛✸☛❈✙❍❄✿✙✼✽✿✙❫☛✼❭◆◗✾✻✿◗✼✹▼✭✾✻❍✹❈❛✿◗▼✱✼❄✼✹❩❯P✂✿✙▼✭✸✻✶❄✿✙✸✻▼✭✼✽✴☛❚➢❍✹✾⑦❾✵❅❆❼⑨❇✷✴☛✶❄✸✻✺❭✼❄✾✻✿❊❉➧❹✭✾⑦❖☛❍❄▼✱❩ ✿◗◆◗✶✹✸✻❈◗❍✹▼✭↔➎❬❪✼❴✸✻P✠✼❘✿◗❫✻✼❘✳✵❀✵❅➨✿◗✴➥▼✱✼❄❍✹❇➥❍❄✾✻❇➥✺✽❍✹✾✻◆◗❖☛✸☛❈✙❍❄✿✙✼❘✿◗❫✻✼❘✴✷▼✱◆◗❜✻◆◗✾✻❍✹❈❺❷➣❅❆❅❆❼❆❇☛✴☛✶❄✸✻✺✽✼✹✾✻✿▲❵✭❾✵❅❆❼ ❇✷✴☛✶❄✸✻✺❭✼❄✾✻✿▲✿◗❫✻❍✹✿➣▼✱✼❄❖☛▼✭✼✹P✠✼❄✾✻✿◗P❿❍❸❇✷❍✹✿◗❍➞✺❭◆◗✾✻◆◗✾✻❜❭✺✽✴☛❇✷✼✹❈◗❝✱↔●✿◗✴✽✼❄↕✻❖☛✴✷▼✱✿➎❍➞✾✻✼❄❬①❷➣❅❆❅❆❼❙❇✷✴☛✶✹✸☛✺❭✼❄✾✻✿➎❍✹✾✻❇ ❍❄❈✙P✂✴❭✿◗✴✽✶✹▼✭✼✹❍✹✿◗✼❸❍✹✾✻❇✽✺❭❍❄✾✻◆◗❖☛✸✻❈◗❍✹✿◗✼❪✿◗❫✻✼❸❜✻▼✭❍✹❖☛❫☛◆✙✶❄❍✹❈✯❏✻◆◗P✠✸✻❍❄❈✙◆◗➩✹❍❄✿✙◆◗✴☛✾❭❵✱❑▲➫✵➭➒❇☛✴☛✶❄✸✻✺✽✼✹✾✻✿◗P✠❝✜❉ ❡✷❢❤➯ ➲▲❦❄➳☛③◗➳☛s▲③◗♥❸➵✵♥✹❦✹q✭❥☛➸❸➺➊➸✹➳✷➻▲➼▲➽◗❦✹➾ ❑➣✶✹❍✹❈◗❍✹❂✷❈✙✼✵➫✵✼❄✶✹✿◗✴☛▼➝➭✵▼✭❍✹❖✷❫✻◆◗✶✹P❿❵✭❑▲➫✵➭✵❝➝◆◗P➢❍❄✾❭❾✵❅❆❼➍❩✱❂☛❍✹P✂✼✹❇➊❈◗❍✹✾✻❜✻✸☛❍✹❜✻✼➞✿◗❫✻❍✹✿➎P✂❖☛✼✹✶❄◆✙❚✱◆✙✼❄P➀❍✹✾✻❇➊❇☛✼❄❚❯◆◗✾✻✼✹P ❏☛✼✹✶✹✿◗✴☛▼❴❜✻▼✱❍❄❖☛❫✻◆◗✶✹P❲✿◗❫✻❍❄✿➔✶✹❍❄✾⑥❂✷✼❨❏✻◆◗P✠✸✻❍❄❈✙◆◗➩✹✼❄❇⑩❂☛➑⑥❍➠❬❸✼✹❂❆❂✷▼✱✴✷❬❪P✂✼✹▼✜❉⑤➚★➁➃➂☛➄➪➅❿✼❄✶✹✴✷✺❭✺✽✼✹✾✻❇✷❍✹✿◗◆✙✴✷✾✻➶✜❉ ➹▲➘ ❇☛✼❄❚❯◆◗✾✻✼✹P➴✿✙❫☛✼➐❚✱✼✹❍✹✿◗✸✻▼✭✼✹P➴❍✹✾✻❇➨P✠➑➧✾☛✿✙❍❄↕➷❚❯✴✷▼➬❑▲➫✵➭✵↔➆❍➐❈✙❍❄✾✻❜✻✸✻❍❄❜✻✼➮❚✱✴☛▼➓❇☛✼✹P✂✶✹▼✭◆✙❂✷◆✙✾☛❜➌✿✙❬❸✴☛❩ ❇✷◆✙✺✽✼✹✾☛P✠◆◗✴☛✾✻❍❄❈➣❏✻✼❄✶✹✿◗✴☛▼❁❍❄✾✻❇➥✺✽◆✙↕☛✼✹❇➥❏✻✼❄✶✹✿◗✴☛▼✭➱✙▼✭❍✹P✂✿✙✼❄▼❁❜✻▼✭❍✹❖✷❫✻◆◗✶✹P➔◆◗✾❲❾✵❅❆❼➍✃➎❉✕❑➣✴☛↔➎✸✻P✂◆✙✾✻❜➊❑▲➫✵➭➓◆◗P❿❏✻✼✹▼✭➑ P✂◆✙✺✽◆◗❈✙❍❄▼❸✿◗✴⑥❬❪✴✷▼✱➛✻◆◗✾✻❜❽❬❪◆◗✿✙❫❽❍✹✾✻➑❨✴✷✿✙❫☛✼✹▼❸✾✻✴☛▼✭✺❭❍❄❈⑤❾✵❅❆❼①❇✷✴☛✶✹✸☛✺❭✼❄✾✻✿✝❉➎❶❸✾❙❑▲➫✵➭❐❇☛✴✷✶✹✸✻✺✽✼✹✾✻✿➢✺✽✸✻P✠✿ ❍❄❈✙P✂✴❨❚❯✴✷❈✙❈◗✴☛❬➟✿◗❫✻✼❒✳✵④❁✳❮❵✱✳✵❍❄✿✙❍➊④❁➑➧❖✷✼➊✳✵✼✹❚❯◆◗✾✻◆◗✿✙◆◗✴☛✾☛❝❿✿✙❫✻❍❄✿➍P✠❖✷✼✹✶✹◆◗❚❯◆◗✼✹P➞✿◗❫✻✼➊❜✻▼✭❍✹❖☛❫☛◆✙✶➊✼✹❈◗✼✹✺✽✼✹✾☛✿✙P➞✿◗❫✻❍✹✿ ✶❄❍✹✾❭❂☛✼❸❖☛▼✭✴☛❇☛✸☛✶✹✼✹❇❛❉ ❶➞❜✻❍❄◆✙✾☛↔➠❬❸✼❰✶✹❍❄✾❮✺✽❍✹✾☛◆✙❖✷✸✻❈✙❍❄✿✙✼➬❑▲➫✵➭Ï❜✻▼✭❍✹❖☛❫☛◆✙✶❄P➦❬❸◆◗✿✙❫➴➫✵❂☛❑▲✶❄▼✱◆◗❖☛✿❨✴✷▼⑩❋✂❍❄❏✻❍✹❑➣✶✹▼✭◆✙❖✷✿❨❵✱✸✻P✂◆✙✾☛❜ ✳✵✴✷✶✹✸✻✺✽✼✹✾☛✿➢❀✵❂✮❃✖✼✹✶✹✿➀❅❆✴☛❇✷✼✹❈◗❝❞❉▲➁➃◆✙✿◗❫❙❑▲➫✵➭✵↔⑤◆✙✿➢◆◗P❴✼✹❍❄P✠➑❨✿◗✴❙❖✷▼✱✴✷❇☛✸✻✶❄✼⑦❍⑦❇✷❍✹✿◗❍⑦❏✻◆◗P✠✸☛❍✹❈◗◆✙➩❄❍✹✿◗◆✙✴✷✾⑧❍✹✾✻❇ ✼❄❏✻✼✹✾✽✺✽❍✹➛✻✼➞◆◗✿➎◆✙✾✻✿◗✼✹▼✭❍✹✶❄✿✙◆◗❏✻✼➞❵✭✶✹✴☛✾☛✿✙▼✭✴☛❈◗❈✙◆◗✾✻❜✽➛✻✼✹➑➧❂✷✴☛❍❄▼✱❇➊✴☛▼➍✺✽✴☛✸☛P✠✼➞✼✹❏✻✼❄✾✻✿◗P✠❝✜❉✮❷❺➜➝❶➞➅➆❜✻✼❄✿✙P➢❇☛❍❄✿✙❍❸❚❯▼✭✴☛✺ ❷❺❅❆❅⑥❼❙❍✹✾✻❇✽❖☛▼✭✼✹P✂✼✹✾✻✿◗P⑤◆◗✿✯✸✻P✠◆◗✾✻❜❭➫✵❂☛❑▲✶❄▼✱◆◗❖☛✿✯❍✹✾☛❇❭❑▲➫✵➭➒❜✻▼✱❍❄❖☛❫✻◆◗✶✹P✉❉ ❡✷❢❤Ð ✐✵➳✷q✱➳✷s▲➳☛➾✂♥❪➳✷♣▲②⑦➲▲Ñ➊Ò ➁➃✼❲✸✻P✂✼❲❍❲▼✱✼❄❈✙❍❄✿✙◆◗✴☛✾☛❍✹❈➝❇✷❍✹✿◗❍✹❂☛❍❄P✠✼❲❵✭❅❆◆✙✶❄▼✱✴✷P✠✴☛❚✱✿➝❶➞✶✹✶❄✼✹P✠P✂❝➔✿◗✴❨P✠✿◗✴☛▼✭✼❲✿✙❫✻✼❲❷❺❅❆❅⑥❼➦✺✽✴☛❇☛✼❄❈➍❍✹✾✻❇❳✿◗❍✹➛✻✼ ❍❄❇☛❏✻❍❄✾✻✿◗❍✹❜✻✼❽✴☛❚❘✸✻P✂◆✙✾☛❜⑩❑▲✿✙▼✭✸✻✶❄✿✙✸✻▼✭✼✹❇➆Ó✵✸✻✼✹▼✭➑❆❼➍❍✹✾✻❜✻✸☛❍✹❜✻✼⑧❵✭❑▲Ó✵❼➍❝❭✿✙✴⑨✴✷❂☛✿◗❍✹◆◗✾⑩P✠✼❄✿✙P➥✴✷❚❘❍✹P✂P✠✴✷✶✹◆◗❍✹✿◗◆✙✴✷✾ ▼✭✸✻❈◗✼✹P✉❉➧➄❿✴☛✺✽❖☛❍❄▼✱✼❄❇♦✿◗✴➠✸✻P✠◆◗✾✻❜⑦✳✵❀✵❅Ô❇✷◆✙▼✭✼✹✶❄✿✙❈◗➑➥✿◗✴⑦✺✽❍✹✾✻◆◗❖☛✸☛❈✙❍❄✿✙✼❭✿✙❫✻✼❭✴☛▼✭◆✙❜☛◆✙✾✻❍❄❈❺❷❺❅❆❅⑥❼⑩❇✷✴☛✶❄✸✻✺❭✼❄✾✻✿◗↔ ❑➣Ó✵❼➡❖☛▼✭✴☛❏✻◆◗❇☛✼❄P❴❍♦❚❯❍✹P✂✿✙✼❄▼❪❍❄✾✻❇❙✼❄❍✹P✂◆✙✼❄▼❪❍❄✶✹✶✹✼❄P✠P✉❉✯❹✱✾❙❷➣➜➝❶➞➅❿↔➝❍❄❈✙❈➢❇☛❍❄✿✙❍❄❂☛❍✹P✂✼⑦✶❄✴☛✾✻✾✻✼❄✶✹✿◗◆✙✴✷✾✻P❴❍❄✾✻❇❙▼✭✼✹❩ Õ ✸✻✼✹P✂✿✙P➢❍✹▼✭✼❸❇☛✴☛✾☛✼❪❬❸◆✙✿◗❫❴❶➞✶❄✿✙◆◗❏✻✼❸❑▲✼✹▼✭❏✻✼✹▼❛❷❺❍✹❜✻✼❄P⑤✴✷✾❴✿◗❫✻✼❪P✂✼✹▼✭❏✻✼✹▼❛P✂◆✙❇✷✼●❉ ❡✷❢❤❡ Ö✵♥❄➻▲➸✹♥❄➾✠♥✹♣➣q✱➽◗♣▲×✽Ø✵➾✠➾✠❥✷❦✹➽◗➳☛q✭➽✙❥✷♣▲➾➢Ö✵❧▲③✙♥❄➾⑤Ù❴➽✙q✭➼⑦Ú➝✈①✈①Ò ❷❺▼✱✼❄❇☛◆◗✶✹✿◗◆✙❏☛✼❲❅⑥✴✷❇☛✼❄❈➝❅❆❍✹▼✭➛✻✸✻❖❨❼➍❍✹✾☛❜✻✸✻❍✹❜☛✼❲❵✱❷❺❅⑥❅❆❼➍❝➔◆◗P✵❍✹✾❳❾✵❅❆❼➍❩❯❂✷❍✹P✂✼✹❇❨❈◗❍✹✾✻❜✻✸☛❍✹❜✻✼✯❉☛❶❮❷➣❅❆❅❆❼ ❇✷✴☛✶❄✸✻✺❭✼❄✾✻✿❱❖✷▼✱✴✷❏✻◆◗❇☛✼✹P⑦❍⑧✾✻✴☛✾☛❩❯❖☛▼✭✴☛✶❄✼✹❇✷✸✻▼✱❍❄❈❱❇✷✼✹❚❯◆◗✾✻◆◗✿✙◆◗✴☛✾⑩✴✷❚❘❚❯✸☛❈✙❈◗➑❆✿✙▼✭❍✹◆◗✾✻✼✹❇➆❇☛❍✹✿◗❍❨✺✽◆✙✾✻◆◗✾✻❜❆✺✽✴☛❇☛✼❄❈✙P ❬❸◆◗✿✙❫⑥P✠✸✻❚✱❚❯◆◗✶✹◆◗✼✹✾✻✿❿◆◗✾✻❚✱✴☛▼✭✺❭❍❄✿✙◆◗✴☛✾⑥❚❯✴✷▼✵❍✹✾⑥❍✹❖☛❖✷❈✙◆◗✶✹❍❄✿✙◆◗✴☛✾❽✿✙✴⑥❇☛✼❄❖☛❈◗✴☛➑⑧✿◗❫✻✼❄✺❳❉▲❹✱✿➀❖☛▼✭✴☛❏☛◆✙❇✷✼✹P✽❍♦❬❸❍✹➑⑧❚✱✴☛▼ 52 Jorge et al. Û✷Ü✹Ý✷Û☛Þ◗Ü❘ß✙Ý⑦à✠á✻â❄ã✱Ü❘ä✽Ý☛å✷Ü✹Þ◗à➔æ☛Ü❄ß✙ç❸Ü✹Ü❄è❲å☛é◗ê❯ê✱Ü✹ã✭Ü✹è✻ß❺â✹Û☛Û✷Þ✙é◗ë✹â❄ß✙é◗Ý☛è☛à✕ì✕í➍é◗î✻Ü❘â✹è☛ï✽ð✵ñ❆í❆å☛Ý✷ë✹ò✻ä✽Ü✹è☛ß✙ó✯â✹Þ◗à✠Ý â❲ô❺ñ❆ñ⑥í➋å☛Ý✷ë✹ò✻ä✽Ü✹è✻ß➍ä✽ò✻à✂ß➍ê❯Ý☛Þ◗Þ◗Ý☛ç❰â➊õ✵â✹ß◗â➊ö❁ï➧Û☛Ü➊õ✵Ü❄ê❯é◗è✻é✙ß◗é◗Ý☛è➠÷✱õ✵ö⑤õ✵ø❿ß✙á✻â❄ß➍å☛Ü❄ê❯é◗è✻Ü✹à➞ß◗á✻Ü➊Ü✹è✻ß◗é✙ù ß◗é◗Ü✹à❿â✹è☛å✽â❄ß✙ß◗ã✭é✙æ✷ò✻ß✙Ü❄à➀ê❯Ý☛ã➍å✷Ý☛ë❄ò✻ä❭Ü❄è✻ß◗é✙è✻ú✽â➞à✠Û✷Ü✹ë✹é◗ê❯é◗ë➞å☛â❄ß✙â➞ä✽é✙è✻é◗è✻ú✽ä❭Ý✷å☛Ü❄Þ✝ì✹û▲Ý☛ã➍é◗è✻à✂ß✙â❄è✻ë✹Ü❄ó✷ß◗á✻Ü✹ã✭Ü➞é✙à ✁❱â✹ï➧Ü✹à ✁✂❸ü➏ä❭Ý✷å☛Ü❄Þ➍â✹è✻å❳à✠Ý❳Ý✷è❺ì✄✂➞è✻ï☎✂➞ü➡ä❭Ý✷å☛Ü❄Þ✇ç❸ã✭é✙ß◗ß✙Ü❄è⑦é◗è⑦ô❺ñ⑥ñ❆í⑨æ✷ï å✷é✙ê✱ê❯Ü❄ã✱Ü❄è✻ß✯Ü✹è✻ß◗é✙ß◗é◗Ü✹à➢ä❭ò✻à✂ß✯ê❯Ý☛Þ◗Þ◗Ý☛ç➃ß✙á☛Ü❪à✂â✹ä✽✆ Ü ➞ ✂ ü⑨à✂Û☛Ü✹ë❄é✙ê✱é✙ë❸õ✵ö❁õ⑦ì ✂①ä✽Ý☛å✷Ü✹Þ✯å☛Ü❄à✠ë❄ã✱é◗æ☛Ü❄å❭ò✻à✂é✙è☛ú❴ô➣ñ❆ñ❆í⑧á✻â❄à⑤ß◗á✻Ü❸ê❯Ý✷Þ✙Þ◗Ý☛ç❸é✙è☛ú❴à✠ß◗ã✭ò✻ë✹ß◗ò✻ã✭Ü✞✝ Ý✷è✻Ü❨õ✵ö❁õ❒ß◗Ý⑩à✠Û☛Ü❄ë✹é◗ê❯ï⑥â❨ü❿Ü✹ú✻ã✭Ü✹à✂à✠é◗Ý☛è❆ä✽Ý☛å☛Ü❄Þ✙ý❱â✹è✻Ý✷ß✙á✻Ü❄ã❘õ✵ö❁õ➐ß✙Ý❆ã✭Ü✹Û✷ã✱Ü❄à✠Ü✹è☛ß❿â❴þ⑦â❄é✙ÿ☛Ü ä✽Ý☛å✷Ü✹Þ◗ý➍Ý☛ß◗á✻Ü✹ã❿ß◗Ý❳å☛Ü❄ê❯é◗è✻Ü➊â✹è ✟ 1) A header, 2) A data schema, 3) A data mining schema, 4) A predictive model schema, 5) Definitions for predictive models, 6) Definitions for ensembles of models, 7) Rules for selecting and combining models and ensembles of models, 8) Rules for exception handling. ✡✠ ☞☛ Ý✷ä❭Û✷Ý☛è✻Ü❄è✻ß✯÷ ✷ø❛é✙à➢ã✱Ü ☛ò✻é◗ã✭Ü✹å✇ì✮ö❁á✻Ü❸Ý☛ß◗á✻Ü❄ã✇ë❄Ý☛ä✽Û☛Ý✷è✻Ü✹è✻ß◗à➢â✹ã✭Ü❪Ý✷Û☛ß◗é✙Ý✷è✻â✹Þ❊ì ö⑤á✻Ü⑦ä✽â✹é◗è⑧ã✭Ü✹â❄à✠Ý☛è☛à❴ß◗á✻â✹ß➢å☛ã✭Ý☛ÿ✻Ü♦ß✙á☛Ü⑦ê❯Ý✷ã✱ä✽ò✻Þ◗â✹ß◗é✙Ý✷è⑧Ý☛ê❱ß✙á✻Ü⑦ô➣ñ❆ñ❆í➦ê✱Ý☛ã❪Û☛ã✭Ü✹å☛é◗ë✹ß◗é◗ÿ✻Ü➥ä✽Ý☛å☛Ü❄Þ✙à ✍✌ ✏✎ ç❸Ü✹ã✭Ü➞ß✙á☛â✹ß➎é◗ß➎ä❭ò✻à✂ß✯æ☛Ü❸ò✻è✻é◗ÿ✻Ü✹ã✭à✠â❄Þ✙ó✷Ü ✻ß◗Ü✹è✻à✂é✙æ✷Þ✙Ü❄ó☛Û✷Ý☛ã✭ß✙â❄æ☛Þ◗Ü❪â❄è✻å❭á☛ò✻ä❭â❄è❴ã✭Ü✹â❄å☛â✹æ✷Þ✙Ü✯ì ✭ß✯â✹Þ◗Þ✙Ý✷ç❪à➢ò✻à✠Ü❄ã✱à ß◗Ý➊å☛Ü❄ÿ✻Ü✹Þ◗Ý☛Û➊ä✽Ý☛å☛Ü❄Þ✙à➀ç❸é✙ß◗á✻é◗è❭Ý✷è✻Ü➞ÿ✻Ü✹è☛å☛Ý☛ã à➀â✹Û☛Û✷Þ✙é◗ë✹â❄ß✙é◗Ý☛è☛ó☛â✹è☛å❭ò✻à✂Ü❪Ý✷ß✙á☛Ü✹ã❛ÿ✻Ü✹è☛å☛Ý☛ã✭à ✜â❄Û☛Û✷Þ✙é◗ë✹â❄ß✙é◗Ý☛è✻à ✕✔ ✒✑ ✖✔ ✓✑ ß◗Ý➥ÿ✻é◗à✠ò☛â✹Þ◗é ❄Ü✹ó➎â✹è☛â✹Þ◗ï ✹Ü✹ó➎Ü❄ÿ✻â✹Þ◗ò✻â❄ß✙Ü❘Ý✷ã❁Ý✷ß✙á✻Ü❄ã✱ç❸é◗à✠Ü❘ò✻à✂Ü❘ß◗á✻Ü❘ä✽Ý☛å☛Ü❄Þ✙à✉ì✕ô➣ã✭Ü✹ÿ✻é◗Ý☛ò☛à✠Þ◗ï➧ó➎ß✙á✻é◗à➔ç❸â✹à❿ÿ✻é◗ã✭ß✙ò✻ù ✍✌ â❄Þ✙Þ◗ï❲é◗ä❭Û✷Ý☛à✠à✂é✙æ✷Þ✙Ü❄ó▲æ☛ò☛ß❺ç❸é✙ß◗á❲ô➣ñ❆ñ❆í➍ó➎ß◗á✻Ü❘Ü ☛ë✹á✻â❄è✻ú✻Ü❘Ý✷ê➝ä❭Ý✷å☛Ü❄Þ✙à➔æ✷Ü✹ß◗ç❪Ü❄Ü✹è❲ë❄Ý☛ä✽Û☛Þ◗é✙â❄è✻ß➣â❄Û☛Û☛Þ◗é◗ë✹â✹ù ß◗é◗Ý☛è✻à❿è✻Ý✷ç➡ç❸é✙Þ◗Þ▲æ☛Ü✵à✂Ü✹â❄ä❭Þ◗Ü✹à✂à✕ì ➞ß▲ß✙á☛é✙à❿ä✽Ý☛ä✽Ü✹è✻ß◗ó✯Ý☛è✻Þ◗ï✽â✵ê❯Ü✹ç➒å☛â✹ß◗â✵ä✽é✙è✻é◗è✻ú➊ß◗Ý☛Ý☛Þ◗à➀â✹è✻å➊â❄Û☛Û☛Þ◗é◗ë✹â✹ù ☞✂ ☞✌ ß◗é◗Ý☛è✻à❭â✹Þ◗Þ✙Ý✷ç❪à❘ß◗Ý⑧Ü ✻Û☛Ý✷ã✱ß⑤ß✙á✻Ü❄é✙ã❪ä❭Ý✷å☛Ü❄Þ✙à❘ß◗Ý⑧ô❺ñ⑥ñ❆í➍ó➍æ✷ò✻ß❁é◗à❘ò✻ã✭ú✻Ü✹è☛ß❁ß◗Ý⑧é✙ä✽Û☛Þ◗Ü✹ä✽Ü✹è☛ß❁é◗ß❁é◗è❨Ý☛ß◗á✻Ü✹ã à✂Ý☛ê✱ß✙ç❸â✹ã✭Ü♦ß◗Ý☛Ý✷Þ✙à✽ß◗Ý❽à✂â✹ß◗é✙à✂ê❯ï❨å☛ã✭â✹ä✽â✹ß◗é◗ë✹â✹Þ◗Þ◗ï❨é✙è✻ë❄ã✱Ü❄â✹à✂é✙è✻ú❙ã✱Ü ☛ò✻é◗ã✱Ü❄ä❭Ü❄è✻ß◗à❴ê❯Ý✷ã❪à✂ß✙â❄ß✙é◗à✠ß◗é◗ë✹â✹Þ➢â✹è☛å❙å☛â❄ß✙â ☞☛ ä✽é◗è✻é✙è☛ú❴ä✽Ý☛å☛Ü❄Þ✙à➢é◗è❴æ☛ò☛à✠é◗è✻Ü✹à✂à⑤à✂ï➧à✠ß◗Ü✹ä✽à✕ì ✘✗✙✂ ✚✂ ✛✂ ô ➞ü❮ë❄â✹è❆ã✭Ü✹â✹å❆â❄è ➞ü➪ä✽Ý☛å✷Ü✹Þ❿à✂Û☛Ü✹ë❄é✙ê✱é✙Ü❄å❆é✙è⑥â❳ô➣ñ❆ñ❆í➒å☛Ý✷ë✹ò✻ä✽Ü✹è☛ß✝ì➣ö❁á✻Ü❳ò✻à✂Ü✹ã✵ç❸é✙Þ◗Þ❿æ☛Ü â❄æ☛Þ◗Ü➊ß✙Ý❳ä✽â✹è✻é◗Û☛ò✻Þ◗â✹ß◗Ü➊ß✙á☛Ü ➞➏ ü ä❭Ý✷å☛Ü✹Þ◗ó✇ë✹ã✭Ü✹â✹ß◗é◗è✻ú♦â➊è✻Ü❄ç❰ã✱ò☛Þ✙Ü➊à✠Û✷â✹ë❄Ü✽æ✷â✹à✂Ü✹å❳Ý☛è➠â➊à✠Ü✹ß❛Ý✷ê⑤Ý✷Û☛Ü✹ã✭â✹ù ✍✌ ß◗Ý☛ã✭à✠ó✷â✹è✻å✽Ü ☛Û☛Ý☛ã✭ß✯â❪à✂ò✻æ☛à✂Ü✹ß✯Ý☛ê✇à✠Ü❄Þ✙Ü❄ë✹ß◗Ü✹å✽ã✱ò☛Þ✙Ü❄à⑤ß◗Ý❭â❸è✻Ü❄ç➦ô❺ñ❆ñ⑥í❙å☛Ý☛ë❄ò✻ä✽Ü✹è✻ß❊ì ✕✔ ✜✔ ✢✎ ö⑤á✻Ü✹ã✭Ü❳é✙à➊à✂Ý☛ä✽Ü➠ç❸Ý☛ã✭î⑥Ý☛è⑥ß✙á✻Ü❳ÿ✻é◗à✠ò☛â✹Þ◗é ❄â✹ß◗é✙Ý✷è❽â❄è✻å❆à✠ò✻ä✽ä❭â❄ã✱é ✹â❄ß✙é◗Ý☛è⑥Ý☛ê➞â❄à✠à✠Ý✷ë✹é◗â✹ß◗é✙Ý✷è❽ã✭ò✻Þ◗Ü✹à✉ì ✭è ß◗á✻é◗à⑤à✂Ü✹ë❄ß✙é◗Ý☛è❭ç❪Ü❸ã✱Ü❄ê❯Ü❄ã✇ß◗Ý✽à✠Ü✹Þ◗Ü✹ë❄ß✙Ü❄å❭ç❸Ý☛ã✭î❴Ý✷è❴ß◗á✻Ü✹ä✽Ü●ì ö⑤á✻Ü✵à✠ï➧à✂ß✙Ü❄ä➟õ ▲ù ➦÷✭ñ❆â✵Ü✹ß▲â❄Þ✝ì ø➝ò✻à✠Ü❄à❿ß✙á☛Ü✵à✠â✹ä✽Ü✵à✠Ý✷ã✱ß➣Ý☛ê➍â✹Û✷Û☛ã✭Ý☛â❄ë✹á➊â✹à❿ß◗á✻Ü✵Ý☛è☛Ü✵ç❪Ü➞Û☛ã✭Ý☛ù ★✎ ✤✣ ✡✥✦✗✧ ✤✣ ✩✥✪✗✙ ✫✗✙✂ Û✷Ý☛à✂Ü❲á✻Ü✹ã✭Ü●ì ✭è➠ë❄Ý☛ä✽ä❭Ý✷è✻ó❛õ ▲ù ➬â❄è✻å❨ô ➞ü➓á✻â✹ÿ☛Ü❲ß✙á✻Ü❲â❄é✙ä➮Ý☛ê❿Û☛Ý✷à✠ß➝Û✷ã✱Ý✷ë✹Ü❄à✠à✠é◗è✻ú➠â➊Þ✙â❄ã✱ú✻Ü à✂Ü✹ß➎Ý✷ê ➞ü⑨ß◗á✻ã✭Ý☛ò✻ú✻á❭ç❪Ü❄æ❭æ✷ã✱Ý✷ç❪à✂é✙è☛ú❴â✹è☛å❭ÿ✻é◗à✠ò☛â✹Þ◗é ❄â✹ß◗é✙Ý✷è❺ì✜õ ▲ù ➆ã✭Ü✹Þ◗é✙Ü❄à⑤Ý✷è❴ß◗á✻Ü❸Û☛ã✭Ü✹à✠Ü❄è✻ß◗â✹ß◗é✙Ý✷è ✘✂ ✕✔ ✤✣ ✡✥✪✗✙ ✤✣ Ü☞✌✻Û☛Þ◗Ý☛ã✭Ü➠ß✙á✻Ü➠ÿ✻â❄ã✱é◗â✹ß◗é✙Ý✷è✻à✽Ý☛ê❸Ü✹â✹ë❄á❙Ý✷è✻Ü♦Ý✷ê❪ß◗á✻Ü❄à✠Ü➠õ✤✣➆ã✭ò✻Þ✙Ü❄à✕ì✬✭✎ è⑧Ý✷ò✻ã❪â❄Û☛Û✷ã✱Ý✷â✹ë✹á☛ó➝á✻Ý☛ç❸Ü✹ÿ☛Ü✹ã✭ó➝ç❪Ü ã✭Ü✹Þ◗ï❴Ý✷è❭â➞à✂Ü✹ß➎Ý☛ê❛ú✻Ü❄è✻Ü✹ã✭â✹Þ➎Ý✷Û☛Ü❄ã✱â❄ß✙Ý✷ã✱à➀ß◗á✻â✹ß➎ë❄â✹è✽æ☛Ü➞â✹Û✷Û☛Þ◗é✙Ü❄å✽ß◗Ý➊â✹è✻ï❭ã✱ò☛Þ✙Ü❄ó✷é◗è✻ë✹Þ◗ò✻å✷é✙è✻ú✽õ✤➠ ✣ ã✱ò✻Þ◗Ü✹à➢â✹à å✷Ü✹ê✱é✙è✻Ü❄å➥ê❯Ý✷ã➝õ✤✣▲ù✡✥✦✗✧❭ì❄ö❁á✻Ü✵à✂Ü✹ß▲Ý✷ê➍Ý☛Û☛Ü❄ã✱â❄ß✙Ý✷ã✱à❿ç❸Ü✵å☛Ü❄ê❯é◗è✻Ü✵é◗à❿æ☛â✹à✂Ü✹å❲Ý✷è✽à✂é✙ä✽Û☛Þ◗Ü✵ä✽â✹ß◗á✻Ü✹ä✽â✹ß◗é✙ë❄â✹Þ Û✷ã✱Ý✷Û☛Ü❄ã✱ß◗é✙Ü❄à➔Ý☛ê❁ß✙á✻Ü❴é✙ß◗Ü✹ä✽à✠Ü❄ß✙à➔â❄è✻å➥á✻â❄ÿ✻Ü❘â❘ë❄Þ✙Ü❄â✹ã⑤â✹è✻å➥é◗è✻ß◗ò✻é◗ß✙é◗ÿ✻Ü❘à✂Ü✹ä✽â✹è✻ß◗é✙ë❄à✕ì✕ô✘✗✙➞ ✂ ü➦â❄Þ✙à✂Ý➥á✻â❄à❿ß✙á✻Ü â❄å☛å✷é✙ß◗é✙Ý✷è✻â✹Þ✯Û✷Ý☛à✠à✂é✙æ✷é✙Þ◗é◗ß✙ï❘Ý✷ê❺ã✭Ü✹â❄å☛é◗è✻ú✭✂➞ü⑨ä✽Ý☛å✷Ü✹Þ◗à⑤â❄à⑤ô❺ñ❆ñ⑥í❱ì ✮ é✜✔✍✥➃é✜♦✔ é◗à❴ß◗á✻Ü♦è✻Ý☛è✻ù✱Ý☛ê✱ê❯é◗ë✹é◗â✹Þ➢è✻â✹ä✽Ü⑦ê✱Ý☛ã❸â⑦ô➣ñ❆ñ❆í①é◗è✻ß✙Ü❄ã✱â❄ë✹ß◗é✙ÿ☛Ü⑦ä✽Ý☛å☛Ü❄Þ⑤ÿ☛é✙à✂ò✻â✹Þ◗✕é ❄✔ Ü✹ã❸é✙ä✽Û☛Þ◗Ü✹ù ä✽Ü✹è☛ß✙Ü❄å⑧é✙è✰✯■â✹ÿ☛â➥÷✡➃ ✥ Ü❄ß✙ß◗à✠á✻Ü❄ã✱Ü❄ë✹î✻ø✜★ì ✱✎ ß❁ú✻ã✱â❄Û☛á✻é◗ë✹â❄Þ✙Þ◗ï➠å☛é◗à✠Û☛Þ◗â✹ï➧à✂ó❛è✻Ý☛ß➝Ý✷è✻Þ◗ï♦â❄à✠à✠Ý✷ë✹é◗â✹ß◗é✙Ý✷è❳ã✱ò✻Þ◗Ü✹à✂ó❛æ☛ò✻ß Ý✷ê❱â♦ã✱Ü❄å☛ò✻ë❄Ü✹å❙à✂Ü✹ß➢Ý☛ê❪ã✱ò✻Þ◗Ü✹à✂ó➝ë✹â❄Þ✙Þ◗Ü✹å❽å☛é◗ã✱Ü❄ë✹ß◗é✙Ý✷è⑧à✠Ü❄ß✙ß◗é◗è✻ú⑧Ý☛ã❸õ ⑨ã✭ò✻Þ◗Ü✹à✠ó❁â✹è✻å❽ß✙á✻Ü❄è⑧ß◗á✻Ü⑦ò✻à✂Ü✹ã❸ë✹â❄è A Post-Processing Environment 53 ✱✛✲✍✳✵✴✷✶✵✸✜✹✄✺☞✻✽✼✵✲✍✸✜✲✾✱❀✿✜✳✄✿✜✳✄❁✁✱❀✶★✼✵✺✍❂✜❃❅❄✵❆✧✹✵✺✛❇★✹✄✿✜❂✕✶★❃✓✶✵❇★✹✄✴✷✶★❈❊❉✪✿✜❋✍❉✪✿✜❋❀❈✡✶✵✻●✼✵✿✜❃✓❇★❂✕✲☞✴✖✿✕✳✄❁☎❍❏■❑✻✡✺☞❂✕✿✜✺✍❃ ✶★✳❀✸✜✹✄✺❏❇✵✻✒✺✍❃▲✺✍✳✄✸✜✲✍✸✜✿✜✶✵✳✛✶✵❈▼✸✕✹✵✺◆❂✜✿✜❃✓✸✢✶✵❈❖✻✒P✄❂✕✺☞❃✓◗★✲✍❂✜❂✕✶★❘✆✿✜✳✄❁✭✸✜✹✄✺◆P✄❃✓✺☞✻▼✸✕✶✛❃✓✺☞✸✞✸✜✹✄✺◆✱❀✿✜✳✄✿✜✱❀✲☞❂✬❃✓P✄❇★❇✵✶✵✻✒✸✬✲✍✳✄✼ ❙☞✶✵✳✄❈✡✿✕✼★✺✍✳✄❙☞✺✚✸✜✹✄✻✒✶✵P✄❁✵✹❯❚✄✺☞✻✡✴❱✿✜✳✄✸✜P✄✿✜✸✕✿✜❚✄✺❲❁✄✲✍P✄❁✵✺✍❃❅❄✙❳✤✿✕❋☞❉✪✿✕❋❲✲✍❂✜❃✓✶✦✲✍❙☞❙✍✶✵✱✛❇✵✲☞✳✄✿✜✺✍❃✁✸✕✹✄✺❲✼✵✿✜❃✓❇★❂✕✲☞✴❱✶✵❈ ✺☞✲✍❙☞✹✚✻✒P✄❂✜✺✁❨★✴❩❙☞✶✵❂✜✶✵✻✤❨✵✲☞✻✡❃✾✻✒✺✍❇★✻✡✺☞❃✓✺✍✳✵✸✕✿✜✳✄❁❲❃✓P✄❇★❇✵✶✵✻✒✸✽✲✍✳✵✼✚❙☞✶✵✳✄❈✡✿✕✼★✺✍✳✄❙☞✺✞❄✢❆❊✹✄✿✜❃❀❚✄✿✜❃✓P✵✲✍❂✜✿✕❋☞✺✍✻❏❙✍✲☞✳❩❨✵✺ P✵❃✓✺✍✼✛✼✵✿✜✻✒✺✍❙✍✸✜❂✜✴❬✿✕✳❀✲✆❘◆✺✍❨✛❨✵✻✒✶✵❘◆❃✓✺☞✻▼✲✍❃❭✲✙❪❫✲✍❚✵✲✆❇★❂✕P✄❁✵❴✩✿✜✳✘❄ ❵✒❛ ✺✍✳✵✸✘✺☞✸✘✲☞❂✘❜★❝✵❞❭✼✵✺✍❃▲❙✍✻✒✿✕❨★✺✭✲☞✳❡✲✍❇★❇✵✻✒✶✵✲✍❙☞✹❡✸✜✶✷✸✜✹✄✺✭❙☞❂✕P✄❃▲✸✕✺☞✻✡✿✜✳✄❁❡✶★❈✧✲☞❃✓❃✓✶★❙✍✿✜✲✍✸✜✿✕✶★✳❡✻✒P✄❂✕✺☞❃❅❄❢❆❊✹✄✺❬✲☞✿✕✱ ✿✜❃❣✸✕✶❤✼✵✺☞✻✡✿✜❚✄✺✐✿✕✳✄❈✡✶✵✻✒✱❀✲☞✸✕✿✜✶✵✳❥✲✍❨★✶✵P✄✸❦✸✜✹✄✺❧❁✄✻✒✶✵P✵❇✵✿✜✳✄❁♠✶✵❈♥✻✡P✵❂✕✺☞❃♦✶✵❨★✸✕✲☞✿✕✳✵✺✍✼♣❈✡✻✡✶★✱q❙☞❂✕P✵❃✓✸✜✺✍✻✒✿✕✳✄❁❖❄ ❍❏❃r✲s❙☞✶✵✳✄❃▲✺✍t✵P✵✺✍✳✄❙☞✺s✶✵✳✄✺✉❙✍✲☞✳❥✻✒✺✍❇★❂✕✲☞❙✍✺✉❙✍❂✜P✄❃▲✸✕✺☞✻✡✺☞✼❤✻✒P✄❂✜✺✍❃✈❨✵✴❥✶✵✳✄✺✉✱✛✶✵✻✒✺✉❁✄✺✍✳✄✺☞✻✡✲☞❂✇✻✡P✵❂✕✺✬❄ ① ✶✵✻②✲✷❁✄✿✕❚✵✺✍✳✰✲✍✸✜✸✜✻✡✿✜❨✵P✄✸✜✺❡✿✜✳✰✸✕✹✵✺❡❙✍✶★✳✄❃✓✺☞t✵P✄✺☞✳✄✸✕◗③✸✜✹✄✺❡❇★✻✡✶★❇✵✶★❃✓✺✍✼✰✲☞❂✕❁✵✶✵✻✒✿✕✸✜✹✄✱♠❙☞✶✵✳✄❃▲✸✕✻✒P✄❙✍✸✜❃✤✲⑤④✵⑥✐❁✄✻✒✿✕✼ ❘◆✹✄✺☞✻✡✺❡✺☞✲✍❙☞✹✰✲✍⑦✄✿✜❃❬❙✍✶★✻✡✻✒✺✍❃▲❇✵✶★✳✄✼✵❃❬✸✜✶⑧✲☞✳✰✲✍✸✜✸✕✻✒✿✕❨★P✄✸✜✺❡✿✕✳✰✸✜✹✄✺❡✲☞✳✄✸✜✺✍❙✍✺☞✼✵✺☞✳✄✸⑨❄★❆❊✹✄✺⑤✲✍❂✜❁✄✶✵✻✒✿✜✸✕✹✄✱♣✸✕✻✒✿✕✺☞❃✤✸✕✶ ❈✡✿✕✳✵✼❶⑩❷✸✜✹✄✺♠❨✵✺☞❃✓✸✜❸♠❙✍❂✜P✄❃▲✸✕✺☞✻✡✿✜✳✄❁❹✶✵❈♦✻✡P✄❂✜✺✍❃❧❈✩✶✵✻♦✳✄✶✵✳✄❴✡✶✵❚✄✺☞✻✡❂✜✲✍❇★❇✵✿✜✳✄❁❺✲✍✻✒✺✍✲☞❃r✶✵❈❻✸✜✹✄✺s④✵⑥❼❁✄✻✒✿✕✼▼❄ ❆❊✹✄✺◆✲✍❇✵❇★✻✡✶★✲✍❙☞✹✭✶✵✳✵❂✕✴❬❙☞✶✵✳✄❃▲✿✕✼★✺✍✻✒❃❊✻✒P✄❂✜✺✍❃❭❘✆✿✜✸✜✹✭✳✄P✄✱✛✺✍✻✒✿✜❙✆✲☞✸✕✸✜✻✡✿✜❨✵P✵✸✕✺☞❃❊✿✜✳✭✸✜✹✄✺◆✲✍✳✄✸✜✺✍❙☞✺✍✼✵✺☞✳✄✸✜❃❅❄ ❍❏❃▲❃✓✶✵❙☞✿✕✲☞✸✕✿✜✶✵✳❩✻✡P✄❂✜✺✷✺☞✳✄❁✄✿✜✳✄✺✍❃❀✲✍✻✒✺✷✶★❈✩✸✜✺✍✳⑧✻✒✿✕❁✵✹✄✸✕❂✜✴✰✲✍❙☞❙✍P✄❃▲✺✍✼✚✶✵❈②✶★❚✄✺✍✻✒❂✜✶✵✲✍✼★✿✕✳✵❁⑧✸✕✹✵✺✷P✄❃▲✺✍✻◆❘✆✿✜✸✕✹❩❚✄✺✍✻✒✴ ❂✜✲✍✻✒❁✄✺❽❃▲✺✍✸✜❃❑✶✵❈❯✻✡P✄❂✜✺✍❃❢❄✛❆✧✹✄✿✜❃❑✲☞❇✵❇✵❂✜✿✜✺✍❃❑✸✜✶♠✲✍✳✵✴✐❃▲✶✵❈✡✸✕❘◆✲✍✻✒✺❽❇✵✲☞❙✍❾✄✲☞❁✄✺✍◗✰❙☞✶✵✱✛✱❀✺☞✻✡❙☞✿✕✲☞❂⑧✶✵✻❯✳✄✶✵✳✄❴ ❙☞✶✵✱✛✱❀✺☞✻✡❙☞✿✕✲☞❂✕◗★✸✕✹✵✲✍✸✬❘✆✺◆❾✄✳✄✶★❘✾❄ ❿ ✳✾✸✕✹✵✿✕❃✽❇★✲✍❇✵✺☞✻✙❘◆✺✤✼✵✺✍❃▲❙✍✻✒✿✕❨★✺✤✲✤✻✡P✵❂✕✺✤❇★✶✵❃✓✸✫❇✵✻✒✶✵❙✍✺☞❃✓❃▲✿✕✳✄❁✾✺☞✳✄❚✄✿✜✻✡✶★✳✄✱❀✺☞✳✄✸➀✸✜✹✄✲✍✸✫✲✍❂✜❂✕✶★❘✆❃✽✸✜✹✄✺❏P✄❃✓✺☞✻③✸✕✶ ❨★✻✡✶★❘✆❃▲✺❬✸✜✹✄✺❬✻✒P✄❂✕✺✭❃✓❇✵✲☞❙✍✺☞◗✢✶✵✻✒❁✄✲✍✳✄✿✜❋✍✺☞✼❡❨✵✴✾❁✄✺☞✳✄✺✍✻✒✲✍❂✜✿✜✸✕✴✖◗✢❨★✴✛❚✵✿✕✺☞❘✆✿✜✳✄❁⑤✶✵✳✵✺✤✻✡✺☞❂✕✺☞❚✄✲✍✳✄✸✫❃✓✺✍✸✫✶✵❈③✻✒P✄❂✕✺☞❃✽✲✍✸ ✲✛✸✜✿✕✱✛✺✞❄✄❍r❃✓✺✍✸❖✶✵❈❊❃✓✿✜✱❀❇★❂✕✺❀✶✵❇★✺✍✻✒✲✍✸✜✶✵✻✒❃②✲✍❂✜❂✜✶✵❘◆❃②✸✜✹✄✺✭P✄❃▲✺✍✻❭✸✕✶☎✱❀✶★❚✄✺✭❈✡✻✡✶★✱➁✶★✳✄✺✭❃▲✺✍✸❖✶✵❈❊✻✡P✄❂✜✺✍❃✆✸✕✶☎✲✍✳✄❴ ✶★✸✕✹✵✺✍✻✏❄✄➂✙✲☞❙✍✹☎❃✓✺✍✸▼✶★❈❊✻✒P✄❂✜✺✍❃◆✿✕❃◆❇✵✻✒✺✍❃▲✺✍✳✄✸✜✺✍✼✁✿✕✳☎✲❀❇★✲✍❁✄✺✛✲✍✳✵✼☎❙☞✲✍✳✷❨★✺❀❁✵✻✡✲☞❇✵✹✄✿✜❙✍✲☞❂✕❂✜✴❡❃✓P✵✱❀✱✛✲✍✻✒✿✕❋☞✺✍✼▼❄ ❿ ✳ ✸✜✹✄✺❡❈✡✶✵❂✜❂✕✶★❘✆✿✜✳✄❁✰❘◆✺❡❃✓P✵✱❀✱✛✲✍✻✒✿✕❋☞✺❡✸✜✹✄✺❡✱✛✲✍✿✜✳✰✲✍✼✵❚✵✲✍✳✄✸✜✲✍❁✄✺☞❃✓◗▼❂✜✿✕✱✛✿✕✸✜✲✍✸✜✿✜✶✵✳✄❃✤✲☞✳✄✼✰❈✩P✄✸✜P✄✻✒✺⑤❘✆✶★✻✡❾➃✶★❈✽✸✕✹✄✺ ❇★✻✡✶★❇✵✶★❃✓✺✍✼✛✲✍❇★❇✵✻✒✶✵✲☞❙✍✹✘❄ ❆❊✹✄✺◆✱❀✲☞✿✕✳❀✲✍✼✵❚✵✲✍✳✄✸✜✲✍❁✄✺☞❃❊✲☞✻✡✺✬➄ • ➅ ➂✙❍❏■❱✺✍✳✵✲✍❨✵❂✜✺✍❃❭❃✓✺☞❂✕✺☞❙✍✸✜✿✕✶★✳✭✲☞✳✄✼❀❨★✻✡✶★❘✆❃▲✿✕✳✵❁✭✲✍❙☞✻✡✶★❃✓❃❭✸✕✹✵✺✆❃▲✺✍✸✬✶✵❈❖✼✵✺☞✻✡✿✜❚✄✺✍✼✛❍❏■✭❄ ❿ • ✸✬✺✍✳✄✲☞❨✵❂✜✺✍❃❭❇✵❂✜✶✵✸✜✸✕✿✜✳✄❁❀✳✄P✄✱✛✺✍✻✒✿✕❙◆❇✵✻✒✶✵❇★✺✍✻✒✸✕✿✜✺✍❃❭✶✵❈❖✺✍✲☞❙✍✹✭❃▲P✄❨✵❃▲✺✍✸✬✶✵❈❖✻✒P✄❂✕✺☞❃❊❈✡✶✵P✄✳✵✼❖❄ • ➆ ✻✡✶★❘✆❃▲✿✕✳✄❁⑤✿✜❃➇✼★✶✵✳✄✺❬❨★✴❀✲✤❃▲✺✍✸➀✶★❈③❘✆✺☞❂✕❂✜❴✩✼★✺✍❈✡✿✕✳✄✺☞✼⑤✶✵❇★✺✍✻✒✲✍✸✜✶✵✻✒❃✽❘✆✿✜✸✕✹✾✲✤❙☞❂✕✺☞✲✍✻✙✲☞✳✄✼⑤✿✕✳✵✸✕P✄✿✜✸✜✿✕❚✄✺ ❃▲✺✍✱✛✲✍✳✄✸✜✿✜❙✍❃❅❄ • ➈ ✺✍❂✜✺✍❙✍✸✜✿✜✶✵✳✛✶✵❈▼❍❏■❯✻✒P✄❂✕✺☞❃●❨✵✴✭✲☞✳❀P✵❃✓✺✍✻③✿✜❃●✲✍✳❀✿✕✱✛❇✵❂✜✿✕❙☞✿✕✸✬❈✡✶✵✻✒✱❣✶★❈✘❇★✻✡✶★❚✄✿✜✼✵✿✜✳✄❁➉❨★✲✍❙☞❾✄❁✄✻✒✶✵P✄✳✄✼ ❾✵✳✄✶✵❘◆❂✕✺☞✼✵❁✄✺☞◗✬✸✕✹✄✲☞✸➀❙✍✲☞✳✾❨✵✺✤❂✜✲✍✸✜✺✍✻✙P✄❃▲✺✍✼★◗✞❈✡✶✵✻✙✺☞⑦✄✲✍✱✛❇✵❂✜✺✍◗✬✿✜✳✾❃✓✺✍❂✜✺✍❙☞✸✕✿✜✳✄❁✾✻✒P✄❂✕✺☞❃✽❈✩✶★✻✙✲✤✲❏❙✍❂✜✲✍❃✓❴ ❃▲✿✕❈✡✿✕✺☞✻▼✱❀✲☞✼✵✺◆✶✵P✄✸✬✶✵❈❖✲◆❃✓P✄❨★❃✓✺✍✸✬✶★❈✘✻✒P✄❂✜✺✍❃❅❄ ❛ • ➅ ➂✙❍❏■r❇✵✻✒✺✍❃▲✺✍✳✄✸✜❃✭✲☞✳⑧✶✵❇★✺✍✳⑧❇★✹✄✿✕❂✜✶✵❃▲✶✵❇★✹✄✴✰❨✵✴✰✻✒✺✍✲☞✼✵✿✜✳✄❁⑧✸✜✹✄✺✷❃▲✺✍✸❭✶✵❈②✻✒P✄❂✜✺✍❃❀✲✍❃✭✲ ➅✫➊❦➊ ✱✛✶✵✼★✺✍❂➋❄ ❆❊✹✄✺◆✱❀✲☞✿✕✳❀❂✕✿✜✱❀✿✜✸✜✲✍✸✜✿✕✶★✳✄❃❊✲☞✻✡✺✬➄ • ❳✤✿✜❃✓P✵✲✍❂✜✿✕❋☞✲✍✸✜✿✕✶★✳⑧✸✕✺☞❙✍✹✄✳✵✿✕t★P✄✺✍❃❀✲✍✻✒✺✷✲✍❂✜❘◆✲✍✴✖❃✭✼★✿✕❈✡❈✩✿✜❙✍P✄❂✜✸✧✸✜✶⑧✺☞❚✄✲✍❂✜P✄✲✍✸✜✺✞❄✞❆❊✹✄✿✜❃❬✶✵✳✄✺✷✿✕❃❬✳✵✶⑧✺✍⑦✄❴ ❙☞✺✍❇★✸✕✿✜✶✵✳✘❄ • ❆❊✹✄✺✤❙✍P✵✻✡✻✒✺✍✳✄✸✫✿✕✱✛❇✵❂✜✺✍✱✛✺✍✳✄✸✜✲✍✸✜✿✜✶✵✳✾✻✡✺☞t✵P✄✿✜✻✒✺✍❃✓◗✬✶★✳✛✸✜✹✄✺✤❃▲✺✍✻✒❚✄✺✍✻✒❴✩❃▲✿✕✼★✺✍◗✬✸✕✹✵✺✤P✄❃✓✺✤✶★❈③✲✍✳✛✶✵❇✵✺☞✻✡✲☞✸✕❴ ✿✜✳✄❁❀❃✓✴✖❃✓✸✜✺✍✱✈❈✡✻✡✶★✱❣✶★✳✄✺◆❃✓❇✵✺☞❙✍✿✜❈✩✿✜❙✆❚✵✺✍✳✄✼★✶✵✻✏❄ ❵ • ❆❊✹✄✺◆✺✍✳✄✸✜✻✡✴✭❇✵✶✵✿✜✳✄✸ ✸✕✹✄✺◆✿✜✳✄✼✵✺☞⑦✭❇✵✲☞❁✄✺✍❞▼✿✜❃❊❃▲✸✕✿✜❂✜❂✞✻✒✺✍❂✜✲✍✸✜✿✜❚✄✺✍❂✜✴❬❘◆✺✍✲✍❾❖❄ ❆❊✹✄✺◆❚✄✿✕❃▲P✄✲✍❂✜✿✜❋✍✲✍✸✜✿✜✶✵✳✭✸✜✺✍❙☞✹✄✳✄✿✜t✵P✄✺☞❃❊✶★❈✩❈✡✺✍✻✒✺✍✼✛✲✍✻✒✺✆❚✄✺☞✻✡✴✭❂✕✿✜✱❀✿✜✸✜✺✍✼❖❄ ① P✄•✸✕P✵✻✡✺◆❘✆✶★✻✡❾❖➄ • ⑥✤✺☞❚✄✺✍❂✜✶✵❇✛✱✛✺✍✸✜✻✡✿✜❙✍❃❭✸✕✶✛✱✛✺✍✲✍❃▲P✄✻✒✺✆✸✜✹✄✺◆❁✄✲✍✿✜✳✄❃❭✶✵❈❖✸✕✹✄✿✜❃❭✲✍❇✵❇★✻✡✶★✲✍❙☞✹✘❄ 54 Jorge et al. • • • • ➌✤➍☞➎✄➍✍➏✜➐✵➑✚➒❀➍☞➓✍➔✄→☞➣✄↔✜↕✓➒✛↕✭➙✜➔✄→✍➙❭→✍➏✜➏✕➐★➛➁➙✜➔✄➍❡↔✜➣✄➓☞➐✵➜✒➑✵➐✵➜✒→✍➙✜↔✜➐✵➣✰➐✵➝②➞✄↕✓➍☞➜②➟✵➍☞➝✩↔✜➣✄➍✍➟❩➎✄↔✕↕▲➞✄→✍➏✜↔✜➠✍→✍➡ ➙✜↔✜➐✵➣✄↕⑧→☞➣✄➟❑➜✒➞✄➏✜➍❱↕✓➍☞➏✕➍☞➓✍➙✜↔✕➐★➣❻➓✍➜✒↔✕➙✜➍✍➜✒↔✕→☞➢✤↕✓➞✄➓☞➔✪→✍↕✰➝✩➐★➜❡➍✍➤✵→✍➒✛➑✵➏✜➍✍➢❏➙✕➔✵➍✇➓✍➐✵➒✛➥✵↔✜➣✄→☞➙✕↔✜➐✵➣✪➐✵➝ ➑★➜✡↔✜➒✛↔✕➙✜↔✕➎✵➍✆➐★➑✵➍✍➜✒→✍➙✜➐✵➜✒↕❅➦ ➧✧➎✄→✍➏✜➞✄→✍➙✜➍✁➙✕➔✄➍✁➓✍➞✵➜✡➜✒➍✍➣✄➙●↔✜➒❀➑★➏✕➍☞➒❀➍☞➣✄➙✜→✍➙✜↔✕➐★➣❩→✍➨✵→✍↔✜➣✄↕✓➙●➐✵➙✜➔✄➍☞➜◆→☞➏✕➙✜➍✍➜✒➣✄→☞➙✕↔✜➎✄➍✍↕❀↕✓➞✄➓☞➔⑧→✍↕➇➩❫→✍➎✄→☞➢ →☞↕❊➛◆➍✍➏✜➏✬→✍↕❭→✍➣❀→✍➏✜➙✕➍☞➜✡➣✄→☞➙✕↔✜➎✄➍◆➙✕➐✛➓✍➏✜↔✜➍✍➣✄➙✜➡✩↕▲➍✍➜✒➎✄➍✍➜✒➢✵↕▲➞✄➓✍➔❀→✍↕❭➑✵➏✜➞✄➨✄➡✩↔✜➣✘➦ ➫✒➣✄➎✄➍☞↕✓➙✜↔✕➨✵→✍➙✜➍✆→☞➣✄➟❀↔✜➒✛➑✵➏✜➍✍➒✛➍✍➣✄➙✬➐✵➙✜➔✄➍☞➜❖➎✵↔✕↕▲➞✄→✍➏✬➜✒➍✍➑✵➜✒➍✍↕▲➍✍➣✄➙✜→✍➙✜↔✜➐✵➣✄↕❭➐✵➝❖↕✓➞✄➥★↕✓➍☞➙✕↕❭➐✵➝❖➜✡➞✵➏✕➍☞↕❅➦ ➭❏➏✜➏✕➐★➛➉➙✜➔✄➍❬➟★➍✍➝✡↔✕➣✄↔✜➙✜↔✕➐★➣⑤➐✵➝✙➜✒➞✄➏✜➍❬↕✓➍☞➏✕➍☞➓✍➙✜↔✕➐★➣⑤➓✍➜✒↔✕➙✜➍✍➜✒↔✕→✭➥✵→✍↕▲➍✍➟❡➐★➣⑤➙✕➔✵➍❬↕✓➞✄➑★➑✵➐★➜✡➙✘→✍➣✄➟❡➓☞➐✵➣✄➝✡↔✕➡ ➟★➍✍➣✄➓☞➍✆➐★➝✘➙✜➔✄➍◆➜✡➞✵➏✕➍☞➢✵↔✜➙✕↕❭→✍➣✄➙✜➍✍➓☞➍✍➟★➍✍➣✄➙✬→✍➣✵➟❀↔✜➙✕↕❭➓✍➐★➣✄↕✓➍☞➯✵➞✄➍☞➣✄➙➋➦ ❅➲ ➳ ❭➵ ➸✍➺➼➻➾➽❊➻➾➚✡➪✞➶ ➳ ➪✞➹✷➻➾➘☞➘❅➴➷➚➷➻✖➪★➬ ➳ ➪▼➮❷➺➼➴➷➱✍➻➾➘☞✃✒➪✵➶ ➳ ➪★❐✢❒❅➴➷❮✍❒❅➘☞❰➾➘✢➪✵➬ ➳ ➪★Ï✽❰▲➺➼➱☞➻▲Ð✆❒✬➪★➵ ➳★Ñ✓➳ ➪★Ò★➻❢Ó✕✃✙Ô●➴✡Ó✕Õ➾❒☞❮☞❰▲➺➼Ö❡❒❅× ➳➋Ú✫Û☞Ü➾Ý☞Þ❅ß➾à▲á✽â Þ②ã③Þ❅ä❅å②æ à▲Û❅ç❅à✧è❬â á⑨ß▲ä❅Ü➾à▲é✕ê◆Ý❅Þ❅Û②è●Ý✬ë Ý②ì✁â Þ✬â Þ❅ç➀í➾î☞ï❅ð☞ñ➼î❅ò✢ó☞➳❅➲❫ô❅ô❅õ✬➳ ò✢➳ ö③✤➵ ➼➺ Ó✒➴➷Ó✕➘✢❒❅➪✢Õ▲➴Ø➮ ➻▲➳ ✃➷➪❅➴Ø❒☞➹✷➘✆❒❅➶③✃➷➽❭Ù☞➚➷➻▲❰✖➘❅Ó ➴✡➪❅➶ ➳ ➪❅÷●➚➷➚➷Ð✆➻▲➘✢➪☞ø ➳ Ô ➳ ➻▲➘❅ù✛❐✧ÓØÙ❅➺✓➪✬➮ ➳ Ô●Ö❷➘❅➻▲Ð✆➴➷Õ✆➴Ø✃➷❰▲Ð✾Ó✕❰▲✃➀Õ➾❒❅Ù☞➘❅✃➷➴➷➘❅➸✭➻➾➘☞ù✛➴➷Ð✆ú☞➚➷➴ØÕ▲➻ ñ ➳þý❅✁ÿ ✫ì✄✘✂ è✆➀☎ à➾ß▲ä❅é⑨Û✞✝ Ú✠✞✟ ìrý✁✡✬à➾ß❢â Ý✬æ☞ÿ➼Þ✢ë à▲é⑨à➾á❷☛ë ✘é⑨✌ä ☞✍✡☎ä❅Þ❬ì❀Ý☞✌Þ ✎ Ý❅✃➷➴➷ç☞❒❅✑à ➘❬✏●➺➼à▲Ù❅Þ✢➚➷❰✖ë➾Ó✢ä✓❭✒ ×û❒☞è●➺❖Ý✢Ð②ë Ý✕➻➾✔ ➺➼➪➱✍ò❅❰➾õ✗✃✬✖➼ü☞ò✌➻✖✔✓Ó✕í ➱✍ò✙❰➾✘✗✃✬✘ ù☞➪ ➻➾➲❫✃➷ô❅➻ ô☞ð✢➳✁✚ ✃➷✃Øú í ✛✜✛ Õ▲➴Ø✃➷❰❢Ó✕❰➾❰▲➺ ➳ ➘✁✢ ➳ ➘☞❰➾Õ ➳ Õ➾❒❅Ð ✛ ü❅➺➼➴➷➘ ô☞ð ù☞Ö✍➘☞➻➾Ð②➴➷Õ ➳ ✚ ✃ØÐ②➚ î✢➳ Ô●➻▲✃Ø➻✧➹✷➴Ø➘☞➴➷➘❅➸✤✣❭➺➼❒☞Ù❅ú ✖✦✥ ➹✷✞➹ ✧✾ù☞❰➾❮✍❰➾➚➷❒❅ú☞Ð✆❰▲➘❅✃ ✔ ➪ ✚ ✃➷✃Øú í ✛✜✛ ➽❊➽❭➽ ➳ ù❅Ð②➸ ➳ ❒☞➺➼➸ ✛ ★✢➳ ñ➼ò❅õ☞ò Ó✕✃➷➻▲➘❅ù❅➻▲➺➼ù ✚ ✃➷✃Øú í ✛✜✛ ➽❊➽❭➽ ➳ ❰➾Õ➾Ð②➻ ➳ Õ ✚✌✛ ❰▲Õ➾Ð②➻ ➲✁✛ ➮✏❐✢➵✭✬●Ô ✛ ✩✫✪③➹✷➵ ñ➼ò☞õ❅ò✢➳ ➬●❐✢➹ ✩✫✪③➹☎➵ ✘❅➳ ➳ ➳➪ ✰ ➳ í ✪③➚➷Ù✢Ó✕✃➷❰▲➺➼➴Ø➘☞➸➃➵✤Ó✒Ó✕❒❅Õ▲➴Ø➻▲✃➷➴Ø❒☞➘✰➶③Ù❅➚➷❰✖Ó✒➪❯➴➷➘✰➵❭➚➷❰✑✰✱✣❭➺➼➻➾Ö★➪ ✥ ❰▲➺ ✦ñ ✲ ➱✍❰ ✧✢❰➾➘☞✃✡➪➀ö ➪✙➮✏➽❭➻▲Ð✆➴✡➪✬➵ ✯ ✮ ➴➷ù❅❒☞Ð ➪✬ø ✖ ➳ ✔✓✴ í ✳➀é⑨ä☞✶ ß ❖ ✵ ✓ä ♥ ✒ ë ✷❅à✹✸✺✬ ✷ â é✏ë à▲à➾Þ✬ë ✷❲ÿ➼Þ✢ë à➾é⑨Þ☞Ý✢ë➷â ä❅Þ☞Ý✢✻ æ ✞✟ ä❅✦Þ ✒þà▲é⑨à➾Þ☞ß➾à➃ä☞Þ❦è●Ý✢ë ✽ Ý ✼✫Þ☞ç✢â Þ❅à▲à➾é❷â Þ☞✌ç ✾ ÿ✧✢✟✞➻➾✿è ➺✓ÓØ✼❁❒❅➘ ❀✯❂ ✩➀ö③ù✢➴➷➺➼Ó Ð②➴Ø➘☞➸ ✚ ➻➾Ð❱÷ ➳ ❃✤➳✓Ñ ✩✫✩✫✩❁✪③❒❅Ð②ú❅Ù❅✃➷❰▲➺✙➮❷❒☞Õ➾➴➷❰➾✃➷Ö ➲❫ô❅ô❅ð ✁ õ✢➳ ➹✷➻✖➪❅❄●➴➷Ð✆➴➷➘❅➸✞➪❅✧✢➴ØÙ✬➪❷ö③➴➷➘☞➸✬❅➪ ✮✰❒❅➘☞➸✬➪ ❃ ➴➷➻➾➘ ✖➼ò❅ï☞ï❅✗ï ✔ ➪✺✮✰❰▲ü✭×✜❒❅➺❖Ô❭➻➾✃➷➻✽➹☎➴➷➘☞➴Ø➘☞➴➷➸ ✺í ❆ ➺➼➸☞➻▲➘❅➴❈❇➾➴➷➘☞➸❏➻➾➘☞ù Ñ ➘❅✃➷❰➾➺ ñ ✚ ý☞✁ÿ ✘ã③è●❊ è ✼●❋❍✧✡ æ ä❅é✒Ý✢ë➷â ä❅Þ❅á ➪➇✠➵ ✪③➹ ú❅➺➼Ñ ❰▲✃➷➴Ø❃ ➘☞➸❑✃ ✚ ❰♥Ô❭➴✒ÓØÕ➾❒❅❮✍ò ❰➾➺➼Ñ ❰▲ù➉➶③Ù☞➲ ➚➷❰✖Ó⑤÷❬ò☞Ó✕➴➷ï❅➘❅ï❅➸❻ï✬➳ ✃ ✚ ❉ ❰ ✰ ✮ ❰▲ü✢➪✭➮❷Õ ❒❅❒☞➚✡➪ Ï✽❒❅➚➷Ù☞Ð✆❰ ✖ ➪ Ó⑨Ó✕✥☛Ù❅✔✯❰ ✚ ➪❫ø⑨Ù❅í ✛✜➚➷✛ Ö ➳ ð✢➳ ➹✷➮ ✣➴ØÕ▲➺➼❒✢Ô●Ó✕❒☞Ô✤×û✶✃ ➪✓✰ ✮ ❰➾ü✛➮✏➴Ø✃➷❰ ➵✤➮ ✃➷✃Øú Ð✛Ó✕➳ ù❅➘ Ð②➴ØÕ▲➺➼❒✢Ó✕❒☞×û✃ ➳ Õ➾❒☞Ð ✛ ➚➷➴➷ü❅➺➼➻▲➺➼Ö ✛ ù☞❰➾×û➻▲Ù❅➚➷✃ ➳ ➻✖Ó✕✯ú ■✩Ù☞➺➼✜➚ ❏ ✛ ➚➷➴Øü☞➺➼➻➾➺➼Ö ✛ ❰➾➘ ñ ✛ ✛✜✚ ✛ ó❅➳ Ù✢➹✷Ó ➴Øù☞Õ▲➺➼➘❅❒✢ü☞Ó✕❰➾❒☞➸✍×û❮☞✃ ü ✃➷✮✰Ð✆❰➾➚ ü ➻➾Õ▲✃Ø➴➷❮✍❰✖➮❷➴➷Ó✕✃➷❰▲❰ ➺➼❮☞❰▲➺➼ú❅✖ ➻▲Ô●➸☞❰❢❰❢Ó✕Ó Õ➾➺➼➻✖➴➷ÓØú☞ú ✃Ø➴➷❒☞➘✢Ó ❒☞× ø⑨➻➾❮✍➻✖➪ ø❷➮❷Õ▲➺➼➴➷ú❅✃✡➪ ➻▲➘❅ù ø⑨➻➾❮✍➻✖➮✏Õ➾➺➼➴➷ú❅✃ ✔ ✚ í ✛❈✛ ➳ ➳ ✛ ➳ ñ ✌➲ ✘✁★✯❅✘ ✗ó ✘ ô✢➳ ➮✏✃➷➻➾✃➷❮✍ú ➻✖Ó✕❰▲Ó✕➺➼Ù❅❰✖ú☞➪➾ú❅➵ ❒☞➺➼➳ ➪✃ ❆ Ð✆Ð②➴➷Õ➾➴Ø➺➼❰▲❒✬Õ➾Ó✕➴➷❒❅➘✢×✜ÓØ✃ ➱☞Õ➾➴✡❒❅◆➪ ✩ Ð ➳ ù❅➻▲➘❅❰▲×û❖ù ➻➾Ù☞✬❭➚Ø➻➾✃ ❮✍➻❢➻➾Ó✕ú✌✃ ✚ ✰☛❰❢■➾➪❢ÓØÕ➾➮ ➴➷➳ ✌ù ➪▲✵❏➵❭➱☞➘✆ü✯❰➾❑ ✩●×û×û✬ ➴➷Õ▲➴Ø÷❬❰▲➘❅➮▲✃✄❑ ▼➻➾➚➷➸✍❒❅➺➼➴➷✃ ✚ Ð❯×û❒❅➺✢Ð②➴➷➘❅➴➷➘❅➸②➻✖Ó✒Ó✕❒❅Õ▲➴Ø➻▲✃➷➴Ø❒☞➘ ✍➳ ✳➀é⑨ä❅✕ ß ✵✓✓ä ✒✭P✙◗✓á✏ë➾ÿ➼Þ✢ë➷❈æ ✵✌✞✟ ä❅Þ✓✒❘✵✓ä❅❚ Þ ❙▲à➾é✕❖ê ❯✵Ý❅é⑨ç☞à❊è●Ý✬ë ✌Ý ❅❱ Ý☞á⑨à➾✿á ✝✍❙◆❯★✭è ❲✶❳ ➲✓ô☞ô✙✘❅➳ ➲✓ï✢➳ ➺➼➮✏Ù❅➴Ø➚➷➚➷ü☞❰❢❰➾Ó✵➺✓➴➷➘✆Ó✕Õ ✚➚➷➻▲➻➾➺➼❈✃ ➸☞❇❢❰✧➪✍➵ù❅➻▲➳ ✃Ø➻▲➻➾ü❅➘☞➻❢ù❀Ó✕❰✖❐✢Ó ✗Ù ❇ ✚ ➴➷➚➷➴➷➘✢➪❷➵ ➳ ➪ ❆ ➘➃ÓØÙ❅✁ü ✢✡❰▲Õ➾✃➷➴➷❮☞❰②Ð✆❰▲➻✖Ó✕Ù☞➺➼❰✖Ó➀❒❅×❖➴➷➘☞✃Ø❰▲➺➼❰✖Ó✕✃➷➴➷➘❅➸✍➘❅❰❢Ó⑨Ó✫➴➷➘✭➱☞➘☞❒❅➪ ➽❊➚Ø❰▲ù❅➸✍❰ ➳ ✳➀é⑨ä❅ß▲à➾à▲Û✢â Þ❅ç☞á③✦ä ✒✭ë ✷☞✿à ❨❊â é⑨á❷ë✏ÿ➼Þ✢ë à➾é⑨Þ☞Ý✢ë➷â ä❅Þ☞Ý✢❘æ ✟✞ä❅✓Þ ✒✏à▲é⑨à➾Þ☞ß➾à❊ä❅Þ◆ã③Þ☞ä❅å②æ à➾Û☞ç❅à❊è❬â á⑨ß➾ä☞Ü➾à➾éØê✤Ý❅Þ☞Û ◆ è●ù❅➴✡Ý✬Ó✕Õ▲ë Ý②❒❅❮✍ì✁❰➾➺➼â Þ✬Ö â Þ❅ç ➪ ➲✓ô☞✙ô ✘ ➪ ò❅✯ð ✘✓ñ➼ò✢ó☞➲❅✍➳ ✚ ✃➷✃➷ú í ✛❈✛ Õ▲➴➷✃Ø❰❢Ó✕❰➾❰▲➺ ➳ ✁➘ ✢ ➳ ➘☞❰➾Õ ➳ Õ➾❒☞Ð ✛ Ó✕➴➷➚➷ü❅❰▲➺✓Ó✕Õ ✚ ➻➾❈✃ ❇ ✯ô ✘ Ó✕Ù☞ü✁✡✢ ❰➾Õ▲✃Ø➴➷❮✍❰ ➳ ✚ ✃➷Ð✆➚ ➲❅➲❅➳ ❐✢❒❅➴➷❮✍❒❅➘☞❰➾➘✢➪✍➬ ➳ ➪✬➮✏➻➾Ð②ú❅➚➷➴➷➘❅➸❬➚➷➻➾➺➼➸✍❰✆ù☞➻➾✃➷➻➾ü☞➻✖Ó✕❰❢Ó✫×û❒☞➺▼➻✖Ó✒Ó✕❒❅Õ▲➴Ø➻▲✃➷➴Ø❒☞➘❀➺➼Ù☞➚➷❰✖Ó ❅➳ ✳✫é✒ä❅✶ß ✵❷✦ä ✒❖P✗P❅Þ❅Û✭ÿ➼Þ✢ë➷❈æ ✵☛✟✞ä☞✦Þ ✒❘✵✍ä☞Þ ❙▲à➾é✕❖ ê ❯✵Ý❅é⑨ç☞à❊è●Ý✬ë ✌Ý ❱❅Ý☞á⑨à➾á✿✝✍❙◆★❯ è✭❲✶❳ ➲✓ô☞ô❅õ✢✍➳ ✚ í ✛❈✛ ➳ ➳ ➳ ✛ ô☞õ ➳✚ ➲✓ò✢➳ ✮ î ✪✁Ô ❆ ➹❩✢✧ ❰➾❮✍❰➾➚ ➲ Ó✕ú❅❰▲Õ➾➴➷×û➴➷Õ➾➻▲✃➷➴Ø❒☞➪ ➘ ✚ ✃➷✃Øú í ✛✜✛ ✃➷➽❊✃➷ú ➽❭➽ Õ➾➳ ➴➷➽ ✃➷❰✖î✢Ó✕➳ ❰▲❒❅❰➾➺➼➺ ➸ ✛✍➘ Ô ✢ ❆ ➘❅❰➾➹ Õ ✛ Õ➾❒☞Ð ✃➷❒❅➴➷❮✍❒❅➘☞❰➾➘ Ó✕➻▲Ð✆ú☞➚➷➴Ø➘☞➸ ✃➷Ð✆➚ ➲✓î✢➳ ✮ î ✪②➪③➮✏Õ➾➻▲➚➷➻➾ü❅➚➷❰✾Ï✽❰▲Õ➾✃➷❒❅✤➺ ✣❊➺➼➻➾ú ✚ ➴➷Õ✖Ó ✖ ➮✏❬Ï ✣ ✔❀➲❅➳ ï ➮❷ú☞❰➾Õ▲➴Ø×✜➴ØÕ▲➻➾✃➷➴➷❒❅➘✬▲➪ ✮ î ✪ ➶③❰▲Õ➾❒❅Ð②Ð②❰➾➘❅ù☞➻➾✃➷➴➷❒❅➘✬➪▼➮❷❰▲ú ñ ò❅ï☞ï✢➲ ✚ í ✛❈✛ ➳ î✬➳ ✛ ✛ ✛ ➲ ★✢➳ ✃➷✮✰❰➾Ð②❰➾✃➷ü❅✃✡❰▲Ó ✚ ➺ ❰➾➺➼❰➾Õ▲➱✬➪❅➪ Ô ✃➷✃➷➳ ú➪❅➵ ➽❭❃ ➽❭Ô●➽ Ô✤➽ ➮✺✩ ñ ❒❅➴➷➘❅➺➼➸ù☞❰➾❐✢ú❅➶ ❰▲➘❅➮❷ù☞Ï❬❰➾➘❅✣ ✃ ✥ ➹✷❭➹ ✧❩Ï✽➴✡Ó✕Ù☞➻➾➚➷❈➴ ❇➾❰▲➺✓➪❅➴Ø➘ ✳✫é✒ä❅✶ß ✵❅✓ä ✒❬ÿ➼è❭è●❪ì ✎✁❫✍P✌✾✢å③ä☞❵é ❴✑✎ ✁ ❵á ✷☞✁ ä ✡❲ä❅Þ✷ÿ➼Þ✢ë à➾ç☞é⑨Ý✢ë➷â ä❅Þ✷Ý❅❛á ✡✬à➾ß❢ë á✆ä✦✒✛è●à▲ß✖â á❷â ä❅Þ✷✌ý ☞✍✡✌✡✬ä❅é❷ë✞Ý☞Þ❅Û✷è●Ý✢ë Ý✷ì➃â Þ✢â Þ ➸✬➪ ✖ ✩✫ù✬Ó ➳ ✔ ö③❒ ✚ ➻▲➘❅❰➾Õ❢➪✵➹ ➳ ➪ ➹✷➚Ø➻▲ù❅❰▲➘❅➴➷Õ✖ò☞➪➇ï❅ï❅Ô ò✬➳ ✻➪➳ ✧✢➻▲❮☞➺➼➻▲Õ✖✤➪ ✬ ➳ ➪②➻❢Ó⑨Ó✕❒☞Õ➾➴➷➻➾✃➷❰➾ù❑✃➷❒❑✃ ✚ ❰❱Õ➾❒☞➘❅×û❰▲➺➼❰➾➘❅Õ▲❰✖❁Ó ✩✫✪③✞➹ ✧ ✛❈✥☛❃ Ô●Ô ï☞ò ➪✽➬●❰➾➚✡Ó✕➴➷➘☞➱☞➴✡➪ Ò✵➴Ø➘☞➚➷➻➾➘❅ù✬➪ 
   
  

 ½
  
 

 

  
       
   
 

        
 
  !" 
# 

  
     $%
 &$' 


   
          
 
(
   
 
% 
  
 
 )
    
 ( 
%
  
  
  Æ % 
*  (
%
 



      +

 

 

 

         
+,# ( (   
      %  
 -%
# $    

 )

   
 -%
 + 
%

     . 
 
$ + +

 %
 + 

  
 +   

  
 +
 %  % /  
    

         
        
    
  

        
   !℄# $      
%  
 

 
 
    Æ      &   

 & 

    #  


    
  &
 

 
 '     #   

      
 (℄ )℄ !*℄
  &

&
  
     & 
  
  &
 
 &     

  
     # +    
      ,℄ !-℄ !.℄ !/℄ !!℄
     
 &      % 

   &    # 0          &    
 
   
     1 
     

 
 '    #  2 
 3  

4 3       

 
 
& 
 
        
 
 # 0    
 34 
 
3 

&   5

 6  -℄    
  
    
 
         
         &
 
 
  # ½ )  

 +   %
 ( 0  2!  &3   4%'  
   ( 1 56 Bruzzese and Davino !   5%//

        
  &
 



   & 0 
 '    0



   
   #           
 
 # 0   -   '      
 
 &
 
 # 7  . 
     
  '

  
     



   # 7     

  
 
'     *#           8 9
½ 
¾     
&
    
  
  ##
   & &
    
 9 ½  ¾      
&
   

   
 

  
&   ##
   
 : &
 # 
 
   

   ;         
       
      %        
 9 # $
       

 &  
 ;  ß ß    ; <  
  ; 
   9         9     
 &  

  &  


            # 
 
     

   
   & 

   
    &       # +    
  


        
 
  & &  
      
 &
&     %  
   #     7
&
   
          

 
          # = 
   
   2   

 2 &    &   
 
  #   

  
  '  
    

  
 
&
 ;  
   
 &        &  
    &  # +   ' 
  
    
   &  !.℄#    &       
 
 &    

   & 
   2   


   
 
  2
    
  
# 5      

    
 & 
 
   
             
     
  '  
          

  #                 
  '

 

     


        &    
 
    
      Visual Post Analysis of Association Rules 57  % 3      $%
 6  2
&     
 
  # 

         
   


    
 
 1   #  2 
 3  

4 3  &
      

 '  
      
     < 
   
    

 

             #       2  &
 
 0  =     
& 
  
   2           
  #   
' 
    
         
 &
 2    # 0    
 34 
 
3
     *℄ &
  
 
    &       5

 6   '        
    
        
  
  -' 
 .℄#   
  &
 



   & 0 0

&    
    
     Æ 
   
 ))>(      &  0           &' 
 #      .) 1     
 
  
    
      #

     & &  &'      

   
2    
  
 
  %
  2   
 
   
%  
 
# 0
              # 
  

   
      &
       

 

   
     
     ))>( 

 #           
     
  
       %  2     /#/!
 2   
 /#/!
 &   .(?,.  &
  #         +      

  
 
    
      &  &  1    
 
  2 
     # 0

 

 
     

  &      2   
 &     
 




            2  
# @ 
  
        &
 
   
   

 2
  
   2   
  
        
     % #   

 

     2 !  
 
  &
 
 
   
      # 5 2 !
    
         
&
  
  
&   !    2   
      2 !&    
& 
 

   
 
   1  

    
 
     2   
# 58 Bruzzese and Davino 7   5%//

      Rule: A & B --> C Rule: A & B --> C Rule: A --> C (b) (c) (a)       


   

 5 %    
%
8
  0    
  
    
       
      
 
       &
   
  &    2              ;  9        <     ! 0  ℄ !< /         &

 
&  

      

 
 < 
  9 /       
 
  &
       2      A

 %

 

 
   & &     <   ℄/< !           
  

&   
    
   % #   
 
 
  
     

  '
    
   &    
   
# 4

  '
      & 0 &  !),! >℄  
   

     

    


'  
'  
 
# 



   

  

  &
    



     &
  B    
   


 # 0    

  

 
 
   

     
    

 

           
 # 7    
   

  
 ?℄
        
  
    &   # 0 
  




 
& 

 &;   ß   
&           &
 2               <  Visual Post Analysis of Association Rules 59  % 3      $%
 9    
            
  
 %          
 
 < ß
   
         
 2                
    

  &
 2   < ß      2                    
 
 < ß 
           &
    &  #  C  

4 3 
 
&
        0

 # 0 2 -
 
             !>(-  
  
  %  ;      #                


   

   
  
 
    &
#    )> 
&   &
     &  
   
 # ß     
  
  
%
 +  

-%

-%   0 2 .
&   !/,*      % %
  
    2   
 
/#(          & B 2  7  *#-   # 0         

 &           
  
 
       2    
  2        
   
       #      
 
       
 

   
    2   2 
     
    
 
 # 60 Bruzzese and Davino "   5%//

          3  
%
 +  

-%

-%      Ê
9 3  
%
 % 
  
.

  %  Visual Post Analysis of Association Rules 61  % 3      $%
 : 5
  *>.     2   
 
/#(
     
    
  
    2 *#   &           

 
   &
  0= 

   

 

    
   

  %
 
     
     
       & 
& 
 

&   #          C4 
 
3 
   4$@       
 

   & 
   .℄ 

 2 

     2  
 


   &       

     
        
 2

   

 
&         
  
  -' 
 #  &
  &
         
 2   

 % 
      
 
      2
    %   
  
   2  #  
   
 

         
 

&     &
  &       

&   #  &  &          #   
  &    #                    2      
        %  
   
  
     2          % 
   #  
  

    
   
 
&  
  

&     % 
 #               '-     
     

   
     

   2
 
 
 

 
 
 
 #    &
   
  &     
    
  
  
 


 
    
   
 #                   2
    2     
     

    

 
     
 
 &  
 #      *℄  &
   
  &    
   
   
    



       2    
 
  
 #  C  3            & 
           

 
  
     &    % #  
 
  2   
 2     2
 
 
C  3 
   9 / (#    C  3   &
        ; ¼ ;  9   ½ ;        62  Bruzzese and Davino   5%//

         
  

 
 &  
 
'  &          &

%
  
' D 
 &    #   
&
  
 &  


       & 
 ;  ;  9 / (
      
& 
 
 

  &  ;   / (<     Æ   
# 0  
 
  

&;    
 9  ;   /(  

& 
 /( !  /(  
 
 
 /(  

&

  
  
 
 /< !
        

   .(?,.      0


 
    
&# 
     (% 
%  
 5

 
  
  
 3%  
 
 ! 
 6 %(
  %
 "; 6!:! "7; 9"!  % .

" ; ; 96 * % .

: 9 9  % %     * % %  9  9  9  9    2 
& !/E   
      F    2
  
 2   
 #    
   
 &   ---.-
        

     # =    C  3      !(>-    
   2   %
  /#(. 
   ,(.>
 
  
 2   
 
/#(.   
      #  
    
       

 

  
      2  
      

 
 
 #  !(>- 
 
   
 &    & 

    &     
 


  

       

  
 
 
    -'  
 
 #  6      
  6  -℄ 
 Visual Post Analysis of Association Rules 63  % 3      $%
 ; 
  G -


      &         
    
 &  1   &   
   

   % 
     < 
 2   


     # $
     &
&



  
 !               

 /   # 1 

    G- 

&   6 ; 
    
    

& 

& 
      

 
2   

 
     %  
 
  2   

     

& 

&    &  
B  
   2  

 
 #     
   
   
 &      
  
   
    
      % 
 # +   6        &       

 

   



  2      
 

&   
  2  

 
   

 
 & 2  &
     # +     &  1 & 6 
  

   
 
  
  
  
  %   # if I NT 7 6 if RBS 5 if CTS if PS A 4 then PRS if I NP then PSA 3 then RBS then PRB 2 if P RM if ITV if P RS if RRE if RCS 0 if RSV if RED then PMG 1 then LOT if DOC 1 then FCM if COM if LOT then FLM if F CM if P CI if P DI if I N C then PDI if FTV then PRM if P M if P RB Gif FL M then INC if RTL if RTG 5 4 3    2 1 0 1 )

 $


  0 2 ( 
  
  %  
     2 

 
  
   
   &   
  
 2 >
    

  #  
64  Bruzzese and Davino   5%//

      1 if DOC then FCM 0.5 if LOT if COM then MAN if FCM then FLM then TG then FLD then FLC then CAR if MAN C if CAR S if CON if INC if MUS then MUS then FTV then MET if FLD if FTV if MET then PCI if FLM if SCA if PCI 0 then PDI 0. 5 if PMG then INC 1 0. 6 0. 4  0. 2   0 0.2 0.4 0.6 0.8 1 1.2  /  

 $


    2   

&    &          
   B    

  

  
 Æ    
 #    
         
    
 2   ;  2 (  2 %


  
 
    2   &    #    &  
          
   
       &    %    ' 
 


  # 5
     & 
    
 %        
    
'      # 5 
   %   58      
      
       +6   6=7         
 &   
 
  % #     2   
  2 *  
'              
 # +      
& 
      

     &   &
  
 
 &      

 
    
   
    2  # 0   
    
               
    
    
 
  1  % 

  
        
   
' &     
       
  B    1  %   # Visual Post Analysis of Association Rules 65  % 3      $%
       0  
    
   '    &  
     &  


 
  2
    

 
  
&
 
&   
 

  #         
       

  
   



 %
 
    &   C  

4 3
 C4 
 
3 
 #   
 
    

    &   '   &   ##    

  
   &'    
 
                 & 6 # 
 
   
       '

   0= 


  
  
  

 
  &
       


           # 5    
 
 
 

    
     
&B              

 &     &  



    


 

   
  
2
  #    +# $# 
 , # ) 8 + # <      $%
 (
+

 
  
  4
(
# 3

   
;;6  = 2 


# # >  # # & ;;6' !:?! " ! 5
/@
 # A3< 4B

 @

# %# 3  & ;:6' 6 5%//

#  8  # < 3% # C*    % /     $%
#     #  !#  7# &!' 7" ?7:!  7 5%//

#  8  # <    3%    

    $%
# %    #  " &! ' 6:?6; 9 D # D 8 > 
# <        $%
 ( 
  
 3  5
 

# A=#  
 D
E
# 3=  &
'<  !  3

   %    # 3  
# D

(
 &!' 7;;?97 " 
(
# <  
  = #    4 
  D

#  5 4 )
 $
 =6!!: # 7 
 5 4 
 . 

 & ; ' : 
(
# <  %    +  3
  
# %    #  6#  # & ;;' 7:?"7  F

 
# #  # D# $, 
# 3# ) 
# D# 8 
,# <    

  %
  

    

    %
# 3

   
)  
  


     F+



 F ;7# & ;;7' 7 ?7: ; 4 %# 5# D%# > 8 # G< 3%   % /  
 

    # 3

   
 =F 
  


 F+

 
     &F ;;'# %% 9 # 
# #  & ;;;'  4 %# 5# D%# ># ># F 8 
# <  %  
 C*  

     $%
 3

   
3 .   


 F+

 
     &3F ;;'  
 C#   !"!# 5
E  & ;;;' 66 ! Bruzzese and Davino   5%//

         # # 4, # 4# $  # F 8 %  < 

  
  3%   
 3
# >,  
  
;;;  = 2 $

 %
      F+

 
 & ;;;' ! ) 
# D# F

 
# # $, 
# 3 # D
# F 8  # D< 3%   %     

    %
 >,  
  
C 4;9 >,     #  
4
 #  F+

 
  (
# D
, # =


#   ;;9 & ;;9' 7:?9! 6 >
(
# < 2 3%  

   
  =

 /
  H%  
    $%
 3

   F+

 
     >, #  
& ;;' 7 ># 3# > 
# 3# ) # A<  % /     $%
  )
*    > # =# F
#  &
'< 3

   CCC     % /  B;;# CCC  3
# 4  #  & ;;;'       
  ) 

 $<  )< %  
 4< 

 )< ) . )< %
  < 
  3<  +  3$5< 
 
3<  $C <
   $)=<  
+ 
< 

)4 < 
  
2 <  
2<  %
 4 < . +    %  < %
 )< (
, C)< +

 
 3 <   %  3$ < 
   $5<  
$$C< 
  % / 
$)4< %
 I  < 
  )4< ) . 2< 
  <   4 < . 3<  
 
+ 42)< 
 < %  3 =< 
3$<     $<   
$<  %
 
$5< 
)=< 
+ )HJ< -% /  + Cooperation between automatic algorithms, interactive algorithms and visualization tools for Visual Data Mining François Poulet ESIEA Recherche 38, rue des Docteurs Calmette et Guérin Parc Universitaire de Laval-Changé 53000 Laval, France poulet@esiea-ouest.fr Abstract. Visual data-mining strategy lies in tightly coupling the visualizations and analytical processes into one data-mining tool that takes advantage of the strengths from multiple sources. This paper presents concrete cooperation between automatic algorithms, interactive algorithms and visualization tools. The first kind of cooperation is an interactive decision tree algorithm called CIAD+. It allows the user to be helped by an automatic algorithm based on a support vector machine (SVM) to optimize the interactive split performed at the current tree node or to compute the best split in an automatic mode. This algorithm is then modified to perform an unsupervised task, the resulting clustering algorithm has the same kind of help mechanism based on another automatic algorithm (the k-means). The last effective cooperation is a visualization algorithm used to explain the results of SVM algorithm. This visualization tool is also used to view the successive planes computed by the incremental SVM algorithm. 1 Introduction Knowledge Discovery in Databases (or KDD) can be defined [1] as the non-trivial process of identifying patterns in the data that are valid, novel, potentially useful and understandable. In most existing data mining tools, visualization is only used during two particular steps of the data mining process: in the first step to view the original data, and in the last step to view the final results. Between these two steps, an automatic algorithm is used to perform the data-mining task. The user has only to tune some parameters before running his algorithm and wait for its results. Some new methods have recently appeared [2], [3], [4], trying to involve more significantly the user in the data mining process and using more intensively the visualization [5], [6], this new kind of approach is called visual data mining. In this paper we present some tools we have developed, which integrate automatic algorithms, interactive algorithms and visualization tools. These tools are two interactive classification algorithms and a visualization tool created to show the 68 Poulet results of an automatic algorithm. The classification algorithms use both human pattern recognition facilities and computer calculus power to perform an efficient user-centered classification. This paper is organized as follows. In section 2 we briefly describe some existing interactive decision tree algorithms and then we present our new interactive algorithms, the first one is an interactive decision tree algorithm called CIAD+ (Interactive Decision Tree Construction) using support vector machine (SVM) and the second is derived from the first one and performs unsupervised classification (clustering). In section 3 we present a graphical tool used to explain the results of support vector machine algorithms. These algorithms are known to be efficient but they are used as "black boxes", there is no explanation of their results. Our visualization tool graphically explains the results of the SVM algorithm. The implemented SVM algorithm can modify an existing linear classifier by both retiring old data and adding new data. We visualize the successive separating planes computed by this algorithm. Section 4 concludes the paper and lists some future work. 2 Interactive decision tree construction Some new user-centered manual (i.e. interactive or non-automatic) algorithms inducing decision trees have appeared recently: Perception Based Classification (PBC) [7], Decision Tree Visualization (DTViz) [8], [9] or CIAD [10]. All of them try to involve the user more intensively in the data-mining process. They are intended to be used by a domain expert not the usual statistician or data-analysis expert. This new kind of approach has the following advantages: - the quality of the results is improved by the use of human pattern recognition capabilities, - using the domain knowledge during the whole process (and not only for the interpretation of the results) allows a guided search for patterns, - the confidence in the results is improved, the KDD process is not just a "black box" giving more or less comprehensible results. The technical part of these algorithms are somewhat different: PBC and DTViz use an univariate decision tree by choosing split points on numeric attributes in an interactive visualization. They use a bar visualization of the data: within a bar, the attribute values are sorted and mapped to pixels in a line-by-line fashion according to their order. Each attribute is visualized in an independent bar (cf. fig.1). The first step is to sort the pairs (attri, class) according to attribute values, and then to map to lines colored according to class values. When the data set number of items is too large, each pair (attri, class) of the data set is represented with a pixel instead of a line. Once all the bars have been created, the interactive algorithm can start. The classification algorithm performs univariate splits and allows binary splits as well as n-ary splits. CIAD is described in the next section and, in section 2.2, we present a new version of CIAD (called CIAD+) with a help tool added to the interactive algorithm allowing the user to perform an automatic computation of the best bivariate split. Cooperation Between Automatic Algorithms 69 [9] uses a two dimensional polygon or more precisely, an open-sided polygon (i.e. a polyline) in a two dimensional matrix. It is interactively drawn in the matrix. The display is made of one 2D matrix and one-dimensional bar graphs (like in PBC). Only PBC provides the user with an automatic algorithm to help him choosing the best split in a given tree node. The other algorithms can only be run in a 100% manual interactive way. Attr.1 1 5 2 9 3 6 Attr.2 5 7 1 3 2 9 Class A A B B A B Attr.1 1 2 3 5 6 9 Class A B A A B B Attr.2 1 2 3 5 7 9 Class B A B A A B Attr.1 Class A Class B Attr.2 Fig. 1. Creation of the visualization bars with PBC 2.1 CIAD CIAD uses a bivariate decision tree using line drawing in a set of two-dimensional matrices (like scatter plot matrices [11]). The first step of the algorithm is the creation of a set of (n-1)2/2 two-dimensional matrices (n being the number of attributes). These matrices are the two dimensional projections of all possible pairs of attributes, the color of the point corresponds to the class value. This is a very effective way to graphically discover relationships between two quantitative attributes. One particular matrix can be selected and displayed in a larger size in the bottom right of the view (as shown in figure 2 using the Segment data set from the UCI repository [12], it is made of 19 continuous attributes, 7 classes and 2310 instances). Then the user can start the interactive decision tree construction by drawing a line in the selected matrix and performing thus a binary, univariate or bi-variate split in the current node of the tree. The strategy used to find the best split is the following. We try to find a split giving the largest pure partition, the splitting line (parallel to the axis or oblique) is interactively drawn on the screen with the mouse. The pure partition is then removed from all the projections. If a single split is not enough to get a pure partition, each half-space created by the first split will be treated alternately in a recursive way (the alternate half-space is hidden during the current one's treatment). 70 Poulet At each step of the classification, some additional information can be provided to the user like the size of the resulting nodes, the quality of the split (purity of the resulting partition) or overall purity. Some other interactions are available to help the user: it is possible to hide / show / highlight one class, one element or a group of elements. Fig. 2. The Segment data set displayed with CIAD 2.2 CIAD+ The first version of the CIAD algorithm was only an interactive algorithm. No help was available for the user, and sometimes, it was difficult to find the best pure partition in the set of two-dimensional matrices. We have decided to provide such help. Our first intention was to use a modified OC1 (Oblique Classifier 1) algorithm [13]: OC1 performs real oblique cuts (we have a real n-dimensional hyperplane with n-dimensional data) and in CIAD, the cuts are only "oblique" in two dimensions. The plane coefficients are null in all the other dimensions. We have made another choice: Cooperation Between Automatic Algorithms 71 we use a support vector machine (SVM). This algorithm is equivalent to OC1 in its simplest use, and will allow us to benefit from all its other possibilities for further developments. 2.2.1 The SVM algorithms The SVM algorithms are kernel-based classification methods. They can be seen as a geometric problem: to find the best separating plane of a two classes data set. A lot of methods can be used to find this best plane, and a lot of algorithms have been published. A review of the different algorithms can be found in [14]. They are used in a wide range of real-world applications such as text categorization, hand-written character recognition, image classification or bioinformatics. We briefly describe here the basis of the algorithm, from the geometrical point of view. Fig. 3. Two possible separating planes The aim of the SVM algorithm is to find the best separating plane between the ndimensional elements of two classes. There are two different cases according to the nature of the data: they are linearly separable or not. In the first case, the data are linearly separable i.e. there exists a plane that correctly classifies all the points in the two sets. But there are infinitely many separating planes as shown in Fig.3. Geometrically, the best plane is defined as being furthest from both classes (i.e. small perturbations of any point would not introduce misclassification errors). The problem is to construct such a plane. It has been shown in [15] that this problem is equivalent to finding the convex hull (i.e. the smallest convex set containing the points) of each class, and then to finding the nearest two points (one from each convex hull); the best plane bisects these closest points. Fig. 4. The best separating plane bisects the closest points In the second case, the data are not linearly separable (i.e. the intersection of the two convex hulls is not empty). There is no clear definition of what is the "best" plane. The solution is to create a misclassification error, and to try to minimize this error. 72 Poulet 2.2.2 SVM in CIAD+ We use two different SVM algorithms in CIAD+. The first one is the geometric version described in section 2.2.1 for the linearly separable case. The convex hulls are computed in two dimensions with the quick hull algorithm [16], then the two closest points of the convex hulls are computed with the rotating calipers algorithm [17]. For the linearly inseparable case, a lot of solutions have been developed and compared, we have chosen one of these algorithms: the RLP (Robust Linear Programming) algorithm [18] because it is the best one when the data are not linearly separable. The RLP algorithm will compute the separating plane wx=γ minimizing the average violations: 1 m 1 k ( Bi w − γ + 1) + ∑ (− Ai w + γ + 1) + + k ∑ m i =1 i =1 (1) of points of A lying on the wrong size of the plane wx=γ+1, and of points of B lying on the wrong side of the plane wx=γ-1 as shown in Fig.5. This algorithm computes the n-dimensional hyperplane, we have modified it to compute the best two-dimensional plane. wx=γ+1 wx=γ Ai wx=γ-1 Bi Fig. 5. Optimal plane for linearly inseparable data SVMs in CIAD+ are used to help the user. The first kind of help is when the user draws interactively the separating line on the screen with a pure partition on one side, the optional help optimizes the line position to reach the best line position (furthest from both groups) with the computation of the closest points of the convex hulls. The second kind of help is the same case except there is no pure partition, the best line position is computed with the RLP algorithm in the two dimensions corresponding to the selected matrix. The last kind of help is used when the user cannot find a separating plane, the help algorithm has to compute the best separating plane among all the ones corresponding to the projections along pairs of attributes. So we compute all the separating lines in the 2D projections and we keep the best one. Cooperation Between Automatic Algorithms 73 This help mechanism can be turned on / off by the user. It slightly improves the accuracy of the results on the training sets (this result may be more significant according to the kind of user), more on the test set, and it considerably reduces the time needed to perform the classification and increases the ease of use. An example is shown on figure 6, the left part is the original line drawn interactively by the user on the screen and the right part shows the transformed line (the best separating plane computed with the convex hulls). Fig. 6. An example of the automatic best separating plane on iris data set 2.3 Clustering The interactive algorithm described in the previous section can also be used for unsupervised classification. The computation of the convex hulls and the nearest points can be computed with or without the class information. The proposed algorithm can perform either usual decision tree (supervised classification) or clustering (unsupervised classification). This kind of approach allows the user to perform clustering of the dataset easily using its pattern recognition capabilities and avoiding the usual complexity of the other algorithms. The same kind of help as for decision tree construction is provided to the user: the separating line drawn in a 2D projection can be optimized to be the furthest from the two clusters. But there is one difference with the decision tree construction algorithm, when the user does not perceive clearly a separating line, this line can be computed automatically (with a modified SVM algorithm). This SVM algorithm cannot be used without the class information, so we have chosen a k-means algorithm. All possible partitions into two clusters are searched for in each matrix and the best one is kept. Then we compute the convex hulls and nearest points to find the best separating line in the same way as for decision tree construction. 74 Poulet As shown in [19], this kind of algorithm may not be very efficient for high dimensional data sets because axis-parallel projections may lead to an important loss of information. This restriction exists anyway because of the graphical representation we use (we cannot display a very large quantity of scatter plot matrices). On the other hand, the results are more comprehensible because we only use one attribute on each axis and not a linear combination of various number of attributes for the axes. 2.4 Some results of interactive algorithms The characteristics of the different algorithms are summarized in table 1. Some results of interactive algorithms compared to automatic ones have been presented by their authors. To summarize them, we can say their results concerning efficiency, are generally at least as good as automatic decision tree algorithms such as CART (Classification And Regression Trees) [20], C4.5 [21], OC1 [13], SLIQ (Supervised Learning In Quest) [22] or LTree (Linear Tree) [23]. The main difference between the two kinds of algorithms is the tree size. Most of the time, interactive algorithms have smaller tree sizes. This tree size reduction can significantly increase the result comprehensibility. Furthermore, as the user is involved in the tree construction, his confidence in the model is increased too. This may be a little less significant for the LTree algorithm because of the open-sided polygon used in the classification: they are easy to understand during the visual construction step of the tree, but without this information, the resulting equations may be not so easy to understand. PBC Vis. technique bar graphs + 1 scatter plot bar graphs DTViz bar graphs CIAD+ set of scatter plot Ware split poly-line binary univariate n-ary univariate n-ary bivariate binary + large dataset tree size large datasets help large datasets result comprehensibility plot of qual. x qual. attr. loss of information tree size help plot of qual. x qual. attr. loss of information Table 1. Comparison of interactive decision tree algorithms If we compare the interactive algorithms, we can say PBC and DTViz are particularly interesting for large data sets (because of the pixelisation technique used), but their pixelisation technique introduces some bias in the data: for example two classes very far one from the other have the same representation as the same two classes very near one to one other. We lose the distance information in this kind of representation. Ware and CIAD+ will provide smaller trees because they can use bivariate splits, but their kind of visualization tool (scatterplot matrices) are not at all suitable for the display of two qualitative attributes (a lot of points have the same projection). Only two algorithms provide the user with a help, PBC and CIAD+, this is a significant advantage of these two algorithms because during the decision tree Cooperation Between Automatic Algorithms 75 construction, there is often a least one particular step where the best split is difficult to visually detect. The kind of cooperation between automatic and manual algorithms in PBC and CIAD+ shows the interest of mixing the human pattern recognition facilities and the computer processing power. The human pattern recognition facilities reduce the cost of the computation of the best separating plane, and the computer processing power can be used at low cost (for a single step, instead of the whole process) when the human pattern recognition fails. 3 Visualization of SVM results Another kind of cooperation is between automatic algorithms and visualization tools used to show the results. As described in the previous section, SVM are today widely used because they give high quality results, but they are used as a "black box". They give high quality results, but there is no explanation of these results. One paper [24] talks about SVM results visualization: they use projection-based tour [25] method to visualize the results. They use a visualization of the distribution of the data predicted class (by the way of histograms), a visualization of the data and the support vectors in 2D projection, and examine the weights of the plane coordinates to find the most important attributes for the classification. The authors recognize that their approach is very "labor intensive for the analyst. It cannot be automated because it relies heavily on the analyst's visual skills and patience for watching rotations and manually adjusting projection coefficients." 3.1 Visualization of the SVM separating plane Our approach is to visualize all the intersections of the 2D planes (of the scatter plot matrices) with the separating plane computed by the SVM algorithm. We have chosen to use the incremental SVM algorithm from [26]. This algorithm gives the coefficient values of the separating hyperplane and the accuracy of the algorithm. We visualize the intersection of this hyperplane with the 2D scatter plot matrices, i.e. a line in each matrix (as shown in Figure 7 with the diabetes data set, from the UCI repository). As we can see on figure 7, the resulting lines do not necessarily separate the two classes, the hyperplane does separate the data (the accuracy of the incremental SVM is 77,8% on this dataset), but not its "2D projections". It is only an approximate interpretation of the results. All 2D representations of one n-dimensional feature will lead to a loose part of the information like the lines we get here or the support vectors displayed in [24]. This kind of representation seems more comprehensible than the support vectors, but it can only be used with a linear kernel function. For large data sets, it is possible to only display the separating plane and not the data. The SVM algorithm used is able to classify 1 billion points, and such a quantity of points cannot be displayed in a reasonable time (furthermore this kind of representation is not at all suitable for such data set size). 76 Poulet For other kinds of kernel functions (not linear), this method cannot be used. Fig. 7. Visualization of the separating hyperplane of the diabetes dataset. 3.2 Visualization of the evolution of SVM separating planes A very interesting feature of the incremental support vector machine we have used is its capability to modify an existing linear classifier by both withdrawing old data and adding new data. We use our visualization tool to show the successive modifications of the separating plane projections. The first plane is calculated (and projected on the 2D matrices) and then blocks of data are successively added and withdrawn. The modification of the plane is computed and the corresponding projections are displayed in the matrices. This kind of visualization tool is very powerful to examine the variations of the separating plane according to the data evolution. Even if the projections used lose some amount of information, we know what the attributes involved in the Cooperation Between Automatic Algorithms 77 modification of the separating plane are. The evolution of the n-dimensional plane is very difficult to show in another way. The authors of the paper describing the algorithm measure the difference between planes by calculating the angle between their normals. These angle values are then displayed like circle radii. Fig. 8. Three linked tools in the FullView environment 4. Conclusion and future work Before concluding, some words about the implementation. All these tools have been developed using C/C++ and three open source libraries: OpenGL, Open-Motif and Open-Inventor. OpenGL is used to easily manipulate 3D objects, Open-Motif for the graphical user interface (menus, dialogs, buttons, etc.) and Open-Inventor to manage the 3D scene. These tools are included in a 3D environment, described in [27], where each tool can be linked to other tools and be added or removed as needed. Figure 8 shows an example with a set of 2D scatter plot matrices, a 3D matrix and parallel coordinates. The element selected in the 3D matrix appeared selected too in the set of scatter plot matrices and in the parallel coordinates. The software program can be run on any platform using X-Window, it only needs to be compiled with a standard C++ compiler. Currently, the software program is developed on SGI O2 and PCs with Linux. 78 Poulet In this paper we have presented two new interactive classification tools and a visualization tool to explain the results of an automatic SVM algorithm. The classification tools are intended to involve the user in the whole classification task in order to: - take into account the domain knowledge, - improve the result comprehensibility, and the confidence in the results (because the user has taken part in the model construction), - exploit human capabilities in graphical analysis and pattern recognition. The visualization tool was created to help the user in understanding the results of an automatic SVM algorithm. These SVM algorithms are more and more frequently used and give efficient results in various applications, but they are used as "blackboxes". Our tool gives an approximate but informative graphical interpretation of these results. A forthcoming improvement will be another kind of cooperation between SVM and visualization tools: an interactive visualization tool will be used to improve the SVM results when we have to classify more than two classes. References 1. U. Fayyad, G. Piatetsky-Shapiro, P. Smyth, R. Uthurusamy, Eds, "Advances in Knowledge Discovery and Data Mining", AAAI Press, 1996. 2. P.Wong, "Visual Data Mining", in IEEE Computer Graphics and Applications, 19(5), 20-21, 1999. 3. F. Poulet, "Visualization in data mining and knowledge discovery," in Proc. HCP'99, 10th Mini Euro Conference "Human Centered Processes" ed. P. Lenca (Brest, 1999), 183-192. 4. M.Ankerst, M.Ester, H-P.Kriegel, "Toward an Effective Cooperation of the Computer and the User for Classification" in proc. of KDD'2001, 179-188. 5. C.Aggarwal, "Towards Effective and Interpretable Data Mining by Visual Interaction", in SIKDD Explorations 3(2), 11-22, accessed from www.acm.org/sigkdd/explorations/. 6. B.Schneiderman, "Inventing Discovery Tools: Combining Information Visualization with Data Mining", in Information Visualization 1(1), 5-12, 2002. 7. M.Ankerst, "Visual Data Mining", PhD Thesis, Ludwig Maximilians University of Munich, 2000. 8. J.Han, N.Cercone, "Interactive Construction of Decision Trees" in proc. of PAKDD'2001, LNAI 2035, 575-580, 2001. 9. M.Ware, E.Franck, G.Holmes, M.Hall, I.Witten, "Interactive Machine Learning: Letting Users Build Classifiers", in International Journal of Human-Computer Studies (55), 281292, 2001. 10.F.Poulet, "CIAD: Interactive Decision Tree Construction" in proc. of VIII Rencontres de la Société Francophone de Classification", Pointe-à-Pitre, 275-282, 2001 (in french). 11.J.Chambers, W.Cleveland, B.Kleiner, P.Tukey, "Graphical Methods for Data Analysis", Wadsworth (1983). 12.C.Blake, C.Merz, UCI Repository of machine learning databases, [http://www.ics.uci.edu/~mlearn/MLRepository.html]. Irvine, CA: University of California, Department of Information and Computer Science, (1998). 13.S.Murthy, S.Kasif, S.Salzberg, "A system for induction of oblique trees", Journal of Artificial Intelligence Research 2, 1-32, 1994. Cooperation Between Automatic Algorithms 79 14.K-R.Muller, S.Mika, G.Ratsch, K.Tsuda, B.Scholkopf, "An Introduction to Kernel-Based Learning Algorithms" in IEEE Transactions on Neural Networks, 12(2), 181-201, 2001. 15.K.Bennett, E.Bredensteiner, "Duality and Geometry in SVM Classifiers", in proc of the Seventeenth International Conference on Machine Learning, Pat Langley Editor, Morgan Kaufmann, San Francisco, 57-64, 2000. 16.C.Barber, D.Dobkin, H.Huhdanpaa, "The Quickhull algorithm for convex hulls", in ACM Transactions On Mathematical Software, 22, 469-483, 1996. 17.G.Toussaint, "Solving geometric problems with the rotating calipers", in proc. of IEEE MELECON'83, Athens, Greece, pp. A10.02/1-4, 1983. 18.K.Bennett, O.Mangasarian, "Robust Linear Programming Discrimination of Two Linearly Inseparable Sets", in Optimization Methods and Software, 1, 23-34, 1992. 19.C.Aggarwal, P.Yu, "Redifining Clustering for High-Dimensional Applications", in IEEE Transactions on Knowledge and Data Engineering, 14(2), 210-225, 2002. 20.L.Breiman, J.Friedman, R.Olsen, C.Stone, "Classification and Regression Trees", Wadsworth, (1984). 21.J.Quinlan, "C4.5: Programs for Machine Learning", Morgan-Kaufman Publishers, 1993. 22.M.Metha, R.Agrawal, J.Rissanen, "SLIQ: A fast scalable classifier for data mining", in proc. of the 5th International Conference on Extending Database Technology, Avignon, France, 1996, 18-32. 23.J.Gama, P.Brazdil, "Linear Tree" in Intelligent Data Analysis, 3, 1-22 (1999). 24.D.Caragea, D.Cook, V.Honavar, "Gaining Insights into Support Vector Machine Pattern Classifiers Using Projection-Based Tour Method", in proc. of KDD'2001 Workshop on Visual Data Mining. 25.D.Asimov, "The Grand Tour: A Tool for Viewing Multidimensional Data" in SIAM Journal of Scientific and Statistical Computing, 6(1):128-143, 1985. 26. G. Fung, O. Mangasarian, "Incremental Support Vector Machine Classification" in proc. of the 2nd SIAM International Conference on Data Mining, Airlington, USA, Apr.11-13, 2002. 27.F.Poulet, "FullView: A Visual Data-Mining Environment", in International Journal of Image and Graphics, 2(1), 127-144, 2002. Defining Like-minded Agents with the Aid of Visualization Penny Noy and Michael Schroeder City University, London EC1V 0HB, UK p.a.noy, msch @soi.city.ac.uk Abstract. Profile carrying agents offer the opportunity of meeting like minds and increasing efficiency in many information search applications. Profiles can also increase the sophistication of relationships and interactions in multi-agent systems in general. Such profiles (feature lists) may be of the agent owner (interests) or of the information sought (specifications). At the same time there is a continuing increase of profiling data of increasing complexity becoming available. The paper first describes various possibilities for defining profiles and selecting similarity metrics in the agent interaction context. General and domain-specific data sets are specially constructed and used to provide a concrete view of the behaviour of the metrics with visualization. Visualization usually results in xy (or xyz) coordinates for each agent, placing them in a profile space. The use of these coordinates as a means of carrying and comparing one’s profile without revealing it to other agents is proposed. Thus this paper extends our previous work on visualizing multivariate data and proximity data in the agent scenario in two ways where visualization is not the end product: using visualization tools in the phase of similarity metric choice; proposing the use of coordinates as a profile. The paper seeks to illustrate, with these applications, an objective of visual data mining, namely the increased integration of visualization and analytic techniques. 1 Introduction Every second that passes witnesses millions of people searching for information via a myriad of means. Increasingly such means and media employed are electronic. Many people are seeking similar information and work on similar problems. Software agents carrying our profile can meet with the representatives of others and exchange important information or provide introductions. Consider the following messages from some of my (hypothetical) personal agents: myNetworkAgent: Good morning. Professor Blake in Helsinki has just started work on one of your main research areas. myShoppingAssistant: Here are the fourteen houses closest to your ideal specification. myInformationSearchAgent: Here are the four most relevant documents in your specified area ’agent matchmaking’ from the survey point of view. 82 Noy and Schroeder These agents may be providing real-time search responses or background information monitoring. The concepts are embodied in many search and classification applications. These may be ordinary searches, not explicitly involving other agents (or agents at all), but agents may assist in improving precision and speed. Of course the above scenario is far fetched in several senses: the degree to which information is searchable (i.e. specified according to agreed ontologies); the accuracy of the metrics in matching seeker with sought; the willingness of humans to allow an agent to carry their profile (and the security issues involved); privacy of information. The focus of this paper is the task of ’matching seeker with sought’. This may involve a user profile or a profile of a task or desired piece of information. It is in this general sense that the word profile is used here. To compare profiles a metric is needed. There are many examples of this in agent systems. For instance: Faratin et al use the maximum value distance in making negotiation trade-offs[10]; The Yenga system uses correlation to determine user interests and then a direct comparison to find a common joint interest [4]; Somlo and Howe use incremental clustering for profile maintenance in information gathering web agents based on term vectors of documents the user has shown interest in [14]; GRAPPA (Generic Request Architecture for Passive Provider Agents) is a configurable matchmaking framework which includes demand and supply profiles and has been applied to matching multidimensional profiles of job applicants to vacancies [18,17]; Ogston and Vassiliadis use minimal agents working without a facilitator to match services and providers [9]. The many possible application areas divide broadly into two areas - matchmaking (e.g. matching services to clients, people connector) and search. The general topic lies within organizational concepts in multi-agent systems and improving learning with communication [21]. The finding and remembering of appropriate, like-minded agents can be centralized, left as a diffusion process or engineered in a computational ecology sense [5,4,9]. In the agent domain, as in others, visualization is often seen as a separate application that one adds on to an application for a variety of purposes. It may be of value from this point of view, but it can do more. Visual datamining seeks, amongst other things, to give the human visual system a more central role in the knowledge discovery process, to increase the integration of visualization and datamining techniques. This can be approached in a variety of ways, taking advantage of the human visual system’s pattern recognition abilities, for instance, or presenting overviews of large amounts of data in novel ways. Related to this, but with a different focus, is another way, which the work presented here seeks to illustrate: the merging of the visualization and analytic processes. Here the question of whether visualization is the end result is not so important. Two examples of this are presented here: – Overview understanding and similarity metric choice merge: Metrics are used for layouts involving dimension reduction, but different metrics produce different layouts. A metric (or other transformation process) may be needed for layout, especially in creating an overview of large complex datsets, but how is the user made aware of the different possibilities and their implications? At the same time many applications use a similarity measure - how can designers choose appropriately? – Viewer and computational object (an agent) merge: Our layout creates a topic space. If such topic spaces are valid (they are usually approximations), can we Defining Like-Minded Agents 83 use this notion of spaces (or surfaces) for software agents to use when meeting or seeking other agents? In this case the visualization concept is being used to assist in agent-orientated computation. Can this help in the pursuit of the use of visualization techniques for reasoning (diagrammatic reasoning [6])? Our earlier work looked at proximity data and multivariate data in the agent domain and indicated possible metric choices for visualization[11,13]. From the agent point of view, the question is how can we apply this and how can we assist in the problem of metric choice for profiling and classification in the agent domain. How can we meaningfully identify like-minded agents and then put this to use? The agent paradigm is considered by some to be a valuable way of looking at problems and therefore of general application. Our work in the agent and visualization fields seeks to use visualization to serve the agent community, but, from the agent-orientated computation point of view, suggests other uses of agent ideas within visualization. The paper first briefly surveys visualization possibilities for different types of data matrices and presents definitions of profiles, considering also the desirability of comparing profiles without revealing them. It then looks at metric choice and suggests strategies to improve choice using visualization techniques and the designing of a classification system for evaluating the metrics. The idea of using the visualization position coordinates as a profile is introduced and an example given. The nature of the examples in this paper is illustrative and the intention is to show the merging process, where visualization is not necessarily the end product, as well as to present the two specific applications. 2 Defining Profiles In general an agent’s profile is considered to be a vector of interests and behaviours (a feature list) or a similarity measure or sets and/or combinations of these [11,13]. The purpose of our work in visualization was to find layouts (in 2D or 3D) which would satisfy (usually approximately) these data either by using mathematical transformations (effective reductions via e.g. Principal Component Analysis (PCA) or distance metrics followed by Principal Coordinates Analysis (PCoA), spring embedding [3] or Selforganizing Map (SOM)[16]) or novel representations (e.g. colour maps, hierarchical axes, ’Daisy’, parallel coordinates[1,2,15]). PCA, SOM and PCoA are described briefly here as they are used in the discussion that follows: – Principal Components Analysis: PCA is a means by which a multivariate data table is transformed into a table of factor values, the factors being ordered by importance. The two or three most important factors, the principal components, can then be displayed in 2D or 3D space. – Self Organizing Map: The SOM algorithm [16] is an unsupervised neural net that can be used as a method of dimension reduction for visualization. It automatically organizes entities onto a two-dimensional grid so that related entities appear close to each other. 84 Noy and Schroeder – Principal Coordinates Analysis: PCoA is used for proximity data, finding first a multivariate matrix which satisfies the distances, then transforming this into its principal components so that the two or three most important factors can be displayed in a similar fashion to PCA. These visual representations may provide meaningful clusters or reveal patterns from which knowledge can be gained. A key problem in this area is that different methods produce different clusters (and cluster shapes). The determination of an appropriate metric 1 is a difficult problem for which general solutions are not evident. We propose the use of constructed data in a process called signature exploration [8] to assist in this area. This process uses specially constructed data sets to increase the user’s understanding of the behaviour of visualization algorithms applied to high dimensional data. Two developments suggest themselves from aspects of our previous visualization work: a tool for metric choice; the use of layout coordinates as a profile. – Tool for metric choice: Agents need to compare profiles, i.e. when they meet they need to be able to compare themselves (or their tasks) and get a measure of similarity which they can interpret. Assuming (for the moment) that they are carrying their profile with them, they will need to apply an algorithm to calculate a similarity measure by both submitting their profiles to the algorithm, either both independently of the other, or via an intermediary. In designing a specific application a decision (by the designer) needs to made about what similarity measure is appropriate. The tool for metric choice developed in application of the principle of signature exploration provides an interactive interface which can help the designer to choose the metric. – Use the layout coordinates as a profile: For layout on the screen, the data transformation or set of similarity measures results in x/y (or x/y/z) coordinates for each entity. For complex data this usually involves a significant error (i.e. it is normally not possible to find a layout which will satisfy the similarity measurements - on the one hand - and matrix transformations and truncations to 2 or 3 attributes rely on a sharp fall off of the relevant eigenvalues, which is unusual for complex data sets on the other hand). Nevertheless such algorithms are commonly used and thus the approximations involved are often adequate. The relevant point here is that the end result is that there is an x/y (or x/y/z) coordinate pair associated with each entity and within the current space of possibilities this locates their interest position. This suggests the possibility of them carrying a much more lightweight position profile with them, that also means they can compare positions without revealing profiles. The use xy or xyz coordinates as the profile avoids revealing the profile, but the implication is that either there must be a central entity which will do the calculation (and thus that one needs to reveal one’s profile to) and then give the agent its coordinates and the bounds of the space (so that it can judge relative similarity). Also this does not deal easily with dynamic situations (i.e. reflecting changing profiles), as it would require a periodic return to base to profile updating. A possible alternative is to calculate one’s own coordinates with respect to a number of reference 1 metric is here used in a general sense to mean a means of measurement which may not result in a numerical measure directly, i.e. possibly indirectly by means of layout position derived directly from SOM or PCA Defining Like-Minded Agents 85 points, i.e. calculate one’s proximity to the reference points and then find a position in space to satisfy this reduced set of distances. For instance for a feature list of length 5, consisting of a set of five possible agent interest areas and interest values in the range 0 to 1 (say), the following is an indication of the bounds of the space. ABCDE agent1 1 0 0 0 0 agent2 0 1 0 0 0 agent3 0 0 1 0 0 agent4 0 0 0 1 0 agent5 0 0 0 0 1 It may be unwise to base the position on a computation that satisfies the similarity measures to all of these vectors (since this increases the inaccuracy of the layout), but the agent could carry the set of coordinates for certain bounds (or other reference vectors) and profile position, having the calculations made back at base. These ideas are illustrated below. 3 Choosing a Metric - Possibilities For specific applications different metrics are used, this means that often an applications area uses one metric only. Measures may be chosen because of time complexity issues, rather than that they provide the most accurate or appropriate measure. There is also a link between the creation of the feature list and the metric choice (i.e. the formulation of the feature list affects which metric provides the most appropriate clustering) which is a further complication. In general terms the choice of metric and creation of the feature list should correspond to the required classification, but in many situations the starting point is an unknown set of data and clustering indications are sought. There is no training set and no classification. It is likely that different classifications exist. In fact there are hidden classifications, that is to say, the user has a set of things they are interested in and they would like to have the entities (other users, documents..) classified according to these groupings. One of the purposes of the signature exploration process that is being developed is to explore the mapping to clusters (via various metrics) of features of interest to the user. Originating as part of work to increase comprehension and choice of algorithms for visualization of complex data, it does not focus on feature list construction but on metric choice for a given feature set. In the process of constructing data sets for evaluation of the different options the user creates an ad hoc classification system for assessment purposes (demonstrated below). 3.1 How to choose - metrics, feature selection and weighting The first issue is to specify the variables to be used in describing the profile and the ways in which pairwise similarities can be derived from the matrix formed by the set of profiles. 86 Noy and Schroeder Many different measures of pairwise similarity have been proposed [7,19]. Some are closely related to one another. Measures are usually presented that are particularly relevant for comparing objects that are described by a single type of variable. This discussion restricts itself to quantitative data type for brevity. Quantitative variable Let xik denote the value that the kth quantitative variable takes n; k 1 p . The Minkowski metric defines a family of for the ith object i 1 dissimilarity measures, indexed by the parameter . Minkowski metric p wk xik di j x jk 1 1 (1) k 1 p are non-negative weights associated with the variables, allowing where wi k 1 standardization and weighting of the original variables. Values of of 1 and 2 give the two commonly used metrics of this family. City block p wk xik di j x jk (2) k 1 Euclidean distance p w2k xik di j x jk 2 1 2 (3) k 1 These measures can be standardized, for instance so that di j is bounded by 1. p k 1 , where k denotes the range of values taken by the kth variable. If wk p maxkp 1 k , which preserves the quantitative One could also consider wk comparison between objects. Also consider not the range, but (assuming it is relp maxni 1 xik or evant to consider the possible minimum value then take wk p n wk p maxk 1 maxi 1 xik . For example:- agent1 agent2 Sport Art Music 5 1 3 4 1 5 Without weighting this gives 1.29, with the range 0.816, with max value 0.086. Sometimes it is the relative magnitudes of the different variables that is of interest the behaviour across the variables rather than the absolute values. Put another way, the variables describing the object define a vector with p components and interest is in the comparison of the directions of the vectors. In the following metric the cosine of the angle between the vectors is used. Since values are between -1 and 1, the measure can 1 si j 2. be transformed to take values between 0 and 1 by defining si j Angular separation p k 1 xik x jk (4) si j p p 2 2 1 2 k 1 xik l 1 x jl For the previous example this metric gives a value for s of 0.0465. Defining Like-Minded Agents 87 Mixed variables For profiling of objects the variables will sometimes be of different types: for example a person can be described in terms of their gender (binary variable), their age (quantitative variable, their amount of interest in a subject (ordinal variable if sectioned quantitative variable is used) and their personality classification (nominal variable). A general measure is:General similarity coefficient p p si jk ; di j si j k 1 di jk (5) k 1 where si jk (and correspondingly di jk ) denotes the contribution to the measure of similarity provided by the kth variable. The values of si jk and di jk can take definitions as appropriate to the variable type. Selection (feature extraction) and standardization (normalization) Sometimes it is clear what variables should be used to describe objects. In our case, with profiling queries, documents, specifications and personal profiles, it is likely that variables have to be selected from many possibilities. Thus the process is not straightforward. The pattern recognition literature describes the appropriate specification of variables as feature extraction. It is tempting to include a large number of variables to avoid excluding anything relevant, but the addition of irrelevant variables can mask underlying structure. Whilst the choice of relevant variables is important, there is also the possibility (particularly here - multidimensional nature of profiles themselves) that there is more than one relevant classification based on different, but possible overlapping, sets of variables. Having determined appropriate variables, there is then the question of standardizing and/or differentially weighting them, followed by the construction of measures of similarity. One aspect to the standardization is that two variables can have very different variability across the dataset. It may or may not be desirable to retain this variability. Standardization may also be with respect to the data set under consideration or with respect to a population from which the samples are drawn. In the case of quantitative variables, standardization can be made by dividing by their standard deviation or by the range of values they take in the data set. The idea of standardization lies within the larger problem of the differential weighting of variables. 4 Choosing a Metric - Visual Exploration To assist the process of metric choice the use of specially constructed data sets in an exploration of the algorithm behaviours is proposed in signature exploration. Thus, by examining known data we gain a concrete idea of the behaviour of the various possible metrics. We have suggested a number of possible constructed data types[8]: generic(provided by the application to illustrate the behaviour of the particular algorithm); constructed(determined by the user to illustrate the behaviours in the data that are of interest to them, for evaluation purposes this represents an ad hoc classification); query (by visualization or sql-type, based on an unknown dataset, to examine clustering 88 Noy and Schroeder of metric in practice); landmark (to provide marker entities in the visualization); feedback (the means to enable the user to enter their assessed similarities and find or modify the appropriate metric). This paper limits itself to the first two, generic and constructed, to illustrate. 4.1 Using generic data sets Generic data sets are those considered to illustrate the behaviour of the visualization algorithms. Simple data sets do not always give an intuitive placement after such transformations. In this examination a small matrix of 7 agents were given a randomly assigned level of interest (of 1 to 10) in 7 topics. Agent1 9 3 4 6 5 5 5 Agent2 1 10 10 1 7 2 0 Agent3 4 1 6 8 0 5 7 Agent4 2 7 8 4 0 2 0 Agent5 3 6 4 7 1 10 6 Agent6 1 7 6 5 0 2 0 Agent7 8 1 7 1 2 5 9 Subsequently three other agents were added to illustrate (a) interests identical to agent1 but scaled, (b) agent1 with the same level of interest in each topic and three other agents as in (a), (c) two of the agents showing reverse behaviours of another two. These data sets were visualized with various distance measures (using the tool SpaceExplorer [11,13,12])and comments noted. The results illustrate the similarity in behaviour of the metrics, whilst indicating the differences obtained with the two basic types - Euclidean 3), City, Euclidean and Angular Separation. The measures used were Minkowski ( and Angular Separation (equations 1-4). These were followed by Principle Coordinates Analysis to find points in 2D-space that satisfied the distances. Note that the accuracy of such layouts for visualization is an issue, since it is often very low. In the case of PCoA, the eigenvalues can be examined - the sum of the values of the first three (for 3D layout) being above 70% of the sum of all the eigenvalues accounts for 70% of the variance in the data and is thus an encouraging indicator. As an illustration of this process, figure 1 shows the three shots of City, Euclidean and Angular separation with agents a,b and c having scaled interest distribution of agent1. 4.2 User-constructed data sets Here the user constructs data sets specific to their application, explicitly or implicitly creating a classification system with which to measure the performance of the metrics in clustering their interest feature(s). This may provide a distance matrix for comparison, or such a matrix may be obtained by an informal assessment. This could be followed by feedback analysis to obtain weightings of the feature list, but here the focus is on metric choice rather than modification. Defining Like-Minded Agents 89 Fig. 1. City (top left), Euclidean (top right) and Angular Separation (bottom) measures followed by layout using Principal Coordinates Analysis. Agents a, b and c have scaled interest distribution of agent1. In the angular separation, agents a,b,c and 1 are located in the same position. Step 1 - decide features of interest The first thing to consider is what features in the data one is interested in. We suppose that the aspects are: overlap of interest; intensity of interest; joint disinterest; similar pattern of interest (irrespective of subject). These elements provide a classification system with which we can construct a system to give numerical values to differences between a pair of agents’ interests. Then these differences can be used to give a comparison measure for the behaviours of the various metrics. Statistical measures indicate the closeness of the match. The metrics may not correspond to the classification, even approximately. It could be that it is useful to use the classification system as the similarity measure itself and dispense with the metrics. However, in general, we are looking for a similarity metric that is not just a simple query, but something more subtle, something that reflects the multidimensional nature of the profiling data available. This corresponds to the scope that lies between the two questions:- Are you interested in sport? and Are you like me? Also, if you are interested in sport, it may be valuable to know if you are a specialist or a generalist and in general terms what level your interest is on. Thus other similarity measures act as discriminators in this situation. Final choice of overall similarity measure may consist of additions of different similarity metrics (which may include results of specific queries) and can be arrived at in the manner of equation 5. 90 Noy and Schroeder Fig. 2. Interests level against category for agents 1 and 2 The use of visualizations of data for pairs of agents can assist in the specification of features of interest. Simple diagrams such as bar and pie charts are helpful in designing a measure with which to make an informal assessment of the similarity between two agents. Figures 2, 3 are illustrative of this process and identify the features interest level and interest intensity that are used in step 2. Fig. 3. Interests level difference against category for agents 1 and 2 (left) and intensity of overlap (right) Step 2 - create a measure for the features to generate test data sets Suppose that types of agent similarity are chosen to examine e.g.: overlaps of three or more interests of high intensity; large overlaps irrespective of intensity with high common disinterest. Data is created for a representative member and edge member of desired clusters so that representative pairs of data can be created (or groups if required) to examine the metric behaviours both visually and by comparing distance matrices. To illustrate, a data set was created to produce examples covering the range of possibilities of overlap extent and intensity with respect to a reference agent’s interests (as suggested by the visual Defining Like-Minded Agents 91 explorations of step 1). Then the metrics were examined to see how they clustered the 3 and intensity of overlap group of similarities with number of overlap subjects 2 3. One would expect the metrics to perform badly against this criterion, which is an example where a simple query would perform better (for instance, in the Yenta system, the matching between profiles is done simply on the basis of matching a single granule, which corresponds to a single interest, the metric is used in deriving the interest categories - if you simply want to exchange information on a subject that’s ok, however such aspects as level of expertise are relevant, and finding like-minds needs greater subtlety). For metric discrimination, the distance comparison should be made by also evaluating the test criteria for a number of other features (such as joint disinterest and large overlaps irrespective of intensity) and combining the similarities. Step 3 - evaluate visually and numerically The visual evaluation consists of visualizing the constructed data set and observing how well clustered the group of interest is. However, since the layout of such visualizations is an approximation (in order to satisfy the distances), and the observations not themselves measurements, evaluation by visualization is inexact. On the other hand, numerical evaluation, based on measuring differences between the estimated differences and the differences arrived at by the metric under consideration, is precise, but relies on the ability of the designer to define or estimate similarities between the data entities. For the example above this was done by awarding points according to number and intensity of topics of joint interest. Figure 4 shows PCoA layout with City, Euclidean and Angular Separation differences, the reference agent is circled and the agents that are in my group of interest (according to the criteria in step 2) are indicated. That there is little difference between City and Euclidean indicates that it would be adequate to use city where time complexity was an issue. The three outlines traced by the points in the City and Euclidean plots correspond closely to the classification system and the group of interest is well clustered in visual terms. The Angular Separation plot does not cluster so well, misplacing three agents. The layout of the angular separation distances is actually a screenshot of a 3D representation as the layout was particularly inaccurate and needed the extra dimension to improve it (the first two eigenvalues accounted for only 38% of the variance in the data and the first three for only 48%). The inaccuracy of this layout highlights the difficulty of using visualization to assess similarity. 5 The Use of Position Instead of Vector for Profile The pictures of information spaces as maps or terrains derived from multivariate data using self-organizing maps [16] provide us with a compelling image of the profile or topic space we are exploring. The metrics discussed above generate similar conceptual spaces when visualized. Yet this is a misleading image, since the data are high dimensional and it is impossible to represent their similarities accurately in 2 or 3D space (direct mapping methods for multivariate data, such as colour maps and parallel coordinate plots, are not included in this comment). Nevertheless, as an approximation and as a representation, an overview perhaps, of a large body of entities, it is being found useful (see e.g. [20]). Suppose we assume the validity of the layout and propose that 92 Noy and Schroeder Fig. 4. Constructed data set with (left to right) PCoA and City, Euclidean and Angular differences. the agent carries with them their xy (or xyz) coordinates and uses them as their profile. When meeting a fellow agent they can ask for the agent’s xy coordinates and compute the Euclidean distance to calculate their similarity. This would be more efficient than carrying a potentially long profile vector and enable them to use their profile without revealing details or requiring encryption. Two different ways of using this idea suggest themselves - calculating back at base and on the fly. 5.1 By base calculation The agents both have the calculations done at a base point and periodically return for updates. Here the error will be that of the layout itself and the agent would be able to have details of the mean error and variance supplied with its coordinates, so that it can take this into account. Figure 5 shows the layout after City distance and PCoA of the seven agents of randomly generated data from above. Thus, if Agent1 meets Agent2 they can compare coordinates, ((-12.30, -5.20),(23.27,-8.44)), to calculate the Euclidean distance to give them the distance they are apart in this map. 5.2 By calculation on the fly Here the agent calculates its position with respect to a number of reference vectors (either dynamically or at an earlier point in time) and then compares with another agent’s position calculated similarly. Using the seven agent random data again, the reference vectors are chosen to be agents 5,6 and 7 illustrated in the generic data section. Three reference agents are the minimum since only two will create two possible arrangements when agents 1 and 2 overlay their positions. Agents 1 and 2 separately calculate their City distances to the three reference vectors and subsequently lay out these distances with PCoA as shown in figure 6. They now have xy coordinates, but in order to compare them they must be scaled (the Euclidean distance between 5 and 6 is used here), centered (here Agent 5 is placed Defining Like-Minded Agents 93 Fig. 5. Illustration of base plot, the three reference agents (5,6 and 7) and the two of interest in this measurement (1 and 2) are circled. Fig. 6. Illustration of plots calculated individually by agents 1 (left) and 2 (right) with respect to the three reference agents (5,6 and 7) as circled and numbered. at 0,0) and finally rotated to bring the agents 5,6 and 7 into position. Now the coordinates of the agent’s position are in a form that they can use for comparisons. The results of the base calculation and on-the-fly calculation of the difference between agents 1 and 2 are given in the table below. (Since these are normalized with respect to the distance between agents 5 and 6, a value of 1 would indicate that they were the same distance away from each other as agents 5 and 6 are) original city dist base dist on-the-fly dist 1.77 1.57 1.64 8%err -4%err exact 6 Conclusions and Future Work Visual datamining seeks to increase the integration of visualization with specific datamining techniques. This paper presents two applications with this in mind. Appropriate clusterings of data are sought, whilst at the same time layouts are required to present overviews. The user needs understanding of the layout algorithm to appreciate the implications of the overview, the implication of arriving at different clusterings with different algorithms needs to be understood by those seeking valid cluster- 94 Noy and Schroeder ings and classifications. These two purposes concern the same process, but are subtly different in their focus. The first application described in this paper, illustrating the use of signature exploration in making the behaviour of the similarity metrics more concrete and assisting in similarity metric choice, is an example of the merging of understanding of overview and determining appropriate metric. Visualization of pairs of data helped in the creation of an ad hoc, user-specific, classification with which to assess the overview and thus also the metrics. An obvious next step is to use feedback to select and modify metrics and features and this is another part of the signature exploration process. Continuing work lies in further developing the interface for exploration, the data construction engine and in conducting usability tests. Visualizations sometimes suggest the idea of a topic or similarity space - looking at a 2D or 3D scatterplot the closeness of entities is intuitively understood as similarity. Where dimension reduction is involved, considerable approximation or abstraction is required. If this is a valid procedure (in the sense of the considerable error sometimes incurred), and such diagrams are widely used without warnings given, then the idea of using location as a form of privacy protection (the transformation is a one-way function) must also hold on some level. The simple example for using position as a profile demonstrated in this paper - a potentially most useful mechanism - is encouraging, now evaluation for many different data sets is required to test its robustness, in terms of whether the original profile is fully protected and the tolerance of approximation in locations of the entities. Evaluation of the position-as-profile concept points to one of our most pressing problems in visualization - how valid are our visualizations when dealing with complex data and involving approximation or abstraction? How can the level of approximation be indicated to the viewer? Correspondingly, how can a measure of confidence in the agent’s location in the interest space be given to the agent? The investigation of the position-as-profile idea is the same investigation as that of the validity of layout. Thus, we begin to think in terms of transfering our picture as a viewer to the agent, so that the two can become one - a kind of viewer/agent entity. The agent thus may be a software agent or a human agent. The question now becomes, how can the boundaries or parameters of the validity be described to the viewer/agent? How can they be encoded visually and in software terms? We interchange the viewer with agent and must express what the user sees (or finds useful) in a form that the software agent can work with. Via the agent paradigm we may thus be helped toward creating programs that can use graphical elements to mimic our visual thinking. 6.1 Acknowledgements This work is supported by the EPSRC and British Telecom (CASE studentship - award number 99803052). Defining Like-Minded Agents 95 References 1. S. K. Card, J. D. Mackinlay, and B. Shneiderman. Readings in Information Visualization: Using Vision To Think. Morgan Kaufmann, 1999. 2. C. Chen. Information Visualisation and Virtual Environments. Springer, 1999. 3. G. di Battista, P. Eades, R. Tamassia, and I. G.Tollis. Graph Drawing: Algorithms for the Visualization of Graphs. Prentice Hall, 1999. 4. L. Foner. Yenta: a multi-agent, referral-based matchmaking system. In The First International Conference on Autonomous Agents, Marina del Rey, California. ACM press, 1997. 5. L. N. Foner. Clustering and information sharing in an ecology of cooperating agents. In AAAI Spring Symposium ’95 on Information Gathering in Distributed, Heterogeneous Environments, Palo Alto, 1995. 6. J. Glasgow, N. Hari Narayanan, and B. Chandrasekaran. Diagrammatic Reasoning. AAAI Press / The MIT Press, 1995. 7. A. D. Gordon. Classification. Chapman and Hall / CRC, 2nd edition edition, 1999. 8. P. Noy and M. Schroeder. Introducing signature exploration: a means to aid the comprehension and choice of visualization algorithms. In ECML-PKDD01 Visual Data Mining Workshop, Freiburg, Germany, Sept 2001. 9. E. Ogston and S. Vassiliadis. Matchmaking among minimal agents without a facilitator. In Proceedings of Autonomous Agents2001, Montreal, Canada, 2001. ACM press. 10. P.Faratin, C.Sierra, and N.R.Jennings. Using similarity criteria to make negotiation tradeoffs. In Proc. of 4th Int. Conf. on Multi-Agent Systems ICMAS-2000, pages 119–126, Boston, USA, 2000. IEEE Computer Society. 11. M. Schroeder. Using singular value decomposition to visualise relations within multi-agent systems. In Proceedings of the third Conference on Autonomous Agents, Seattle, USA, 1999. ACM Press. 12. M. Schroeder, D. Gilbert, J. van Helden, and P. Noy. Approaches to visualisation in bioinformatics: from dendrograms to Space Explorer. Information Sciences, 139:19–57, 2001. 13. M. Schroeder and P. Noy. Multi-agent visualization based on multivariate data. In Proceedings of Autonomous Agents2001, Montreal, Canada, 2001. ACM press. 14. G. Somlo and A. Howe. Incremental clustering for profile maintenance in information gathering web agents. In Proceedings of Autonomous Agents2001, Montreal, Canada, 2001. ACM press. 15. R. Spence. Information Visualization. Addison-Wesley, 2001. 16. T.Kohonen. Self-organising maps. Springer-Verlag, 2nd edition edition, 1997. 17. D. Veit. Matchmaking algorithms for autonomous agent systems. Master’s thesis, Institute of Computer Science, University of Giessen, Germany, 1999. 18. D. Veit, J. Muller, M. Schneider, and B. Fiehn. Matchmaking for autonomous agents in electronic marketplaces. In Proceedings of Autonomous Agents2001, Montreal, Canada, 2001. ACM press. 19. A. Webb. Statistical Pattern Recognition. Arnold, 1999. 20. Websom. http://websom.hut.fi/websom/. 21. G. Weiss, editor. Multiagent Systems. MIT Press, 1999. Visual Data Mining of Clinical Databases: an Application to the Hemodialytic Treatment based on 3D Interactive Bar Charts Luca Chittaro 1 , Carlo Combi2 , Giampaolo Trapasso1 1 HCI Lab, Dept of Math and Computer Science, University of Udine, via delle Scie nze 206, 33100 Udine, Italy chittaro@dimi.uniud.it 2 Department of Computer Science, University of Verona, strada le Grazie 15, 37134 Verona, Italy combi@sci.univr.it Abstract. The capability of interactively mining clinical databases is an increasingly urgent need. This paper considers a relevant medical application (i.e., hemodialysis) and proposes a system for the visualization and vis ual data mining (VDM) of the collections of time -series acquired during hemodialytic treatments. Our proposal adopts bar charts as the basic visualization technique (because it is very familiar for clinicians) and augments them with several interactive features, exploiting a 3D space to significantly increase both the number of time -series that can be simultaneously analyzed in a convenient way and the number of values associated with each series. 1 Introduction The capability of interactively mining patient clinical information is an increasingly urgent need in the clinical domain, due to the continuous growth in the number of parameters that can be automatically acquired and in the size of the databases where they accumulate [5]. This is particularly critical for the success of medical research projects which generate massive databases of patient data. Some techniques for visual data mining (VDM) of multidimensional clinical databases are illustrated in [7]. They are mainly based on 3D versions of parallel coordinate plots. Graphical connections between points in adjacent planes are drawn in such a way that each patient's case is visually represented by a line connecting individual points referring to it. This allows for VDM of interesting patterns (e.g., a group of patients with the same profile results in parallel lines). 98 Chittaro et al. A different approach is presented by [10] and is based on tables displaying records of the clinical database and their attributes in highly compressed format such that they fit onto the screen. Users directly manipulate the table (e.g., performing zoom and filter operations) that dynamically rearranges itself. To compress the tables, the system relies on visualization criteria such as (i) neighboring cells with identical values are combined into a larger cell, or (ii) if there is no space to display a numeric value in its cell, the value is substituted by a small horizontal line whose position indicates relative size. This paper explores a third possibility, especially suited to clinical databases containing time-series data. Since, historically, bar charts are a widely adopted approach to display time-series and are a very familiar representation for clinicians, we chose them as the basis of our visual approach. Unfortunately, while a bar chart allows for an easy comparison among the data values for a single time-series, when the considered task requires to compare a collection of time-series (such as a monitored signal from the same patient in different sessions of the same clinical test or treatment), traditional bar charts (as other historical approaches) become unfeasible. Therefore, we augment bar charts exploiting a 3D space and adding several interactive features. A 3D space can significantly increase both the number of time-series that can be simultaneously analyzed in a convenient way and the number of values associated with each timeseries, but poses well-known problems such as occlusions, 3D navigation, difficulties in comparing heights, proper use of space, and the need for effective interaction techniques to aid the user in the analysis of large datasets (e.g., highlighting interesting patterns, checking trends,…). The limited capabilities of commercial tools that generate 3D bar charts have led well-known researchers (e.g. [9]) to classify these visualizations as “chartjunk 3D”. However, solutions to the problems of 3D bar charts are emerging from research: e.g., Cichlid offers temporal animation capabilities of 3D stacked bar charts [2], while ADVIZOR allows one to interactively link the 3D bar chart representation with related 2D representations, compare heights with a “water level” plane (perpendicular to the bars) and use filtering tools [6]. Alternative approaches to time-series visualization have been recently proposed, e.g. drawing the timeline along spiral structures [12] is reported to allow for an easier detection of cyclic phenomena. However, we preferred to adopt bar charts, because they were familiar to clinicians. Moreover, we did not have a focus on a specific pattern such as cycles. In the following, we first introduce the real-world clinical context we are working in and motivate the need for VDM in that context. Then, we illustrate the system we have built and its main features. Finally, we show some examples of how our system is being applied to the clinical context. 2 Hemodialysis and Visual Data Mining Hemodialysis is the widely used treatment for patients with acute or chronic endstage renal failure. During an hemodialysis session, the blood passes through an extra- Visual Data Mining of Clinical Databases 99 corporeal circuit where metabolites (e.g., urea) are eliminated, the acid-base equilibrium is re-established, and water in excess is removed. In general, hemodialysis patients are treated 3 times a week and each session lasts about 4 hours. Hemodialysis treatment is very costly and extremely demanding both from an organizational viewpoint [8] and from the point of view of the patient’s quality-of-life. A medium-size hemodialysis center can manage up to 60 patients per day, i.e. more than 19000 hemodialytic sessions per year. Unfortunately, the number of patients that need hemodialysis is constantly increasing [12]. In this context, it is very important to be able to evaluate the quality of (i) each single hemodialysis session, (ii) all the sessions concerning the same patient, and (iii) sets of sessions concerning a specific hemodialyzer device or a specific day, for the early detection of problems in the quality of the hemodialytic treatment. Modern hemodialyzers are able to acquire up to 50 different parameters from the patient (e.g., heart rate, blood pressure, weight loss due to lost liquids,…) and from the process (e.g., pressures in the extra-corporeal circuit, incoming blood flow,…), with a configurable sampling time whose lower bound is 1 sec. As an average example, considering only 25 parameters with a sampling time of 30 seconds, 12000 values (4*120*25) are collected in each session, and a medium-sized center collects more than 228 millions of values per year (considering 19000 provided treatments). While the daily accumulation of huge amounts of data prompts the need for suitable techniques to detect and understand relevant patterns, hemodialysis software is more concerned with acquiring and storing data, rather than visualizing and analyzing it. Data mining applications can thus play a crucial role in this context. More specifically, visual data mining applications are of particular interest for three main reasons. First, clinicians’ abilities in recognizing interesting patterns are used suboptimally or not used at all in the current context. Visual mining of hemodialytic data would allow clinicians to take decisions affecting different important aspects such as therapy (personalizing the individual treatment of specific patients), management (assessing and improving the quality of care delivered by the whole hemodialysis centre), medical research (discovering relations and testing hypothesis in nephrology research). Second, since data mining on the considered database is (at least, at initial stages) intrinsically vague for clinicians, the adoption of VDM techniques can be more promising than fully automatic techniques, because it supports clinicians in discovering structures and finding patterns by freely exploring the datasets as they see fit. Third, the clinical context is characterized by a need for user interfaces that require minimal technical sophistication and expertise to the users, while supporting a wide variety of information intensive tasks. A proper exploitation of visual aspects and interactive techniques can greatly increase the ease of use of the provided solutions. In summary, a clinical VDM system has to achieve two possibly conflicting goals: (i) offering powerful data analysis capabilities, while (ii) minimizing the number of concepts and functions to be learned by clinicians. In the following, we illustrate how our system attempts to achieve these two goals. 100 Chittaro et al. 3 The Proposed Approach The system we have built, called IPBC (Interactive Parallel Bar Charts) connects to the hemodialysis clinical database, produces a visualization that replaces tens of separate screens used in traditional hemodialysis systems, and extends them with a set of interactive tools that will be described in detail in this section. Each hemodialysis session returns a time-series for each recorded clinical parameter. In IPBC, we visually represent each time-series in a bar chart format where the X axis is associated with time and the Y axis with the value (height of a bar) of the series at that time. Then, we layout the obtained bar charts side by side, using an additional axis to identify the single time-series, and we draw them in a 3D space, using an orthogonal view. It must be noted that also the additional axis has typically a temporal dimension, e.g. it is important to order the series by date of the hemodialysis session to analyze the evolution of a patient. An example is shown in Fig. 1, that illustrates a vis ualization of 50 time-series of 50 values each, resulting in a total of 2500 values (the axis on the right is the time axis for single sessions, while the axis on the left identifies the different time-series, ordered by date). Hereinafter, we refer to this representation as a parallel bar chart. Fig. 1. A Parallel Bar Chart. 3.1. The RoundToolbar widget In designing how the different interactive functions of IPBC should be invoked by the user, we wanted to face two different problems: • First, one well-known limitation of many 3D visualizations is the possible waste of screen space towards the corners of the screen; Visual Data Mining of Clinical Databases 101 • Second, the traditional menu bar approach would require long mouse movements from the visualization to the menu bar and vice versa. To this purpose, we designed a specific round-shaped pop-up menu (see Fig. 2), called RoundToolbar (RT), that appears where the user clicks with the right mouse button. The RT can be easily positioned in the unused screen corners, thus allowing a better usage of the screen space (e.g., see Fig. 1) and a reduction of the distance between the visualization and the menu. Moreover, to further improve selection time of functions with respect to a traditional menu, the organization of modes in the toolbar is inspired by Pie Menus [3]: in particular, the main modes are on the perimeter of the RT, and when a mode is selected, the center of the RT contains the corresponding tools (which are immediately reachable by the user, who can also quickly switch back from the tools to a different mode). Fig. 2. Viewpoint mode. Fig. 3. Viewpoint movements: A) Low; B) Rotate; C) Far. 3.2. Changing Viewpoint It is well-known that free navigation in a 3D space is difficult for the average user, 102 Chittaro et al. because (s)he has to control 6 different degrees of freedom and can follow any possible trajectory. To make 3D navigation easier, when the Viewpoint mode is selected in the RT (as in Fig. 2), the proposed controls for viewpoint movement (Rotate, HighLow and Near-Far) cause movement along limited pre-defined trajectories which can be useful to examine the visualization: in particular, Fig. 3 shows how viewpoint movement is constrained. The remaining Vertical scale control in the Viewpoint mode is used to scale the bars on the Y axis. Vertical scaling has been included in the Viewpoint mode, because it has been observed that when users scaled the bars, they typically changed the viewpoint as the following operation. Fig. 4. Dynamic Query mode. 3.3. Dynamic Queries IPBC uses color to classify time-series values into different ranges. In particular, at the beginning of a session, the user can define units of measure and her general range of interest for the values, specifying its lowest and highest value. These will be taken as the lower and upper bounds for an IPBC dynamic query control in the RT (as shown in Fig. 4) that allows the user to interactively partition the specified range into subranges of interest. Different colors are associated to the subranges and when the user moves the slider elements, colors of the affected bars in the IPBC change in realtime. Possible bars with values outside the specified general range of interest are highlighted with a proper single color. For example, Fig. 1 shows a partition that includes the three subranges corresponding to the colors shown by the slider in Fig. 4, and also some bars which are outside the user’s predefined range. The color coding scheme can be personalized by the user with the Colors mode in the RT. The dynamic query control allows the user to: • move the two slider elements independently (to change the relative size of adja- Visual Data Mining of Clinical Databases • 103 cent subranges). For example, in Fig. 4, one has been set to 130 mmHg and the other to 180 mmHg. This can be done both by dragging the edges or (more easily) the tooltips which indicate the precise value. Plus and minus signs in the tooltips also allow for a fine tuning of the value. Move the two slider elements together by clicking and dragging the area between the two bounds. This can be particularly useful (especially when the other areas are associated to the same color), because it will result in a “spotlight” effect on the vis ualization: as we move the area, our attention is immediately focused on its corresponding set of bars, highlighted in the visualization. Fig. 5. Tide mode. 3.4. Comparing data with (time-varying) thresholds A frequent need in VDM is to quickly perceive how many and which values are below or above a given threshold. This can be easily done with the previously described dynamic queries when the threshold is constant. However, the required threshold is often time-varying, e.g. one can be interested in knowing how many and which values are not consistent with an increasing or decreasing trend. For this need, IPBC offers a mode based on a tide metaphor. As it can be seen in Fig. 5, the Tide mode adds a semitransparent solid to the visualization: the solid metaphorically represents a mass of water that floods the bar chart, highlighting those bars which are above the level of water. The slope of the top side of the solid can be set by moving two tooltips shown in the RT (that specify the initial and final values for the solid height), thus determin- 104 Chittaro et al. ing the desired linearly increasing or decreasing trend. The height of the solid can be also changed without affecting the slope by clicking and dragging the blue area in the RT. An opaque/transparent control allows the user to choose how much the solid should hide what is below the threshold. When the Tide mode is activated, all the bars in the user’s range of interest are turned to a single color to allow the user to more easily perceive which bars are above or below the threshold; if multiple colors were maintained, the task would be more difficult, also because the chromatic interaction between the semitransparent surface and the parts of bars inside it adds new colors to the visualization. The Tide mode can be also used to help compare sizes of bars by selecting a zero slope and changing the height of the solid (in this special case, Tide becomes analogous to the “water level” function of other visualization systems). Fig. 5 illustrates this latter case, while Fig. 9 shows a positive slope case. Implementing a non-linear Tide would be relatively straightforward (only linear trends are anyway used by clinicians in the considered hemodialysis domain). Fig. 6. Matrix Visualization. 3.5. Managing Occlusions As any 3D visualization, IPBC can suffer from occlusion problems. To face them, the approach offers two possible solutions. First, by clicking on the 2D/3D label on the RT, the user can transform the parallel bar chart into a matrix format and vice versa. For example, Fig. 6 shows the same data as Fig. 1 in the matrix format. The transformation is simply obtained by automatically Visual Data Mining of Clinical Databases 105 moving the viewpoint over the 3D visualization (and taking it back to the previous position when the user deselects the matrix format). This can solve any occlusion problem (and the dynamic query control can still be used to affect the color of the matrix cells), but the information given by the height of the bars is lost. Transitions to matrix format and back are animated to avoid disorienting the user and allow her to keep her attention on the part of the visualization (s)he was focusing on. Second, by directly clicking on any time-series in the 3D visualization, only the timeseries which can possibly occlude the chosen one collapse into a flat representation analogous to the matrix one, as illustrated in Fig. 7. Fig. 7. Removing occlusions. 3.6. Pattern Matching When the user notices an interesting sequence of values in one of the time-series, IPBC offers her the opportunity to automatically search for and highlight occurrences of a similar pattern in all the visualization (a detailed example will be described in Section 4.4). The user selects her desired sequence of values in a time-series by simply dragging the mouse over it, then (s)he can specify how much precise the search should be, by indicating two tolerance values in the RT: (i) how much a single value can differ in percentage from the corresponding one in the given pattern, (ii) the maximum number (possibly zero) of values in a pattern that can violate the given percentage. 106 Chittaro et al. 3.7. Mining Multidimensional Data If multiple variables are associated to the considered time-series, IPBC can organize the screen into multiple windows, each one displaying a parallel bar chart for one of the variables. The visualizations in all the windows are linked together, e.g. if one selects a single time-series in one of the windows (or a specific value in a time-series), that time-series (or the corresponding value) is automatically highlighted in every other window. This (as some other features of IPBC) will be shown in more detail in the next section. 4. Mining Hemodialytic data In the following, we will show how IPBC can be used during real clinical tasks, to help physicians evaluating the quality of the hemodialytic treatments given to single patients, on the basis of the clinical parameters acquired during the sessions. Each hemodialysis session returns a time-series for each parameter; different time-series are displayed side by side in the parallel bar chart according to date (in this case, the axis on the left chronologically orders the sessions). The following examples are ordered according to the complexity of the related task: in particular, the first two tasks are relatively simple and are taken from the daily activity of clinicians, while the last two tasks are more complex and are performed by clinicians only in specific occasions (in the two considered examples, they are related to a detailed evaluation of the quality of care provided by nurses). 4.1. Mining patient signs data A first task consists in analyzing patient signs, as the systolic and diastolic blood pressures and the heart rate; indeed, these parameters are important both for the health status of the patient and for the management of device settings during the hemodialytic session. Let us consider, for example, the task of analyzing all the systolic pressures of a given patient: Fig. 8 shows a parallel bar chart (containing more than 5000 bars), representing the systolic pressure measurements (about 50 per session) during more than 100 hemodialytic sessions. In this figure, we can observe that the presence of out-ofscale values, usually related to measurement errors (e.g., the patient was mo ving; the measurement device was not properly operating), has been highlighted by specifying a proper range of interest (that highlights them in a suitable color) and hiding their height. In the specific situation represented in the figure, the presence of several outof-scale values at the beginning of each session is due to the fact that nurses activate the measurement of patient’s blood pressure with some delay with respect to the beginning of the session. In the figure, the user is focusing on a specific session, avoiding occlusion problems (as described in Section 3.5). At the same time, with a dynamic query, (s)he is able Visual Data Mining of Clinical Databases 107 to distinguish low, normal, and high blood pressures. In this case, the clinician can observe that the systolic pressure in the chosen session, after a period of low values (yellow bars), was in the range of normal values (orange bars). While the values for the chosen session correspond to a normal state, it is easy to observe that several sessions among those in the more recent half part of the collection contain several high values (in red) for the systolic blood pressure. Thus, the clinician can conclude that in those sessions the patient had some hypertension, i.e. a clinically undesired situation. Fig. 8. Analyzing systolic blood pressures. 4.2. Mining bl ood volume data Another task is related to observing the percentage of reduction of the blood volume during hemodialysis, mainly due to the removal of the water in excess. This reduction is sometimes slowed down to avoid situations in which the patient has too low blood pressures. In this case, VDM can benefit from the usage of the Tide mode. Fig. 9 shows an IPBC with more than 9000 bars, representing 36 hemodialytic sessions, containing about 250 values each. In this case, being the percentage of reduction of the blood volume increasing during a session, Tide allows the physician to distinguish those (parts of) sessions characterized by a percentage of reduction above or below the desired trend. In the figure, for example, the selected session has a first part emerging from the tide, while the last part is below. At the same time, it is possible to observe that one of the last sessions has the percentage of reduction above the tide during almost the entire session. The clinician can thus easily identify those (parts of) sessions with a satisfying reduction of the blood volume as the emerging (parts of) 108 Chittaro et al. sessions. Fig. 9. Visualizing the time-varying reduction of the blood volume in the Tide mode. 4.3. Mining related clinical parameters The next task we consider is related to the analysis of three related parameters: the systolic and diastolic blood pressures (measured on the patient) and the blood flow (QB) entering the hemodialyzer. QB is initially set by the hemodyalizer, but it can be manually set (reduced) by nurses when the patient’s blood pressures are considered too low by the medical staff. It is thus interesting to visually relate QB and blood pressures, to check whether suboptimal QBs are related to low pressures. Otherwise, suboptimal values of QB would be due to human errors during the manual setting of the hemodialyzer. Fig. 10 shows the coordinated visualization of three clinical parameters for the same patient: the diastolic blood pressure (small window in the upper left part), the systolic blood pressure (small window in the lower left part), and QB (right window). The user can freely organize the visualization, switching the different charts from the smaller to the larger windows (by clicking on the arrow in the upper right part of the smaller windows). In the figure, the clinician is focusing on a session where the QB was below the prescribed value during the first two hours of hemodialysis (yellow color for QB) and (s)he has selected a specific value (the system highlights that value and the corresponding values in the other windows with black arrows). It is easy to notice that the suboptimal QB was related to low blood pressures (yellow bars in the corresponding time-series in the two small windows); then, QB was set to the correct value by nurses (see black arrow in the right window) only after blood pressures reached normal values (orange color in the corresponding charts). In this case, the Visual Data Mining of Clinical Databases 109 physician can conclude that the suboptimal QB has been correctly set by nurses because of the patient’s hypotension. Fig. 10. Coordinated analysis of blood pressures and incoming blood flow. 4.4. Mining for similar patterns Finally, let us consider a task concerning the analysis of QB. As previously mentioned, the value of QB can be manually set by nurses and it may happen that this value is below the optimal one, due to hypotensive episodes. Fig. 11 shows a visualization where the clinician noticed a change of QB from a lower value to the correct one in a session: this means that, after a period of suboptimal treatment, the proper setting had been entered. Therefore, the clinician asks IPBC to identify QB patterns similar to the one (s)he noticed, by indicating it with the mouse, and setting the tolerance parameters (see Section 3.6). Fig. 11 shows the selected pattern (see the area near the black arrow) and the similar patterns automatically found by IPBC (two are in the lower right part of the figure, one in the upper left part): these patterns are identified by a line of a suitable color, which highlights the contours of the first and last bar of the pattern and intersects the inner bars. To avoid possible occlusion problems in visually detecting the patterns, the physician can move the viewpoint or switch to the matrix representation, where each pattern can be easily observed. 110 Chittaro et al. Fig. 11. Automatic Pattern Matching. 5. Conclusions and Future Work In this paper, we described the main features of IPBC (Interactive Parallel Bar Charts), a VDM system devoted to interactively analyze collections of time-series, and showed its application to a real clinical database of hemodialytic data. We are currently carrying out a field evaluation of IPBC with the clinical staff of the hemodialysis center at the Hospital of Mede, PV, Italy. One of the major advantages of IPBC that is emerging is that the visualization and its interactive features are very quickly learned and remembered by clinicians, the major disadvantage is that usage of screen space becomes difficult if a clinician tries to relate more than 3 collections of time-series simultaneously (Section 4.3 dealt with the analysis of 3 collections). This early feedback received from the field evaluation is helping us in identifying new research directions. Besides facing the problem of analyzing more than 3 collections in a convenient way, we aim to face another problem (that is considered very relevant by clinicians), i.e. dealing with time-series at different abstraction levels, allowing for both a fine exploration of time-series (e.g., to detect specific unusual values) and their coarse exploration (to focus on more abstract derived information). In both cases, we are working at the integration of parallel bar charts with other visualizations that can provide a synthetic view of data (e.g., the medical literature is proposing some comp utation methods to derive some quality indexes of the hemodialytic session from the time-series of that session). In particular, we are experimenting with Parallel Coordinate Plots, e.g. a trajectory in a plot could connect the quality indexes (typically, 5-7 values) of a session, and this high-level perspective would be linked to the much Visual Data Mining of Clinical Databases 111 of a session, and this high-level perspective would be linked to the much more detailed perspective of the parallel bar chart. Acknowledgements This work is partially supported by a MURST COFIN 2000 project (“Analysis, Information Visualization, and Visual Query in Databases for Clinical Monitoring”). References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. Ahlberg, C., Williamson, C., Shneiderman B.: Dynamic queries for information exploration: An implementation and evaluation. Proc. of the CHI ‘92 Co nference on Human Factors in Computing Systems, ACM Press, New York (1992) 619-626 Brown, J.A., McGregor, A.J., Braun H-W.: Network Performance Visualization: Insight Through Animation. Proc. of PAM2000: Passive and Active Measurement Workshop, Hamilton, New Zealand (2000) 33-41 Callahan, J., Hopkins, D., Weiser, M., Shneiderman, B.: An empirical comparison of pie vs. linear menus. Proc. of the CHI ‘88 Conference on Human Factors in Computing Systems, ACM Press, New York (1998) 95-100 Chittaro L. (ed.), Special issue on Information Visualization in Medicine, Artificial Intelligence in Medicine Journal, 22(2) (2001) Chittaro L.: Information Visualization and its Application to Medicine, in [4] 81-88 Eick, S. G.: Visualizing Multi-Dimensional Data. ACM SIGGRAPH Computer Graphics, 34(1) (2000) 61-67 Falkman, G.: Information Visualization in Clinical Odontology: Multidimensional Analysis and Interactive Data Exploration, in [4] 133-158 McFarlane, P.A., Mendelssohn, D.C.: A call to arms: economic barriers to optimal hemodialysis care. Perit Dial Int 20 (2000) 7-12. Shneiderman, B.: 3D or Not 3D: When and Why Does it Work?, invited talk at Web3D: 7th Internatio nal Conference on 3D Web Technology, Tempe, AZ (2002) Spenke, M.: Visualization and Interactive Analysis of Blood Parameters with InfoZoom, in [4] 159-172 Tufte, E.R.: The Visual Display of Quantitative Informatio n, Graphics Press (1982) USRDS, The United States Renal data system, http://www.usrds.org Weber, M., Alexa, M., Mueller, W.: Visualizing Time-Series on Spirals. Proc. of the IEEE InfoVis Symposium, IEEE Press, Los Alamitos, CA (2001) Sonification of time dependent data Monique Noirhomme-Fraiture1, Olivier Schöller1, Christophe Demoulin1, Simeon Simoff2 1 2 University of Namur, Institut d’Informatique, 5000 Namur, Belgium mno@info.fundp.ac.be University of Technology Sydney, Faculty of Information Technology, NSW2007 Sydney, Australia simeon@it.uts.edu.au Abstract. This paper presents the results of experiments with sonification of 2D and 3D time dependent data. A number of sonification means for these experiments have been implemented. An Internet Web site was created where sound sequences were presented and could be evaluated by the participants in the experiment. All participants that performed the tests also needed to fill an evaluation questionnaire. The purpose of the experimentation was to determine how the sonification of two and three-dimensional graphs can support or be an alternative to visually displayed graphs. The paper concludes discussion of the results and the issues related with the experiments. 1 Introduction Visual data mining is a part of the KDD process [1], which places an emphasis on visualisation techniques and human cognition to identify patterns in a data set. [1] identified three different scenarios for visual data mining, two of which are connected actually with the visualisation of final or intermediate results and one operates directly with visual representation of the data. The design of data visualisation techniques, in broad sense, is the formal definition of the rules for translation of data into graphics. Generally, the term ‘information visualisation’ has been related to the visualisation of large volumes of abstract data. The basic assumption is that large and normally incomprehensible amounts of data can be reduced to a form that can be understood and interpreted by a human through the use of visualisation techniques. The process of finding the appropriate visualisation is not a trivial one. A number of works offer some results that can be applied as guiding heuristics. For example, [2] defined the Proximity Compatibility Principles (PCP) for various visualization methods in terms of tasks, data and displays - if a task requires the integration of multiple data variables, they should be bundled in proximity in an integrated display. Based on this principle authors have concluded that 3D graphs do not have an advantage over 2D graphs for scientific visualisation (which may not necessarily hold for visual data mining). 114 Noirhomme-Fraiture et al. Visual data mining relies heavily on human visual processing channel and utilises human cognition overall. The visual data mining cycles are shown in Fig. 1. In most systems, visualisation is used to represent the output of conventional data mining algorithms (the path shown in Fig. 1a). Fig. 2 shows an example of visualisation of the output of an association rule mining algorithm. In this case, visualisation assists to comprehend the output of the data mining algorithms. Fig. 1b shows the visual data mining cycle when visualisation is applied to the original or pre-processed data. In this case, the discovery of the patterns and dependencies is left to the capacity of the human visual reasoning system. The success of the exercise depends on the metaphor selected to visualise the input data [3]. Knowledge Timelines Timelines Knowledge 0 100 200 300 400 500 600 700 800 U tterances 0 100 200 300 400 500 600 700 800 Utterances Visualisation of the output Visualisation of the output output output Data Mining Algorithms Data Mining Algorithms Visualisation of the input input input Data Data a. b. Fig. 1. Visualisation and visual data mining Although human visual processing system remains a powerful ‘tool’ that can be used in data mining, there are other perceptual channels that seem to be underused. Our capability to distinguish harmonies in audio sequences (not necessarily musical ones) is one possibility to complement the visual channel. Such approach can be summarised as ‘What You Hear Is What You See’. The idea of combining the visual and audio channels is illustrated in Fig. 3. The conversion of data into a sound signal is known as sonification. Similar to the application of visualisation techniques in Fig. 1b, sonification can be used both for representing the input and/or the output of the data mining algorithms. Sonification of Time Dependent Data 115 Fig. 2. Example of visualisation of the output of an association rule miner. Timelines Knowledge 0 100 200 300 400 500 600 700 800 Uttera nces Visualisation of the output output Sonification of the output output Data Mining Algorithms Sound Analysis Algorithms Visualisation of the input Sonification of the input input Data Fig. 3. Combining visual data mining and sonification In visual data mining, sonification should be synchronised with the visualisation technique. Further, in this paper we discuss the issues connected with designing such data mining techniques and present an example of a practical implementation of combined technique. We briefly discuss the characteristics of the sound that are suitable for such approach, the actual sonification process, the design of the overall combined technique and the results of the experiments conducted with proposed technique. 116 Noirhomme-Fraiture et al. 2 Characteristics of sound for time dependent data representation Several researchers have investigated the use of sound as means for data representation [4-11]. In this context, the important feature of the sound is that it has a sequential nature, having particular duration and evolving as a function of time. A sound sequence has to be heard in a given order, i.e. it is not possible to hear the end before the beginning1. Similarly, a time series depends on time and have the same sequential characteristics. Consequently sound provides good means to represent time series. 3 Sonification The easiest way to transform time dependent data into sound is to map the data to frequencies by using linear as well as chromatic scale mappings. We call this process a pitch-based mapping. We compute the minimum and maximum data values from the chosen series and map this data interval into a frequency range, chosen in advance. Each value of the series is then mapped into a frequency. To avoid too large, non-realistic intervals, we first discard outliers (see below). Another pre-treatment is the smoothing of the series. In fact, if we map all the points of a series into a sound, we will hear rather inconsistent sounds. A first treatment consists in smoothing the series by a standard mean, for example, by moving average method. After that, we map the smoothed curve into pitch. Beat drums can be used to enhance the shape of the curve (see below). 3.1 Detection of outliers To detect statistically the values of the outliers, a confidence interval is computed at each time t., based on the normal distribution. Once a data value is detected outside the confidence interval, the corresponding time value is stored and sonified at the experiment phase. 3.2 Beat drums mapping The rhythm of a beat drum increases with respect to the rate of growth of the curve (i.e. the first derivative). 1 Images and drawings do not have such constraint. Strictly speaking digital sound recording can provide access to an arbitrary section of the sound fragment, even reproduce the sound in reverse order, which is beyond the scope of this paper. Sonification of Time Dependent Data 117 3.3 Stereo panning Variation of the stereo acoustics is introduced, for example, an increase of the volume of the right speaker and decrease of the volume of the left speaker. 3.4 3D Curve When the time series is given at each time, not by a single value but by a function of values, we decide to “hear” at each discrete time the function. We can also choose to cut the surface at a certain level and to hear “continuously” the obtained curve as a function of time. We call these transformations respectively horizontal and vertical travelling. An example of a 3D data surface for sonification is shown in Fig. 4. Fig. 4. Example of a surface that can be sonified 4. Prototype implementation The prototype has been implemented in Java programming language, using the MIDI package of the Java Sound API2 [12]. The MIDI sequence is constructed before the actual playback. When the designer starts the sonification, the whole sequence is computed. Then computed sequence is sent to the MIDI sequencer for playback. 5. Experimentation The purpose of the experimentation is to determine how the sonification of two and three-dimensional graphs can complement or be an alternative to visually displayed 2 API – Application Programming Interface 118 Noirhomme-Fraiture et al. graphs. An Internet Web site has been created, where sound sequences are presented and can be evaluated by the visitors. The site contains questionnaire that has to be filled in by visitors performing the test. The structure of the questionnaire is the following one: A. Identification of the user: name, age, gender, title/position, e-mail address. These data are used to identify the subject and to validate the answer. B. Ability: field of activity, musical experience (instrument played, practicing period), self-evaluation of musical level (from ‘no experience’ to ‘expert level’). C. 2D evaluation: 2D evaluation is divided into three subtasks: Part C.a: Explanation about the four sonification techniques used: pitch based only, beat drums, stereo and extreme values detection. Each of them is briefly described and at least one example is given. Part C.b: Application test: four sequences are presented to the user. Each time precise questions are asked: Question 1: Annual sheep population in England and Wales between 1867 and 1939 (see Fig. 5). − Were there more sheep in 1867 then in 1939? − In your opinion, when (which year) did the sheep population reach the minimum? Fig. 5. Annual sheep population in England and Wales between 1867 and 1939 This question aims to evaluate if subject can perceive a global trend in the series and to understand if the relation with the time scale is done. For each sequence, beat drums and stereo mapping are added to enhance the pitch-based sonification. Question 2 aims to identify whether extreme values are detected. Question 3 aims to identify whether seasonal trend can be detected. Question 4 is focused on trend Sonification of Time Dependent Data 119 identification. For each question, the subject must specify the number of times he listened to the sonified data before answering the question. Part C.c: Subject preferences: four other questions aim to evaluate subject preference: 1. Choice of instrument for pitch mapping. The user is asked to hear the sonification of the same data, but with different MIDI instruments for the pitch mapping. The user is asked to grade on the scale 0 to 10 the different instruments (acoustic grand, steel string guitar, violin, synthstrings 2, pan flute). The instruments proposed are very different and belong to a specific MIDI group such as piano, guitar or to string group. 2. Choice of instrument for drums mapping. Different instruments for the beat drums mapping are presented and have to be marked (Celesta, Slap Bass 1, Timpani, Tinkle Bell, Woodstock). 3. Choice of sonification technique. The subject has to give his opinion on the different mapping techniques: pitch, beat drums, stereo mapping, extreme values detection and on the sonification in general. The four levels are proposed: useless, sometimes useful, always useful and essential. 4. Open question: “Please tell us what you think about our project, our applications, this web page or anything else that comes to mind”. D. 3D Evaluation. The same schema is used for 3D curves: explanation about the sonification method, the application test (3 questions), subject preferences. The sonification method is based on a cutting of the initial surface following some direction: vertical (see Fig. 6), horizontal (see Fig. 7) or diagonal. Each 2D line obtained by the cutting process can be heard, similar to the 2D case. The different lines, proposed one after the other, are separated by a specific sound. Beat drums can still be added to pitch mapping. 120 Noirhomme-Fraiture et al. Fig. 6. Data for vertical travelling Fig. 7. Data for horizontal travelling 6 Results Below we present the results of the experiments. 6.1 The sample 23 visitors answered the questions for the case of a 2D visualisation and 18 visitors for the case of 3D visualisation. A large part of the sample (9) includes people working in the computer science area, who have limited musical experience or no experience at all. To see if people with musical experience get a better score, we have compared the average score in both groups. The influence of the musical and computer science background on the results is presented in Table 1 and Table 2, respectively. The score obtained for each question is the number of good answers, normalised on a Sonification of Time Dependent Data 121 score out of 100. The average score is the mean of the scores for the different questions. Table 1. The influence of musical background on results Musical background No experience 2D sonification 3D sonification Av. score/100 Av. score/100 76 66 N 9 14 23 40 42 The same comparison was done for computer scientist. Table 2. The influence of computer science background on results Computer science Others 2D sonification 3D sonification Av. score/100 Av. score/100 72 69 N 11 12 23 41 40 Having a musical or a computer science background gives a minor advantage in using sonification of 2D curves. The differences are not significant. We do not observe any difference for the 3D sonification case. The way 3D case has been implemented is rather complex and uses a good spatial representation. Musical or computer experience has a minor influence on the result in this case. 6.2 Results in 2D The following questions were included in the questionnaire, targeting 2D sonification: Qu1 Annual sheep population in England and Wales between 1867 and 1939 1.1 Were there more sheep in 1867 than in 1939? 1.2 About which year did the sheep population reach the minimum? Qu2 Daily morning temperature of an adult woman during two months 2.1 Did she have fever during the period? 2.2 If yes, for how long did she have the fever? Qu3 Monthly electricity production in Australia between January 1956 and August 1995 3.1 Is the electricity production in Australia lower in 1956 than in 1995? 3.2 How would you categorise the evolution of electricity production in Australia: as linear or as exponential? 122 Noirhomme-Fraiture et al. 3.3 Is the evolution of electricity production in Australia characterised by seasonal trend? Qu4 Monthly Minneapolis public drunkenness intakes between January 1966 and July 1978 (151 months) 4.1 Were there more intakes in 1966 than in 1978 ? 4.2 Is the evolution of public drunkenness intakes linear? The results are summarised in Table 3. Table 3. Summary of the results for 2D Correct Qu1 Qu2 Qu3 Qu4 1.1 1.2 2.1 2.2 3.1 3.2 3.3 4.1 4.2 17 17 23 13 22 12 16 20 19 Wrong 5 6 0 10 1 11 6 2 3 No idea 1 0 0 0 0 1 1 1 6.3 Results in 3D The following questions were included in the questionnaire, targeting 2D sonification: Qu1 A 3D graph containing 2 bumps has been sonified. The selected mapping is the vertical travelling and the sonification starts from the bottom right corner. - If the grid below (3 x 3) represents the graph, where are these 2 bumps located? - Do they have the same height? Qu2 Same kind of questions with respect to horizontal travelling. Qu3 Same kind of questions with respect to diagonal travelling. The results are summarised in Table 4. Sonification of Time Dependent Data 123 Table 4. Summary of the results for 3D Correct Qu1 Qu2 Qu3 1.1 Trend 1.2 Value 2.1 Outlyers 2.2 3.1 General trend 3.2 2 correct 4 1 correct 6 11 4 6 12 1 11 10 Wrong No idea 8 - 4 8 33 6 6 3 0 0 2 7 Discussion There are some issues related to the design of the experiments that could have influenced the outcome of the experimentation: − In a graphical representation, if you want to identify a particular point, you need to find the information concerning that point on each axis. In the experiment, we provided little information about the scale (for example, see Fig. 6 and Fig. 7). The wording gives the limits for the time period. The lack of scaling information could have caused some difficulty in identifying particular points or sub-periods. − The outcomes in the case of sonification of a 3D graph are worse than in the case of 2D. It is necessary to take in account that the sonification of a 3D graphical representation is more difficult then the sonification of a 2D graphical representation. A possible reason could be that the sonification technique is based on visual representation and does not use sound properties, but surface properties, as seen in a 3 axis referential. An important issue becomes the correspondence between visual and audio representations of the data. Consistent representations should provide audio representations that allow transitions from 3D to 2D projections in terms of corresponding sound representations. 8 Conclusions Overall, the results of the experimentation on sonification of time dependent data leave optimism for further investigation of sound as medium for presenting information. The sound can be an effective complementary interface to the visual interface for data representation. Similar results were presented by Alty [5] for people with disabilities. On the other hand, the experimentation with sonification of surfaces in 124 Noirhomme-Fraiture et al. 3D space did not efficiently support the visual representation and certainly could not replace it (at least for the way it had been implemented). In general, this experimental work contributes to the research efforts on bringing other (non-visual) channels for information and data processing. This research area, that can be labelled as ‘perceptual data mining’, is focused on interactive systems that support rich perceptual – visual, audio, tactile – interaction between the human and the data representation. Such systems are expected to play significant role in assisting data understanding and supporting pattern discovery process, utilising human information processing capabilities. Acknowledgements We thank our colleagues Anne de Baenst and Florence Collot for their help in the preparation of this paper. References 1. 2. 3. 4. 5. 6. 7. 8. Ankerst, M.: Visual Data Mining: Ph.D. thesis. Faculty of Mathematics and Computer Science, University of Munich, Dissertation.de, 2000. Wickens, C.D. and Carswell, C.M.: The proximity compatibility principle: Its psychological foundation and relevance to display design. Human Factors. 37 (1995) 473-495 Simoff, S.J.: Towards the development of environments for designing visualisation support for visual data mining. Proceedings Int. Workshop on Visual Data Mining, 12th European Conference on Machine Learning and 5th European Conference on Principles and Practice of Knowledge Discovery in Databases ECML/PKDD2001. Freiburg, Germany (2001) 93-106 Anrijs, K.: The use of sound in 3d representations of symbolic objects. Namur, Facultés Universitaires Notre-Dame de la Paix, 1999. Alty, J.L., Rigas, D. and Vickers, P.: Using music as a communication medium. Proceedings of CHI'97 (1997) Brewster, S.A., Wright, P.C. and Edwards, A.D.N.: An evaluation of earcons for use in auditory human-computer interfaces. Proceedings of Inter CHI'93 (1993) Conversy, S. and Beaudouin-Lafon, M.: Le son dans les applications interactives. Paris, Laboratoire de Recherche en Informatique, Université de ParisSud, 1995. Hermann, T.: Data exploration by sonification, 1999. Available from [http://www.techfak.unibielefeld.de/techfak/ags/ni/projects/datamining/datamin_e.htm]. Sonification of Time Dependent Data 9. 10. 11. 12. 125 Kramer, G.: An introduction to auditory display, Auditory Display: Sonification, Audification, and Auditory Interfaces. Santa Fe Institute Studies in the Sciences of Complexity (1994) Noirhomme-Fraiture, M.: Le son dans les interfaces IHM: Application à la représen-tation de données multivariées complexes. Actes des 2èmes Journées Multimédia. Namur (2000) Sahyun, S.C.: A comparison of auditory and visual graphs for use in physics and mathematics, Oregon State University, 1999. Available from [http://www.physics.orst.edu/sahyun/thesis/]. Java: Javasound API programmer's guide, Sun Microsystems, 2000. Author Index Paulo Azevedo ………………………………………… 43 Dario Bruzzese ………………………………………… 55 Tizianna Catarci ………………………………………… 27 Luca Chittaro ………………………………………… 97 Carlo Combi ………………………………………… 97 Cristina. Davino ………………………………………… 55 Christophe Demoulin ………………………………………… 113 Alipio Jorge ………………………………………… 43 George Katopodis ………………………………………… 11 Stephen Kimani ………………………………………… 27 Kan Liu ………………………………………… 1 Monique Noirhomme-Fraiture ………………………………………… 113 Penny Noy ………………………………………… 81 João Poças ………………………………………… 43 François Poulet ………………………………………… 67 Giuseppe Santucci ………………………………………… 27 Olivier Schöller ………………………………………… 113 Michael Schroeder ………………………………………… 11, 81 Simeon. J. Simoff ………………………………………… 113 Giampaolo Trapasso ………………………………………… 97 Dongru Zhou ………………………………………… 1 Xiaozheng Zhou ………………………………………… 1