International Journal of Information Systems for Logistics and Management Vol. 6, No. 1 (2010) 25-30 http://www.knu.edu.tw/academe/englishweb/web/ijislmweb/index.html An Evaluation Method Based on Multi-Attributes Analysis with Stochastic Dominances for Improving the Information Quality Mei-Chen Lo1* and Jerzy Michnik2 1Department of Business Management, National United University No. 1, Lienda, Miaoli 36003, Taiwan 2Operations Research Department, University of Economics in Katowice ul. 1 Maja 15, 40-287 Katowice, Poland Received 18 October 2010; received in revised form 26 November 2010; accepted 3 December 2010 ABSTRACT The virtual business work flow depends on information quality (IQ). Based on discussion with experts, six alternative strategies, these can be used for improving the IQ, have been designed. This study has presented the quantitative model suitable for group decision making. It takes into account multiple aspects of analyzed decision problem and uses PROMETHEE methodology for setting the preferences between alternatives. Results have been compared with the multiple criteria model based on stochastic dominances. Knowledge-based strategies for obtaining the benefits of this approach are demonstrated using the high-tech industry as a case study. This study suggests that external types of strategies like strategic networked alliance (suppliers or/and customers) are suitable to develop fair relationships with external subjects and explore the market. The case study showed that the proposed methodology is workable and useful for managerial practice. Keywords: decision analysis, multiple attributes analysis, stochastic dominance (SD), information quality (IQ), Preference Ranking Organization methods for Enrichment Evaluations (PROMETHEE). 1. INTRODUCTION The high-tech industry, as many other electronic sectors, relies heavily on continuous innovation activities to sustain a higher market share. Due to increasing global economic integration and technological competition, knowledge management (KM) has become a vital business tool in recent years. Since that, the virtual business work flow depends on information quality. The proposed approach uses PROMETHEE (Preference Ranking Orga- *Corresponding author: meichen_lo@yahoo.com nization methods for Enrichment Evaluations) to enable internal users to identify generic process elements that are relevant to the current business flow. Knowledge-based strategies for obtaining the benefits of this approach are demonstrated using the high-tech institutes as a case study. The value of information circulation and utilization is in the background of both: promoting the use of information platforms to establish communication channels, and strengthening and opening communication channels. 26 International Journal of the Information Systems for Logistics and Management (IJISLM), Vol. 6, No. 1 (2010) However, in the rapidly changing and competitive commercial environment, the proliferation of information often causes confusion. The data were collected from twelve independent units in the science-based park in Taiwan, using both interview and email survey. This study suggests that operation management positively influences the quality of information. Then information quality (IQ) is transformed into core competence, which improves firm performance. Strategic networked alliance (suppliers or/and customers) is the trend to develop fair external relationships to provide satisfied customer services and explore new market demand. The rest of this paper is organized as follow: In Section 2, the basic concepts of information quality and its evaluation are introduced and described. In Section 3 multiple attribute aggregations with PROMETHEE methodology are proposed. An empirical study is illustrated to explore the proposed method in Section 4. Then discussions are conducted in Section 5. Finally, conclusions are presented in Section 6. 2. INFORMATION QUALITY AND ITS EVALUATION IQ describes the quality of the content of information systems. Also, it may term as the fitness for use of the information provided. Most information system practitioners use the term synonymously with data quality. The relatively small scale of many enterprises limits their ability to access the type and volume of information that they require. For this reason, maintaining adequate control over organizational processes is a critical management issue since managers must be able to detect, diagnose and resolve exceptions in their business processes. Traditionally, managers have relied on their own experience and understanding of a process in order to handle deviations from the expected flow of events. Information quality assurance (IQA) is the process to guarantee confidence that particular information meets some context specific quality requirements. However, the rising complexity of modern business processes and the accelerating pace with which these processes evolve and change has made the reliance on individual managers’ experience and intuition an increasingly unsatisfactory way to deal with exceptions. Business process modeling has been used successfully via well established information systems in order to increase understanding, facilitate analysis and enhance IQA’s communication among the various stakeholders involved in the design and enactment of an “ideal” business process. 2.1 Hierarchical Evaluation Model The hierarchical model was developed for the KM evaluation and presented in previous work (Michnik and Lo, 2009). It applies the structure (Wang and Strong, 1996; Table 1. Decision-making hierarchy structure for evaluation model Aspects Attributes C1 Intrinsic c1 c2 c3 c4 Accuracy Objectivity Believability Reputation 0.168 0.052 0.076 0.066 C2 Contextual c5 c6 c7 c8 c9 Relevancy Value added Timeliness Completeness Amount of information 0.028 0.033 0.037 0.057 0.055 Interpretability Ease of understanding Concise representation Consistent representation 0.050 0.075 0.040 0.078 c10 c11 C3 Representational c12 c13 C4 Accessibility c14 Access c15 Convenience c16 Security Weights 0.101 0.044 0.055 Huang et al., 1999) and the concepts (Tzeng, 1977; Tzeng and Shiau, 1987; Tzeng et al., 1992; Teng and Tzeng, 1994) which establish the multiple attributes to study objects within the frame of quantitative model. In our approach we incorporated the revised fouraspect representation of IQ proposed by Wang and Strong (1996) in order to draw up four evaluation aspects as shown in Table 1: (1) intrinsic IQ; (2) contextual IQ; (3) representational IQ; and (4) accessibility IQ. The phase of intrinsic IQ includes the criteria of accuracy, objectivity, believability and reputation. The phase of contextual IQ includes the five criteria such as relevancy, value added, timeliness, completeness and amount of information. Representational IQ covers the four attributes: interpretability; ease of understanding, concise representation and consistent representation. Accessibility IQ concerns the three attributes: access, convenience and security. This study adopts this framework (Table 1) as a basis to measure, analyze, and improve data quality. The model shows a decision-making hierarchy structure with three levels, four aspects and sixteen evaluation attributes of IQ. 2.2 Alternatives The limited resources prevent particular institute to establish perfect system for IQ management and assurance procedures. The discussion with experts allowed us to formulate six different strategies Ak which are designated for improvement direction of IQ. In our model they play the role of decision alternatives. Fig. 1 visualizes the placement of these strategies on the background of schematic structure of the institute and its close environment. M. C. Lo and J. Michnik: An Evaluation Method Based on Multi-Attributes Analysis with Stochastic Dominances... 27 A6 A1 A2 Institute Data flow Suppliers Customers A5 Data A4 Information A3 Knowledge Intellectual Property Fig. 1. Strategies’ placement Below we present its short characteristics. Each of alternatives (Michnik and Lo, 2009) is described as it implies in this study and the data flow within the functional organization. A1: Suppliers related. This strategy concentrates on building relations with suppliers to improve and strengthen their own product design, production, marketing and system development. A2: Customers related. This strategy strengthens relationship with customer and is combined with insight into customer values. Institute is able to make quick response by bringing all aspects of resources, including IT solutions, to solve the customer’s problems when the customer needs help. Being customer-centric, it is necessary to operates at all times with the customer in mind, focusing on quality, innovation and value (Gibbert et al., 2002). A3: Intellectual property related (such as innovation, knowledge implementation). Organizations need to empower knowledge workers so that they are constantly encouraged to seek improvements, engage in innovative thinking, take necessary risks, and focus on creating solutions for customers. Besides, competing for intellectual influence, firm must set and lever up the information and knowledge management system. A4: Knowledge-oriented activities. Knowledge is largely derived from information in context. Companies need to organize knowledge about customers and processes, particularly for front-line knowledge workers. Corporate knowledge is an asset of increasing value (Chu et al., 2007). A5: Data collection and information database building. This strategy focuses on internal operation management and concentrate effort on data collection and system platform. A6: Strategic networked alliance (suppliers, customers). A strategic networked alliance is a powerful, systematic process that can configure different competencies to form a business alliance to deliver innovative solutions and dominate the market. Strategic alliances provide a launching pad for greater innovation, faster time to market, and higher profitability (Yim et al., 2004). 3. MULTIPLE ATTRIBUTE AGGREGATION WITH PROMETHEE METHODOLOGY The PROMETHEE methodology (Brans and Mareschal, 2005) has been applied for the sake of comparison of alternatives. The finite set of alternatives A = {A1, A2, ..., An} is evaluated with the aid of finite set of criteria {g1, g2, ..., gm}. The value of criterion gj for alternative Ak is denoted by gj{Ak}. For each criterion, the preference function needs to be defined on the difference of evaluations. It has a form Pj(A, B) = Pj(dj(A, B)), where dj(A, B) = gj(A) – gj(B), A, B - a pair of two different alternatives. The pair (gj, Pj) is called the generalized criterion. The decision maker should also provide the set of positive weights {w1, w2, ..., wm} which present the relative importance of criteria and sum to unity. With the aid of preference functions and weights, the aggregated preference indices are defined, m π (A, B) = Σ P j(A, B)w j j=1 m π (B, A) = Σ P j(B, A)w j j=1 π (A, B) is expressing with which degree A is preferred to B over all the criteria; And π (B, A) states how the degree B is preferred to A. 28 International Journal of the Information Systems for Logistics and Management (IJISLM), Vol. 6, No. 1 (2010) In the next step the positive and negative outranking flows are calculated, Φ +(A) = 1 n–1 X∈A Φ –(A) = 1 n–1 X∈A Σ π (A, X) Σ (1) A is preferred over B Φ +(A) > Φ +(B) and Φ –(A) < Φ –(B) or Φ +(A) = Φ +(B) and Φ –(A) < Φ –(B) or Φ +(A) > Φ +(B) and Φ –(A) = Φ –(B) (2) A and B are indifferent AIB iff Φ+(A) = Φ+(B) and Φ–(A) = Φ–(B) (3) A and B are incomparable ARB iff Φ +(A) > Φ +(B) and Φ –(A) > Φ –(B) or Φ +(A) < Φ +(B) and Φ –(A) < Φ –(B) In the case of PROMETHEE II, the net outranking flow represents the balance between the positive and negative outranking flows, the values of net outranking flows give the complete order of alternatives. Φ(A) = Φ+(A) – Φ–(A) 4. CASE STUDY: HIGH-TECH INSTITUTES IN TAIWAN The questionnaires were sent by email to twelve independent units in the science-based parks in Taiwan. The sample data for model calculations were taken from 28 completed answers. The scale 0-100 has been used for the evaluation. The weights of attributes and the evaluations of alternatives were calculated as the average values from all sample data. In our example for all criteria, the preference definition has been divided into three categories (d ≤ 0, 0 ≤ d ≤ p, and d ≤ p), which the generalized criterion has been taken according to the formula, P(d) = 3 2 4 6 π (X, A) The positive outranking flow expresses how an alternative A is outranking all the others. The negative outranking flow expresses how an alternative A is outranked by all the others. The PROMETHEE I partial ranking is given by following relations, APB iff 1 0 if d ≤ 0 d/p if 0 ≤ d ≤ p 1 if d ≥ p 5 Fig. 2. Preference relations of alternatives received by PROMETHEE I where p is a threshold of strict preference. For further calculations the arithmetic averages has been taken. For all criteria the threshold of strict preference was assumed to be 10. According to PROMETHEE I methodology we get the following partial order of alternatives (Fig. 2), A6PA1, A6PA2, A6PA3, A6PA4, A6PA5, A4PA1, A4PA2, A4PA3, A4PA5, A2PA1, A2PA3, A2PA5, A1PA3, A1PA5, A3PA5 where P denotes performance relation, and R denotes as incomparable relation. The above preference relations can be presented graphically as follows, According to Table 2, PROMETHEE II provides a complete ranking. It is based on the balance of the two preference flows. Both PROMETHEE I and II help the decision-maker to finalize the selection of a best compromise. With Table 3 data, the positive flow and the negative flow were calculated, instantly, the net flows are obtained and provides a clear view of the outranking relations between the alternatives. The positions of the first three alternatives are the same as in the ranking received from the weighted average calculations, but the order of the least three is different. Additionally, we present the complete ranking given by weighted sum of average attribute scores (Fig. 3). The PROMETHEE I partial ranking presented on Fig. 3, it can be compared with partial ranking received within our model presented in previous paper (Michnik and Lo, 2009), which the example rankings with ELECTRE approach (Roy, 1985) received for various parameters are presented on Table 4. 5. DISCUSSIONS For the sake of practical application of our model, the hierarchical structure of IQ evaluation model is developed. The number of subjects who take part in this survey, presented their opinion abut the relative importance of different aspects of the IQ concerns. The result is summarized in a set of weights assigned to 16 attributes. 29 M. C. Lo and J. Michnik: An Evaluation Method Based on Multi-Attributes Analysis with Stochastic Dominances... Table 2. Aggregate preference indices Alternatives A1 A2 A3 A4 A5 A6 Positive flow A1 A2 A3 A4 A5 A6 0.00 0.38 0.37 0.49 0.30 0.62 0.26 0.00 0.06 0.29 0.07 0.56 0.30 0.28 0.00 0.45 0.18 0.73 0.08 0.15 0.06 0.00 0.03 0.41 0.27 0.29 0.17 0.42 0.00 0.69 0.00 0.07 0.01 0.07 0.00 0.00 0.18 0.24 0.13 0.34 0.12 0.60 0.43 0.25 0.39 0.14 0.37 0.03 – Negative flow Table 3. PROMETHEE II complete ranking Net flow Complete ranking -0.25 -0.01 -0.26 0.20 -0.25 0.57 A1 A2 A3 A4 A5 A6 Table 4. The examples of rankings with various values of PT and V Item 4 3 6 2 5 1 Threshold Alternatives 1 PT = 1 N/A 2 PT = 0.9 A3 3 PT = 0.8 A6 A3 A6 A4 A2 7.6 Weighted Average 7.35 4 A3 A4 A6 A5 VT = 0.3 7.2 A1 7.01 A6 6.82 6.8 6.59 6.58 6.48 6.4 6.0 PT = 0.6 A6 A4 A2 A5 Alternatives A3 A1 Fig. 3. Ranking of alternatives by weighted sum of average attribute scores the matured institutes with accumulated experience (more than 15 years in average). For example, strategies A5 and A3 can be considered as more important in case of new set or emerging units. Strategies A4 and A5 help the result of IQA processes to be adopted/delivered more reliably and effectively through both internal and external IQ activities. 6. CONCLUSIONS Based on the exhaustive discussion with experts, the six alternative strategies have been designed which can be used for improving the IQ. Then we asked the surveyed subjects about their estimates to how extend the given alternative give rise to improvement of each attribute. With the aid of PROMETHEE methodology, we received the picture of group preferences between alternatives. The results are similar to those received within different multiple criteria approach. From this study, it is necessary to stress that strategy evaluation can depend on the position of institute within the industry. The results point out the preference of external strategies (like A6, A1 and A2) to internal ones (like A3, A4 and A5) is reliable. Such result indicates that the subjects, which took part in the survey, represented IQ exists the high impact on performance evaluation if the effective work can be fulfilled in time. We developed the hierarchical structure for quantitative analysis of IQ management. This gives a basis for evaluation multiple attributes grouped in a few aspects at the higher level of hierarchy. We also have presented the quantitative model suitable for group decision making. It takes into account multiple aspects of analyzed decision problem and uses PROMETHEE methodology for setting the preferences between alternatives. The results confirm the previous findings that used stochastic dominances and multiple criteria aggregation model similar to ELECTRE approach. It seems that the hierarchical structure and weights developed for criteria are more important for 30 International Journal of the Information Systems for Logistics and Management (IJISLM), Vol. 6, No. 1 (2010) the final ranking than the aggregation method. It means that the analyst should pay more attention to this part of analysis. On the other hand, the final ranking is quite robust against the method of aggregation, which gives some room for flexible choice of most suitable method. REFERENCES Brans, J. P. and Mareschal, B. (2005) Promethee Methods. In: Multiple Criteria Decision Analysis, State of the Art Surveys. International Series in Operations Research & Management Science, 78(3), 163-186. DOI: 10.1007/0387-23081-5_5. Chu, M. T., Shyu, J., Tzeng, G. H. and Khosia, R. (2007) Comparison among three analytical methods for knowledge communities group-decision analysis. Expert Systems with Applications, 33(4), 1011-1024. Gibbert, M., Leibold, M. and Probst, G. (2002) Five Styles of Customer Knowledge Management, and How Smart Companies Use Them To Create Value. European Management Journal, 20(5), 459-469. Huang, K. T., Lee, Y. W. and Wang, R. Y. (1999) Quality Information and Knowledge. New Jersey: Prentice Hall PTR. Michnik, J. and Lo, M. C. (2009) The Assessment of the Information Quality with the Aid ofé Multiple Criteria Analysis. European Journal of Operation Research, 195(3), 850856. Roy , B. (1985) Méthodologie Multicritère d’Aide à la Décision. Economica, Paris. Tzeng, G. H. (1977) A study on the PATTERN Method for the Decision Process in the Public System. Japan Journal of Behaviormetrics, 4(2), 29-44. Tzeng, G. H. and Shiau, T. A. (1987) Energy Conservation Strategies in Urban Transportation: Application of Multiple Criteria Decision-Making. Energy Systems and Policy, 11(1), 1-19. Tzeng, G. H., Shian, T. A. and Lin, C. Y. (1992) Application of Multicriteria Decision Making to the Evaluation of New Energy-System Development in Taiwan. Energy, 17(10), 983-992. Teng, J. Y. and Tzeng, G. H. (1994) Multicriteria Evaluation for Strategies of Improving and Controlling Air-Quality in the Super City: A Case of Taipei City. Journal of Environmental Management, 40(3), 213-229. Wang, R. Y. and Strong, D. M. (1996) Beyond Accuracy: What Data Quality Means to Data Consumers. Journal of Management Information Systems, 12(4), 5-34. Yim, N. H., Kim, S. H., Kim, H. W. and Kwahk, K. Y. (2004) Knowledge based decision making on higher level strategic concerns: system dynamics approach. Expert Systems with Applications, 27(1), 143-158.