International Journal of Information Systems for Logistics and Management
Vol. 6, No. 1 (2010) 25-30
http://www.knu.edu.tw/academe/englishweb/web/ijislmweb/index.html
An Evaluation Method Based on Multi-Attributes
Analysis with Stochastic Dominances for
Improving the Information Quality
Mei-Chen Lo1* and Jerzy Michnik2
1Department
of Business Management, National United University
No. 1, Lienda, Miaoli 36003, Taiwan
2Operations Research Department, University of Economics in Katowice
ul. 1 Maja 15, 40-287 Katowice, Poland
Received 18 October 2010; received in revised form 26 November 2010; accepted 3 December 2010
ABSTRACT
The virtual business work flow depends on information quality (IQ). Based on discussion with
experts, six alternative strategies, these can be used for improving the IQ, have been designed. This
study has presented the quantitative model suitable for group decision making. It takes into account
multiple aspects of analyzed decision problem and uses PROMETHEE methodology for setting the
preferences between alternatives. Results have been compared with the multiple criteria model based
on stochastic dominances. Knowledge-based strategies for obtaining the benefits of this approach are
demonstrated using the high-tech industry as a case study. This study suggests that external types of
strategies like strategic networked alliance (suppliers or/and customers) are suitable to develop fair
relationships with external subjects and explore the market. The case study showed that the proposed
methodology is workable and useful for managerial practice.
Keywords: decision analysis, multiple attributes analysis, stochastic dominance (SD), information
quality (IQ), Preference Ranking Organization methods for Enrichment Evaluations
(PROMETHEE).
1. INTRODUCTION
The high-tech industry, as many other electronic
sectors, relies heavily on continuous innovation activities
to sustain a higher market share. Due to increasing global
economic integration and technological competition,
knowledge management (KM) has become a vital business tool in recent years. Since that, the virtual business
work flow depends on information quality. The proposed
approach uses PROMETHEE (Preference Ranking Orga-
*Corresponding author: meichen_lo@yahoo.com
nization methods for Enrichment Evaluations) to enable
internal users to identify generic process elements that are
relevant to the current business flow. Knowledge-based
strategies for obtaining the benefits of this approach are
demonstrated using the high-tech institutes as a case
study.
The value of information circulation and utilization
is in the background of both: promoting the use of information platforms to establish communication channels,
and strengthening and opening communication channels.
26
International Journal of the Information Systems for Logistics and Management (IJISLM), Vol. 6, No. 1 (2010)
However, in the rapidly changing and competitive commercial environment, the proliferation of information often causes confusion.
The data were collected from twelve independent
units in the science-based park in Taiwan, using both interview and email survey. This study suggests that operation management positively influences the quality of
information. Then information quality (IQ) is transformed
into core competence, which improves firm performance.
Strategic networked alliance (suppliers or/and customers)
is the trend to develop fair external relationships to provide satisfied customer services and explore new market
demand.
The rest of this paper is organized as follow: In
Section 2, the basic concepts of information quality and
its evaluation are introduced and described. In Section
3 multiple attribute aggregations with PROMETHEE
methodology are proposed. An empirical study is illustrated to explore the proposed method in Section 4. Then
discussions are conducted in Section 5. Finally, conclusions are presented in Section 6.
2. INFORMATION QUALITY AND
ITS EVALUATION
IQ describes the quality of the content of information systems. Also, it may term as the fitness for use of
the information provided. Most information system practitioners use the term synonymously with data quality. The
relatively small scale of many enterprises limits their ability to access the type and volume of information that they
require. For this reason, maintaining adequate control over
organizational processes is a critical management issue
since managers must be able to detect, diagnose and resolve exceptions in their business processes. Traditionally,
managers have relied on their own experience and understanding of a process in order to handle deviations from
the expected flow of events. Information quality assurance (IQA) is the process to guarantee confidence that
particular information meets some context specific quality requirements. However, the rising complexity of modern business processes and the accelerating pace with
which these processes evolve and change has made the
reliance on individual managers’ experience and intuition
an increasingly unsatisfactory way to deal with exceptions.
Business process modeling has been used successfully via
well established information systems in order to increase
understanding, facilitate analysis and enhance IQA’s communication among the various stakeholders involved in
the design and enactment of an “ideal” business process.
2.1 Hierarchical Evaluation Model
The hierarchical model was developed for the KM
evaluation and presented in previous work (Michnik and
Lo, 2009). It applies the structure (Wang and Strong, 1996;
Table 1. Decision-making hierarchy structure for
evaluation model
Aspects
Attributes
C1 Intrinsic
c1
c2
c3
c4
Accuracy
Objectivity
Believability
Reputation
0.168
0.052
0.076
0.066
C2 Contextual
c5
c6
c7
c8
c9
Relevancy
Value added
Timeliness
Completeness
Amount of information
0.028
0.033
0.037
0.057
0.055
Interpretability
Ease of understanding
Concise representation
Consistent representation
0.050
0.075
0.040
0.078
c10
c11
C3 Representational
c12
c13
C4 Accessibility
c14 Access
c15 Convenience
c16 Security
Weights
0.101
0.044
0.055
Huang et al., 1999) and the concepts (Tzeng, 1977; Tzeng
and Shiau, 1987; Tzeng et al., 1992; Teng and Tzeng,
1994) which establish the multiple attributes to study objects within the frame of quantitative model.
In our approach we incorporated the revised fouraspect representation of IQ proposed by Wang and Strong
(1996) in order to draw up four evaluation aspects as
shown in Table 1: (1) intrinsic IQ; (2) contextual IQ; (3)
representational IQ; and (4) accessibility IQ. The phase
of intrinsic IQ includes the criteria of accuracy, objectivity,
believability and reputation. The phase of contextual
IQ includes the five criteria such as relevancy, value added,
timeliness, completeness and amount of information.
Representational IQ covers the four attributes: interpretability; ease of understanding, concise representation and
consistent representation. Accessibility IQ concerns the
three attributes: access, convenience and security.
This study adopts this framework (Table 1) as a
basis to measure, analyze, and improve data quality. The
model shows a decision-making hierarchy structure with
three levels, four aspects and sixteen evaluation attributes
of IQ.
2.2 Alternatives
The limited resources prevent particular institute to
establish perfect system for IQ management and assurance procedures. The discussion with experts allowed
us to formulate six different strategies Ak which are designated for improvement direction of IQ. In our model they
play the role of decision alternatives. Fig. 1 visualizes the
placement of these strategies on the background of schematic structure of the institute and its close environment.
M. C. Lo and J. Michnik: An Evaluation Method Based on Multi-Attributes Analysis with Stochastic Dominances...
27
A6
A1
A2
Institute
Data flow
Suppliers
Customers
A5
Data
A4
Information
A3
Knowledge
Intellectual
Property
Fig. 1. Strategies’ placement
Below we present its short characteristics.
Each of alternatives (Michnik and Lo, 2009) is described as it implies in this study and the data flow within
the functional organization.
A1: Suppliers related. This strategy concentrates on building relations with suppliers to improve and strengthen
their own product design, production, marketing and
system development.
A2: Customers related. This strategy strengthens relationship with customer and is combined with insight into
customer values. Institute is able to make quick response by bringing all aspects of resources, including
IT solutions, to solve the customer’s problems when
the customer needs help. Being customer-centric, it
is necessary to operates at all times with the customer
in mind, focusing on quality, innovation and value
(Gibbert et al., 2002).
A3: Intellectual property related (such as innovation,
knowledge implementation). Organizations need to
empower knowledge workers so that they are constantly encouraged to seek improvements, engage in
innovative thinking, take necessary risks, and focus
on creating solutions for customers. Besides, competing for intellectual influence, firm must set and
lever up the information and knowledge management
system.
A4: Knowledge-oriented activities. Knowledge is largely
derived from information in context. Companies need
to organize knowledge about customers and processes, particularly for front-line knowledge workers.
Corporate knowledge is an asset of increasing value
(Chu et al., 2007).
A5: Data collection and information database building.
This strategy focuses on internal operation management and concentrate effort on data collection and
system platform.
A6: Strategic networked alliance (suppliers, customers). A
strategic networked alliance is a powerful, systematic
process that can configure different competencies
to form a business alliance to deliver innovative solutions and dominate the market. Strategic alliances
provide a launching pad for greater innovation, faster
time to market, and higher profitability (Yim et al.,
2004).
3. MULTIPLE ATTRIBUTE AGGREGATION
WITH PROMETHEE METHODOLOGY
The PROMETHEE methodology (Brans and
Mareschal, 2005) has been applied for the sake of comparison of alternatives. The finite set of alternatives A =
{A1, A2, ..., An} is evaluated with the aid of finite set of
criteria {g1, g2, ..., gm}. The value of criterion gj for alternative Ak is denoted by gj{Ak}. For each criterion, the
preference function needs to be defined on the difference
of evaluations. It has a form Pj(A, B) = Pj(dj(A, B)), where
dj(A, B) = gj(A) – gj(B), A, B - a pair of two different
alternatives. The pair (gj, Pj) is called the generalized
criterion.
The decision maker should also provide the set of
positive weights {w1, w2, ..., wm} which present the relative importance of criteria and sum to unity. With the
aid of preference functions and weights, the aggregated
preference indices are defined,
m
π (A, B) =
Σ P j(A, B)w j
j=1
m
π (B, A) =
Σ P j(B, A)w j
j=1
π (A, B) is expressing with which degree A is preferred to
B over all the criteria; And π (B, A) states how the degree
B is preferred to A.
28
International Journal of the Information Systems for Logistics and Management (IJISLM), Vol. 6, No. 1 (2010)
In the next step the positive and negative outranking flows are calculated,
Φ +(A) =
1
n–1
X∈A
Φ –(A) =
1
n–1
X∈A
Σ π (A, X)
Σ
(1) A is preferred over B
Φ +(A) > Φ +(B) and Φ –(A) < Φ –(B) or
Φ +(A) = Φ +(B) and Φ –(A) < Φ –(B) or
Φ +(A) > Φ +(B) and Φ –(A) = Φ –(B)
(2) A and B are indifferent
AIB iff Φ+(A) = Φ+(B) and Φ–(A) = Φ–(B)
(3) A and B are incomparable
ARB iff
Φ +(A) > Φ +(B) and Φ –(A) > Φ –(B) or
Φ +(A) < Φ +(B) and Φ –(A) < Φ –(B)
In the case of PROMETHEE II, the net outranking
flow represents the balance between the positive and negative outranking flows, the values of net outranking flows
give the complete order of alternatives.
Φ(A) = Φ+(A) – Φ–(A)
4. CASE STUDY: HIGH-TECH INSTITUTES
IN TAIWAN
The questionnaires were sent by email to twelve independent units in the science-based parks in Taiwan. The
sample data for model calculations were taken from 28
completed answers. The scale 0-100 has been used for
the evaluation. The weights of attributes and the evaluations of alternatives were calculated as the average values
from all sample data.
In our example for all criteria, the preference definition has been divided into three categories (d ≤ 0, 0 ≤ d ≤
p, and d ≤ p), which the generalized criterion has been
taken according to the formula,
P(d) =
3
2
4
6
π (X, A)
The positive outranking flow expresses how an alternative A is outranking all the others. The negative outranking flow expresses how an alternative A is outranked
by all the others.
The PROMETHEE I partial ranking is given by
following relations,
APB iff
1
0
if d ≤ 0
d/p if 0 ≤ d ≤ p
1
if d ≥ p
5
Fig. 2. Preference relations of alternatives received by PROMETHEE I
where p is a threshold of strict preference.
For further calculations the arithmetic averages has
been taken. For all criteria the threshold of strict preference was assumed to be 10. According to PROMETHEE
I methodology we get the following partial order of alternatives (Fig. 2),
A6PA1, A6PA2, A6PA3, A6PA4, A6PA5, A4PA1, A4PA2,
A4PA3, A4PA5, A2PA1, A2PA3, A2PA5, A1PA3, A1PA5,
A3PA5
where P denotes performance relation, and R denotes as
incomparable relation.
The above preference relations can be presented
graphically as follows,
According to Table 2, PROMETHEE II provides a
complete ranking. It is based on the balance of the two
preference flows. Both PROMETHEE I and II help the
decision-maker to finalize the selection of a best compromise.
With Table 3 data, the positive flow and the negative flow were calculated, instantly, the net flows are
obtained and provides a clear view of the outranking
relations between the alternatives. The positions of the
first three alternatives are the same as in the ranking received from the weighted average calculations, but the
order of the least three is different.
Additionally, we present the complete ranking given
by weighted sum of average attribute scores (Fig. 3).
The PROMETHEE I partial ranking presented on
Fig. 3, it can be compared with partial ranking received
within our model presented in previous paper (Michnik and
Lo, 2009), which the example rankings with ELECTRE
approach (Roy, 1985) received for various parameters
are presented on Table 4.
5. DISCUSSIONS
For the sake of practical application of our model,
the hierarchical structure of IQ evaluation model is
developed. The number of subjects who take part in this
survey, presented their opinion abut the relative importance of different aspects of the IQ concerns. The result
is summarized in a set of weights assigned to 16 attributes.
29
M. C. Lo and J. Michnik: An Evaluation Method Based on Multi-Attributes Analysis with Stochastic Dominances...
Table 2. Aggregate preference indices
Alternatives
A1
A2
A3
A4
A5
A6
Positive flow
A1
A2
A3
A4
A5
A6
0.00
0.38
0.37
0.49
0.30
0.62
0.26
0.00
0.06
0.29
0.07
0.56
0.30
0.28
0.00
0.45
0.18
0.73
0.08
0.15
0.06
0.00
0.03
0.41
0.27
0.29
0.17
0.42
0.00
0.69
0.00
0.07
0.01
0.07
0.00
0.00
0.18
0.24
0.13
0.34
0.12
0.60
0.43
0.25
0.39
0.14
0.37
0.03
–
Negative flow
Table 3. PROMETHEE II complete ranking
Net flow
Complete ranking
-0.25
-0.01
-0.26
0.20
-0.25
0.57
A1
A2
A3
A4
A5
A6
Table 4. The examples of rankings with various values of PT and V
Item
4
3
6
2
5
1
Threshold
Alternatives
1
PT = 1
N/A
2
PT = 0.9
A3
3
PT = 0.8
A6
A3
A6
A4
A2
7.6
Weighted Average
7.35
4
A3
A4
A6
A5
VT = 0.3
7.2
A1
7.01
A6
6.82
6.8
6.59
6.58
6.48
6.4
6.0
PT = 0.6
A6
A4
A2
A5
Alternatives
A3
A1
Fig. 3. Ranking of alternatives by weighted sum of average attribute
scores
the matured institutes with accumulated experience (more
than 15 years in average). For example, strategies A5 and
A3 can be considered as more important in case of new set
or emerging units. Strategies A4 and A5 help the result
of IQA processes to be adopted/delivered more reliably
and effectively through both internal and external IQ
activities.
6. CONCLUSIONS
Based on the exhaustive discussion with experts, the
six alternative strategies have been designed which can
be used for improving the IQ. Then we asked the surveyed subjects about their estimates to how extend the
given alternative give rise to improvement of each
attribute. With the aid of PROMETHEE methodology,
we received the picture of group preferences between
alternatives. The results are similar to those received
within different multiple criteria approach.
From this study, it is necessary to stress that strategy evaluation can depend on the position of institute
within the industry. The results point out the preference
of external strategies (like A6, A1 and A2) to internal ones
(like A3, A4 and A5) is reliable. Such result indicates that
the subjects, which took part in the survey, represented
IQ exists the high impact on performance evaluation if the effective work can be fulfilled in time. We
developed the hierarchical structure for quantitative analysis of IQ management. This gives a basis for evaluation
multiple attributes grouped in a few aspects at the higher
level of hierarchy. We also have presented the quantitative model suitable for group decision making. It takes
into account multiple aspects of analyzed decision problem and uses PROMETHEE methodology for setting the
preferences between alternatives. The results confirm the
previous findings that used stochastic dominances and
multiple criteria aggregation model similar to ELECTRE
approach. It seems that the hierarchical structure and
weights developed for criteria are more important for
30
International Journal of the Information Systems for Logistics and Management (IJISLM), Vol. 6, No. 1 (2010)
the final ranking than the aggregation method. It means
that the analyst should pay more attention to this part of
analysis. On the other hand, the final ranking is quite
robust against the method of aggregation, which gives
some room for flexible choice of most suitable method.
REFERENCES
Brans, J. P. and Mareschal, B. (2005) Promethee Methods. In:
Multiple Criteria Decision Analysis, State of the Art
Surveys. International Series in Operations Research &
Management Science, 78(3), 163-186. DOI: 10.1007/0387-23081-5_5.
Chu, M. T., Shyu, J., Tzeng, G. H. and Khosia, R. (2007) Comparison among three analytical methods for knowledge
communities group-decision analysis. Expert Systems
with Applications, 33(4), 1011-1024.
Gibbert, M., Leibold, M. and Probst, G. (2002) Five Styles of
Customer Knowledge Management, and How Smart Companies Use Them To Create Value. European Management Journal, 20(5), 459-469.
Huang, K. T., Lee, Y. W. and Wang, R. Y. (1999) Quality Information and Knowledge. New Jersey: Prentice Hall
PTR.
Michnik, J. and Lo, M. C. (2009) The Assessment of the Information Quality with the Aid ofé Multiple Criteria Analysis.
European Journal of Operation Research, 195(3), 850856.
Roy , B. (1985) Méthodologie Multicritère d’Aide à la Décision.
Economica, Paris.
Tzeng, G. H. (1977) A study on the PATTERN Method for the
Decision Process in the Public System. Japan Journal of
Behaviormetrics, 4(2), 29-44.
Tzeng, G. H. and Shiau, T. A. (1987) Energy Conservation Strategies in Urban Transportation: Application of Multiple
Criteria Decision-Making. Energy Systems and Policy,
11(1), 1-19.
Tzeng, G. H., Shian, T. A. and Lin, C. Y. (1992) Application of
Multicriteria Decision Making to the Evaluation of New
Energy-System Development in Taiwan. Energy, 17(10),
983-992.
Teng, J. Y. and Tzeng, G. H. (1994) Multicriteria Evaluation
for Strategies of Improving and Controlling Air-Quality
in the Super City: A Case of Taipei City. Journal of Environmental Management, 40(3), 213-229.
Wang, R. Y. and Strong, D. M. (1996) Beyond Accuracy: What
Data Quality Means to Data Consumers. Journal of Management Information Systems, 12(4), 5-34.
Yim, N. H., Kim, S. H., Kim, H. W. and Kwahk, K. Y. (2004)
Knowledge based decision making on higher level strategic concerns: system dynamics approach. Expert Systems
with Applications, 27(1), 143-158.