Abstract
Pairwise comparisons have been a long-standing technique for comparing alternatives/criteria and their role has been pivotal in the development of modern decision-making methods such as the Analytic Hierarchy/Network Process (AHP/ANP), the Best-Worst method (BWM), PROMETHEE and many others. Pairwise comparisons can be performed within several frameworks such as multiplicative, additive and fuzzy representations of preferences, which are particular instances of a more general framework based on Abelian linearly ordered groups. Though multiplicative, additive and fuzzy representations of preferences are widely used in practice, it is unknown whether decision makers are equally precise in the three aforementioned representations when they measure objective data. Therefore, the aim of this paper is to design, carry out and analyse an experiment with over 200 respondents (undergraduate university students) from two countries, Czechia and Italy, to compare precision of the respondents in all three representations. In the experiment, respondents pairwise compared (by approximation) the areas of four geometric figures and then, the imprecision of their assessments was measured by computing the distance with the exact pairwise comparisons. We grouped the respondents in such a way that each participant was allowed to deal with a unique type of representation. The outcomes of the experiment indicate that the multiplicative approach is the most precise.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
1 Introduction
Let \(X=\{x_{1},x_{2},\ldots ,x_{n}\}\) be a set of decision elements, such as alternatives or criteria; the entry of a Pairwise Comparisons Matrix (PCM) \({\textbf{G}}=[g_{ij}]_{n \times n}\) quantifies the preference intensity of \(x_i\) over \(x_j\). The aggregation of the entries of a PCM allows us to obtain a weighted ranking on X; that is a mapping that assigns a real value to each \(x_i\) (Barzilai et al., 1987). Origins of the PCMs can be dated back to the early works of Catalan medieval scholar Llull (1274). The first notable work with an application in psychology on pairwise comparisons can be, however, attributed to Thurstone (1927) and his Law of Comparative Judgments. Currently, PCMs constitute the core of the Analytic Hierarchy Process (AHP) and the Analytic Network Process (ANP) (Saaty, 1977, 1980), and of popular theoretical frameworks for the Multiple Criteria Decision Making (MCDM). For further details about PCMs, the reader can refer to Ramík (2020).
The main advantage in using PCMs is that they allow a Decision Maker (DM) to compare two alternatives at a time, thus reducing the complexity of a decision-making problem; indeed, it is easier to perform pairwise comparisons than to provide directly a weighted ranking on all the alternatives, especially when the set X is large.
In the literature, several types of PCMs are proposed:
-
if \(g_{ij}\in ]0, +\infty [\) represents a preference ratio, then \(G=[g_{ij}]_{n \times n}\) is a multiplicative PCM (Barzilai and Golany, 1990). In a multiplicative PCM, \(g_{ij}=1\) if there is indifference between \(x_{i}\) and \(x_{j}\), \(g_{ij}>1\) if \(x_{i}\) is strictly preferred to \(x_{j}\), whereas \(g_{ij}<1\) expresses the reverse preference;
-
if \(g_{ij}\in {\mathbb {R}} =] -\infty , +\infty [\) represents a preference difference, then \(G=[g_{ij}]_{n \times n}\) is an additive PCM (Barzilai, 1997). In an additive PCM, \(g_{ij}=0\) if there is indifference between \(x_{i}\) and \(x_{j}\), \(g_{ij}>0\) if \(x_{i}\) is strictly preferred to \(x_{j}\), whereas \(g_{ij}<0\) expresses the reverse preference;
-
if \(g_{ij} \in ]0, 1[\) reflects a preference degree, then \(G=[g_{ij}]_{n \times n}\) is a fuzzy PCM (Tanino, 1988). In a fuzzy PCM, \(g_{ij}=0.5\) if there is indifference between \(x_{i}\) and \(x_{j}\), \(g_{ij}>0.5\) if \(x_{i}\) is strictly preferred to \(x_{j}\), whereas \(g_{ij}<0.5\) expresses the reverse preference.
Multiplicative, additive and fuzzy PCMs share the same algebraic structure; that is, they are PCMs over Abelian linearly ordered (Alo)-groups (Cavallo and D’Apuzzo, 2009). Several authors follow this approach based on Alo-groups (e.g. Hou, 2016, Koczkodaj et al., 2016, Ramík, 2015 and Xia and Chen, 2015).
The foremost type of PCM, at least with respect to the number of real-world applications, is probably the multiplicative representation, used among others by Saaty (1980) in the theory of the Analytic Hierarchy Process (AHP) by using the finite semantic scale \(\{\frac{1}{9},\frac{1}{8},\frac{1}{7},\frac{1}{6},\frac{1}{5},\frac{1}{4},\frac{1}{3},\frac{1}{2},1,2,3,4,5,6,7,8,9\}\) instead of \(]0, +\infty [\).
Though multiplicative, additive and fuzzy approaches are widely used in practice, to best of our knowledge, in the literature no study has been conducted in order to compare their reliability, with the exclusion of an experiment performed by Cavallo et al. (2019); the authors compare the DMs’ coherence when they express subjective preferences by means of multiplicative, additive and fuzzy PCMs. However, no comparative study has been conducted in order to establish for which type of PCM, DMs are more precise when they measure objective data. This paper aims at filling this gap. We stress that if a DM measures objective data then (s)he can be perfectly coherent but not necessarily perfectly precise; on the other hand, if (s)he is perfectly precise then (s)he is perfectly coherent, too. In particular, we perform an experiment in order to compare the precision of these approaches when areas of four geometric figures have to be compared. In addition, we provide an appropriate methodology for the comparison of the approaches. The experiment is inspired by a validation study conducted by Saaty (2008); where, multiplicative pairwise comparisons have been applied to geometric areas in order to show the effectiveness of the AHP. In this paper, precision of additive and fuzzy pairwise comparisons, in addition to multiplicative ones, are analyzed.
The remainder of the paper is organized as follows: Sect. 2 provides preliminaries about Alo-groups and PCMs defined over Alo-groups. Section 3 describes the problem, the methodology and the results of the experiment. Section 4 provides conclusions and future work.
2 Preliminaries
In this section, we provide preliminaries about Alo-groups and PCMs defined over Alo-groups useful in the sequel; for further details the reader can refer to (Cavallo and D’Apuzzo, 2009).
Let G be a non-empty set, \(\odot : G\times G \rightarrow G\) a binary operation on G, \(\le\) a weak order on G. Then, \({\mathcal {G}}= (G, \odot , \le )\) is an Abelian linearly ordered group, Alo-group for short, if \((G, \odot )\) is an Abelian group and, for all \(g_1, g_2, g_3 \in G\), it holds:
Let us denote with e the identity element, \(g^{(-1)}\) the inverse element of \(g\in G\) with respect to \(\odot\), and \(\div\) the inverse operation defined by \(g_1 \div g_2= g_1 \odot g_2^{(-1)}\) for all \(g_1,g_2 \in G\).
Let \(n\in {\mathbb {N}} \cup \{0\}\) and \(g \in G\); then, the (n)-natural- power \(g^{(n)}\) of \(g\in G\) is:
By definition, an Alo-group \({\mathcal {G}}= (G, \odot , \le )\) is a lattice ordered group (Cavallo and D’Apuzzo, 2009), that is, for each \(g_1, g_2 \in G\), there exists a maximal element \(max\{g_1, g_2 \}\). Cavallo and D’Apuzzo (2009) propose the following notions of \({\mathcal {G}}\)-norm and \({\mathcal {G}}\)-distance
the above notions are generalizations to \({\mathcal {G}}\) of the usual concepts of norm and distance.
2.1 Real continuous Alo-groups
An Alo-group \({\mathcal {G}}= (G, \odot , \le )\) is called continuous if the operation \(\odot\) is continuous, and real if G is a subset of the real line \({\mathbb {R}}\) and \(\le\) is the weak order on G inherited from the usual order on \({\mathbb {R}}\). From now on, we will assume that \({\mathcal {G}}= (G, \odot , \le )\) is a real continuous Alo-group, with G being an open interval. Under these assumptions, the equation \(x^{(n)}=g\) has a unique solution; thus, it is reasonable to consider the notions of (n)-root and \({\mathcal {G}}\)-mean (Cavallo and D’Apuzzo, 2009).
For each \(n\in {\mathbb {N}}\) and \(g \in G\), the (n)-root of g, denoted by \(g^{(\frac{1}{n})}\), is the unique solution of the equation \(x^{(n)}=g\), that is:
and the \({\mathcal {G}}\)-mean \(m_{{\mathcal {G}}}( g_{1},g_{2},\ldots , g_{n})\) of the elements \(g_{1},g_{2},\ldots , g_{n}\) of G is expressed as follows:
Two real continuous Alo-groups \({\mathcal {G}}= (G, \odot , \le )\) and \({\mathcal {H}}= (H, \circ , \le )\), with G and H being open intervals, are isomorphic (Cavallo and D’Apuzzo, 2009); thus, there exists a bijection \(\phi :G\rightarrow H\) that is both a lattice isomorphism and group isomorphism, that is, for all \(g_1, g_2 \in G\), it holds:
Note that, the isomorphisms allows us to pass from an Alo-group to another isomorphic Alo-group; as an example, Cavallo and D’Apuzzo (2009) prove that:
with \(g_1, \ldots , g_n \in G\).
Examples of real continuous Alo-groups are the following ones:
-
Multiplicative Alo-group \({\mathcal {M}}\) = \(({\mathbb {R}}^+, \cdot , \le )\), where \(\cdot\) is the usual multiplication on \({\mathbb {R}}\), \(e=1\) and the \({\mathcal {M}}\)-mean of \(m_{1},m_2,\ldots ,m_{n} \in {\mathbb {R}}^+\) is the geometric mean:
$$\begin{aligned} m_{{\mathcal {M}}}\left( m_{1}, m_2,\ldots ,m_{n} \right) = \root n \of {\prod _{i=1}^{n}m_{i}}. \end{aligned}$$(9)\({\mathcal {M}}\)-distance between \(m_1\) and \(m_2\) is \(d_{{\mathcal {M}}}(m_1,m_2)= \max \{\frac{m_1}{m_2},\frac{m_2}{m_1} \}\).
-
Additive Alo-group \({\mathcal {A}} = ({\mathbb {R}}, +, \le )\), where \(+\) is the usual addition on \({\mathbb {R}}\), \(e=0\) and the \({\mathcal {A}}\)-mean of \(a_{1}, a_2,\ldots , a_{n} \in {\mathbb {R}}\) is the arithmetic mean:
$$\begin{aligned} m_{{\mathcal {A}}}( a_{1}, a_2,\ldots , a_{n})=\dfrac{\sum _{i=1}^{n}a_{i}}{n}. \end{aligned}$$(10)\({\mathcal {A}}\)-distance between \(a_1\) and \(a_2\) is \(d_{{\mathcal {A}}}(a_1,a_2)= \max \{a_1- a_2,a_2 - a_1\} = |a_1-a_2|\).
-
Fuzzy Alo-group \({\mathcal {F}}=(]0,1[, \otimes , \le )\), where \(\otimes : ]0,1[^{2}\rightarrow ]0,1[\) is the operation defined by
$$\begin{aligned} f_1\otimes f_2=\frac{f_1f_2}{f_1f_2+(1-f_1)(1-f_2)}, \end{aligned}$$(11)\(e=0.5\) and the \({\mathcal {F}}\)-mean of \(f_{1}, f_2,\ldots , f_{n} \in ]0,1[\) is the following one:
$$\begin{aligned} m_{{\mathcal {F}}}( f_{1}, f_2,\ldots , f_{n})=\frac{\root n \of {\prod _{i=1}^{n}f_{i}}}{\root n \of {\prod _{i=1}^{n}f_{i}} + \root n \of {\prod _{i=1}^{n}(1-f_{i})}}. \end{aligned}$$(12)\({\mathcal {F}}\)-distance between \(f_1\) and \(f_2\) is the following one:
$$\begin{aligned} d_{{\mathcal {F}}}(f_1,f_2)= max \left\{ \frac{f_1(1-f_2)}{f_1(1-f_2)+(1-f_1)f_2}, \frac{f_2(1-f_1)}{f_2(1-f_1)+(1-f_2)f_1} \right\} . \end{aligned}$$(13)
Isomorphisms between multiplicative and additive Alo-groups are the following logarithmic and exponential functions:
with a be a real number greater than 1.
Isomorphisms between multiplicative and fuzzy Alo-groups are the following ones:
For isomorphisms between further Alo-groups, the reader can refer to Cavallo and D’Apuzzo (2009) and Ramík (2015).
2.2 PCMs over an Alo-group
Let \({\mathcal {G}}= (G, \odot , \le )\) be a real continuous Alo-group, with G an open interval. A PCM over \({\mathcal {G}}= (G, \odot , \le )\) is defined as follows:
where \(g_{ij} \in G\) quantifies the preference intensity of \(x_{i}\) over \(x_{j}\) (Cavallo and D’Apuzzo, 2009). We assume that each PCM \({{\textbf {G}}}=[g_{ij}]_{n \times n}\) satisfies the following \({\mathcal {G}}\)-reciprocity property:
Therefore, a multiplicative PCM is a PCM over the multiplicative Alo-group \({\mathcal {M}} = ({\mathbb {R}}^+, \cdot , \le )\), an additive PCM is a PCM over the additive Alo-group \({\mathcal {A}} = ({\mathbb {R}}, +, \le )\), and a fuzzy PCM is a PCM over the fuzzy Alo-group \({\mathcal {F}}=(]0,1[, \otimes , \le )\).
Under the assumption of \({\mathcal {G}}\)-reciprocity, we set:
where \(x_{i} \succ _{{\textbf {G}}} x_{j}\) and \(x_{i} \sim _{{\textbf {G}}} x_{j}\) stand for “\(x_{i}\) is strictly preferred to \(x_{j}\)” and “\(x_{i}\) is indifferent to \(x_{j}\)”, respectively, and we denote:
where \(x_{i}\succsim _{{\textbf {G}}} x_{j}\) stands for “\(x_{i}\) is weakly preferred to \(x_{j}\)”.
The minimal level of coherence that a \({\mathcal {G}}\)-reciprocal PCM should satisfy is the \({\mathcal {G}}\)-transitivity. \({{\textbf {G}}}=[g_{ij}]_{n \times n}\) is \({\mathcal {G}}\)-transitive if verifies the following condition:
\({\mathcal {G}}\)-transitivity ensures a rearrangement \((i_{1},i_{2},\ldots , i_{n})\) of \(\{1, 2,\ldots , n\}\) such that:
2.2.1 Distance between PCMs
Let \({\textbf{G}}^1=[g^1_{ij}]_{n \times n}\) and \({\textbf{G}}^2=[g^2_{ij}]_{n \times n}\) be PCMs over \({\mathcal {G}}= (G, \odot , \le )\); then, Cavallo (2019) provides the following \({\mathcal {G}}\)-distance between \({\textbf{G}}^1\) and \({\textbf{G}}^2\):
The \({\mathcal {G}}\)-distance in (21) can be interpreted as a \({\mathcal {G}}\)-mean (5) of \({\mathcal {G}}\)-distances \(d_{{\mathcal {G}}}(g^1_{ij}, g^2_{ij})\), with \(i<j\), and it satisfies the following properties:
-
1.
\(d_{{\mathcal {G}}}({\textbf{G}}^1,{\textbf{G}}^2)\ge e;\)
-
2.
\(d_{{\mathcal {G}}}({\textbf{G}}^1,{\textbf{G}}^2)=e \Leftrightarrow {\textbf{G}}^1={\textbf{G}}^2\);
-
3.
\(d_{{\mathcal {G}}}({\textbf{G}}^1,{\textbf{G}}^2)= d_{{\mathcal {G}}}({\textbf{G}}^2,{\textbf{G}}^1)\);
-
4.
\(d_{{\mathcal {G}}}({\textbf{G}}^1,{\textbf{G}}^2) \le d_{{\mathcal {G}}}({\textbf{G}}^1,{\textbf{G}}^3) \odot d_{{\mathcal {G}}}({\textbf{G}}^3,{\textbf{G}}^2).\)
Example 1
Let us consider the following multiplicative PCMs:
Then, the distance between them is calculated as
where \(d_{{\mathcal {M}}}(m^1_{ij}, m^2_{ij})=\max \bigg \{\frac{m^1_{ij}}{m^2_{ij}}, \frac{m^2_{ij}}{m^1_{ij}} \bigg \}\).
Example 2
Let us consider the following additive PCMs:
Then, the distance between them is calculated as
where \(d_{{\mathcal {A}}}(a^1_{ij}, a^2_{ij})=|a^1_{ij}-a^2_{ij}|\). Note that, the distance can also be computed by applying the isomorphism in (14) as follows:
with \(h_a^{-1}({\textbf{A}}^1)= [h_a^{-1}(a_{ij}^1)]_{4 \times 4}\) and \(h_a^{-1}({\textbf{A}}^2)= [h_a^{-1}(a_{ij}^2)]_{4 \times 4}\) be multiplicative PCMs.
Example 3
Let us consider the following fuzzy PCMs:
Then, the following equality holds:
where \(d_{{\mathcal {F}}}(f^1_{ij},f^2_{ij})=\max \left\{ \frac{f^1_{ij}(1-f^2_{ij})}{f^1_{ij}(1-f^2_{ij})+(1-f^1_{ij})f^2_{ij}}; \frac{f^2_{ij}(1-f^1_{ij})}{f^2_{ij}(1-f^1_{ij})+(1-f^2_{ij})f^1_{ij}} \right\}\). Note that, the distance can also be computed by applying the isomorphism in (15) as follows:
with \(g^{-1}({\textbf{F}}^1)= [g^{-1}(f_{ij}^1)]_{4 \times 4}\) and \(g^{-1}({\textbf{F}}^2)= [g^{-1}(f_{ij}^2)]_{4 \times 4}\) be multiplicative PCMs.
Applications of the \({\mathcal {G}}\)-distance in (21) are provided by Brunelli and Cavallo (2020) for measuring incoherence of PCMs and by Cavallo and Ishizaka (2023) for computing matching errors in PCMs.
3 A comparative study
As introduced in Sect. 1, the study has been inspired by a validation study conducted by Saaty (2008). The goal is to compare the imprecision of multiplicative, additive and fuzzy approaches when the areas of four geometric figures have to be compared. It means that the geometric figures have to be pairwise compared by means of area ratios on the interval \(]0, \;+ \infty [\), area differences on the interval \(]- \infty , \;+ \infty [\) and relatives intensities of the areas on the interval \(]0, \;1[\). The study has been conducted by designing an experiment, where a set of respondents pairwise compared (by approximation) the areas of the geometric figures and then, the imprecision of their assessments was measured by computing the distance with the exact pairwise comparisons. We grouped the respondents in such a way that each participant was allowed to deal with a unique type of approach.
3.1 Geometric figures and exact PCMs for the experiment
In this section, we provide the geometric figures used in the experiment and the exact multiplicative, additive and fuzzy PCMs. Let us consider the circle, the rectangle, the triangle and the square in Figure 1, which areas are \(A_1=9.126\), \(A_2=2.553\), \(A_3=4.503\) and \(A_4=1.3\), respectively.
By pairwise comparing the areas, we obtain the following multiplicative and fuzzy PCMs:
\({\textbf{M}}=[m_{ij}]_{4 \times 4}\) and \({\textbf{F}}=[f_{ij}]_{4 \times 4}\) represent the area ratios on the interval \(]0, \;+ \infty [\) and the relative intensities of the areas on the interval \(]0, \;1[\), respectively. As an example, the fourth columns of the PCMs in (22) and (23) represent the pairwise comparisons between the area of each geometric figure in Fig. 1 and the area of the square.
Note that for all real positive numbers c, figures with areas equal to \(c \cdot A_1\), \(c \cdot A_2\), \(c \cdot A_3\) and \(c \cdot A_4\) generate the PCMs in (22) and (23).
The multiplicative PCM \({\textbf{M}}=[m_{ij}]_{4 \times 4}\) and the fuzzy PCM \({\textbf{F}}=[f_{ij}]_{4 \times 4}\) are isomorphic; indeed, for each \(i,j \in \{1, \ldots , 4\}\), we have:
where g is the isomorphism in (15).
Let a be a real number greater than 1. Then, the following additive PCM:
is isomorphic to both \({{\textbf {M}}}\) and \({{\textbf {F}}}\); indeed, for each \(i,j \in \{1, \ldots , 4\}\), we have:
and
where \(h_a\) and g are the isomorphisms in (14) and (15), respectively. Note that, the additive PCM with entries equal to \(A_i-A_j\) is isomorphic neither to \({{\textbf {M}}}\) nor to \({{\textbf {F}}}\).
Let us set \(a=A_4\); then, we obtain the following additive PCM:
\({\textbf{A}}=[a_{ij}]_{4 \times 4}\) represents the area differences on the interval \(]- \infty , \;+ \infty [\) of the geometric figures in Fig. 2, which areas are \(\log _{A_4} A_1=8.428\), \(\log _{A_4} A_2=3.573\), \(\log _{A_4} A_3=5.735\) and \(\log _{A_4} A_4=1\). As an example, the fourth column of the PCM in (27) represents the pairwise comparisons between the area of each geometric figure in Fig. 2 and the area of the square.
3.2 The methodology
We applied an opinion survey with a sample of 108 students of master courses in economics and management at the School of Business Administration in Karvina of the Silesian University, Czech Republic, and 120 students of the Department of Architecture of University of Naples Federico II, Italy. The students completed the survey during a class period and received instructions to fill it out. On average, the experiment lasted twenty minutes.
As done by Cavallo et al. (2019), in the design of the experiment, we created four surveys and we grouped the respondents by survey, in such a way that each participant was allowed to answer only one. Thus, each survey was filled out by \(N= 57\) participants (i.e., 27 Czech students and 30 Italian students). This “between-subject” designed experiment was more appropriate for our purpose than a “within-subject” designed experiment, where participants were exposed to more than one survey (i.e., all the four surveys). It has been long observed that within-subject cannot be used when independence of multiple exposure is not warrant; in fact, the respondents have a reference point when they are responding the second survey. The reader can refer to the literature mentioned by Cavallo et al. (2019).
The survey forms are the following ones:
- \(Q_M\):
-
The respondent has to perform 6 pairwise comparisons; that is, for each pair of figures in Fig. 1, (s)he has to select the bigger one and to establish the ratio between the biggest area and the smallest (i.e., a real number in \([1, +\infty [\)). As an example, if two areas are equal to 5 and 3 then the ratio is equal to \(\frac{5}{3}=1.667\);
- \(Q_A\):
-
The respondent has to perform 6 pairwise comparisons; that is, for each pair of figures in Figure 2, (s)he has to select the bigger one and to establish the difference between the biggest area and the smallest (i.e., a real number in \([0, +\infty [\)), by assuming that area of the square, which represents the measure unit, is equal to 1. As an example, if two areas are equal to 5 and 3 then the difference is equal to \(5-3=2\);
- \(Q_F\):
-
The respondent has to perform 6 pairwise comparisons; that is, for each pair of figures in Fig. 1, (s)he has to select the bigger one and to establish the relative intensity of the biggest area on the interval \([0.5, \;1 [\) in such a way that the sum of the intensities of the two areas is equal to 1. As an example, if two areas are equal to 5 and 3 then the biggest intensity is equal to \(\frac{5}{5+3}=0.625\) (the smallest is equal to 0.375);
- \(Q_S\):
-
The respondent has to perform 6 pairwise comparisons; that is, for each pair of figures in Fig. 1, (s)he has to select the bigger one and to establish how much the biggest figure is bigger than the other one by using a positive integer value in Table 1. As an example, if two areas are equal to 5 and 3 then the closest value in the Saaty’s scale to the ratio \(\frac{5}{3}\) is equal to 2.
Note that, although there is not an Alo-group defined on Saaty’s scale, since multiplicative PCMs over Saaty’s scale are used in several real world applications, they will be also considered in our experiment; thus, we assume that the exact pairwise comparisons are provided by the multiplicative PCM in (22) and we set \({\textbf{S}}=[s_{ij}]_{4 \times 4}={\textbf{M}}=[m_{ij}]_{4 \times 4}\).
The survey forms have been developed by means of Google Forms (https://www.google.it/intl/en/forms/about/) in such a way that each question is mandatory (each PCM is complete); moreover, we added constraints that allows us to obtain a suitable range for the entries of the PCMs. As an example, Fig. 3 provides a question in the multiplicative survey \(Q_M\).
In order to measure and compare the imprecision of the four types of PCMs (i.e., multiplicative, additive, fuzzy, Saaty), we perform the following steps:
- STEP 1.:
-
By the pairwise comparisons of the areas of circle, rectangle, triangle and square in Fig. 1 and in Fig. 2 performed by the students by filling out the assigned survey, we obtain the following four sets:
-
\(M= \{{{\textbf {M}}}^k=[m_{ij}^k]_{4 \times 4}\; |k \in \{1, \ldots , N\}\}\), with \({{\textbf {M}}}^k=[m_{ij}^k]_{4 \times 4}\) be a multiplicative PCM, approximation of \({{\textbf {M}}}\) in (22), obtained by the k-th student who filled out \(Q_M\);
-
\(A= \{{{\textbf {A}}}^k=[a_{ij}^k]_{4 \times 4}\; |k \in \{1, \ldots , N\}\}\), with \({{\textbf {A}}}^k=[a_{ij}^k]_{4 \times 4}\) be an additive PCM, approximation of \({{\textbf {A}}}\) in (27), obtained by the k-th student who filled out \(Q_A\);
-
\(F= \{{{\textbf {F}}}^k=[f_{ij}^k]_{4 \times 4}\; |k \in \{1, \ldots , N\}\}\), with \({{\textbf {F}}}^k=[f_{ij}^k]_{4 \times 4}\) be a fuzzy PCM, approximation of \({{\textbf {F}}}\) in (23), obtained by the k-th student who filled out \(Q_F\);
-
\(S= \{{{\textbf {S}}}^k=[s_{ij}^k]_{4 \times 4}\; |k \in \{1, \ldots , N\}\}\), with \({{\textbf {S}}}^k=[s_{ij}^k]_{4 \times 4}\) be a multiplicative PCM, approximation of \({{\textbf {M}}}\) in (22) with entries on the Saaty’s scale, obtained by the k-th student who filled out \(Q_S\).
-
- STEP 2.:
-
For each \(k\in \{1, \ldots , N\}\), we compute the following multiplicative PCMs:
$$\begin{aligned} h_{A_4}^{-1}({\textbf{A}}^k)= & {} [h_{A_4}^{-1}({a_{ij}^k})]_{4 \times 4};\\ g^{-1}( {\textbf{F}}^k)= & {} [g^{-1}(f_{ij}^k) ]_{4 \times 4}, \end{aligned}$$that are isomorphic to \({\textbf{A}}^k\) and \({\textbf{F}}^k\), respectively, and they are approximations of \({\textbf{M}}\) in (22).
- STEP 3.:
-
For each \(k\in \{1, \ldots , N\}\), by applying the distance (21) between two PCMs over the multiplicative Alo-group, we compute the following imprecisions of the approximations:
$$\begin{aligned}{} & {} d_{{\mathcal {M}}}({\textbf{M}},{\textbf{M}}^k)= \root \frac{n(n-1)}{2} \of { \prod _{i=1}^{n-1} \prod _{j=i+1}^n d_{{\mathcal {M}}}(m_{ij}, m^k_{ij})};\\{} & {} d_{{\mathcal {M}}}({\textbf{M}},h_{A_4}^{-1}({\textbf{A}}^k))= \root \frac{n(n-1)}{2} \of { \prod _{i=1}^{n-1} \prod _{j=i+1}^n d_{{\mathcal {M}}}\bigg (m_{ij}, h_{A_4}^{-1}({a_{ij}^k}) \bigg )};\\{} & {} d_{{\mathcal {M}}}({\textbf{M}}, g^{-1}( {\textbf{F}}^k))= \root \frac{n(n-1)}{2} \of { \prod _{i=1}^{n-1} \prod _{j=i+1}^n d_{{\mathcal {M}}}\bigg (m_{ij}, g^{-1}(f_{ij}^k) \bigg )};\\{} & {} d_{{\mathcal {M}}}({\textbf{M}},{\textbf{S}}^k)= \root \frac{n(n-1)}{2} \of { \prod _{i=1}^{n-1} \prod _{j=i+1}^n d_{{\mathcal {M}}}(m_{ij}, s^k_{ij})}. \end{aligned}$$ - STEP 4.:
-
For each approach, we compute the geometric mean of the imprecisions:
$$\begin{aligned}{} & {} m_{{\mathcal {M}}}\bigg (d_{{\mathcal {M}}}({\textbf{M}},{\textbf{M}}^1), \ldots , d_{{\mathcal {M}}}({\textbf{M}},{\textbf{M}}^N) \bigg ) = \root N \of {\prod _{k=1}^{N}d_{{\mathcal {M}}}({\textbf{M}},{\textbf{M}}^k)};\\{} & {} m_{{\mathcal {M}}}\bigg (d_{{\mathcal {M}}}\bigg ({\textbf{M}},h_{A_4}^{-1} ({\textbf{A}}^1)\bigg ), \ldots , d_{{\mathcal {M}}}\bigg ({\textbf{M}}, h_{A_4}^{-1}({\textbf{A}}^N)\bigg )\bigg ) = \root N \of {\prod _{k=1}^{N} d_{{\mathcal {M}}}\bigg ({\textbf{M}},h_{A_4}^{-1}({\textbf{A}}^k)\bigg )};\\{} & {} m_{{\mathcal {M}}}\bigg (d_{{\mathcal {M}}}\bigg ({\textbf{M}}, g^{-1} ( {\textbf{F}}^1)\bigg ), \ldots , d_{{\mathcal {M}}}\bigg ({\textbf{M}}, g^{-1}( {\textbf{F}}^N)\bigg ) \bigg ) = \root N \of {\prod _{k=1}^{N}d_{{\mathcal {M}}} \bigg ({\textbf{M}}, g^{-1}( {\textbf{F}}^k)\bigg )};\\{} & {} m_{{\mathcal {M}}}\left( d_{{\mathcal {M}}}({\textbf{M}},{\textbf{S}}^1), \ldots , d_{{\mathcal {M}}}({\textbf{M}},{\textbf{S}}^N) \right) = \root N \of {\prod _{k=1}^{N}d_{{\mathcal {M}}}({\textbf{M}},{\textbf{S}}^k)}. \end{aligned}$$They represent the averages (i.e., geometric means) of the imprecisions, computed on the multiplicative Alo-group, obtained by the N surveys \(Q_M\), the N surveys \(Q_A\), the N surveys \(Q_F\) and the N surveys \(Q_S\), respectively.
Remark 1
The choices done in STEP 2, STEP 3 and STEP 4 to choose the multiplicative Alo-group for computing the approximations, the imprecisions and the averages of the imprecisions, do not affect the results. Indeed, if, instead to compare the geometric means of the imprecisions in STEP 4, we compare the isomorphic arithmetic means on the additive Alo-group or the isomorphic fuzzy means on the fuzzy Alo-group, then, we obtain equivalent results, because by properties of \({\mathcal {G}}\)-distance and \({\mathcal {G}}\)-mean in (7) and (8), by relationships among \({{\textbf {M}}}\), \({{\textbf {A}}}\) and \({{\textbf {F}}}\) in (24), (25) and (26), and isomorphisms in (14) and (15), we have that:
In particular, since the isomorphisms \(h_{A_4}\) and g are strictly increasing functions, the order is preserved by passing from multiplicative approach to arithmetic or fuzzy one. Thus, the choice of the approach, among multiplicative, additive and fuzzy, to compare the imprecisions could be a measurement option in our methodology; indeed, for each choice of approach, we reach the same result on the imprecision. However, we stress that our choice (i.e., multiplicative approach) is the most efficient because, in STEP 1, both the PCMs \(M_k\) and the PCMs \(S_k\) are multiplicative PCMs; thus, the multiplicative choice allows us to compute the smallest number of isomorphic PCMs in STEP 2.
The following example shows how the above steps are performed in the experimentation.
Example 4
Let us suppose that eight students fill out the questionnaires \(Q_M\), \(Q_A\), \(Q_F\) and \(Q_S\).
- STEP 1.:
-
By the questionnaires, we obtain the following multiplicative, additive, fuzzy and Saaty PCMs:
$$\begin{aligned} \mathbf {M^1}= & {} \left[ \begin{array}{cccc} 1&{} 4 &{} 1.9 &{} 6.5\\ 0.25 &{} 1 &{} 0.5 &{} 2\\ 0.526 &{} 2 &{} 1 &{} 3\\ 0.154 &{} 0.5 &{} 0.333 &{} 1 \end{array}\right] , \quad \mathbf {M^2}=\left[ \begin{array}{cccc} 1&{} 3.8 &{} 2 &{} 6\\ 0.263 &{} 1 &{} 0.333 &{} 1.5\\ 0.5 &{} 3 &{} 1 &{} 3.2\\ 0.167 &{} 0.667 &{} 0.313 &{} 1 \end{array}\right] ;\\ {\textbf{A}}^1= & {} \left[ \begin{array}{cccc} 0 &{} 4 &{} 3.5 &{} 8.3\\ -4 &{} 0&{} -2.5 &{} 2\\ -3.5 &{} 2.5 &{} 0 &{} 6\\ -8.3&{} -2 &{} -6 &{} 0 \end{array}\right] , \quad {\textbf{A}}^2=\left[ \begin{array}{cccc} 0 &{} 3.2 &{} 1 &{} 7\\ -3.2 &{} 0&{} -3&{} 3.5\\ -1 &{} 3 &{} 0 &{} 4\\ -7 &{} -3.5 &{} -4 &{} 0 \end{array}\right] ;\\ {\textbf{F}}^1= & {} \left[ \begin{array}{cccc} 0.5 &{} 0.8 &{} 0.6 &{} 0.9\\ 0.2 &{} 0.5 &{} 0.4 &{} 0.6\\ 0.4 &{} 0.6 &{} 0.5 &{} 0.8\\ 0.1 &{} 0.4 &{} 0.2&{} 0.5 \end{array}\right] , \quad {\textbf{F}}^2=\left[ \begin{array}{cccc} 0.5 &{} 0.75 &{} 0.6 &{} 0.85\\ 0.25 &{} 0.5 &{} 0.35 &{} 0.65\\ 0.4 &{} 0.65 &{} 0.5 &{} 0.7\\ 0.15 &{} 0.35 &{} 0.3&{} 0.5 \end{array}\right] ;\\ \mathbf {S^1}= & {} \left[ \begin{array}{cccc} 1&{} 3 &{} 2 &{} 7\\ \frac{1}{3} &{} 1 &{} \frac{1}{3} &{} 2\\ \frac{1}{2} &{} 3 &{} 1 &{} 3\\ \frac{1}{7} &{} \frac{1}{2} &{} \frac{1}{3} &{} 1 \end{array}\right] , \quad \mathbf {S^2}=\left[ \begin{array}{cccc} 1&{} 5 &{} 2 &{} 6\\ \frac{1}{5} &{} 1 &{} \frac{1}{3} &{}3\\ \frac{1}{2} &{} 3 &{} 1 &{} 4\\ \frac{1}{6} &{} \frac{1}{3} &{} \frac{1}{4} &{} 1 \end{array}\right] . \end{aligned}$$ - STEP 2.:
-
We compute the following multiplicative PCMs:
$$\begin{aligned} h_{A_4}^{-1}({\textbf{A}}^1)= & {} \left[ \begin{array}{cccc} 1 &{} 2.856 &{}2.505 &{} 8.825\\ 0.350 &{} 1&{} 0.519 &{} 1.690\\ 0.399 &{}1.927 &{} 1 &{} 4.827\\ 0.113&{} 0.592&{} 0.207&{} 1 \end{array}\right] , \quad h_{A_4}^{-1}({\textbf{A}}^2)= \left[ \begin{array}{cccc} 1 &{} 2.315 &{} 1.3 &{} 6.275\\ 0.432 &{} 1&{} 0.455 &{} 2.505\\ 0.769 &{}2.197 &{} 1 &{} 2.856\\ 0.159&{} 0.399 &{} 0.350 &{} 1 \end{array}\right] ;\\ g^{-1}( {\textbf{F}}^1)= & {} \left[ \begin{array}{cccc} 1 &{} 4 &{}1.5 &{} 9\\ 0.25 &{} 1&{} 0.667 &{} 1.5\\ 0.667&{}1.5 &{} 1 &{} 4\\ 0.111&{} 0.667&{} 0.25&{} 1 \end{array}\right] , \quad g^{-1}( {\textbf{F}}^2)=\left[ \begin{array}{cccc} 1 &{} 3 &{}1.5 &{} 5.667\\ 0.333 &{} 1&{} 0.538 &{} 1.857\\ 0.667&{}1.857 &{} 1 &{} 2.333\\ 0.176&{} 0.538&{} 0.429&{} 1 \end{array}\right] . \end{aligned}$$ - STEP 3.:
-
We compute the following multiplicative imprecisions:
$$\begin{aligned}{} & {} d_{{\mathcal {M}}}({\textbf{M}},{\textbf{M}}^1)= 1.094, \quad d_{{\mathcal {M}}}({\textbf{M}},{\textbf{M}}^2)= 1.203;\\{} & {} d_{{\mathcal {M}}}({\textbf{M}},h_{A_4}^{-1}({\textbf{A}}^1))=1.229, \quad d_{{\mathcal {M}}}({\textbf{M}},h_{A_4}^{-1}({\textbf{A}}^2))=1.316;\\{} & {} d_{{\mathcal {M}}}({\textbf{M}}, g^{-1}( {\textbf{F}}^1))=1.229, \quad d_{{\mathcal {M}}}({\textbf{M}}, g^{-1}( {\textbf{F}}^2))=1.220;\\{} & {} d_{{\mathcal {M}}}({\textbf{M}},{\textbf{S}}^1)=1.159, \quad d_{{\mathcal {M}}}({\textbf{M}},{\textbf{S}}^2)=1.307. \end{aligned}$$ - STEP 4.:
-
We compute the following geometric means \(m_{{\mathcal {M}}}\) of the multiplicative imprecisions \(d_{{\mathcal {M}}}\):
$$\begin{aligned}{} & {} m_{{\mathcal {M}}}(1.094, \; 1.203 )=1.148;\\{} & {} m_{{\mathcal {M}}}(1.229, \; 1.316 )=1.271;\\{} & {} m_{{\mathcal {M}}}(1.229, \; 1.220 )=1.224;\\{} & {} m_{{\mathcal {M}}}(1.159, \; 1.307 )=1.230; \end{aligned}$$obtained by surveys \(Q_M\), \(Q_A\), \(Q_F\) and \(Q_S\), respectively.
By \(1.148<1.224<1.230<1.271\), we have that the multiplicative PCMs are on average more precise than fuzzy PCMs that are on average more precise than Saaty PCMs that are on average more precise than additive PCMs.
As stressed in Remark 1, we would have obtained the same order if we had computed the isomorphic additive imprecisions \(d_{{\mathcal {A}}}\) or fuzzy imprecisions \(d_{{\mathcal {F}}}\) and then, compared the isomorphic arithmetic means \(m_{{\mathcal {A}}}\) or the fuzzy means \(m_{{\mathcal {F}}}\).
3.3 Results and discussion
In each survey, by selecting the bigger figure for each pair of figures (e.g., see Fig. 3), we gave students the possibility to provide an ordinal ranking of the areas of the figures even when wrong areas (obtained by wrong entries of the PCMs) were provided. Thus, we discarded about \(10\%\) of the surveys, i.e., surveys where the students were not transitive or provided an incorrect ranking of the figures (see (19) and (20)); indeed, the selection of the wrong figure shows incoherence or an error that does not depend by the method (multiplicative, additive, fuzzy or Saaty). The averages of the imprecisions obtained from the surveys \(Q_M\), \(Q_A\), \(Q_F\) and \(Q_S\), by performing STEP 4 of the our methodology, are shown in Table 2.
Table 2 shows that the multiplicative approach is always the most precise because the average of the imprecisions obtained from \(Q_M\) is the smallest for both Italy and Czechia. Note that, for Italian respondents, the biggest average of the imprecisions is provided by Saaty’s scale and, for Czech respondents, it is provided by the fuzzy approach. Of course, as stressed in Remark 1, by applying the isomorphisms in (14), with \(a=1.3\), and (15), we obtain the same rankings of the averages of the imprecisions (see Table 3 and Table 4).
To analyze if the differences among the means are significant, we performed ANalysis Of VAriance (ANOVA) tests at a significance level \(\alpha =0.05\). A significance level of 0.05 indicates a \(5 \%\) risk of concluding that a difference exists when there is no actual difference. An ANOVA test (King, 2010) is a way to find out if survey or experiment results are significant. It is a type of statistical test used to determine if there is a statistically significant difference between categorical groups by testing for differences of means using variance. We recall that if ANOVA test provides a p-value less than the significance level then the null hypothesis is rejected, and we can conclude that not all of the group means are equal. The F value is used along with the p-value in deciding whether the results are significant enough to reject the null hypothesis. If ANOVA test provides a large F value (i.e. one that is bigger than the critical value \(F_{crit}\)), it means the difference is significant. Statistical analyses were conducted using StatPlus (https://www.analystsoft.com/en/).
ANOVA results are shown in Table 5; thus, we have that:
-
for Italy, the null hypothesis is rejected and there are highly significant differences among all the means provided by \(Q_M\), \(Q_A\), \(Q_F\) and \(Q_S\) because p-value is equal to 0 and F value is higher than the critical value \(F_{crit}\);
-
for Czechia, there are not significant differences among the means provided by \(Q_M\), \(Q_A\), \(Q_F\) and \(Q_S\) because p-value is greater than the significance level \(\alpha\) and F value is smaller than the critical value \(F_{crit}\).
Since, for Italian respondents, the null hypothesis is rejected, we performed all pairwise comparisons in order to determine which differences are statistically significant. The results are shown in Table 6. In particular, we stress that the difference between the mean provided by \(Q_M\) and each other mean is significant.
Thus, we can conclude that statistically, while for Czech respondents all the approaches provide the same imprecision, for Italian respondents the multiplicative approach provides the least imprecision. A possible reason for the different results obtained for the two groups of respondents may be found in different background knowledge and former experience concerning pairwise comparisons method. Indeed, the students from the Silesian University had some former experience with AHP, and so with Saaty’s scale, therefore, they could be more experienced when using pairwise comparisons. Saaty’s approach should to be avoided for measuring objective data both whenever the additional notion of ratio is not given and whenever the exact values of the ratios are not close to the values in the Saaty’s scale.
4 Conclusions and future work
To best of our knowledge, this paper represents the first comparative study conducted in order to establish for which type of PCM (i.e., multiplicative, additive, fuzzy), DMs are more precise when they compare objective data. By following the idea proposed by Cavallo et al. (2019), where the authors compare the coherence of multiplicative, additive and fuzzy approaches for subjective preferences, and inspired by a validation study conducted by Saaty (1980), where the author shows the effectiveness of the multiplicative PCMs for the prioritization of geometric figures, we performed an experiment in order to compare the precision of these types of PCMs when areas of geometric figures have to be pairwise measured.
The experiment involves students from Silesian University and University of Naples Federico II, and it shows that the multiplicative approach is the most precise; the difference is statistically significant for Italian respondents.
It should be noted that in an experiment setting the order of comparisons and the size of the problem cannot be neglected, see (Bozóki et al., 2013), hence our results cannot be simply generalized and further research is needed.
We stress that our experimental results are valid in comparisons of areas of several geometric figures but no experiment has been performed for establishing which type of comparison (i.e. multiplicative, additive, fuzzy or linguistic) is the most precise in comparison of further objective data, e.g. sizes of objects and weights of objects. The application of a specific approach can also depend on the DM’s problem, his/her personal preferences and views; thus, our future work will be the application of our comparative methodology to further real decision making problems concerning the measure of objective data, e.g., weights of objects and electricity consumption of household appliances (Whitaker, 2007), proposed in validation examples of the AHP. Finally, in usual decision making problems, the comparisons between alternatives is often different from the comparisons of areas; indeed, in these problems there are not exact values to approximate because the decision maker has to express subjective preferences between alternatives; indeed, experimental results obtained by Cavallo et al. (2019) show that the additive comparisons provide the worst results when decision maker expresses subjective preferences.
Data availability
The data that support the findings of this study are available from the corresponding authors upon request.
Code availability
Our research did not involve any code.
References
Barzilai, J. (1997). Deriving weights from pairwise comparison matrices. The Journal of the Operational Research Society, 48(12), 1226–1232.
Barzilai, J., & Golany, B. (1990). Deriving weights from pairwise comparison matrices: The additive case. Operations Research Letters, 9(6), 407–410.
Barzilai, J., Cook, W., & Golany, B. (1987). Consistent weights for judgements matrices of the relative importance of alternatives. Operations Research Letters, 6(3), 131–134.
Bozóki, S., Dezsö, L., Poesz, A., et al. (2013). Analysis of pairwise comparison matrices: an empirical research. Annals of Operations Research, 211(1), 511–528.
Brunelli, M., & Cavallo, B. (2020). Distance-based measures of incoherence for pairwise comparisons. Knowledge-Based Systems, 187, 104808.
Cavallo, B. (2019). G-distance and G-decomposition for improving G-consistency of a pairwise comparison matrix. Fuzzy Optimization and Decision Making, 18(1), 57–83.
Cavallo, B., & D’Apuzzo, L. (2009). A general unified framework for pairwise comparison matrices in multicriterial methods. International Journal of Intelligent Systems, 24(4), 377–398.
Cavallo, B., & Ishizaka, A. (2023). Evaluating scales for pairwise comparisons. Annals of Operations Research, 325, 951–965.
Cavallo, B., Ishizaka, A., Olivieri, M. G., et al. (2019). Comparing inconsistency of pairwise comparison matrices depending on entries. Journal of the Operational Research Society, 70(5), 842–850.
Hou, F. (2016). A multiplicative alo-group based hierarchical decision model and application. Communications in Statistics - Simulation and Computation, 45(8), 2846–2862.
King, B. (2010). Analysis of variance. In P. Peterson, E. Baker, & B. McGaw (Eds.), International Encyclopedia of Education (Third Edition), third (edition, pp. 32–36). Oxford: Elsevier.
Koczkodaj, W. W., Szybowski, J., & Wajch, E. (2016). Inconsistency indicator maps on groups for pairwise comparisons. International Journal of Approximate Reasoning, 69, 81–90.
Llull R (1274-1283) Artifitium electionis personarum (The method for the elections of persons). https://www.math.uni-augsburg.de/htdocs/emeriti/pukelsheim/llull/
Ramík, J. (2015). Isomorphisms between fuzzy pairwise comparison matrices. Fuzzy Optimization and Decision Making, 14(2), 199–209.
Ramík, J. (2020). Pairwise Comparisons Method: Theory and Applications in Decision Making, vol Lecture Notes in Economics and Mathematical Systems (Vol. 690). Cham: Springer International Publishing.
Saaty, T. L. (1977). A scaling method for priorities in hierarchical structures. Journal of Mathematical Psychology, 15(3), 234–281.
Saaty, T. L. (1980). Analytic Hierarchy Process. New York: McGraw-Hill.
Saaty, T. L. (2008). Relative measurement and its generalization in decision making why pairwise comparisons are central in mathematics for the measurement of intangible factors the analytic hierarchy/network process. RACSAM - Revista de la Real Academia de Ciencias Exactas, Fisicas y Naturales Serie A Matematicas, 102(2), 251–318.
Tanino, T. (1988). Fuzzy Preference Relations in Group Decision Making (pp. 54–71). Berlin Heidelberg: Springer.
Thurstone, L. L. (1927). A law of comparative judgments. Psychological Reviews, 34(4), 273–286.
Whitaker, R. (2007). Validation examples of the Analytic Hierarchy Process and Analytic Network Process. Mathematical and Computer Modelling, 46(7), 840–859.
Xia, M., & Chen, J. (2015). Consistency and consensus improving methods for pairwise comparison matrices based on Abelian linearly ordered group. Fuzzy Sets and Systems, 266, 1–32.
Funding
Open access funding provided by Università degli Studi di Napoli Federico II within the CRUI-CARE Agreement.
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Conflict of interest
We have no conflict of interest to disclose.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Cavallo, B., Mazurek, J. & Ramík, J. A comparative study on precision of pairwise comparison matrices. Fuzzy Optim Decis Making 23, 179–198 (2024). https://doi.org/10.1007/s10700-023-09416-4
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10700-023-09416-4