On the Validity of the “Importance Minus Performance” Construct—A Genuine Contribution of the Tourism Literature or a Mishap?

This art icle was downloaded by: [ Nam hyun Kim ] On: 14 August 2012, At : 14: 44 Publisher: Rout ledge I nform a Lt d Regist ered in England and Wales Regist ered Num ber: 1072954 Regist ered office: Mort im er House, 37- 41 Mort im er St reet , London W1T 3JH, UK Journal of Travel & Tourism Marketing Publicat ion det ails, including inst ruct ions for aut hors and subscript ion informat ion: ht t p:/ / www.t andfonline.com/ loi/ wt t m20 On the Validity of the “Importance Minus Performance” Construct—A Genuine Contribution of the Tourism Literature or a Mishap? a Namhyun Kim , SangSoo Choi b & Zvi Schwart z c a Depart ment of Recreat ion Sport and Tourism, Universit y of Illinois, Champaign, IL, 61820, USA b Depart ment of Tourism and Hospit alit y, Semyung Universit y in Jecheon Cit y, Chungbuk, Sout h Korea c Depart ment of Hospit alit y and Tourism Management of t he Pamplin College of Business, Virginia Tech in Blacksburg, VA, USA Version of record first published: 08 Aug 2012 To cite this article: Namhyun Kim, SangSoo Choi & Zvi Schwart z (2012): On t he Validit y of t he “ Import ance Minus Performance” Const ruct —A Genuine Cont ribut ion of t he Tourism Lit erat ure or a Mishap?, Journal of Travel & Tourism Market ing, 29:6, 599-610 To link to this article: ht t p:/ / dx.doi.org/ 10.1080/ 10548408.2012.703039 PLEASE SCROLL DOWN FOR ARTI CLE Full t erm s and condit ions of use: ht t p: / / www.t andfonline.com / page/ t erm s- and- condit ions This art icle m ay be used for research, t eaching, and privat e st udy purposes. Any subst ant ial or syst em at ic reproduct ion, redist ribut ion, reselling, loan, sub- licensing, syst em at ic supply, or dist ribut ion in any form t o anyone is expressly forbidden. The publisher does not give any warrant y express or im plied or m ake any represent at ion t hat t he cont ent s will be com plet e or accurat e or up t o dat e. The accuracy of any inst ruct ions, form ulae, and drug doses should be independent ly verified wit h prim ary sources. The publisher shall not be liable for any loss, act ions, claim s, proceedings, dem and, or cost s or dam ages what soever or howsoever caused arising direct ly or indirect ly in connect ion wit h or arising out of t he use of t his m at erial. Journal of Travel & Tourism Marketing, 29:599–610, 2012 Copyright © Taylor & Francis Group, LLC ISSN: 1054-8408 print / 1540-7306 online DOI: 10.1080/10548408.2012.703039 ON THE VALIDITY OF THE “IMPORTANCE MINUS PERFORMANCE” CONSTRUCT—A GENUINE CONTRIBUTION OF THE TOURISM LITERATURE OR A MISHAP? Downloaded by [Namhyun Kim] at 14:44 14 August 2012 Namhyun Kim SangSoo Choi Zvi Schwartz ABSTRACT. Measuring service quality in a reliable and valid manner is crucial. Accordingly, this study explores whether the tourism-oriented Importance-Performance construct is a theoretically sound measure of service quality. The conceptual analysis is followed by an empirical test of the construct’s prediction capability within the realm of service quality/satisfaction framework in the tourism context of a large festival. Both the conceptual and empirical results clearly indicate that the validity of the Importance-Performance construct should be strongly doubted, and that tourism and hospitality managers as well as researchers would be better off avoiding the use of the Importance-Performance construct. KEYWORDS. Service quality measure, satisfaction, importance-performance analysis, validity INTRODUCTION Customer satisfaction and service quality are fundamental components of any successful marketing strategy, and assessing customers’ level of satisfaction in a reliable manner is crucial to firms who strive to establish a competitive advantage in the market place. Two instruments—SERVQUAL (1988) and the Importance-Performance Analysis (IPA; 1977)—gained wide popularity in connection with these two concepts of service quality and consumer satisfaction. Despite this popularity, the theoretical and methodological validity of the measurement of perceived quality using SERVQUAL has been widely criticized (e.g., Carman, 1990; Cronin & Taylor, 1992; Smith, 1995; Teas, 1993) and its usefulness is still being debated. Not surprisingly, given the importance of these two concepts, the tourism and hospitality literature reflects several attempts to explore the validity of quality and satisfaction Namhyun Kim is a PhD candidate in the Department of Recreation, Sport and Tourism at the University of Illinois, 104 Huff Hall, 1206 South Fourth Street, Champaign, IL 61820, USA (E-mail: nkim34@ illinois.edu). SangSoo Choi, PhD, is Assistant Professor in the Department of Tourism and Hospitality at Semyung University in Jecheon City, Chungbuk, South Korea (E-mail: tourdoc@semyung.ac.kr). Zvi Schwartz, PhD, is Associate Professor in the Department of Hospitality and Tourism Management of the Pamplin College of Business at Virginia Tech in Blacksburg, VA, USA (E-mail: zvi@vt.edu). Address correspondence to: Namhyun Kim at the above address. 599 Downloaded by [Namhyun Kim] at 14:44 14 August 2012 600 JOURNAL OF TRAVEL & TOURISM MARKETING measurements (e.g., Crompton & Love, 1995; Hudson, Hudson, & Miller, 2004; Yuksel & Rimmington, 1998). Interestingly, the tourism and hospitality literature seems to have a unique contribution in this area. While the general marketing literature explored mainly Importance and/or Performance and/or the product (multiplication) of these two constructs (Importance × Performance), a new derived construct was suggested and explored in the tourism literature. This unique, derived construct is Importance minus Performance. It is the difference between Importance and Performance, and was used in several tourism studies as an alternative measure to service quality (e.g., Crompton & Love, 1995; Hudson et al., 2004). Unfortunately, none of these tourism publications outlined the theoretical foundation, nor justified the use of this derived Importance minus Performance construct. The goal of the current study is twofold: First, we analyze this derived construct from a conceptual perspective in an attempt to establish whether its use is indeed justified and theoretically grounded. Second, using data collected in a field study, we empirically test the validity of this Importance-Performance construct. If this new construct is a valid measure of service quality, it could be used as an adequate indicator of customer satisfaction in a service quality/satisfaction framework. Accordingly, this study’s main contribution is answering the question of whether the tourism-oriented Importance-Performance construct is a theoretically sound measure of service quality, and empirically testing this Importance-Performance construct’s prediction capability within the realm of the service quality/satisfaction framework in a tourism context. LITERATURE REVIEW AND THEORETICAL BACKGROUND Measuring Service Quality Quality is considered a key marketing factor within the service industry because it is a crucial product aspect used to differentiate a service provider from its competitors. Two main instruments have been developed and extensively used to measure service quality and satisfaction. One service quality measuring instrument was proposed by Parasuraman, Zeithaml, and Berry (1988) who argued that perceived quality of service is best represented by the gap between perceived service levels and consumers’ expectations. The authors defined perception as consumers’ beliefs concerning the service received or experienced, and expectations as the desires or wants of consumers (that is, what providers should offer in the eyes of customers). The service quality construct (called SERVQUAL) consisted of 22 items designed to load on five dimensions: tangibles, reliability, responsiveness, assurance, and empathy. SERVQUAL has been widely used in a variety of fields including education (e.g., Hussain & Birol, 2011), IT service (e.g., Jia, Reich, & Pearson, 2008), public health (e.g., John, Yatim, & Mani, 2011), sport marketing (e.g., Tsuji, Bennett, & Zhang, 2007), and hospitality and tourism (e.g., Crompton & MacKay, 1989; Ostrowski, O’Brien, & Gordon, 1993; Urdang & Howey, 2001; Wright, Duray, & Goodale, 1992). The second main research instrument to measure service quality and customer satisfaction is the Importance-Performance Analysis (IPA) framework (Hudson et al., 2004). IPA is a technique that shows the relative importance of certain product attributes and the performance or level of these attributes in a twodimensional grid. Developed by Martilla and James (1977), IPA has been used extensively in strategic marketing, as it proved most useful in making marketing resource allocation decisions. For example, IPA is of great benefit in addressing decisions regarding whether a firm should continue focusing on certain attributes of a marketing program or should redirect its resources to another. While originally designed for evaluating “consumer acceptance of a marketing program” (Martilla & James, 1977) as discussed above, the IPA framework was adopted in other areas such as banking services (Ennew, Reed, & Binks, 1993), the hotel industry (Chu & Choi, 2000), and festival and event settings (Smith & Kim, Choi, and Schwartz Costello, 2009) and is now widely used as a tool to measure service quality and satisfaction. Downloaded by [Namhyun Kim] at 14:44 14 August 2012 The Validity of Service Quality Measurements SERVQUAL has been subjected to some criticisms in terms of the operationalization and conceptualization of perceived quality, the measurement of quality, and the dimensionality of service quality. Some studies show that theoretical and methodological problems exist with regard to the concept of the expectation construct (e.g., Carman, 1990; Cronin & Taylor, 1992; Ennew et al., 1993; Smith, 1995; Teas, 1993). For example, the validity of the expectation measure was questioned when consumers evaluate services with which they are inexperienced. Carman (1990) argued that expectation constructs in SERVQUAL are useful “in situations where norms for expectations are wellformulated in the respondent’s mind from past experience with similar services” (p. 48) and established the relevance of the importance of a particular service attribute based on attitude theory. The author suggested that the original SERVQUAL model be modified to include the importance construct as a multiplier in the function of service quality, the function that refers to differences between perceptions and expectations. Numerous additional studies focused on the validity of the measurement of SERVQUAL, exploring, for example, whether a discrepancy measurement is superior to a performance-only measurement in predicting satisfaction and behavior (e.g., Brady, Cronin, & Brand, 2002; Cronin & Taylor, 1992; Smith, 1995; Teas, 1993). In this context, Cronin and Taylor (1992) compared four forms of service quality measurement: SERVQUAL, importance∗ (performanceexpectation), performance (SERVPERF), and importance∗ performance. They found that the SERVPERF scale explained more of the variation in service quality than SERVQUAL in testing their structural models. The authors concluded that a performance-based measure (SERVPERF) of quality is an improved means of measuring the service quality construct. 601 The use of the performance-only construct in recent tourism and hospitality studies seems to indicate that its value is gaining recognition (e.g., Kim & Lee, 2004; Lee, Graefe, & Burns, 2004; Tkaczynski & Stokes 2010; Tsang, Lai, & Law, 2010). The debate on measuring service quality and satisfaction as outlined above was reflected in some ways in the discussion on the IPA framework. Since many studies suggested that when measuring service quality the performance-only measurement is more valid (Cronin & Taylor, 1992; Lee et al., 2004; Tkaczynski & Stokes, 2010; Tsang et al., 2010), researchers wondered if within the IPA framework both importance and performance attributes should be used to better describe customer satisfaction. In addition, questions were asked about timing the measurements of two sets of attributes; i.e., when the data should be collected (e.g., Deng, 2007; Matzler, Sauerwein, & Heischmidt, 2003; Mount, 1997; Oh, 2001). Recently, Matzler et al. (2003) and Deng (2007) suggested a revision of the IPA platform, according to which the survey only asks about the attributes’ performance, not their importance. The authors argued that by eliminating the importance elements of the questionnaire, the revised and shortened IPA survey could overcome inefficiency in measuring importance attributes before purchasing services, and increase consumers’ response rates. With regard to assessing the importance element, the authors suggested that importance scores of attributes could be implicitly derived from the consumers’ answers to performance questions. Specifically, they argued that this could be done through partial correlation analysis between attributes’ performance and measurement of overall performance satisfaction. It was concluded that the proposed modified IPA methodology is more effective and can better support managers in assessing service quality and customer satisfaction. Importance Minus Performance Construct Given the concerns regarding the validity of various service quality measurements as outlined in the previous section, it is not Downloaded by [Namhyun Kim] at 14:44 14 August 2012 602 JOURNAL OF TRAVEL & TOURISM MARKETING surprising that efforts similar to Cronin and Taylor’s (1992) work appeared in the tourism and hospitality literature. Crompton and Love (1995) empirically tested the validity of seven quality evaluation measures: expectations, importance minus performance, importance times expectations, importance times performance, performance minus expectations (SERVQUAL), importance times (performance minus expectations), and performance. They hypothesized that the performance-only measurement would be a better predictor of quality than other evaluation measuring tools, and that importance weights do not improve the predictive validity of a quality. Interestingly, as noted earlier, one of the seven tested alternatives was a new measure: Importance minus Performance. This same new construct of Importance – Performance, as a quality measure, appeared in later tourism studies such as Hudson et al. (2004) and Yuksel and Rimmington (1998). Similar to the Crompton and Love (1995) approach, Yuksel and Rimmington (1998) compared the reliability and validity of six quality-constructs, adding direct confirmation/ disconfirmation, a measure that is subjectively evaluated by customers. Their six measures included: performance-only, performance weighted by importance, importance minus performance, direct confirmation/disconfirmation, confirmation/disconfirmation weighted by importance, and performance minus expectations (SERVQUAL). Both studies found that the performance-only construct was the most reliable and valid measure, a finding consistent with Cronin and Taylor (1992). Hudson et al. (2004) conducted a similar study in the travel industry and employed four constructs: SERVQUAL (Performance– Expectation), IPA (Performance–Importance), SERVQUAL∗ IMPORTANCE ([P-E]∗ I), and SERVPERF (Performance-Only). Using Friedman’s two-way ANOVA, they compared four service quality measurements of each service quality dimension and found no significant statistical difference among any of them. However, there were differences in the rankings of the 13 quality dimensions across four different measurements. Note that the Hudson et al. version of the new construct is in reverse order; that is, Performance–Importance rather than the original Importance–Performance. Whether the new service quality construct first suggested by Crompton and Love (1995) is valid and whether it should be used to predict customer satisfaction and behavior is yet to be determined. While tourism studies that have adopted this Importance–Performance construct seem to have been aware of the relevance of “Importance” in service quality, none discussed the theoretical foundation of this new modified construct nor provided the rationale for using it. This is especially surprising given that the traditional role of “Importance” in the service quality and satisfaction function is of a weight, since “Importance” is a multiplier as in (P-E)∗ I. As noted above, this role of a multiplier is attributed to Carman (1990) who was first to argue that although the SERVQUAL may not be useful in measuring the quality of newto-the-consumer services or products, it can be advanced by multiplying the attribute level by its importance. Accordingly, Carman further argued that marketers should collect data on all three constructs: importance, expectations, and perceptions. It is difficult to overstate the importance of fully understanding the construct used. As Hudson et al. (2004) demonstrated, different service quality dimensions (or attributes) rank differently according to measurement constructs, and the interpretation of each measurement can differ. It follows that without solid understanding and proper validation of the quality measure, misinterpretation and misuse are possible, and this could result in misguided marketing and policy decisions. Accordingly, this study aims to explore the validity of this new Importance–Performance construct. Our investigation includes both an analytical/conceptual examination, as well as an extensive empirical test. METHODOLOGY Theoretical Foundation As outlined above, the traditional approach to testing the connection between service quality Downloaded by [Namhyun Kim] at 14:44 14 August 2012 Kim, Choi, and Schwartz and satisfaction focused on three types of quality perception measures. The first is the “quality only” measure, and the second is the difference between perceived and desired (or anticipated) levels of quality. A third later development added importance as a multiplier, meaning that the tested construct was the product performance∗ importance. The rationale behind this performance∗ importance product has to do with a standard traditional assumption about how consumers evaluate a product. According to that specific approach, consumers weigh and add all available product information when they derive a utility value for the product. According to rational decision-making theories, options with higher utility are preferred. Within this framework a compensatory product selection strategy (e.g., the model suggested by Bass & Talarzyk, 1972) allows a higher value of one attribute to compensate for a lesser value of another attribute. An important element of this strategy is that the consumer assigns a weight to each of the product’s attributes. This weight represents the importance of the attribute to the consumer. Mathematically, the expected utility from a good or a service is the sum of the value∗ weight products across all of the good’s or service’s attributes. Clearly the performance∗ importance construct of the service quality and satisfaction domain parallels this value∗ weight approach. The perceived performance of the service corresponds with the (expected) value, the importance is the same as the weight, and the expected utility is replaced by satisfaction. Once it is realized that importance is a weight, the rationale behind a performance∗ importance term becomes clear: the higher the importance, the more the quality should affect the satisfaction. This is achieved by multiplying the performance by the importance level. Consider Figure 1 where the Y-axis represents performance (P), the X-axis holds the importance values (I), and the vertical axis holds the product P∗ I. The shape of the plane reveals how the two (P and I) work “together” to form the P∗ I construct in the same manner. That is, the higher P or I the higher P∗ I as the relation is monotonic and linear. Now consider two versions of the “tourism proposed” construct of the gap between the 603 FIGURE 1. Importance × Performance (color figure available online) FIGURE 2. Performance-Importance (color figure available online) FIGURE 3. |Performance-Importance| (color figure available online) perceived quality of performance and the importance of each of the service attributes. The first is the sign-sensitive, directional P-I (performanceimportance) as used by Hudson et al. (2004) and the second is the sign free, non-directional absolute value of the differential performanceimportance |P-I|. While the sign-free version was not used before, we propose analyzing it in order to remove any doubt arising from possible confusion about directionality. The charts depicting the two are shown in Figures 2 and 3 below. 604 JOURNAL OF TRAVEL & TOURISM MARKETING TABLE 1. The Combinations of End Values of P and I I Downloaded by [Namhyun Kim] at 14:44 14 August 2012 Low Low High High P P-I |P-I| Low High Low High Medium High Low Medium Low High High Low These two charts do not provide any clear rationale for the use of the P-I nor |P-I| constructs nor why these constructs are expected to be determinants of satisfaction. Exploring the extreme values of Figures 2 and 3 and investigating the meaning of the edge values of P and I in relation to satisfaction best demonstrate that these two versions of the suggested gap construct do not make sense. Table 1 lists the combinations of end values of P and I (named High and Low here) and the corresponding values of the constructs P-I and |P-I|. The arbitrary nature of this gap construct is obvious at first glance. For example, it is difficult to reconcile the notion that a Low importance/Low performance combination results in the same Medium P-I construct level as the High importance/High performance pair. Why would unimportant low-quality service attributes generate the same level of satisfaction as important and high-quality service attributes? Similarly, why would a person be more satisfied with a High performance/Low importance service compared to a High performance/High importance service? This implies that when holding the level of high performance constant, the lower the importance the higher the satisfaction. Similar logical inconsistencies are observed when we replace the suggested P-I construct with the absolute version |P-I|. The underlying “logic” of the gap construct seems to imply that consumers’ utility formation is irrational as it clearly violates the rationality rules. However, none of the tourism empirical studies that used this measure in the past offered any compelling arguments as to why such a seemingly erratic relation should be postulated. In summary, it seems that the suggested differential construct of P-I has no apparent intuitive quality when it comes to serving as an explanatory variable. The next step is to closely examine the construct’s performance in an empirical study. Formally, we test the hypothesis that P-I is a significantly inferior construct compared to the traditional P-only concept. Empirical Test The empirical portion of this investigation into the validity of the Importance– Performance construct was conducted as a field study contrasting the P-I with the P-only construct as predictors of tourists’ satisfaction. Specifically, we adopted the service “quality → satisfaction” causal relationship framework assuming that a valid measure will perform well as a predictor of satisfaction. The relationships among quality, satisfaction, and behavioral intention were explored extensively in marketing, service, and tourism literature (Cronin & Taylor, 1992; Lee et al., 2004). This study uses the “quality → satisfaction” causal relationship framework and not the behavioral intention concept because some empirical studies do not support the relationship between quality perceptions and behavioral intention (Cronin & Taylor, 1992; Lee & Beeler, 2007; Yuan & Jang, 2008). Study Setting and Sample The empirical test to assess the validity of the P-I differential construct was conducted in a festival/event setting. This was deemed most appropriate because the first study to employ the Importance-Performance construct as a measure of quality (Crompton & Love, 1995) was performed in a festival setting as well. As a reminder, Crompton & Love’s study compared seven constructs in a quest to identify the predictive validity of alternative measures of quality: (a) Expectations, (b) Importance times Expectations, (c) Performance, (d) Importance minus Performance, (e) Importance times Performance, (f) Performance minus Expectation, and (g) Importance times (Performance minus Expectation). The current study setting was the 2010 World Oriental Medicine-Bio Expo (called Bio Expo) held in Jecheon, South Korea. The Downloaded by [Namhyun Kim] at 14:44 14 August 2012 Kim, Choi, and Schwartz area is a center of traditional and oriental medicine in the Chungcheongbuk-do province of Korea (http://bioexpo.wordpress.com/). This first oriental medicine and therapy world expo was organized by Jecheon city and the Chungcheongbuk-do province. It took place between September 16 and October 16. The Bio Expo provided a variety of activities including exhibitions and museums (e.g., the oriental medicine bioscience museum and medical industrial pavilion), international and oriental therapies, herb gardens, performances, and free health checks. More than 1,300,000 domestic and foreign visitors attended the event (http:// www.hanbang-expo.org). Data was collected on the grounds of the Bio Expo. Trained interviewers intercepted event participants in a random manner and asked them to participate in this study by responding to a self-administered questionnaire. A total of 199 usable questionnaires were completed out of 220 requests, representing a usable response rate of 90%. A little over half of the samples were male respondents (53.3%) and 54.3% of the respondents were married. Approximately one-third of respondents were between 20 and 29 years of age (32.7%), followed by those between 30 and 39 (26.6%), and individuals between 40 and 49 (26.1%). Among the respondents 32.7% were residents of Jecheon, the city where the event took place. Measurement Scale Development The service quality attributes of the event were identified based on an extensive review of quality-oriented festival and event literature (e.g., Childress & Crompton, 1997; Crompton, 2003; Cole & Illum, 2006; Lee & Beeler, 2007; Lee, Lee, Lee, & Babin, 2008; Taylor & Shanka, 2002; Wicks & Fesenmaier, 1993). The survey instrument consisted of three parts: perceived quality, satisfaction, and demographic characteristics. Two subsets of perceived quality questions (Importance and Performance) were identified in the literature review. The items were then modified to reflect the advice of festival organizers and professionals who were interviewed for this purpose. Finally, 20 attributes on a 7-point scale (1 = strongly disagree and 7 = 605 strongly agree) were selected (Table 2). A single statement was used to measure participants’ overall satisfaction. Data Analyses The first step in the process of comparing the differential measure of P-I with the performance-only construct was to calculate the P-I score for each one of the attributes. Event quality items were analyzed using factor analysis, and Cronbach’s alpha coefficient was calculated to assess the reliability of the factors. Two descriptive statistics, skewness and kurtosis, were used to compare the characteristics of two data series. Finally, a regression model with satisfaction as the dependent variable was employed to further explore and compare the predictive power of the P-I construct with that of the performance-only one. Given that the “quality →satisfaction” relation is well documented in the literature with the performance-only construct, it was expected that if the P-I measure were indeed a valid one, it would clearly show in the results of the comparative analysis of the two regression models. In other words, we tested whether the P-I construct key regression indicators are at least as good as those of the P-only measure. RESULTS Measurement Reliability and Data Properties Factor analysis using the principal component analysis method with varimax rotation was employed first. Three factors were identified based on an eigenvalue greater than one, explaining 68.3% of the total variance. They were (see Table 2): 1. comfort amenities, food & beverage, and program content; 2. information, staff, and souvenirs; 3. general features. One of the 20 items was eliminated to allow for better interpretation of the factor because it was loaded on two factors with factor loading of 606 JOURNAL OF TRAVEL & TOURISM MARKETING TABLE 2. Results of Factor Analysis and Reliability Factors and items Downloaded by [Namhyun Kim] at 14:44 14 August 2012 Factor 1. Comfort amenities and program Cleanliness of restrooms Interesting program contents Availability of restrooms Variety of program, events Quality of food and beverage Fair price of food and beverage Ease of parking Availability of space to sit and rest Factor loading Eigenvalue Variance explained 10.464 1.379 Reliability (M/SD) P-only P-I 55.07% .929 (4.33/1.68) (4.61/1.52) (4.73/1.69) (4.72/1.54) (4.53/1.54) (4.31/1.59) (4.72/1.68) (4.69/1.59) .9 (−.74/1.94) (−.58/1.60) (−.47/1.82) (−.50/1.64) (−.37/1.70) (−.49/1.72) (−.40/1.75) (−.41/1.55) 7.26% .808 .774 .754 .709 .706 .659 .656 .591 Factor 2. Information, staff, and souvenirs Information and signage Pamphlets Variety of souvenirs Friendliness of guides and staff Guides and staff knowledge about the festival Guides and staff availability Information booth Quality of souvenirs .775 .741 .740 .709 .653 .922 (5.06/1.34) (4.99/1.37) (5.01/1.48) (5.23/1.38) (4.92/1.47) .632 (−.40/1.37) (−.30/1.22) (−.37/1.50) (−.61/4.51) (−.43/1.44) .639 .619 .584 (5.10/1.37) (5.08/1.37) (5.01/1.35) (−.35/1.59) (−.10/1.40) (−.45/1.55) Factor 3. General feature Accessibility of event site Festival advertising Timing of the event .798 .791 .756 .825 (4.36/1.48) (4.82/1.39) (4.82/1.36) .763 (−.50/1.65) (−.31/1.67) (−.19/1.38) .909 (4.60/1.39) (4.59/1.57) (4.50/1.36) – – – – 1.137 Total variance Satisfaction I will recommend to others I would like to visit again Overall I am satisfied 5.98% 68.32% 2.548 .944 .924 .895 Total variance over .4. Bartlett’s test of sphericity was significant at p < .001, and the Kaiser-Meyer-Oklin measure of sampling adequacy of .91 indicates that the data set is suitable for the factors. All of the reliability coefficients of the performanceonly construct exceeded the minimum cutoff value of .7, while Factor 2 of the P-I construct didn’t exceed that threshold. All factors of the P-I construct show lower values than those of the performance-only construct indicating less reliability and poor internal consistency of the P-I construct. The satisfaction construct was identified as one factor (Bartlett’s test, p < .001; KMO = .734), explaining 84.9% of the total variance. The Cronbach’s alpha coefficient for reliability was .91 (see Table 2). Two descriptive statistics, skewness and kurtosis, were used to measure the properties 84.93% of the two data series (Performance-only construct and P-I construct) and to evaluate whether these data series approximate a particular probability distribution such as the normal. Table 3 contrasts the skewness and kurtosis of the two constructs across all factors. Factors 1 and 2 of the P-I construct are more negatively skewed, indicating that the distribution of the P-I data for Factor 1 and 2 is highly skewed (less than −1) and less likely to be symmetrical. Kurtosis provides “a measure of the thickness of the tails of a distribution” (Pindyck & Rubinfeld, 1997, p. 47) or the degree of peakedness of a distribution. The P-I construct’s kurtosis values are greater than 1 and the distribution is more leptokurtic (higher peak, narrower, and fatter tails than normal). For example, Factor 2 of the P-I construct has an extreme positive Kim, Choi, and Schwartz 607 TABLE 3. Skewness and Kurtosis: P Versus P-I Statistics Factor 1 Number M SD Skewness SES Kurtosis SEK Factor 2 Factor 3 P-only P-I P-only P-I P-only P-I 199 4.579 1.314 −0.369 .172 −0.377 .343 199 −0.496 1.316 −1.004 .172 1.524 .343 199 5.049 1.1201 −0.29 .172 −0.305 .343 199 −0.375 1.1058 −2.635 .172 17.706 .343 199 4.67 1.218 −0.418 .172 0.04 .343 199 −0.335 1.295 −0.342 .172 1.471 .343 Downloaded by [Namhyun Kim] at 14:44 14 August 2012 Note. SES = standard error of skewness; SEK = standard error of kurtosis. kurtosis (17.7) indicating a distribution where more of the values of the data series are located in the tails of the distribution rather than near the mean. The Predictive Power of the Two Constructs To contrast the prediction capability of the two event-quality constructs, we fit a multivariate linear regression to model the relationship between quality and satisfaction where satisfaction is the dependent variable and the three factors are the independent variables. The results show clear evidence of the predictive power of two constructs. The performance-only construct regression model was significant with F = 85.4 and p < .001 (Table 4). As indicated by the coefficient of determination R2 , 56.8% of the variance in satisfaction was explained by the performance-only set of independent variables. In contrast, although the model was significant (F = 7.014, p < .001), only a mere 9.7% of the variance in satisfaction was explained by the independent variables when P-I constructed the measures. Clearly, the P-I construct model is less capable of predicting the variation in satisfaction. In addition, while all performance-only construct factors were statistically significant at the p < .01 level, only a single factor (comfort amenities and program) had a positive impact on satisfaction at the significance level of .01. These findings indicate that the validity of the P-I construct is highly questionable and might be risky to use as a measurement of quality in event/festival settings. TABLE 4. Results of Regression Models of Each Construct Factors Unstandardized coefficients B SE Standardized coefficients Sig. R2 F (p) −14.299 5.351 2.894 3.230 .000∗ .000∗ .004∗ .001∗ .568 85.418 (.000) 1.636 2.643 .084 1.514 .104 .009∗ .933 .132 .097 7.014 (.000) B Regression model of the Performance-only construct (Constant) −3.250 .227 Factor 1 .317 .059 .416 Factor 2 .208 .072 .233 Factor3 .161 .050 .196 Regression model of the P-I construct (Constant) .120 .073 Factor1 .175 .066 Factor2 .006 .074 Factor3 .093 .061 t value .231 .007 .120 Note. Factor 1 = Comfort amenities, F&B, and program; Factor 2 = Information, staff, and souvenirs; Factor 3 = General features. ∗ p < .01. 608 JOURNAL OF TRAVEL & TOURISM MARKETING Downloaded by [Namhyun Kim] at 14:44 14 August 2012 CONCLUSION Service quality in tourism and hospitality has been long recognized as an important element in attracting and retaining customers. As such, it is clearly of utmost importance for industry managers and policy makers to effectively measure and monitor customers’ perception of quality. It follows that destination marketers and researchers should be very careful when selecting the instruments they use to evaluate service quality. The use of inappropriate measures to assess visitors’ perceptions about service quality and their satisfaction with their tourism experience might lead to misleading conclusions on behalf of the policy makers and as a result to misguided policy and management at the destination. The tourism literature has suggested and utilized a unique construct, Importance– Performance, of service quality in the past without providing any rationale for using this construct. The purpose of this study was to explore the validity of this new construct and to assess its relevance to tourism research on visitors’ quality perceptions and satisfaction. The main contribution of this study is that it provides clear analytical and empirical evidence that question this construct’s external validity. Being the first study to explore the possible logical justification for using this construct, we argue that given the demonstrated inconsistency and the contradictory nature of the construct, its external validity is minimal at best. In our study, an empirical test follows this conceptual discussion, comparing the construct under investigation to that of the widely accepted one of “performance-only.” The various statistics strongly suggest that the construct in question is indeed inferior. Given this study’s conceptual and empirical observations, and given that none of the previously published empirical tests show that this construct is better than others, it might be safe to suggest that the “Performance–Importance” construct is of inferior validity and perhaps should not be considered by practitioners and researchers in the future. The findings contribute to the enhancement of the measurement of service quality in the tourism and hospitality literature as it critically evaluates the validity of the construct, both from theoretical and empirical points of view. From a managerial perspective, it provides managers with a better understanding of which measurement should be employed in assessing service quality and satisfaction and how the measurement should be interpreted in line with its theoretical foundation. This study has its limitations. While the conceptual analysis is generic in nature and is not limited to a specific area of application, our empirical test is narrow in that it was limited in scope and focused on a single event in a single destination. Follow-up empirical tests could broaden the support for our finding. In addition, given its established superior performance of the “Performance-only” construct, this study used it as a benchmark for comparison with the construct in question. Future research could use a wider selection of service quality measurements for comparisons. Such a replication and extension of our effort where a meaningful comparison of possible quality measurement are tested, compared to each other and across diverse tourism and hospitality environments could prove very useful for practitioners and researchers who need to establish validity levels of quality and satisfaction measures in tourism research. REFERENCES Bass, F., & Talarzyk, W. W. (1972). An attitude model for the study of brand preference. Journal of Marketing Research, 9, 93–96. Brady, M. K., Cronin, J. J., & Brand, R. R. (2002). Performance-only measurement of service quality: A replication and extension. Journal of Business Research, 55(1), 17–31. doi:16/S0148-2963(00)00171-5 Carman, J. M. (1990). Consumer perceptions of service quality: An assessment of the SERVQUAL dimensions. Journal of Retailing, 66(1), 33–55. Childress, R. D., & Crompton, J. L. (1997). A comparison of alternative direct and discrepancy approaches to measuring quality of performance at a festival. Journal of Travel Research, 36(2), 43–57. Chu, R. K. S., & Choi, T. (2000). An importanceperformance analysis of hotel selection factors in the Hong Kong hotel industry: A comparison of business Downloaded by [Namhyun Kim] at 14:44 14 August 2012 Kim, Choi, and Schwartz and leisure travellers. Tourism Management, 21(4), 363–377. Cole, S. T., & Illum, S. F. (2006).Examining the mediating role of festival visitors’ satisfaction in the relationship between service quality and behavioral intentions. Journal of Vacation Marketing, 12(2), 160–173. Crompton, J. L. (2003). Adapting Herzberg: A conceptualization of the effects of hygiene and motivator attributes on perceptions of event quality. Journal of Travel Research, 41(3), 305–310. Crompton, J. L., & Love, L. L. (1995). The predictive validity of alternative approaches to evaluating quality of a festival. Journal of Travel Research, 34(1), 11–24. Crompton, J. L., & MacKay, K. J. (1989). Users’ perceptions of the relative importance of service quality dimensions in selected public recreation programs. Leisure Sciences, 11(4), 367–375. Cronin, J. J., Jr., & Taylor, S. A. (1992). Measuring service quality: A reexamination and extension. Journal of Marketing, 56(3), 55–68. Deng, W. (2007). Using a revised importance-performance analysis approach: The case of Taiwanese hot springs tourism. Tourism Management, 28(5), 1274–1284. Ennew, C. T., Reed, G. V., & Binks, M. R. (1993). Importance-performance analysis and the measurement of service quality. European Journal of Marketing, 27(2), 59–70. Hudson, S., Hudson, P., & Miller, G. (2004). The measurement of service quality in the tour operating sector: A methodological comparison. Journal of Travel Research, 42(3), 305–312. Hussain, K., & Birol, C. (2011). The assessment of non-academic and academic service quality in higher education. EgitimArastirmalari-Eurasian Journal of Educational Research, 11(42), 95–116. Jia, R., Reich, B. H., & Pearson, J. M. (2008). IT service climate: An extension to IT service quality research. Journal of the Association for Information Systems, 9(5), 294–320. John, J., Yatim, F. M., & Mani, S. A. (2011). Measuring service quality of public dental health care facilities in Kelantan, Malaysia. Asia-Pacific Journal of Public Health, 23(5), 742–753. Kim, W. G., & Lee, H. Y. (2004). Comparison of web service quality between online travel agencies and online travel suppliers. Journal of Travel & Tourism Marketing, 17(2), 105–116. Lee, J., & Beeler, C. (2007). The relationships among quality, satisfaction, and future intention for first-time and repeat visitors in a festival setting. Event Management, 10(4), 197–208. Lee, J., Graefe, A., & Burns, R. (2004). Service quality, satisfaction, and behavioral intention among forest visitors. Journal of Travel & Tourism Marketing, 17(1), 73–82. 609 Lee, Y., Lee, C., Lee, S., & Babin, B. J. (2008). Festivalscapes and patrons’ emotions, satisfaction, and loyalty. Journal of Business Research, 61(1), 56–64. Martilla, J. A., & James, J. C. (1977). Importanceperformance analysis. Journal of Marketing, 41(1), 77–79. Matzler, K., Sauerwein, E., & Heischmidt, K. A. (2003). Importance-performance analysis revisited: The role of the factor structure of customer satisfaction. Service Industries Journal, 23(2), 112–129. Mount, D. J. (1997). Introducing the relativity to traditional importance-performance analysis. Journal of Hospitality & Tourism Research, 21(2), 111–119. Oh, H. (2001). Revisiting importance-performance analysis. Tourism Management, 22(6), 617–627. doi:10.1016/S0261-5177(01)00036-X Ostrowski, P. L., O’Brien, T. V., & Gordon, G. L. (1993). Service quality and customer loyalty in the commercial airline industry. Journal of Travel Research, 32(2), 16–24. Parasuraman, A., Zeithaml, V. A., & Berry, L. L. (1988). SERVQUAL: A multiple-item scale for measuring consumer perceptions of service quality. Journal of Retailing, 64(1), 12–40. Pindyck, R. S., & Rubinfeld, D. L. (1997). Econometric models and economic forecasts (4th ed.). New York, NY: McGraw-Hill/Irwin. Smith, A. M. (1995). Measuring service quality: Is SERVQUAL now redundant? Journal of Marketing Management, 11(1–3), 257–276. Smith, S., & Costello, C. (2009). Culinary tourism: Satisfaction with a culinary event utilizing importanceperformance grid analysis. Journal of Vacation Marketing, 15(2), 99–110. Taylor, R., & Shanka, T. (2002). Attributes for staging successful wine festivals. Event Management, 7, 165–175. Teas, R. K. (1993). Expectations, performance evaluation, and consumers’ perceptions of quality. Journal of Marketing, 57(4), 18–34. Tkaczynski, A., & Stokes, R. (2010). Festperf: A service quality measurement scale for festivals. Event Management, 14(1), 69–82. Tsang, N. K. F., Lai, M. T. H., & Law, R. (2010). Measuring e-service quality for online travel agencies. Journal of Travel & Tourism Marketing, 27(3), 306–323. Tsuji, Y., Bennett, G., & Zhang, J. (2007). Consumer satisfaction with an action sports event. Sport Marketing Quarterly, 16, 199–208. Urdang, B., & Howey, R. (2001). Assessing damages for non-performance of a travel professional—A suggested use of “servqual.” Tourism Management, 22(5), 533–538. Wicks, B. E., & Fesenmaier, D. R. (1993). A comparison of visitor and vendor perceptions of service quality at a 610 JOURNAL OF TRAVEL & TOURISM MARKETING Downloaded by [Namhyun Kim] at 14:44 14 August 2012 special event. Festival Management & Event Tourism, 1, 19–26. Wright, B. Z., Duray, N., & Goodale, T. L. (1992). Assessing perceptions of recreation center service quality: An application of recent advancements in service quality research. Journal of Park & Recreation Administration, 10(3), 33–47. Yuan, J., & Jang, S. (2008). The effects of quality and satisfaction on awareness and behavioral intentions: Exploring the role of a wine festival. Journal of Travel Research, 46(3), 279–288. Yuksel, A., & Rimmington, M. (1998). Customersatisfaction measurement. Cornell Hotel & Restaurant Administration Quarterly, 39(6), 60–70. SUBMITTED: September 20, 2011 FINAL REVISION SUBMITTED: February 27, 2012 ACCEPTED: April 3, 2012 REFEREED ANONYMOUSLY