Customer Satisfaction in IS Projects: Assessing the Role of Process and Product Performance

Communications of the Association for Information Systems Volume 37 Article 22 8-2015 Customer Satisfaction in IS Projects: Assessing the Role of Process and Product Performance Dirk Basten University of Cologne, basten@wiso.uni-koeln.de Oleg Pankratz University of Cologne Follow this and additional works at: htp://aisel.aisnet.org/cais Recommended Citation Basten, Dirk and Pankratz, Oleg (2015) "Customer Satisfaction in IS Projects: Assessing the Role of Process and Product Performance," Communications of the Association for Information Systems: Vol. 37, Article 22. Available at: htp://aisel.aisnet.org/cais/vol37/iss1/22 his material is brought to you by the Journals at AIS Electronic Library (AISeL). It has been accepted for inclusion in Communications of the Association for Information Systems by an authorized administrator of AIS Electronic Library (AISeL). For more information, please contact elibrary@aisnet.org. C ommunications of the A ssociation for I nformation S ystems Research Paper ISSN: 1529-3181 Customer Satisfaction in IS Projects: Assessing the Role of Process and Product Performance Dirk Basten Oleg Pankratz Department of Information Systems and Systems Development, University of Cologne basten@wiso.uni-koeln.de Department of Information Systems and Systems Development, University of Cologne Abstract: Despite extensive research over the past several decades, assessing information system (IS) project success is still a challenging endeavor. While the traditional approach takes process performance (time and budget) and product performance (functional and non-functional requirements) into account, the contemporary perspective acknowledges the more comprehensive character of project success and emphasizes the criticality of stakeholder satisfaction. Continuing previous research, we propose and test a model with customer satisfaction as the uppermost criterion of IS project success and process performance and product performance as its determinants. Following recent calls for researchers to investigate the explicit linkage between success factors and success criteria, we also analyze the influence of process transparency on process and product performance. We conducted a survey via a questionnaire with IS experts in Germany. We contribute to a deeper understanding of IS project success by indicating that customer satisfaction is less a matter of time and budget and that a stronger emphasis should be placed on product performance. Moreover, our results illuminate the role of process transparency in IS projects showing that it contributes to both process and product performance. Keywords: Project Management, Information Systems, Project Success, Process Performance, Product Performance, Customer Satisfaction, Process Transparency. The manuscript was received 08/28/2014 and was with the authors 3 months for 2 revisions. Volume 34 Paper 22 pp. 430 – 447 August 2015 431 1 Customer Satisfaction in IS Projects: Assessing the Role of Process and Product Performance Introduction Information systems (IS) as sets of “interrelated components that collect (or retrieve), process, store, and distribute information to support decision making and control” (Laudon & Laudon, 2009, p. 46) are highly important for organizations. Despite decades of research calling for an extended set of success criteria (Ika, 2009; Judgev & Müller, 2005), organizations still widely rely on adherence to planning (i.e., adherence to budget, adherence to schedule, and conformance with requirements) to assess IS project success (Collins & Baccarini, 2004; Joosten, Basten, & Mellis, 2014; Thomas & Fernández, 2008). This phenomenon can be explained by (among other reasons) the plainness of measuring these criteria (Joosten et al., 2014; Pinto & Slevin, 1988a). However, IS projects are typically commissioned by a customer organization, whose satisfaction is important (Anderson, Fornell, & Legmann, 1994; Anderson & Sullivan, 1993; Nelson, 2005). Consequently, project managers strive to meet the traditional criteria while also satisfying the customer1. While budget and schedule are closely related indices that have a rather short-term character (i.e., they are directly linked to the project), requirements are more likely to have a long-term character (i.e., they define the product, which the customer organization will apply in the long run). As such, project managers need to know whether to prioritize the process (i.e., budget and schedule) or the product (i.e., requirements) in IS projects. The unsettled question of how to measure IS project success is also reflected in IS research. Whereas IS success is often assessed in terms of the model by DeLone and McLean (1992, 2003), “determining the contribution of (IS) projects is a difficult endeavour” (Barclay, 2008, p. 331), and project success in general might require a new theory (Barclay, 2008; Cuellar, 2010; Glass, 1999). As Ika (2009) emphasizes, research is particularly in need of studies linking project success criteria and project success factors. Whereas success criteria are measures by which success is assessed, success factors are aspects contributing to project success (Cooke-Davies, 2002). Concerning the former (i.e., project success criteria), a major contrast is the difference between using traditional criteria such as a project’s adherence to budget, adherence to schedule, and conformance with requirements (Atkinson, 1999; Lech, 2013; Pinto, 2004) and contemporary criteria such as stakeholder satisfaction (Baker, Murphy, & Fisher, 2008; Ika, 2009). While the traditional criteria can be seen as the sole constant in the evolution of IS project success, the general understanding has shifted to a strategic perspective that focuses on the customer (Ika, 2009; Judgev & Müller, 2005). Nelson (2005) even equals stakeholder satisfaction to project success. In this context, the customer organization’s satisfaction can be assumed to be the uppermost criterion because it is crucial for contractor reputation and decisions about follow-up projects (Anderson et al., 1994; Anderson & Sullivan, 1993). Considering the research shift towards success criteria related to customer satisfaction (Baker et al., 2008; Basten, Joosten, & Mellis, 2012; Nelson, 2005) and taking into account that adherence-to-planning criteria are still most widely applied in practice (Collins & Baccarini, 2004; Joosten et al., 2014; Thomas & Fernández, 2008), we need to analyze the linkage between these two concepts. Concerning the latter (i.e., project success factors), both research (Joosten et al., 2014; Pankratz & Loebbecke, 2011; Stavrou, Pankratz, & Basten, 2014) and practice (Böhm & Haselberger, 2012; Cohn, 2009) see transparency of the development process (henceforth: process transparency) as a topic that is of high importance yet not explicitly addressed in common literature so far. Process transparency refers to the stakeholders’ perception to have all information relevant to them. Emphasizing the importance of process transparency in IS projects (Stavrou et al., 2014), managers point to the need of further analyses with regard to IS project success (Joosten et al., 2014). Such analyses will contribute to a better understanding of the IS development, which is often treated as a black box (Siau, Long, & Ling, 2010). Based on the above considerations, we investigate the coherence between customer satisfaction as the uppermost criterion and its determinants, process performance, and product performance. More precisely, we assess the relation between process performance and customer satisfaction and the relation between product performance and customer satisfaction. In this regard, we also analyze whether and which one of these two determinants is more important for assessing IS project success (i.e., has a higher influence on customer satisfaction). We also analyze the role of process transparency for IS project success, or, more precisely, the influence of process transparency on process performance and product performance. 1 In this paper, we differentiate between the customer as organization that commissioned the IS project and end users as that organization’s individuals who use the developed product. Volume 34 Paper 22 Communications of the Association for Information Systems 432 To do so, we conducted a questionnaire survey among managers of IS projects and assess their view on the concepts under consideration. For our investigation, we apply partial least squares (PLS) path analysis (Lohmöller, 1989). Concerning our core concepts, we build on prior research in IS and marketing to operationalize process performance and product performance (Wallace, Keil, & Rai, 2004) and customer satisfaction (Caruana, 2002). Concerning process transparency, we apply items that we adapted from previous literature (Joosten et al., 2014; Mao, Lee, & Deng, 2008; Pankratz & Loebbecke, 2011). Our contribution is threefold. First and following contemporary research that equals project success to stakeholder satisfaction (Nelson, 2005), particularly customer satisfaction (Basten et al., 2012; DeCotiis & Dyer, 1977; Pankratz & Loebbecke, 2011), we provide a differentiated analysis of the influence of both process and product performance on customer satisfaction. Second, we go one step further and analyze the coherence between process performance and product performance. Finally, we contribute to theory and practice by explicitly accounting for the linkage of a success factor—process transparency—to IS project success criteria. This paper proceeds as follows: in Section 2, we present the theoretical underpinnings of IS project success. In Section 3, we present our research model and related hypotheses. In Section 4, we explain the strategy applied to collect data from contractors in IS projects and the operationalization of the research variables. In Section 5, we validate the measurement properties of the constructs and test the proposed research model. In Section 6, we discuss our findings. Finally, in Section 6, we conclude the paper with implications for practitioners and directions for future research. 2 Success of Information System Projects A project in general is “a temporary endeavor undertaken to create a unique product, service, or result” (Project Management Institute, 2008, p. 5). As such, an IS project can be seen as a temporary and unique endeavor with the objective to develop, extend, or adapt an information system. In previous research, many terms have been used to denote the concept of IS project success. Examples include software project success (Agarwal & Rathod, 2006), project performance (Aladwani, 2002), development project success (Saarinen & Sääksjärvi, 1992), IT project success (Thomas & Fernández, 2008; Wateridge, 1995), and IS development success (Banker & Kemerer, 1992). In this paper, we use the term IS project success. 2.1 A Retrospective View IS project success has been a topic in IS research for decades (Agarwal & Rathod, 2006; Brittain White & Leifer, 1986; Ginzberg, 1981; Narayanaswamy, Grover, & Henry, 2013; Wateridge, 1995; Yetton, Martin, Sharma, & Johnston, 2000). Ika (2009) and Judgev and Müller (2005) reflect on studies concerning (IS) project success published since the 1960s. They reveal that the understanding of (IS) project success has evolved from the traditional short-term evaluation to a contemporary assessment from a strategic perspective. The traditional approach to assess IS project success—transferred from the engineering discipline to IS projects, probably due to its easy measurability (Pinto & Slevin, 1988b)—considers a project’s adherence to budget, adherence to schedule, and conformance with requirements (Agarwal & Rathod, 2006). It is referred to as iron triangle (Atkinson, 1999), triple constraint (Pinto, 2004), or adherence to planning (Basten et al., 2012), the latter of which we use in this paper. Whereas adherence to planning has a straightforward and objective character, it has been widely criticized to cover a limited perspective of success. The criticism stems from the project management and the information system disciplines and refers to the following shortcomings: 1) adherence to planning is a short-term measure that does not include customer benefits (Agarwal & Rathod, 2006; Atkinson, 1999); 2) scholars criticize that projects primarily initiated to realize business benefits should be evaluated in terms of according criteria (Shenhar, Dvir, Levy, & Maltz, 2001); 3) effort estimation as a fundamental activity of project conceptualizing (Jurison, 1999) is often defective (Moløkken-Østvold et al., 2004), lacks adequate methods (Agarwal and Rathod, 2006), and is often distorted by political negotiations (Lederer et al., 1990); and 4) solely relying on adherence to planning to assess IS project success does not account for the perspective of all stakeholders (Agarwal & Rathod, 2006). The contemporary perspective acknowledges the comprehensive character of project success and includes criteria such as strategic customer objectives, end user satisfaction, and stakeholder benefits Volume 34 Paper 22 433 Customer Satisfaction in IS Projects: Assessing the Role of Process and Product Performance (Ika, 2009). Prior research has emphasized the criticality of stakeholder satisfaction (Baker et al., 2008), which has led scholars to consider success as a matter of perspective (Basten et al., 2012; Myers, 1995). Such perceptions may differ from success assessments in terms of adherence to planning. Anecdotal and empirical evidence exist for projects failing the traditional criteria but being perceived successful and for projects satisfying the traditional criteria but being perceived as failures (Baker et al., 2008; Ika, 2009; Pinto & Slevin, 1988b). Nelson (2005) refers to such mismatching projects as successful failures and failed successes, respectively. Moreover, Nelson (2005) equals stakeholder satisfaction to success. In this context, we can assume satisfaction of the customer organization to be the uppermost criterion because it is crucial for contractor reputation and decisions about follow-up projects (Anderson et al., 1994; Anderson & Sullivan, 1993). While research is continuously moving away from relying on adherence to planning (Barclay, 2008), this criterion is the only constant in the focus of research considering the timely development of IS project success (Ika, 2009; Judgev & Müller, 2005). Despite the criticism towards this concept, scholars have argued for supplementing it with additional criteria instead of eliminating planning-related ones (Agarwal & Rathod, 2006). Considering the research shift towards success criteria related to customer satisfaction (Baker et al., 2008; Basten et al., 2012; Nelson, 2005) and considering that adherence to planning is still most widely applied in practice (Collins & Baccarini, 2004; Joosten et al., 2014; Thomas & Fernández, 2008), an analysis of the linkage between these two concepts is likely to reveal important insights for IS research. To analyze its relation to customer satisfaction, one can divide adherence to planning into two components (Keil, Rai, & Liu, 2013; Wallace et al., 2004): 1) process performance as the first component concerns the project input and is measured in terms of adherence to schedule and budget (Keil et al., 2013; Wallace et al., 2004), and 2) meeting requirements represents the project output. Whereas several researchers have explicitly differentiated between meeting functional and non-functional requirements (Agarwal & Rathod, 2006; Baccarini, 1999; Kappelman, McKeeman, & Zhang, 2006), others have not (Atkinson, 1999). However, researchers in both cases refer to conformance with (non-)functional requirements as the deviation between the specified (non-)functional requirements and their actual implementation (Joosten et al., 2014). To avoid confusion, Agarwal and Rathod (2006) suggest emphasizing the conceptual difference of functional requirements and non-functional requirements. Following their argumentation, we include both concepts in our study and, following a majority of authors (Atkinson, 1999; Jørgensen, 2004; Paulk, Weber, Curtis, & Chrissis, 1995; Rai & Al-Hindi, 2000; Westerveld, 2003), refer to a project’s conformance with (non-)functional requirements as the concept of product performance. 2.2 Process Transparency in the Context of IS Projects Considering growing research on project management alternatives such as agile approaches, critical success factors have evolved by focusing on stakeholder interaction (Barclay, 2008). In this context, transparency is a topic in both recent exploratory research (Joosten et al., 2014; Pankratz & Loebbecke, 2011; Stavrou et al., 2014) and the practitioner community (Böhm & Haselberger, 2012; Cohn, 2009). One can see transparency as highly relevant for project management, but it has been widely neglected so far. Pankratz and Loebbecke (2011, p. 6) define transparency as “team members incl. [including] project manager are informed about project plan, status and all events important to them” and uncover that its extent considerably affects IS project success. While transparency is related to success factors such as communication (Hyväri, 2006) and stakeholder involvement (Petter, 2008), it has a different focus. For instance, stakeholders can contribute their ideas but still feel that they are insufficiently informed about the project progress. As managers have previously emphasized the importance of process transparency in IS projects (Joosten et al., 2014; Stavrou et al., 2014), we need to further analyze its impact on success criteria to better understand the IS development process (Siau et al., 2010). 3 Research Model and Hypotheses Figure 1 presents our research model, which posits relationships between process performance and customer satisfaction (H1a) and between product performance and customer satisfaction (H1b). Another relation is postulated between product performance and the relationship between process performance and customer satisfaction (H2). This moderating effect is modeled as a negative one. Finally, the model also posits that process transparency can strengthen process performance (H3a) and product performance (H3b). Volume 34 Paper 22 Communications of the Association for Information Systems 3.1 434 Impact of Process and Product Performance on Customer Satisfaction Following Myers’ (1995) hermeneutical view, success is a matter of perception. Considering IS projects, various stakeholders exist, including contractors, customers, project teams, and end users (Baker et al., 2008). Accordingly, Nelson (2005) equals IS project success to stakeholder satisfaction. Following this view, two main reasons exist to consider customer satisfaction as the uppermost criterion in IS projects. First, the customer decides on follow-up projects and a satisfied customer is, thus, crucial for contractor reputation (Anderson et al., 1994; Anderson & Sullivan, 1993). Second, considering the customer perspective and the general differentiation between organizational and end user level (Baker et al., 2008), end users’ (dis-)satisfaction is likely to be reflected in the (dis-)satisfaction of the customer organization. Following this argumentation, the dependent variable in our study is customer satisfaction, which is in line with previous research emphasizing the importance of this particular stakeholder group (Basten et al., 2012; DeCotiis & Dyer, 1977; Pankratz & Loebbecke, 2011). +H3a Process Performance +H1a ‐H2 Process Transparency +H3b Product Performance Customer Satisfaction +H1b Figure 1. Postulated Research Model of Customer Satisfaction Incorporating traditional research that deems adherence to planning an important success criterion for IS projects (Pinto & Slevin, 1988b), we differentiate—in line with studies by Wallace et al. (2004) and Keil et al. (2013)—between process performance and product performance. Regarding both components, previous research indicates a positive relation to customer satisfaction in IS projects (Baccarini, 1999; Thomas & Fernández, 2008). Concerning the former, “[s]everal empirical studies show a strong correlation between [process performance] and customer satisfaction” (Dvir & Lechler, 2004, p. 6). Concerning the latter, several studies suggest that meeting requirements leads to enhanced customer satisfaction since product performance is a measure of effectiveness (Baker et al., 2008; Basten et al., 2012; Kraut & Streeter, 1995; Pinto, 2004). As such, we hypothesize that: Hypothesis 1a. The higher process performance, the higher the customer’s satisfaction with the project. Hypothesis 1b. The higher product performance, the higher the customer’s satisfaction with the project. 3.2 Relationship between Process and Product Performance Analyzing the relation between process performance and customer satisfaction and between product performance and customer satisfaction raises the question of whether one of the determinants is more important. We assume that process performance is subordinate to product performance for the following reasons. First, organizations initiate projects to realize business benefits (Shenhar et al., 2001). Accordingly, fulfilling these long-term objectives is likely to be more important than the short-term criteria of project time and budget. Second, time and budget are project management criteria and, in most cases, Volume 34 Paper 22 435 Customer Satisfaction in IS Projects: Assessing the Role of Process and Product Performance do not reflect the needs of all stakeholders (e.g., end users) (Baker et al., 2008). Instead, those needs are typically addressed by product performance. Finally, time and budget are project management indices that are often incorrect due to underestimated effort (Moløkken-Østvold et al., 2004) and political negotiations (Lederer et al., 1990). While these reasons indicate a larger influence of product performance on customer satisfaction compared to process performance, we also consider the interplay of these components concerning customer satisfaction. On the one hand, projects that cost more and take longer to complete yet deliver usable products that satisfy the customer “have become a classic scenario” (Nelson, 2005, p. 369). On the other hand, projects exist that are completed on-time and on-budget but fail to satisfy customers due to unfulfilled requirements (Shenhar, Levy, & Dvir, 1997). Both scenarios show that product performance is likely to moderate the relation of process performance to customer satisfaction. As such, we hypothesize that: Hypothesis 2. The higher product performance, the lower the influence of process performance on customer satisfaction. 3.3 Impact of Process Transparency on Process and Product Performance The development process’s transparency is of high importance for project management. Based on a previous study (Joosten et al., 2014), we hypothesize that process transparency has a positive influence on both process performance and product performance. The beneficial influence of transparency on IS project success in general (Stavrou et al., 2014) is shown by the causal mapping of success factors to success criteria in Pankratz and Loebbecke’s (2011) study. In case of process performance, transparency reduces the effort needed to gain required information and, thus, also reduces the waste of resources (Joosten et al., 2014). In case of product performance, process transparency increases the likelihood that project stakeholders have all information important to them (Pankratz & Loebbecke, 2011). Following Pankratz & Loebbecke (2011), this leads to more effective customer involvement, which, in turn, is likely to result in a product that meets customer needs and expectations. As such, we hypothesize that: Hypothesis 3a. The higher process transparency, the higher the process performance. Hypothesis 3b. The higher process transparency, the higher the product performance. 4 4.1 Research Design Data Collection We gathered data via a questionnaire survey with IS experts in Germany. We used a call for participation by the GPM (Deutsche Gesellschaft für Projektmanagement e.V.), the German chapter of the International Project Management Association (IPMA), and personal contacts to gain participants. Overall, 52 IS project managers on behalf of the contractor participated in our survey to report on their experience in IS projects. Since we contacted individuals and organizations, we do not have an exact number of individual potential respondents who received our call. Thus, our sampling approach does not provide insights into the response rate and a potential non-response bias. However, we did not find a significant difference between early and late respondents. Accordingly, we assume the potential related bias to be minor. All participants were knowledgeable in the IS field with many years of professional experience (17 mean; 16 median). They gathered their experience on average in 20 IS projects in general (31 median) and in 17 (8 median) in their current position. Almost 80 percent had a university degree. In our sample, internally (56%) and externally (44%) contracted projects are almost equally distributed. There are also about as many fixed-price (54%) as time-and-material (46%) projects. The majority of projects (92%) were conducted to develop new or extend existing IS. Table 1 overviews the projects’ business sectors. Table 2 provides indices of project cost, duration, and number of team members. Volume 34 Paper 22 Communications of the Association for Information Systems 436 Table 1. Projects’ Distribution over Industries Industry Share Financial services 46% Telecommunication 15% Transport 12% Other 5% Retail 4% Chemistry 4% Manufacture 4% Supply 2% Research and development 2% Public services 2% Defense 2% Media / Print 2% Table 2. Projects’ Cost, Duration, and Number of Team Members Median Mean 500,000 3,009,282 Project duration in months 12 17 Number of team members 10 19 Project cost in € 4.2 Measurement Scales Table 3 overviews the applied constructs and the according items and their references. For process performance and product performance, we used measures that are typically used in IS research (Keil et al., 2013; Wallace et al., 2004). Concerning product performance, we selected the items to account for both functional and non-functional requirements (see our differentiation in Section 2). In contrast to Wallace et al. (2004), we omitted the item concerning reliability since this aspect is included in the item concerning overall system quality (PROD2). Moreover, we do not refer only to response time, but also to performance in general (PROD3). Table 3. Constructs and Corresponding Items Construct Item no. Process performance Product performance Customer satisfaction Item: to which extent do you agree/disagree with the following statements concerning the considered project? References Adopted from Wallace et al. (2004) PROC1 The system was developed within budget. PROC2 The system was developed within schedule. PROD1 The users perceive that the system meets intended functional requirements. PROD2 The overall quality of the developed application is high. PROD3 The system meets user expectations with respect to performance. PROD4 The system is easy to maintain. CS1 Based on the experience with the project and the product, the customer is satisfied. CS2 In general, the customer is satisfied. Volume 34 Adopted from Wallace et al. (2004) Adapted from Caruana (2002) Paper 22 437 Customer Satisfaction in IS Projects: Assessing the Role of Process and Product Performance Table 3. Constructs and Corresponding Items Construct Item no. Process transparency Item: to which extent do you agree/disagree with the following statements concerning the considered project? References CS3 Based on overall experience, the customer is dissatisfied. (reversed) PT1 I had access to all necessary information at all times. PT2 I was informed about the project status at all times. PT3 My information needs were often unmet. (reversed) Adapted from Mao et al. (2008) We adapted the items concerning customer satisfaction from Caruana (2002). Because we did not have access to the customers of the analyzed projects, we measured customer satisfaction as perceived by the project managers (cf. Ooi, Lin, Tan, and Chong (2011) for a similar approach). Since we did not identify existing items concerning the construct process transparency, we adapted those for the related construct of information sharing (Mao et al., 2008) based on our understanding of process transparency derived from literature (Joosten et al., 2014; Pankratz & Loebbecke, 2011; Stavrou et al., 2014). Although these items are applicable to any stakeholder in IS projects (the definition applied in this study refers to team members in general; see Section II), we collect data from the perspective of project managers since they are in charge of completing a project in terms of adherence to planning, which is what we assess in terms of process performance and product performance. Four IS experts successively pretested our questionnaire and approved it or suggested marginal changes in wording. Additionally, three researchers (authors’ colleagues working as graduates in the IS discipline) checked our items to ensure face validity (Gravetter & Forzano, 2012). We selected our respondents with regard to their experience in managing IS projects (we preferred most experienced). All items are reflective and were measured on a 7-point Likert scale (1—total disagreement; 7—total agreement). 5 Data Analysis and Results In contrast to covariance-based modeling approaches, partial least squares (PLS) path modeling inhibits minimal limitations on sample size and residual distribution (Chin, Marcolin, & Newsted, 2003). Due to our sample size and the fact that we followed an explorative approach (Hair, Ringle, & Sarstedt, 2011), we applied PLS path modeling by using SmartPLS 2.0 (Ringle, Wende, & Will, 2005) for data analysis. As such, we validated our scales’ psychometric properties to measure the constructs and falsify the hypothesized relations as stated above. 5.1 Measurement Model We first conducted an exploratory factor analysis using R, a free software environment for statistical computing and graphics. Using the threshold of eigenvalues greater than 1, we extracted four components that explain a total of 76 percent of the variance (see the convergent validity criteria below concerning the unidimensionality of our constructs). In this section, we elaborate on the four reliability and validity criteria that are relevant for measurement models containing only reflective indicators: internal consistency, indicator reliability, convergent validity, and discriminant validity. First, one can assess two criteria to evaluate internal consistency. All constructs need to exhibit a Cronbach’s alpha and composite reliability greater than 0.700 (Nunnally, 1978; Werts, Linn, & Jöreskog, 1974). As the lowest Cronbach’s alpha amount was 0.706 and the lowest composite reliability amount was 0.832 (see Table 4), the data passed both criteria. Second, indicators are considered reliable if the associated latent construct explains more than half of the indicator’s variance (Henseler, Ringle, & Sinkivocs, 2009). Considering standardized factor loadings, one can evaluate this criterion in terms of the squared factor loadings. Considering the lowest loading (i.e., 0.753), the data passed the criterion of indicator reliability because the according squared loading was 0.567. Third, one can apply three criteria to assess convergent validity (Fornell & Larcker, 1981): all item factor loadings should exceed 0.700, composite construct reliabilities should exceed 0.800, and average variance extracted (AVE) should exceed 0.500 for each construct. One of the items for product Volume 34 Paper 22 Communications of the Association for Information Systems 438 performance (PROD4) showed a low factor loading (0.364). Accordingly, we removed this item and reestimated the model parameters. As Table 5 shows, standardized item loadings exceeded the threshold of 0.700. Additionally, Table 4 demonstrates the composite reliabilities of all constructs exceeded the required minimum of 0.800. Table 4 also shows that AVE values of all constructs exceeded the threshold of 0.500. Thus, convergent validity conditions are met. Fourth, to confirm discriminant validity, a latent variable needs to explain its indicators’ variances to a higher degree than the variances of other latent variables (Fornell & Larcker, 1981). Accordingly, the square root of each construct’s AVE needs to exceed the construct’s correlation with all other constructs. As Table 4 shows, the highest correlation between any pair of constructs was 0.739 (between process performance and customer satisfaction), whereas the lowest square root of AVE was 0.789 (product performance). We also assessed discriminant validity by examining the factor loadings of each indicator. According to Chin (1998), each indicator needs to load higher on the associated construct compared to all other factors. In our case, factor loadings and cross-loadings (see Table 5) corroborated discriminant validity. Table 4. Scale Properties and Descriptive Statistics Inter-construct correlations (1) (2) (3) (4) Construct Mean Standard deviation Cronbach’s alpha Composite reliability AVE (1) Process transparency 5.813 1.659 0.913 0.945 0.851 0.922 (2) Process performance 4.598 2.167 0.801 0.909 0.834 0.348 0.913 (3) Product performance 5.902 0.925 0.706 0.832 0.623 0.530 0.575 0.789 (4) Customer 5.576 1.490 0.927 0.954 0.873 0.401 0.739 0.729 satisfaction Note: Diagonal elements in bold represent the square root of AVE for the respective construct 0.934 Table 5. Factor Loadings and Cross-loadings Scale items PROC1 PROC2 PROD1 PROD2 PROD3 CS1 CS2 CS3 PT1 PT2 PT3 5.2 Process performance Product performance Customer satisfaction Process transparency 0.902 0.924 0.523 0.350 0.452 0.720 0.687 0.665 0.302 0.295 0.359 0.530 0.520 0.778 0.753 0.835 0.658 0.711 0.672 0.465 0.356 0.602 0.669 0.680 0.623 0.528 0.549 0.931 0.963 0.909 0.381 0.273 0.429 0.216 0.408 0.604 0.276 0.292 0.344 0.345 0.435 0.929 0.875 0.960 Common Method Bias Because we used a self-report approach to assess the indicators of independent and dependent variables, our study may suffer from common method variance (CMV). Although method biases seem to be less serious in IS research compared to other disciplines (Malhotra, Kim, & Patil, 2006), we designed our study in a way that reduces their likelihood. Following Podsakoff, MacKenzie, Lee, and Podsakoff (2003), we reduced the biases of social desirability or respondent acquiescence by requesting them to answer each question as honestly as possible, by not providing incentives for participation, by guaranteeing anonymity, and by assuring that no right or wrong answers existed. Moreover, we included reversed formulated items to reduce the likelihood of common method variance. Additionally, we tested for common method variance downstream. Following Malhotra et al. (2006), we conducted Harmon’s singlefactor test and applied a marker-variable technique in a post hoc fashion (Lindell & Brandt, 2000). To Volume 34 Paper 22 439 Customer Satisfaction in IS Projects: Assessing the Role of Process and Product Performance apply Harman’s single-factor test, we conducted an exploratory factor analysis of all items (Podsakoff et al., 2003). Because none of the resulting four factors accounted for a majority of the variance, we assume no substantial CMV to exist. We applied the marker-variable technique to check for the correlation between theoretically uncorrelated dimensions (Malhotra et al., 2006). According to Lindell and Whitney (2001), one can use the second-smallest positive correlation between manifest variables as an indicator to assess CMV because it provides an adequate proxy. Considering that the second-smallest correlation between manifest variables in our sample amounts 0.04, we argue that CMV was not prevalent (Malhotra et al., 2006) in our study. 5.3 Hypotheses Testing Testing structural models with PLS requires a sample size of at least ten cases per predictor (Chin, 1998). The sample size is then at least ten times larger than either the highest number of indicators per scale or the highest number of paths directed at any construct in the structural model. Although our sample of 52 cases is rather low, it is sufficient to adequately calculate the model since the highest number of indicators per construct is four and the highest number of paths to any construct is two. Moreover, our study is in line with other recent studies relying on rather low sample sizes (e.g., Keil et al., 2013). Structural models are typically evaluated in terms of path coefficients and explained variances (R2). Whereas path coefficients represent the strength of relationships between independent and dependent variables, R2 values indicate the predictive power of the model. We used SmartPLS (Ringle et al., 2005) to calculate path coefficients applied SmartPLS’s bootstrapping (5000 samples) to retrieve the according t-values. Figure 2 shows the overall result (path coefficients are shown as standardized β). As we can see, our model is in line with Falk and Miller’s (1992) recommendation that, concerning nomological validity, endogenous latent constructs should provide an R2 of at least 0.10 in order to be adequately judged. R2 = 0.121 Process Performance R2 = 0.712 Process Transparency Customer Satisfaction R2 = 0.281 Product Performance Figure 2. Estimated Model of Customer Satisfaction (n = 52) According to Chin (1998), the R2 of our target construct customer satisfaction (71.2%) is substantial. The R2 values of process performance (12.1%) and product performance (28.1%) are rather low. However, process transparency is only one of the many success factors associated with process and product performance in IS projects (Byers & Blume, 1994; Kendra & Taplin, 2004; Nelson, 2007; Reel, 1999) and explaining these constructs is not our primary purpose. Using blindfolding, we calculated the predictive relevance Q2. This value indicates how far the underlying empirical data can be rebuilt using the model and PLS parameters (Fornell & Cha, 1994). We obtained a value of 0.555 (omission distance = 7; crossvalidated redundancy), which represents the predictive nature (Chin, 2010). As Figure 2 shows, the estimated model corroborates all our hypotheses. Hypothesis 1a tested a positive relationship between process performance and customer satisfaction, which was supported (standardized β = 0.660, t = 3.730, p = 0.000, Cohen’s (1988) f 2 = 0.918). Hypothesis 1b tested a positive relationship Volume 34 Paper 22 Communications of the Association for Information Systems 440 between product performance and customer satisfaction, which was supported (standardized β = 0.793, t = 5.104, p = 0.000, f 2 = 0.993). Hypothesis 2 tested a negative moderation (i.e., the relationship between product performance and the relationship between process performance and customer satisfaction), which was supported (standardized β = -0.229, t = 2.646, p = 0.008). Following the guidelines for moderating effects provided by Carte and Russell (2003), we also calculated f 2 (0.133) and the according ∆R2 (3.8%). Hypothesis 3a tested a positive relationship between process transparency and process performance, which was supported (standardized β = 0.697, t = 2.555, p = 0.011, f 2 = 0.138). Hypothesis 3b tested a positive relationship between process transparency and product performance, which was supported (standardized β = 0.401, t = 4.836, p = 0.000, f 2 = 0.391). 6 Discussion We developed and empirically tested an IS project success model that suggests customer satisfaction to be the uppermost criterion that is associated with process performance (i.e., budget and schedule) and product performance (i.e., functional and non-functional requirements). Furthermore, the model suggests product performance to moderate the relation between process performance and customer satisfaction and hypothesizes process transparency to be related to both components (i.e., process performance and product performance). As Figure 2 shows, our data from a questionnaire survey with IS experts in Germany widely corroborate the hypothesized model. Thus, we advance the understanding of measuring IS project success and contribute to a better understanding of IS development and explicitly link a success factor to success criteria (Siau et al., 2010). In this section, we discuss implications of the findings and our study’s limitations. The analysis concerning data on IS project success revealed that we can explain customer satisfaction in terms of process performance and product performance to the degree of 71.2 percent. Other aspects related to customer satisfaction might, for instance, be whether the developed project contributes to strategic goals of the customer organization (Cleland, 1986) or the efficiency of the development process (Basten et al., 2012). The latter is not to be equated with adherence to planning (which, in our model, is represented by process and product performance; see Section 2). As Section 2 explains, project plans are often incorrect. Therefore, a project performed as efficiently as possible can still not meet its plans if these were unrealistic in the first place. Empirical findings confirm the necessity to differentiate between adherence to planning and process efficiency (Basten et al., 2012). Our model shows that, even when considering customer satisfaction as the uppermost criterion of IS project success (Anderson et al., 1994; Anderson and Sullivan, 1993; Nelson, 2005), adherence to planning—the sole constant in IS project success evolution (Ika, 2009; Judgev & Müller, 2005)—is still a central component in assessing IS project success. Our findings support the call for an extended set of success criteria (Ika, 2009; Judgev & Müller, 2005) since adherence to planning in terms of process performance and product performance does not fully explain our target construct customer satisfaction (see Figure 2). Since adherence-to-planning criteria are typically perceived as easy to measure (Pinto & Slevin, 1988b) and further ones are required, our findings also corroborate Barclay’s (2008) assessment that measuring project success is a difficult endeavor. While a difference exists between the traditional adherence-to-planning criteria (Atkinson, 1999; Lech, 2013; Pinto, 2004) and the contemporary criterion of customer satisfaction (Baker et al., 2008; Ika, 2009), the traditional criteria explain the majority of the variance in customer satisfaction (see Figure 2). In line with previous research, two aspects in our analysis indicate product performance to be more important than process performance with regard to customer satisfaction. First, the relation between product performance and customer satisfaction was stronger than the one between process performance and customer satisfaction (standardized β = 0.793 vs. 0.660). Second, product performance significantly moderated the relation between process performance and customer satisfaction (standardized β = -0.229, t = 2.646). The negative path coefficient reveals that a high product performance reduces the positive influence of high process performance on customer satisfaction. We can explain the higher importance of product performance compared to process performance by the problematic nature of time and budget as success criteria, which results from projects being conceptualized based on effort estimates (Jurison, 1999). Since comparisons between planned and actual time and budget indices are only valid and reliable if underlying estimates are correct, which they rarely are (Moløkken-Østvold et al., 2004), one can consider not meeting time and budget objectives to be only marginally important. Instead, customers seem to place emphasis on product performance. Moreover, whereas time and budget are rather shortterm criteria, the developed product typically represents a long-term perspective (see Section 1), which is Volume 34 Paper 22 441 Customer Satisfaction in IS Projects: Assessing the Role of Process and Product Performance in line with researchers’ view that projects primarily initiated to realize business benefits should be evaluated in terms of according criteria (Shenhar et al., 2001). According to our model, process transparency is positively associated with both process performance and product performance (see the path coefficients and levels of significance in Figure 2). Thus, we also corroborate researchers and practitioners who claim the relevance of process transparency for IS project success (Böhm & Haselberger, 2012; Cohn, 2009; Joosten et al., 2014; Pankratz & Loebbecke, 2011; Stavrou et al., 2014). Regarding process performance, process transparency reduces the time needed to gather information and, thus, contributes to meeting time and budget objectives. If required information is available throughout the development process, more time can be spent on developing the product, which leads to increased product performance. Because process transparency concerns the availability of information to team members including the project manager throughout the development process, the relation between process transparency and process performance is stronger compared to the relationship between process transparency and product performance. Furthermore, as we mention in Section 1, transparency is not unrelated to other success factors such as communication and stakeholder involvement (Stavrou et al., 2014), which prior research has extensively emphasized (Hyväri, 2006; Petter, 2008). However, while likely to be affected by these factors, process transparency has a different focus. For instance, stakeholders can be involved to a large extent to make suggestions regarding the developed product but still feel that they are insufficiently informed about the project progress. Along those lines, there can be regular and detailed communication, which, however, might only serve information needs of specific stakeholders but neglect others who need a different kind of information and feel uninformed as a result. Thus, transparency refers to the stakeholders’ state of mind; that is, their feeling that they have all information relevant to them, which can, in turn, be achieved by several means such as communication and involvement (Stavrou et al., 2014). By illuminating the role of process transparency as a success factor, we make a first step to open the black box of IS project success (Siau et al., 2010) by explicitly analyzing the relation of success factors to success criteria. As with every empirical study, ours has limitations. First, we measured the constructs in our model from the perspective of only project managers. While process performance and product performance are likely to be adequately assessed by project managers, biases are possible concerning process transparency and customer satisfaction. Concerning the former, other project stakeholders might have different perceptions concerning the availability of necessary information. While future research might address transparency on different stakeholder levels, we believe the perspective of project managers to be sufficient in our case since they are in charge of managing budget, schedule, and requirements and since we explicitly analyze the relation between transparency and those criteria. Concerning the latter, customer satisfaction might be best assessed by asking the customer directly. For the match of project managers and customers required in this approach, we depend on project managers providing contact details of customers of the respective projects. However, our experience shows that project managers are unlikely to pass contact details of customers who are dissatisfied with the course or the outcome of the project. As such, this approach would lead to a bias towards successful projects. Consequently, we chose to rely on assessments on behalf of project managers since they have insights into the satisfaction of the customers as well. The bias towards successful projects is in this case less likely to occur to the same extent. Second, we refer to IS projects in general. However, such an approach contradicts research streams in favor of contingency theory (Donaldson, 1996; Tidd, 2001), which states that no single way of managing and organizing fits best in all situations. Accordingly, future research should assess whether our model is applicable across different project types. Finally, our sample comprises 52 projects only. However, this sample size conforms to the requirements for PLS estimation of the model. Despite the limited sample size, we are confident that we provide a substantial model of IS project success. 7 Theoretical Contributions, Management Implications, and Future Research Avenues With this study, we contribute to the literature in three ways. First, we show that, to gain customer satisfaction, attention should be paid to both process performance and product performance. Second, while demonstrating that product performance is more strongly associated with customer satisfaction compared to process performance, we also advance the understanding of IS project success by demonstrating the moderating effect of product performance on the influence of process performance. Finally, our results indicate the importance of process transparency in IS development projects and so Volume 34 Paper 22 Communications of the Association for Information Systems 442 contribute to a better understanding concerning the linkage between project success factors and project success criteria. IS project managers should more significantly emphasize developing a high-quality product that meets users’ needs instead of emphasizing adherence to budget and schedule. Also, process transparency requires significant effort as a result, for instance, of the active communication of project plans and progress. Future research should accordingly cope with the identification of means to increase process transparency and address the question of which information is predominantly required by different project team members. While most projects in our sample were conducted in financial services, the influence of process performance and product performance on customer satisfaction in other domains should be addressed in other studies. Financial services highly depend on qualitative systems, which may potentially increase the importance of product performance. As product performance is a matter of functional and non-functional requirements in this context (see Section 2), future research on this topic should explicitly differentiate between these concepts. While both process and product performance are important components of IS project success, our results indicate that product performance has a more decisive influence when it comes to achieving customer satisfaction. Volume 34 Paper 22 443 Customer Satisfaction in IS Projects: Assessing the Role of Process and Product Performance References Agarwal, N., & Rathod, U. (2006). Defining “success” for software projects: An exploratory revelation. International Journal of Project Management, 24(4), 358-370. Aladwani, A. M. (2002). An integrated performance model of information systems projects. Journal of Management Information Systems, 19(1), 185-210. Anderson, E. W., Fornell, C., & Lehmann, D. R. (1994). Customer satisfaction, market share, and profitability: Findings from Sweden. Journal of Marketing, 58(3), 53-66. Anderson, E. W., & Sullivan, M. W. (1993). The antecedents and consequences of customer satisfaction for firms. Marketing Science, 12(2), 125-143. Atkinson, R. (1999). Project management: Cost, time and quality, two best guesses and a phenomenon, its time to accept other success criteria. International Journal of Project Management, 17(6), 337342. Baccarini, D. (1999). The logical framework method for defining project success. Project Management Journal, 30(4), 25-32. Baker, B. N., Murphy, D. C., & Fisher, D. (2008). Factors affecting project success. In D. I. Cleland & W. R. King (Eds.), Project management handbook (902-919). New York: John Wiley & Sons. Banker, R. D., & Kemerer, C. F. (1992). Performance evaluation metrics for information systems development: A principal-agent model. Information Systems Research, 3(4), 379-400. Barclay, C. (2008). Towards an integrated measurement of IS project performance: The project performance scorecard. Information Systems Frontiers, 10(3), 331-345. Basten, D., Joosten, D., & Mellis, W. (2012). Managers’ perceptions of information system project success. Journal of Computer Information Systems, 52(2), 12-21. Böhm, C., & Haselberger, D. (2012). The concept of transparency in Agile project management. Retrieved from http://p-a-m.org/2012/03/the-concept-of-transparency-in-agile-project-management/ Brittain White, K., & Leifer, R. (1986). Information systems development success: Perspectives from project team participants. MIS Quarterly, 10(3), 215-223. Byers, R. C., & Blume, D. (1994). Tying critical success factors to systems development. Information & Management, 26(1), 51-61. Paulk, M. C., Weber, C. V., Curtis, B., & Chrissis, M. B. (1995). In Carnegie Mellon University Software Engineering Institution (Ed.), The capability maturity model: Guidelines for improving the software process. Reading: Addison-Wesley. Carte, A., & Russell, C. J. (2003). In pursuit of moderation: Nine common errors and their solutions. MIS Quarterly, 27(3), 479-501. Caruana, A. (2002). Service loyalty—the effects of service quality and the mediating role of customer satisfaction. European Journal of Marketing, 36(7/8), 811-828. Chin, W. W. (1998). The partial least squares approach to structural equation modeling. In G. A. Marcoulides (Ed.), Modern methods for business research (pp. 295-358). Mahwah: Erlbaum. Chin, W. W. (2010). How to write up and report PLS analyses. In V. Esposito Vinzi, W. W. Chin, J. Henseler, & H. Wang (Eds.), Handbook of partial least squares: Concepts, methods and application (pp. 645-689). Berlin: Springer. Chin, W. W., Marcolin, B. L., & Newsted, P. R. (2003). A partial least squares latent modeling approach for measuring interaction effects. Results from a monte carlo simulation study and an electronicmail emotion / adoption study. Information Systems Research, 14(2), 189-217. Cleland, D. I. (1986). Measuring success: The owner’s viewpoint. In Project Management Institute (Ed.), Proceedings of the 18th Annual Seminar/Symposium (pp. 6-12). Upper Darby: Project Management Institute. Cohen, J. W. (1988) Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale: Erlbaum. Volume 34 Paper 22 Communications of the Association for Information Systems 444 Cohn, M. (2009) Succeeding with Agile: Software development using Scrum. Upper Saddle River: Addison-Wesley. Collins, A., & Baccarini, D. (2004). Project success—a survey. Journal of Construction Research, 5(2), 211-231. Cooke-Davies, T. (2002). The “real” success factors on projects. International Journal of Project Management, 20(3), 185-190. Cuellar, M. (2010). Assessing project success: Moving beyond the triple constraint. In AIS Special Interest Group on Information Technology Project Management (Ed.), International research workshop on IT project management (pp. 19-28). AIS Electronic Library. DeCotiis, T. A., & Dyer, L. (1977). The dimensions and determinants of project performance. Industrial Marketing Management, 6(5), 370-378. DeLone, W. H., & McLean, E. R. (1992). Information system success: The quest for the dependent variable. Information Systems Research, 3(1), 60-95. DeLone, W. H., & McLean, E. R. (2003). The DeLone and McLean model of information systems success: A ten-year update. Journal of Management Information Systems, 19(4), 9-30. Donaldson, L. (1996). For positivist organization theory. London: Sage. Dvir, D., & Lechler, T. (2004). Plans are nothing, changing plans is everything: The impact of changes on project success. Research Policy, 33(1), 1-15. Falk, R. F., & Miller, N. B. (1992). A primer for soft modeling (1st ed.). Akron, Ohio: University of Akron Press. Fornell, C., & Cha, J. (1994). Partial least squares. In R. P. Bagozzi (Ed.), Advanced methods of marketing research (pp. 52-78). Cambridge: Basil Blackwell. Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39-50. Ginzberg, M. J. (1981). Early diagnosis of MIS implementation failure: Promising results and unanswered questions. Management Science, 27(4), 459-478. Glass, R. L. (1999). Evolving a new theory of project success. Communications of the ACM, 42(11), 1719. Gravetter, F. J., & Forzano, L.-A. B. (2012). Research methods for the behavioral sciences (4th ed.). Belmont, CA: Cengage Learning. Hair, J. F., Ringle, C. M., & Sarstedt, M. (2011). PLS-SEM: Indeed a silver bullet. The Journal of Marketing Theory and Practice, 19(2), 139-152. Henseler, J., Ringle, C. M., & Sinkovics, R. R. (2009). The use of partial least squares path modeling in international marketing. Advances in International Marketing, 20(IV), 277-319. Hyväri, I. (2006). Success of projects in different organizational conditions. Project Management Journal, 37(4), 31-41. Ika, L. A. (2009). Project success as a topic in project management journals. Project Management Journal, 40(4), 6-19. Joosten, D., Basten, D., & Mellis, W. (2014). Measurement of information system project success in German organizations. International Journal of Information Technology Project Management, 5(3), 1-20. Jørgensen, M. (2004). Regression models of software development effort estimation accuracy and bias. Empirical Software Engineering, 9(4), 297-314. Judgev, K., & Müller, R. (2005). A retrospective look at our evolving understanding of project success. Project Management Journal, 36(4), 19-31. Jurison, J. (1999). Software project management: The manager's view. Communications of the Association for Information Systems, 2, 1-57. Volume 34 Paper 22 445 Customer Satisfaction in IS Projects: Assessing the Role of Process and Product Performance Kappelman, L. A., McKeeman, R., & Zhang, L. (2006). Early warning signs of IT project failure: The dominant dozen. Information Systems Management, 23(4), 31-36. Keil, M., Rai, A., & Liu, S. (2013). How user risk and requirements risk moderate the effects of formal and informal control on the process performance of IT projects. European Journal of Information Systems, 22(6), 650-672. Kendra, K., & Taplin, L. J. (2004). Project success: A cultural framework. Project Management Journal, 35(1), 30-45. Kraut, R. E., & Streeter, L. A. (1995). Coordination in software development. Communications of the ACM, 38(3), 69-81. Laudon, K. C., & Laudon, J. P. (2009). Management information systems. Managing the digital firm (11th ed.). Upper Saddle River: Pearson. Lech, P. (2013). Time, budget, and functionality?—IT project success criteria revised. Information Systems Management, 30(3), 263-275. Lederer, A. L., Mirani, R., Neo, B. S., Pollard, C., Prasad, J., & Ramamurthy, K. (1990). Information system cost estimating: A management perspective. MIS Quarterly, 14(2), 159-176. Lindell, M. K., & Brandt, C. J. (2000). Climate quality and climate consensus as mediators of the relationship between organizational antecedents and outcomes. Journal of Applied Psychology, 85(3), 331-348. Lindell, M. K., & Whitney, D. J. (2001). Accounting for common method variance in cross-sectional research designs. Journal of Applied Psychology, 86(1), 114-121. Lohmöller, J.-B. (1989). Latent variable path modeling with partial least squares. Heidelberg: PhysicaVerlag. Malhotra, N. K., Kim, S. S., & Patil, A. (2006). Common method variance in IS research: A comparison of alternative approaches and a reanalysis of past research. Management Science, 52(12), 18651883. Mao, J.-Y., Lee, J.-N., & Deng, C.-P. (2008). Vendors’ perspectives on trust and control in offshore information systems outsourcing. Information & Management, 45(7), 482-492. Moløkken-Østvold, K., Jørgensen, M., Tanilkan, S. S., Gallis, H., Lien, A. C., & Hove, S. E. (2004). A survey on software estimation in the Norwegian industry. In IEEE Computer Society (Ed.), Proceedings of the 10th International Symposium on Software Metrics (pp. 208-219). New York: IEEE Computer Society Press. Myers, M. D. (1995). Dialectical hermeneutics: A theoretical framework for the implementation of information systems. Information Systems Journal, 5(1), 51-70. Narayanaswamy, R., Grover, V., & Henry, R. M. (2013). The impact of influence tactics in information system development projects: A control-loss perspective. Journal of Management Information Systems, 30(1), 191-226. Nelson, R. (2005). Project retrospectives: Evaluating project success, failure, and everything in between. MIS Quarterly Executive, 4(3), 361-372. Nelson, R. (2007). IT project management: Infamous failures, classic mistakes, and best practices. MIS Quarterly Executive, 6(2), 67-78. Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York: McGraw-Hill. Ooi, K.-B., Lin, B., Tan, B.-I., Chong, A. Y.‐L. (2011). Are TQM practices supporting customer satisfaction and service quality? Journal of Services Marketing, 25(6), 410-419. Pankratz, O., & Loebbecke, C. (2011). Project managers’ perception of is project success factors—a repertory grid investigation. In V. Tuunainen, J. Nandhakumar, M. Rossi, & W. Soliman (Eds.), Proceedings of the 19th European Conference on Information Systems. Petter, S. (2008). Managing user expectations on software projects: Lessons from the trenches. International Journal of Project Management, 26(7), 700-712. Volume 34 Paper 22 Communications of the Association for Information Systems 446 Pinto, J. K. (2004). The elements of project success. In D. I. Cleland (Ed.), Field guide to project management (pp. 14-27). Hoboken: Wiley. Pinto, J. K., & Slevin, D. (1988a). Critical success factors across the project life cycle. Project Management Journal, 19(3), 67-75. Pinto, J. K., & Slevin, D. (1988b). Project success: Definitions and measurement techniques. Project Management Journal, 19(1), 67-72. Podsakoff, P. M., MacKenzie, S. B., Lee, J. Y., & Podsakoff, N. P. (2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5), 879-903. Project Management Institute. (2008). A guide to the project management body of knowledge (4th ed.). Newton Square: Project Management Institute. Rai, A., & Al-Hindi, H. (2000). The effects of development process modeling and task uncertainty on development quality performance. Information & Management, 37(6), 335-346. Reel, J. S. (1999). Critical success factors in software projects. IEEE Software, 16(3), 18-23. Ringle, C. M., Wende, S., & Will, S. (2005). SmartPLS 2.0 (M3) beta. Retrieved from http://www.smartpls.de. Saarinen, T., & Sääksjärvi, M. (1992). Process and product success in information systems development. Journal of Strategic Information Systems, 1(5), 266-275. Shenhar, A. J., Dvir, D., Levy, O., & Maltz, A. C. (2001). Project success: A multidimensional strategic concept. Long Range Planning, 34(6), 699-725. Shenhar, A. J., Levy, O., & Dvir, D. (1997). Mapping the dimensions of project success. Project Management Journal, 28(2), 5-13. Siau, K., Long, Y., & Ling, M. (2010). Toward a unified model of information systems development success. Journal of Database Management, 21(1), 80-101. Stavrou, G., Pankratz, O., & Basten, D. (2014). Increasing customer satisfaction—how to manage expectations in the process of developing information systems. In AIS Special Interest Group for Information Technology Project Management (Ed.), International research workshop on IT project management (pp. 14-27). Thomas, G., & Fernández, W. (2008). Success in IT projects: A matter of definition. International Journal of Project Management, 26(7), 733-742. Tidd, J. (2001). Innovation management in context: Environment, organization and performance. International Journal of Management Reviews, 3(3), 169-183. Wallace, L., Keil, M., & Rai, A. (2004). How software project risk affects project performance: An investigation of the dimensions of risk and an exploratory model. Decision Sciences, 35(2), 289321. Wateridge, J. (1995). IT projects: A basis for success. International Journal of Project Management, 13(3), 169-172. Werts, C. E., Linn, R. L., & Jöreskog, K. G. (1974). Intraclass reliability estimates: Testing structural assumptions. Educational and Psychological Measurement, 34(1), 25-33. Westerveld, E. (2003). The project excellence model®: Linking success criteria and critical success factors. International Journal of Project Management, 21(6), 411-418. Yetton, P., Martin, A., Sharma, R., & Johnston, K. (2000). A model of information systems development project performance. Information Systems Journal, 10(4), 263-289. Volume 34 Paper 22 447 Customer Satisfaction in IS Projects: Assessing the Role of Process and Product Performance About the Authors Dirk Basten is an Assistant Professor at the Department of Information Systems and Systems Development at the University of Cologne, Germany. His research focuses on IS project success, software development effort estimation, knowledge management in project and program contexts, and organizational agility. His research has been published in journals such as Communications of the Association for Information Systems, IEEE Computer, Journal of Computer Information Systems, International Journal of Information Technology Project Management, and the proceedings of conferences such as HICSS, ECIS, AMCIS, and ICIS. Oleg Pankratz is a postdoc researcher at the Department of Information Systems and Systems Development at the University of Cologne, Germany. His research focuses on IS project success and has been published in International Journal of Information Systems and Project Management and the proceedings of conferences such as HICSS, ECIS, AMCIS, and ICIS. Copyright © 2015 by the Association for Information Systems. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and full citation on the first page. Copyright for components of this work owned by others than the Association for Information Systems must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists requires prior specific permission and/or fee. Request permission to publish from: AIS Administrative Office, P.O. Box 2712 Atlanta, GA, 30301-2712 Attn: Reprints or via email from publications@aisnet.org. Volume 34 Paper 22