article

Assessing common method bias: problems with the ULMC technique

Authors:

Jason Bennett Thatcher,

Ryan T. WrightAuthors Info & Claims

MIS Quarterly, Volume 36, Issue 3

Pages 1003 - 1019

Published: 01 September 2012 Publication History

Abstract

Recent work, in journals such as MIS Quarterly and Management Science, has highlighted the importance of evaluating the influence of common method bias (CMB) on the results of statistical analysis. In this research note, we assess the utility of the unmeasured latent method construct (ULMC) approach in partial least squares (PLS), introduced by Liang et al. (2007). Such an assessment of the ULMC approach is important, because it has been employed in 76 studies since it appeared in MIS Quarterly in early 2007. Using data generated via Monte Carlo simulations, we use PLS structural equation modeling (SEM) to demonstrate that the ULMC approach of Liang et al. is neither able to detect, nor control for, common method bias. Method estimates using this approach resulted in negligible estimates, regardless of whether there were some, large, or no method bias introduced in the simulated data. Our study contributes to the IS and research methods literature by illustrating that, and explaining why the ULMC approach does not accurately detect common method bias in PLS. Further, our results build on prior work done using covariance-based SEM questioning the usefulness of the ULMC technique for detecting CMB.

References

[1]

Bagozzi, R. P. 2011. "Measurement and Meaning in Information Systems and Organizational Research: Methodoloigcal and Philosophical Foundations," MIS Quarterly (35:2), pp. 261-292.

[2]

Bollen, K. A. 1989. Structural Equations with Latent Variables, New York: Wiley.

[3]

Burton-Jones, A. 2009. "Minimizing Method Bias Through Programmatic Research," MIS Quarterly (33:3), pp. 445-471.

[4]

Chin, W. W. 1998a. "Issues and Opinion on Structural Equation Modeling.," MIS Quarterly (22:1), p. vii.

[5]

Chin, W. W. 1998b. "The Partial Least Squares Approach to Structural Equation Modeling," in Modern Methods for Business Research, G. A. Marcoulides (ed.), Mahwah, NJ: Lawrence Erlbaum Associates, pp. 295-336.

[6]

Chin, W. W. 2010. "How to Write Up and Report PLS Analyses," in Handbook of Partial Least Squares Concepts, Methods and Applications, V. E. Vinzi, W. W. Chin, J. Henseler, and H. Wang (eds.), Berlin: Springer-Verlag, pp. 650-690.

[7]

Chin, W. W., and Gopal, A. 1995. "Adoption Intention in GSS: Relative Importance of Beliefs," Data Base (26:2/3), pp. 42-64.

[8]

Chin, W. W., Marcolin, B. L., and Newsted, P. R. 2003. "A Partial Least Squares Latent Variable Modeling Approach for Measuring Interaction Effects: Results from a Monte Carlo Simulation Study and an Electronic-Mail Emotion/Adoption Study," Information Systems Research (14:2), pp. 189-217.

[9]

Cohen, J. 1988. Statistical Power Analysis for the Behavioral Sciences (1st ed.), New York: Academic Press.

[10]

Cote, J. A., and Buckley, M. R. 1987. "Estimating Trait, Method, and Error Variance: Generalizing Across 70 Construct Validation Studies," Journal of Marketing Research (25), pp. 315-318.

[11]

Davis, F. D. 1989. "Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology," MIS Quarterly (13:3), pp. 319-340.

[12]

Doty, D. H., and Glick, W. H. 1998. "Common Method Bias: Does Common Method Variance Really Bias Results?," Organizational Research Methods (1:4), pp. 374-406.

[13]

Goodhue, D., Lewis, W., and Thompson, R. L. 2007. "Statistical Power in Analyzing Interaction Effects: Questioning the Advantage of PLS with Product Indicators," Information Systems Research (18:2), pp. 211-227.

[14]

Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E., and Tatham, R. L. 2006. Multivariate Data Analysis, Upper Saddle River, NJ: Prentice Hall.

[15]

Jöreskog, K. G., and Sorbom, D. 1993. PRELIS 2 User's Reference Guide, Lincolnwood, IL: Scientific Software International.

[16]

Kline, T. 2005. Psychological Testing: A Practical Approach to Design and Evaluation (1st ed.), London: Sage Publications.

[17]

Liang, H., Saraf, N., Hu, Q., and Xue, Y. 2007. "Assimilation of Enterprise Systems: The Effect of Institutional Pressures and the Mediating Role of Top Management," MIS Quarterly (31:1), pp. 59 - 87.

[18]

Lindell, M., and Whitney, D. 2001. "Accounting for Common Method Variance in Cross-Sectional Research Designs," Journal of Applied Psychology (86:1), pp. 114-121.

[19]

Little, T. D., Cunningham, W. A., Shahar, G., and Widaman, K. F. 2002. "To Parcel or Not to Parcel: Exploring the Question, Weighing the Merits," Structural Equation Modeling: A Multidisciplinary Journal of Applied Psychology (9:2), pp. 151-173.

[20]

Lohmöller, J.-B. 1989. Latent Variable Path Modeling with Partial Least Squares, Heidelberg, Germany: Physica-Verlag.

[21]

Malhotra, N. K., Kim, S. S., and Patil, A. 2006. "Common Method Variance in IS Research: A Comparison of Alternative Approaches and a Reanalysis of Past Research," Management Science (52:12), pp. 1865-1883.

Digital Library

[22]

Mattson, S. 1997. "How to Generate Non-Normal Data for Simulation of Structural Equation Models," Multivariate Behavioral Research (32), pp. 355-373.

[23]

Murphy, K. R., Cleveland, J. N., Skattebo, A. L., and Kinney, T. B. 2004. "Raters Who Pursue Different Goals Give Different Ratings," Journal of Applied Psychology (89:1), pp. 158-164.

[24]

Pace, V. L. 2010. "Method Variances From the Perspectives of Reviewers: Poorly Understood Problem or Overemphasized Complaint?," Organizational Research Methods (13:3), pp. 421-434.

[25]

Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y., and Podsakoff, N. P. 2003. "Common Method Biases in Behavioral Research: A Critical Review of the Literature and Recommended Remedies," Journal of Applied Psychology (88), pp. 879-903.

[26]

Reinartz, W. J., Echambadi, R., and Chin, W. W. 2002. "Generating Non-Normal Data for Simulation of Structural Equation Models Using Mattson's Method," Multivariate Behavioral Research (37:2), pp. 227-244.

[27]

Richardson, H. A., Simmering, M. J., and Sturman, M. C. 2009. "A Tale of Three Prespectives: Examining Post Hoc Statisical Techniques for Detection and Correction of Common Method Variance," Organizational Research Methods (12:4), pp. 762-800.

[28]

Sharma, R., Yetton, P., and Crawford, J. 2009. "Estimating the Effect of Common Method Variance: The Method-Method Pair Technique with an Illustration from TAM Research," MIS Quarterly (33:3), pp. 491-512.

[29]

Siemsen, E., Roth, A., and Oliveira, P. 2010. "Common Method Bias in Regression Models with Linear, Quadratic, and Interaction Effects," Organizational Research Methods (13:3), pp. 456-476.

[30]

Spector, P. E. 2006. "Method Variance in Organizational Research: Truth or Urban Legend?," Organizational Research Methods (9), pp. 221-232.

[31]

Vance, A., Elie-Dit-Cosaque, C., and Straub, D. W. 2008. "Examining Trust in Information Technology Artifacts: The Effects of System Quality and Culture," Journal of Management Information Systems (24:4), pp. 73-100.

Digital Library

[32]

Williams, L. J., and Brown, B. K. 1994. "Method Variance in Organizational Behavior and Human Resources Research: Effects on Correlations, Path Coefficients, and Hypothesis Testing," Organizational Behavior and Human Decision Processes (57), pp. 185-209.

[33]

Williams, L. J., Cote, J. A., and Buckley, M. R. 1989. "Lack of Method Variance in Self-Reported and Perceptions at Work: Reality or Artifact?," Journal of Applied Psychology (74), pp. 323-331.

[34]

Williams, L. J., Edwards, J. R., and Vandenberg, R. J. 2003. "Recent Advances in Causal Modeling Methods for Organizational and Management Research," Journal of Management (29:6), pp. 903-936.

[35]

Williams, L. J., Hartman, N., and Cavazotte, F. 2010. "Method Variance and Marker Variables: A Review and Comprehensive CFA Marker Technique," Organizational Research Methods (13), pp. 477-514.

[36]

Wold, H. 1985. "Partial Least Squares," in Encyclopedia of Statistical Sciences, S. Kotz and N. L. Johnson (eds.), New York: Wiley, pp. 581-591.

Cited By

Zou HSun HFang Y(2023)Satisfaction to Stay, Regret to SwitchInformation Systems Research10.1287/isre.2022.118834:4(1455-1475)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1287/isre.2022.1188
Hsu CLee JFang YStraub DSu NRyu H(2022)The Role of Vendor Legitimacy in IT Outsourcing PerformanceInformation Systems Research10.1287/isre.2021.105933:1(337-361)Online publication date: 1-Mar-2022
https://dl.acm.org/doi/10.1287/isre.2021.1059
Cram WD'Arcy JProudfoot J(2019)Seeing the forest and the treesMIS Quarterly10.25300/MISQ/2019/1511743:2(525-554)Online publication date: 1-Jun-2019
https://dl.acm.org/doi/10.25300/MISQ/2019/15117
Show More Cited By

Recommendations

Common Method Bias in PLS-SEM: A Full Collinearity Assessment Approach

The author discusses common method bias in the context of structural equation modeling employing the partial least squares method PLS-SEM. Two datasets were created through a Monte Carlo simulation to illustrate the discussion: one contaminated by ...
Advanced Mediating Effects Tests, Multi-Group Analyses, and Measurement Model Assessments in PLS-Based SEM

Use of the partial least squares PLS method has been on the rise among e-collaboration researchers. It has also seen increasing use in a wide variety of fields of research. This includes most business-related disciplines, as well as the social and ...
Assessing the effects of consumer involvement and service quality in a self-service setting

The aim of this study is to assess how consumer involvement and service quality within the Technology Acceptance Model (TAM) influence consumers' adoption of self-service technologies (SSTs). A model is proposed to test the antecedent factors of SSTs in ...

Comments

Information & Contributors

Information

Published In

cover image MIS Quarterly

MIS Quarterly Volume 36, Issue 3

September 2012

338 pages

ISSN:0276-7783

Issue’s Table of Contents

Publisher

Society for Information Management and The Management Information Systems Research Center

United States

Publication History

Published: 01 September 2012

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

24
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 14 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zou HSun HFang Y(2023)Satisfaction to Stay, Regret to SwitchInformation Systems Research10.1287/isre.2022.118834:4(1455-1475)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1287/isre.2022.1188
Hsu CLee JFang YStraub DSu NRyu H(2022)The Role of Vendor Legitimacy in IT Outsourcing PerformanceInformation Systems Research10.1287/isre.2021.105933:1(337-361)Online publication date: 1-Mar-2022
https://dl.acm.org/doi/10.1287/isre.2021.1059
Cram WD'Arcy JProudfoot J(2019)Seeing the forest and the treesMIS Quarterly10.25300/MISQ/2019/1511743:2(525-554)Online publication date: 1-Jun-2019
https://dl.acm.org/doi/10.25300/MISQ/2019/15117
Aguirre-Urreta MHu J(2019)Detecting Common Method BiasACM SIGMIS Database: the DATABASE for Advances in Information Systems10.1145/3330472.333047750:2(45-70)Online publication date: 6-May-2019
https://dl.acm.org/doi/10.1145/3330472.3330477
Müller LMattke JMaier CWeitzel T(2019)Conversational Agents in Healthcare: Using QCA to Explain Patients’ Resistance to Chatbots for MedicationChatbot Research and Design10.1007/978-3-030-39540-7_1(3-18)Online publication date: 19-Nov-2019
https://dl.acm.org/doi/10.1007/978-3-030-39540-7_1
Gu RJiang ZOh LWang K(2018)Exploring the Impact of Optimum Stimulation Level on Individual Perceptions of IT InnovationsACM SIGMIS Database: the DATABASE for Advances in Information Systems10.1145/3184444.318445049:1(67-92)Online publication date: 1-Feb-2018
https://dl.acm.org/doi/10.1145/3184444.3184450
Queiroz MTallon PSharma RColtman T(2018)The role of IT application orchestration capability in improving agility and performanceThe Journal of Strategic Information Systems10.1016/j.jsis.2017.10.00227:1(4-21)Online publication date: 1-Mar-2018
https://dl.acm.org/doi/10.1016/j.jsis.2017.10.002
Moon YChoi MArmstrong D(2018)The impact of relational leadership and social alignment on information security system effectiveness in Korean governmental organizationsInternational Journal of Information Management: The Journal for Information Professionals10.1016/j.ijinfomgt.2018.01.00140:C(54-66)Online publication date: 1-Jun-2018
https://dl.acm.org/doi/10.1016/j.ijinfomgt.2018.01.001
Yoon C(2018)Extending the TAM for Green ITComputers in Human Behavior10.1016/j.chb.2018.01.03283:C(129-139)Online publication date: 1-Jun-2018
https://dl.acm.org/doi/10.1016/j.chb.2018.01.032
Schwarz ARizzuto TCarraher-Wolverton CRoldán JBarrera-Barrera R(2017)Examining the Impact and Detection of the "Urban Legend" of Common Method BiasACM SIGMIS Database: the DATABASE for Advances in Information Systems10.1145/3051473.305147948:1(93-119)Online publication date: 8-Feb-2017
https://dl.acm.org/doi/10.1145/3051473.3051479
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents