Structural Equation Modeling (SEM)

Khan Sarfaraz Ali Structural Equation Modeling (SEM) as a statistical technique has increased in popularity since it was first conceived by Wright (1918), a biometrician who developed the path analysis method to analyze genetic theory in biology. SEM enjoyed a renaissance in the early 1970s, particularly in sociology and econometrics (Goldberger & Duncan, 1973), and later spread to other disciplines, such as psychology, political science, and education (Kenny, 1979; Werts & Linn, 1970). It was believed that the growth and popularity of SEM was attributed to a large part to the advancement of software development that have made SEM readily accessible to substantive researchers who have found this method to be well-suited to addressing a variety of research questions (MacCallum & Austin, 2000). Some examples of such software include LISREL (LInear Structural RELations), EQS, AMOS (Analysis of Moment Structures), and MPlus. The combination of methodological advances and improved interfaces in various SEM software has resulted in the diverse usage of SEM. Hershberger (2003) examined major journals in psychology from 1994 to 2001 and found that over 60% of these journals contained articles using SEM, more than doubled the number of articles published from 1985 to 1994. Although SEM continues to undergo refinement and extension, it continues to be popular among applied researchers in other fields. Some examples include communication sciences (Holbert & Stephenson, 2002), operations management (Shah & Goldstein, 2006), and counseling psychology (Martens, 2005). Michael , Holbert and Zimmerman (2006) mentioned, structural equation modeling (SEM) is a multivariate technique suited for testing proposed relations between variables. In this article, the authors discuss the potential for SEM as a tool to advance health communication research both statistically and conceptually. Specifically, the authors discuss the advantages that latent variable modeling in SEM affords researchers by extracting measurement error. In addition, they argue that SEM is useful in understanding communication as a complex set of relations between variables. Moreover, the authors articulate the possibility for examining communication as an agent, mediator, and an outcome. Finally, they review the application of SEMto recursive models, interactions, and confirmatory factor analysis. According to Khine and Timothy (2009), in recent years, the use of SEM has increased among educational researchers. To a large part, this could be attributed to the easy access to SEM literature and resources. A simple search on the internet would reveal numerous links to websites that contains teaching and learning materials, bibliographies, journal articles, blogs, book reviews, and free demo versions of SEM software, to name a few. Such preponderance of freely available resources serves not only to initiate new and potential users but also allow current and experience researchers to update themselves of the latest thinking and findings on SEM-related issues. As more educational researchers take to SEM as a tool to analyze complex relationships that exist, more dedicated resources are needed to meet their needs. This book aspires to be one such resource. It brings together the major thinkers and theorists in SEM who draws from their wealth of experience and knowledge to share ideas that are considered to be progressive in the application of SEM. This process of SEM is, as described in the wonderful textbook by John Loehlin (2004), not unlike the crafting of an essay: So long as we want to try to describe complex real-life phenomena as they occur in their natural settings, it seems that our

PhD (Fellow), Universiti Utara Malaysia By: Khan Sarfaraz Ali / sarfarazbim@gmail.com / 01714899542 / 01817528067

Page 1 of 7

chief alternatives are the literary essay and the path model (p. 232).To paraphrase Byrne (1994), Structural Equation Modeling (SEM) is a statistical methodology that takes a hypothesis-testing (i.e. confirmatory) approach to the multivariate analysis. Contributions to the SEM literature over the last three decades have really focused on enhancing our ability to make our case. Certainly much work has taken on the task of expanding this form of argument to be able to address new types of research questions and challenges. Other works have tackled the methods underlying assumptions, either establishing when our inferences are robust to their violations or devising new processes that effectively circumvent such tenuous foundations. Considerable effort has also be directed, especially in the last decade or so, toward the clear communication of SEM and its best practices in order to ensure that our processes of argument are as immune as possible to our overly strong foresights, and to our overly tempting hind sights (Khine & Timothy, 2009). Over the last two decades, structural equation modeling (SEM) has become a common statistical tool for modeling relationships between variables which cannot be observed directly, but only with measurement error. The relationships between these unobservable, latent variables are formulated in structural equations, and they are measured with errors by indicator variables in a measurement model. By the development of software packages for covariance structure analysis such as Amos (Arbuckle, 1997), EQS (Bentler, 1995; Bentler & Wu, 1993), LISREL (Jreskog & Srbom, 1993, 1996), and Mplus (Muthn & Muthn, 1998-2007), SEM has become available to a large community of researchers. SEM is used to estimate simultaneously a system of hypothesized relationships among observable and latent variables to determine whether these associations are consistent with an obtained sample of data (Bollen, 1989). This multivariate analytical technique emerged from three separate lines of mathematical and statistical analysis: path analysis, factor analysis, and simultaneous equation modeling. Wright (1934) offered the first breakthrough in path analysis by demonstrating how the associations among variables are related to model parameter estimates that take the form of a path diagram (Kaplan, 2000). As Wright was refining his method of path analysis, Thurstone (1947) was advancing the study of factor analysis (see Mulaik, 1972, for a review). Similarly, individuals in the field of econometrics began developing and refining a technique of simultaneous equation modeling (see Blalock, 1971). Thework of Karl Jreskog bridged these three areas (Cudeck, du Toit, & Srbom, 2001). His earliest contribution in the development of SEM, confirmatory factor analysis (CFA; Jreskog, 1967, 1969), linked previous work by Lawley on maximum likelihood and restricted factor analysis to create the basic measurement tool that is common to all major SEM programs (i.e., LISREL, EQS, AMOS; Jreskog & Lawley, 1968). However, the present-day technique has evolved beyond the study of just measurement models to become a melding of factor analysis and path analysis (Kaplan, 2000, p. 3). Indeed, SEM has been called the single most important contribution of statistics to the social and behavioral sciences during the past twenty years (Lomax, 1989, p. 171). There are two common components to a structural equation model: the measurement model and the structural model. The measurement model analyzes relationships among a set of observable variables and a predetermined number of latent variables. Observable variables are those collected in the researchers measurement instrument, while latent variables exist beyond human measurement. The use of latent variables cannot be overemphasized because, All observation is fallible, no matter how refined the measuring instrument we never measure exactly the true variables discussed in our theories. In this same strict sense, all (true) variables are unobserved (Duncan, 1975, p. 113). The associations among the observable variables and latent variables in a model are established a priori and tested against a data set to see if the hypothesized measurement relationships match
By: Khan Sarfaraz Ali / sarfarazbim@gmail.com / 01714899542 / 01817528067 Page 2 of 7

the data that have been collected. Beyond the associations analyzed by the measurement model, the structural component of the model analyzes a series of a priori relationships established between latent variables. The structural component of the model can also analyze relationships among observable variables if a researcher is treating all single items or additive indexes as observable. The latter scenario is oftentimes referred to as path analysis (Kline, 1998), but SEM-based analyses of this type are distinct from earlier forms of path analysis given the more common use of full-information estimation procedures in SEM software. There are three primary steps in SEM: specification, estimation, and evaluation. Specification involves (among other things) identifying the set of relationships one wants to examine, determining how to specify these variables within the model (see Stephenson & Holbert, 2003), and screening the data set so that it adheres to the assumptions of the statistical theory underlying SEM analyses. Estimation is the first logical step that follows specification (Chou & Bentler, 1995, p. 37). In this step, researchers make decisions about what data structure will be used (i.e., co-variances, correlations, or raw data), partial- versus full information estimation procedures, and how to evaluate the hypothesized model against a data set. In a social science research context, a linear model structure sometimes provides only a questionable representation of reality. This is particularly the case when in a cross-sectional investigation the relationship between a predictor and the criterion variable itself depends on the outcome of a third variable. While ordinary SEM incorporates linear relationships among latent variables, models with nonlinear structural equations have attracted increasing attention over the last decade. Several researchers have called for estimation methods for nonlinear latent variable models, and numerous substantive theories in education and psychology call for analysis of nonlinear models (Ajzen, 1987; Ajzen & Fishbein, 1980; Ajzen & Madden, 1986; Cronbach, 1975; Cronbach & Snow, 1977; Karasek, 1979; Lusch & Brown, 1996; Snyder & Tanke, 1976). Also, a need for nonlinear extensions of ordinary SEM has been expressed from a methodological perspective (Aiken & West, 1991; Busemeyer & Jones, 1983; Cohen & Cohen, 1975; Jaccard, Turisi & Wan, 1990), and different estimation approaches have been proposed: Approaches that involve the forming of products of indicators (Hayduk, 1987; Bollen, 1995, 1996; Joereskog & Yang, 1996, 1997; Marsh, Wen & Hau, 2004), a moment-based 2-step estimation technique (Wall & Amemiya, 2003), a maximum likelihood estimation (Klein & Moosbrugger, 2000), and an approximate ML estimation (Klein & Muthn, 2007). With the LMS (latent moderated structural equations) method, Klein and Moosbrugger (2000) introduced a maximum likelihood estimation technique for latent interaction models with multiple latent product and quadratic terms. LMS does not require the researcher to form product indicators for the latent product terms, and it takes the mathematical structure of the nonnormal distributions that are implied by the product terms into account. In this approach, the latent independent variables and the error variables are assumed to be normally distributed. In contrast to non-ML estimation methods, LMS also allows for likelihood ratio model difference tests which can test for the significance of one or several nonlinear effects simultaneously. LMS has been implemented in the Mplus structural equation modeling software (Muthn & Muthn, 1998-2007). Tempelaar, Loeff and Gijselaers (2007) described that a SEM model is distinct from a path or regression model in that it hypothesizes that crucial variables, such as attitudes in this study, are not directly observable and are better modeled as latent variables than as observable ones. In doing so, a SEM model makes it possible to distinguish two different types of errors: errors in equations, as does the path model, and errors in the observation of variables. Making this distinction is especially worthwhile when errors in important constructs have rather different sizes. Studying reliabilities of several achievement motivations, and their variation
By: Khan Sarfaraz Ali / sarfarazbim@gmail.com / 01714899542 / 01817528067 Page 3 of 7

over subjects, suggests that this argument applies to this study. The standard approach to estimate a SEM distinguishes two steps (Schumacker & Lomax, 2004). In the first phase of the two-step model building approach, measurement models for all latent variables in the model are estimated. In the second model building step, the structural part of the SEM is estimated. This structural part specifies the relationships between the independent and dependent latent variables. In contrast to the estimation of the measurement models, the estimation of structural relationships is to some extent explorative in nature. The structural part of the full structural equation model is not a priori restricted, except for several hypotheses with regard to the direction of the relationship. For the estimation of these structural parts, two different model modification procedures are applied. The first is called model trimming (Kline, 2005) or backward search (Schumacker & Lomax, 2004). Starting from a full matrix of structural path coefficients, one by one, parameters are restricted to zero if they prove non-significant, until all remaining structural parameters are significant. The second approach is called model building (Kline, 2005) or forward search (Schumacker & Lomax, 2004). It starts from a zero matrix of structural paths coefficients, and frees parameters one by one, in the order indicated by the value of the modification indices, up to point where no more significant improvement in fit is achieved. Because in both approaches subsequent models are nested, the chi-square difference statistic can be used to assess model fit. In all five subjects, both forward and backward searches converge to the same final model. Model modification is a form of explorative analysis, and brings along the risk of capitalization on chance. References Aiken, L. S., & West, S. G. (1991). Multiple regression: Testing and interpreting interactions. Newbury Park: Sage Publications. Ajzen, I. (1987). Attitudes, traits, and actions: Dispositional prediction of behavior in personality and social psychology. In L. Berkowitz (Ed.), Advances in experimental social psychology (Vol. 20, pp. 163). New York: Academic. Ajzen, I., & Fishbein, M. (1980). Understanding attitudes and predicting social behavior. Englewood Cliffs, NJ: Prentice-Hall. Ajzen, I., & Madden, T. J. (1986). Prediction of goal-directed behavior: Attitudes, intentions, and perceived behavioral control. Journal of Experimental Social Psychology, 22, 453 474. Arbuckle, J. L. (1997). AMOS users guide version 3.6. Chicago: Small Waters Corporation. Bentler, P. M. (1995). EQS structural equations program manual. Encino, CA: Multivariate Software. Bentler, P. M., & Wu, E. J. C. (1993). EQS/Windows users guide. Los Angeles: BMDP Statistical Software. Blalock, H. M., Jr. (Ed.). (1971). Causal models in the social sciences. Chicago: Aldine. Bollen, K. A. (1989). Structural equations with latent variables. New York: Wiley.
By: Khan Sarfaraz Ali / sarfarazbim@gmail.com / 01714899542 / 01817528067 Page 4 of 7

Bollen, K. A. (1995). Structural equation models that are non-linear in latent variables: A least squares estimator. In P. V. Marsden (Ed.), Sociological methodology (Vol. 25, pp. 223 251). Washington, DC: American Sociological Association. Bollen, K. A. (1996). An alternative two stage least squares (2SLS) estimator for latent variable equations. Psychometrika, 61, 109121. Busemeyer, J. R., & Jones, L. E. (1983). Analyses of multiplicative combination rules when the causal variables are measured with error. Psychological Bulletin, 93, 549562. Byrne, Barbara M. (1994) Structural Equation Modelling with EQS and EQS/Windows - Basic Concepts, Applications and Programming SAGE Publications. Chou, C. P., & Bentler, P. M. (1995). Estimates and tests in structural equation modeling. In R. H. Hoyle (Ed.), Structural equation modeling (pp. 3759). Thousand Oaks, CA: Sage. Cohen, J., & Cohen, P. (1975). Applied multiple regression/correlational analysis for the behavioral sciences (1st ed.). Hillsdale, NJ: Erlbaum. Cronbach, L. J. (1975). Beyond the two disciplines of scientific psychology. American Psychologist, 30, 116127. Cronbach, L. J., & Snow, R. E. (1977). Aptitudes and instructional methods. A handbook for research on interactions. New York: Irvington. Cudeck, R., du Toit, S., & Srbom, D. (2001). Structural equation modeling: Present and future. Chicago: SSI Scientific Software. Duncan, O. D. (1975). Introduction to structural equation models. New York: Academic. Hayduk, L. A. (1987). Structural equation modeling with LISREL. Baltimore: Johns Hopkins University Press. Hershberger, S. L. (2003). The growth of structural equation modeling: 19942001. Structural Equation Modeling, 10(1), 3546. Holbert, R. L., & Stephenson, M. T. (2002). Structural equation modeling in the communication sciences, 19952000. Human Communication Research, 28, 531551. Jaccard, J., Turrisi, R., & Wan, C. K. (1990). Interaction effects in multiple regression. Newbury Park, CA: Sage publications. Jreskog, K. G. (1967). Some contributions to maximum likelihood factor analysis. Psychometrika, 32, 443482. Jreskog, K. G., & Lawley, D. N. (1968). New methods in maximum likelihood factor analysis. British Journal of Mathematical and Statistical Psychology, 21, 8596. Jreskog, K. G. (1969). A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 36, 183202.
By: Khan Sarfaraz Ali / sarfarazbim@gmail.com / 01714899542 / 01817528067 Page 5 of 7

Jreskog, K. G., & Srbom, D. (1993). New features in LISREL 8. Chicago: Scientific Software. Jreskog, K. G., & Srbom, D. (1996). LISREL 8: Users reference guide. Chicago: Scientific Software. Jreskog, K. G., & Yang, F. (1997). Estimation of interaction models using the augmented moment matrix: Comparison of asymptotic standard errors. In W. Bandilla & F. Faulbaum (Eds.), SoftStat 97 (Advances in Statistical Software 6, pp. 467478). Stuttgart: Lucius & Lucius. Kaplan, D. (2000). Structural equation modeling: Foundations and extensions. Thousand Oaks, CA: Sage. Karasek, R. A. (1979). Job demands, job decision latitude, and mental strain: Implications for job redesign. Administrative Quarterly, 24, 285307. Kenny, D. A. (1979). Correlation and causality. New York: Wiley. Khine, M.S. & Timothy. T. (2009). Structural Equation Modeling in Educational Research Concepts and Applications. The Netherlands: Sense Publishers. Kline, R. B. (1998). Principles and practice and structural equation modeling. New York: Guilford. Kline, R. B. (2005). Principles and practice of structural equation modelling (2nd ed.). New York: Guilford Press. Klein, A., & Moosbrugger, H. (2000). Maximum likelihood estimation of latent interaction effects with the LMS method. Psychometrika, 65(4), 457474. Klein, A. G., & Muthn, B. O. (2007). Quasi maximum likelihood estimation of structural equation models with multiple interaction and quadratic effects. Multivariate Behavioral Research, 42(4), 647673. Loehlin, J. C. (2004). Latent variable models: An introduction to factor, path, and structural equation analysis (4th ed.). Mahwah, NJ: Lawrence Erlbaum. Lomax, R. (1989). Covariance structure analysis: Extensions and developments. In B. Thompson (Ed.), Advances in social scientific methodology (Vol. 1, pp. 171204). Greenwich, CT: JAI. Lusch, R. F., & Brown, J. R. (1996). Interdependency, contracting, and relational behavior in marketing channels. Journal of Marketing, 60, 1938. MacCallum, R. C., & Austin, J. T. (2000). Applications of structural equation modeling in psychological research. Annual Review of Psychology, 51, 201222. Marsh, H. W., Wen, Z., & Hau, K. T. (2004). Structural equation models of latent interactions:
By: Khan Sarfaraz Ali / sarfarazbim@gmail.com / 01714899542 / 01817528067 Page 6 of 7

Evaluation of alternative estimation strategies and indicator construction. Psychological Methods, 9(3), 275300. Martens, M. P. (2005). The use of structural equation modeling in counseling psychology research. The Counseling Psychologist, 33, 269298. Michael T. Stephenson, R. L. (2006). On the Use of Structural Equation Modeling in Health Communication Research. 20(2), 159167. Texas. Lawrence Erlbaum Associates, Inc. Mulaik, S. (1972). The foundations of factor analysis. New York: McGraw-Hill. Muthn, B. O., & Muthn, L. (19982007). Mplus users guide. Los Angeles: Muthn & Muthn. Shah, R., & Goldstein, S. M. (2006). Use of structural equation modeling in operations management research: Looking back and forward. Journal of Operations Management, 24(1), 148169. Schumacker, R. E., & Lomax, R. G. (2004). A beginners guide to structural equation modeling. Mahwah, NJ: Lawrence Erlbaum. Snyder, M., & Tanke, E. D. (1976). Behavior and attitude: Some people are more consistent than others. Journal of Personality, 44, 501517. Stephenson, M. T., & Holbert, R. L. (2003). A Monte Carlo simulation of observable- versus latent-variable structural equation modeling techniques. Communication Research, 30, 332354. Tempelaar, D., Schim van der Loeff, S., Gijselaers, W., & De Crombrugghe, D. (2007). Preferred learning approaches and statistical reasoning. Unpublished manuscript. Thurstone, L. L. (1947). Multiple-factor analysis. Chicago: University of Chicago Press. Wall, M. M., & Amemiya, Y. (2003). A method of moments technique for fitting interaction effects in structural equation models. British Journal of Mathematical and Statistical Psychology, 56, 4764. Werts, C. E., & Linn, R. L. (1970). Path analysis: Psychological examples. Psychological Bulletin, 74, 193212. Wright, S. (1918). On the nature of size factors. Genetics, 3, 367374. Wright, S. (1934). The method of path coefficients. Annals of Mathematical Statistics, 5, 161 215.

By: Khan Sarfaraz Ali / sarfarazbim@gmail.com / 01714899542 / 01817528067

Page 7 of 7