Graduate School of Education University of California Los Angeles, California
Graduate School of Education University of California Los Angeles, California
Graduate School of Education University of California Los Angeles, California
1, 115-132
MARCH1984
BENGT MUTI~N
GRADUATE SCHOOL OF EDUCATION
UNIVERSITY OF CALIFORNIA
LOS ANGELES, CALIFORNIA
A structural equation model is proposed with a generalized measurement part, allowing for
dichotomous and ordered categorical variables (indicators) in addition to continuous ones. A
computationally feasiblethree-stage estimator is proposed for any combination of observed vari-
able types. This approach provides large-sample chi-square tests of fit and standard errors of
estimates for situations not previously covered. Two multiple-indicator modeling examples are
given. One is a simultaneous analysis of two groups with a structural equation model underlying
skewed Likert variables. The second is a longitudinal model with a structural model for multi-
variate probit regressions.
Key words: polychoriccorrelations, probit regressions,generalized least-squares, weight matrix.
1. Introduction
This article considers the specification and estimation of multiple-group (population)
structural equation models with latent variables having multiple indicators, not all of
which are continuous. A linear structure for continuous latent variables will be con-
sidered. However, in the measurement part dichotomous and ordered polytomous ob-
served variables (indicators) will be allowed in addition to continuous indicators. Such
categorical indicators are frequent in m a n y types of applications, and it seems important
to provide the powerful structural equation modeling tool also for these cases.
The methodology to be presented unifies and generalizes several lines of psycho-
metric, econometric and biometric work. For an overview, see Muthrn (1983). In particu-
lar, the paper extends the Muthrn-Christoffersson methodology for factor analysis of di-
chotomous variables (see e.g. Muthrn, 1978; Muthrn and Christoffersson, 1981) to handle
ordered categorical and continuous indicators and general multiple-group structural
equation models with estimation of latent variable means. Hence, the paper also gener-
alizes the J r r e s k o g - S r r b o m ("LISREL") methodology for structural equation models (see
e.g., Jrreskog, 1973, 1977; Srrbom, 1982) to handle properly categorical indicators in
addition to continuous ones. New results also include a general estimation approach for
all cases of the model. A three-stage, limited information, generalized least-squares (GLS)
estimator is proposed, which gives large-sample chi-square tests of model fit and large-
sample standard errors of estimates. Some examples of analyses that the new techniques
make possible are G L S factor analysis with (mixtures of continuous and) ordered poly-
tomous indicators, testing hypotheses of both correlation and level ("mean") structures in
multiple-group structural equation models, and multivariate structural regression with
ordered categorical response variables (such as multivariate probit regression). In the
This research was supported by Grant No. 81-1J-CX-0015 from the National Institute of Justice, by Grant
No. DA 01070 from the U.S. Public Health Service,and by Grant No. SES-8312583 from the National Science
Foundation. I thank Julie Honig for drawing the figures. Requests for reprints should be sent to Bengt Muthrn,
Graduate School of Education, University of California, Los Angeles,California 90024.
0033-3123/84/0300-4058500.75/0 115
© 1984 The PsychometricSociety
i 16 PSYCHOMETRIKA
latter case, the new estimator provides a computationally feasible alternative to the
maximum-likelihood estimator of M u t h r n (1979).
O, if y* < zi 1
(5) vanishes. With categorical y variables, the scale of the latent response variables y* is
indeterminate and we may consider standardized y*, such that instead of the left sides of
(5) and (6), we have
AE(y* I x), (7)
AV(y*Ix)A, (8)
where A is a diagonal matrix of scaling factors,
diag (A) = [diag (V(y* I x))]- 1/2. (9)
In a single group (population) with categorical variables we usually standardize to A = I.
With categorical y variables, the distribution of the observed variables is deduced by
integrating over the corresponding latent response variables y* (c.f., Muth6n, 1979;
Muth6n and Christoffersson, 1981). The integration limits involve the expression [c.f. Eq.
(1), (11)]
6i[z,.c - [E(y* I x)]i], (10)
where 6i is the ith diagonal element of A, c = 0, 1, ..., Ci - 1, and [E(y* I x)]~ is the ith
row of (5).
symmetric matrix elements it pre-multiplies, where diagonal elements are only included if
the corresponding observed variable is continuous.
Variable Yi contributes Ci - 1 elements to ~i if it is categorical with Ci categories,
and a single element (the mean) if it is continuous. Variable y~ contributes q elements to
0.2- The pair of variables yi and yj contribute a single element to 0.3 if both variables are
categorical (a correlation), and may contribute one more element for each y that is con-
tinuous (a variance). Any model fitting into this general framework is identified if and
only if its parameters are identified in terms of 0"1, 0"2,0"3.
The model of (11), (12), and (13) utilizes the common structure of Case A and Case B.
Case A considers means and covariances/correlations of y*; see (5), (6), and (7), (8). For
Case A, part 1 and part 3 would normally be used. Part 2 is not needed. Part 3 corre-
sponds to the covariance/correlation structure, while part 1 would also be used when a
structure on the levels of the latent variables is desired such as in a multiple-group analy-
sis with estimation of latent variable means. Case A with part 1 and part 3 covers all
J6reskog-S6rbom (LISREL) models. Case B considers reduced-form regression structures
for the regression of y* on x; see (5), (6), and (7), (8). Part 1 contains the intercept structure
(the first two terms of (5) and (7)), part 2 contains the regression slope structure (the last
term of (5) and (7)), while part 3 contains the residual covariance/correlation structure ((6)
and (8)).
univariate probit regressions of each Yl on q x's give the p estimates of tr~ (the reduced-
form regression intercepts) and the p x q estimates of tr2 (the reduced-form slopes),
whereas the second stage produces estimates of the p(p - 1)/2 reduced-form residual cor-
relations of a 3 (see e.g. Muth6n, 1979).
A consistent estimator of the asymptotic covariance matrix of the estimates 6 ob-
tained from (15) is given by the following large-sample approximations. Consider the
asymptotic expansion
OF 02F
a--~ + ~ (d -- o') ~ O, (16)
where r runs over the observation units in the particular sample of size N that is con-
sidered (with multiple groups, the groups are considered separately). With the partitioning
FAll Alz]
O2F/Oa Otr' = LA21 Az2A , (18)
All is a block diagonal matrix involving the singly subscripted F's and A22 is a diagonal
matrix with rows corresponding to the doubly subscripted F's. Note that A~2 = 0. In
large samples, we may use the following common approximations to the probability limit
of(18). Each matrix on the diagonal of At, may be approximated as
u rOV,(r)/(~(r,,7 , ,
~, LOVi(r)/OCrzij[OFi(r)/Otrii OFi(r)/Oa2J, (19)
Let B denote the above approximation to the A matrix of (18). It then follows that in
large samples a consistent estimator of the limiting covariance matrix of 6 - tr is
B_I ~ OF(r) OF(r)
• =1 2~ Oa' B-a', (22)
calculated at tr = 6.
mation stage, the model parameters are estimated by minimization of the weighted least-
squares fitting function
G
V 3 = ~ (S (g) -- ~r(O))'W(°)-l(S(°) - - a(°)), (23)
g=l
where the superscript g denotes the 9th group and W t°) is a positive definite weight
matrix. When the W(°)'s are formed as in (22), a limited-information generalized least
120 PSYCHOMETRIKA
where
A~g)= [c~a~gVO0]0=o . (25)
The computations of the three-stage estimator for the examples below have been
carried out by a general computer program LACCI (Latent Variable Analysis with Di-
chotomous, ordered Categorical, and Continuous Indicators), which is being developed
by the author. The optimizations are carried out by the Fletcher and Powell method as
modified by Gruvaeus and J6reskog (1970, Note 1), using only first-order derivatives and
allowing constraints on the parameters in the form of equalities or fixed values.
4. Examples
In this section we wilt consider two applications of the general methodology. With
the first data set we illustrate Case A with ordered categorical variables, using both single-
group analyses and simultaneous multiple-group analyses. With the second data set we
illustrate Case B with dichotomous indicators in a longitudinal model.
Blood Services, Los Angeles-Orange Counties Region. L. Cooper and I. Currim, faculty
supervisors of that project, provided these data for the current research. For this illustra-
tion we consider a sub-set of the data consisting of 662 individuals of age less than 35,
further divided into 354 females and 308 males. Preliminary analyses suggested a certain
interpretable factor structure among a large set of attitude variables. In our illustration,
TABLE I
we have for simplicity chosen two indicators of each of three important factors. All six
variables are five-category Likert scales which range from Strongly Disagree to Strongly
Agree. In Table 1 is given the wording of the six variables and their univariate distri-
butions.
The first two variables are of particular importance since they indicate a propensity
to want to repeat blood donations (the latent variable construct r/1 in Figure 1 below). We
~32
FIGURE 1.
A Structural Equation Model.
BENGT MUTHI~N 123
may note that Females have higher sample proportions for the category Strongly Agree
on both of the two dependent variables, Yl and Y2. For Males the proportions are 0.584
(yl) and 0.779 (Y2), while for Females they are 0.661 (Yl) and 0.825 (Y2). Tests for signifi-
cant differences in proportions, obtain t-values of 0.690 (Y0 and 0.956 (Y2). This does not
give a strong indication of sex difference in levels.
It is of interest to relate yl and Y2 to the other variables, which intend to measure
various aspects of attitudes towards the donation experience, such as perceived physical
discomfort (r/2) and pleasantness of treatment (~/a). The structural equation model address-
ing this, is given in Figure 1. It is of interest to study this model for the two sex groups
separately and to compare model characteristics across the two sex groups in simulta-
neous, two-group analyses.
The above Likert variables are presumably not suitable for ordinary structural equa-
tion modeling, assuming continuous or continuous multivariate normal indicators
(scored, say, 0, 1, 2. . . . ). First, these Likert variables may not have equidistant scale steps
and may be better viewed as ordered categorical variables than continuous ones. Second,
and more importantly, the strong skewness of several of the variables (taken together with
the small number of categories) will distort ordinary Pearson product moment corre-
lations (or covariances); see e.g., Olsson (1979b).
Here, we will instead analyze polychoric correlations, applying Case A of the general
model. In an initial step, we will assess the appropriateness of the normality specification
for the latent response variables of y*. For each pair of indicators we may use the full ML
estimator, see Olsson (1979a), to estimate the polychoric correlation and the thresholds
for each indicator, and obtain a large sample Pearson chi-square test of bivariate nor-
mality in each marginal two-way table. To avoid low bivariate frequencies in these tests, it
was decided to collapse categories for certain variables. For y~ the two left-most catego-
ries were combined, while for Y2, Ys, and Y6 the three left-most categories were combined;
see Table 1. The results of these tests are given for each of the two groups and all pairs of
variables in Table 2.
There seems to be no strong overall indication of misspecification, although for
Males, Y6 seems to produce rather low probability values throughout. It may be noted
that conservative approximations to the chi-square tests may be obtained by instead
using cell frequencies as predicted from the two-stage estimator (which is not fully ef-
ficient). For these data, very similar results were observed for the two approaches.
While some sex differences are anticipated, a first analysis may test for sex invariance
o f a~ and 0-3 (not applying the structural model of Figure 1 to a~ and 0-3)- This will be
carried out here, providing a generalization of tests of group-invariant mean vectors and
covariance matrices for normal continuous variables (see e.g. J6reskog and S6rbom, Note
2). In what follows we will use the same collapsing of categories as was done for the tests
of Table 2; see Table 1. For each group there are 17 thresholds and 15 correlations. The
hypothesis of sex invariant a 1 and a 3 was tested in a simultaneous analysis of the two
groups via the GLS estimator of (23). A chi-square value of 73.53 with 32 degrees of
freedom was obtained. Allowing a~ to vary over sex, while still restricting 0 3 to be sex
invariant, resulted in a chi-square of 19.43 with 15 degrees of freedom. Hence, there seems
to be sex differences in levels (0"~), but there is no indication of sex differences in associ-
ations (0-3).
Next, we consider the structural equation model of Figure 1. First, we estimate this
model for Females and Males separately. In the measurement relation (3) we standardize
to v = 0 since all y's are categorical. In the structural relation (4), ct is a vector repre-
senting one mean for each of the two independent latent variable constructs and one
intercept. In the single-group analyses we may standardize to ~ = 0. The matrix B con-
124 PSYCHOMETRIKA
TABLE 2
Yl Y2 Y3 Y4 Y5
FEMALES (N = 354)
Y2 .06
Y3 .98 .07
MALES (N = 308)
Y2 .22
Y3 .I0 .13
tains the two slopes. In (6), tp is a matrix containing the single residual variance and the
covariance matrix for the independent latent variable constructs. N o t e that elements of O
are not included as parameters to be estimated. In the single-group analyses we set A of
(9) as the identity matrix (see also Muth6n and Christoffersson, 1981, p. 410).
With the third stage G L S estimator, a chi-square test of model fit to sl and s 3 ob-
tains six degrees of freedom and indicated well-fitting models for each sex; the values were
8.99 for Females and 1.49 for Males. Given these results, it is of interest to study sex-
invariance of the structural equation model parameters. Since the same measurement in-
strument was used for both sexes, it is relevant to hypothesize sex-invariant measurement
parameters ~ and A. Sex differences will be allowed for in the structural parameters of B
and W. Also, with more than one group, we can identify and estimate group differences in
the vector ~, fixing ~ = 0 for Females. Allowing for group level differences in the latent
variable constructs would seem to be necessary to account for the group differences in
estimated thresholds found above. Furthermore, we will allow the measurement error
variances of 19 to be different over sex, which is consistent with fixing A = I for Females,
BENGT MUTHI~N 125
TABLE 3.1
aEntries a r e : Estimate
(Standard error,
Estimate/St. error)
and allowing the diagonal A elements to be free for Males (group-variant error variances
is frequently observed in multiple-group analysis of continuous indicators). The chi-
square for this model was 27.68 with 23 degrees of freedom, indicating a good fit.
We may also compare the above model with the one that in addition postulates sex
invariant B and ~P, still allowing for variant c( and A. The chi-square difference was 29.55
with six degrees of freedom and gives a strong indication of misfit.
In Table 3 we report the estimates for the model with 23 degrees of freedom, allowing
for sex differences in B, u/, c(, and A. We also give structural parameter estimates in the
standardization of unit latent variable construct variances. Males have significantly lower
means for both q2 (which has a negative influence on r/i) and r/3 (which has a positive
influence on ql). Of particular interest is the estimated sex difference in the rh mean. We
deduce that there is a significantly smaller value for Males, - . 3 4 3 with a standard error
126 PSYCHOMETRIKA
TABLE 3.2
of .065, corresponding to a t ratio of -5.3. This may be contrasted with the insignificant
sex differences for the Yl, Y2 proportions, reported at the beginning of this example.
For comparison, it is also of interest to give results analogous to those of Table 3 for
the case of scoring the observed variables 0, 1, 2. . . . . and applying continuous variable,
BENGT MUTI-IEN 127
TABLE 4.1
TABLE 4.2
data. A random sample of Canberra electors were interviewed four times with four-month
intervals in 1977 and 1978. The data are fully described by Henderson, Byrne, and
Duncan-Jones (1981). F o r our illustrations we will analyze data from the first and last
occasion, for 231 individuals with complete data. At each occasion we consider three
dichotomous items intended to measure "neurotic illness,"
BENGT MUTHEN 129
TABLE 5
N L1 L2 L3 L4
Correlations
N 1.0000
LI 0.2162 1.0000
Variances
Means
"In the last month have you suffered from any of the following?"
Anxiety
Depression
Irritability
The response Yes was denoted by y = 1 and No by y = 0. Below the abbreviations A, D, I
will be used for these three indicators. It was hypothesized that at each time point (one
and four) these three items may be viewed as indicators of latent variable constructs t/1
and r/4 , "current level of neurotic illness." The latent variable constructs are related to a
set of observed continuous variables: the N scale from the Eysenck Personality Inventory
(N) and number of "life events" occurring to the respondent in the four months prior to
each interview (L1, L2, L3, L4). The variable N is intended to measure long-term liability
to neurosis and is taken here as the average score from occasions two and four. Descrip-
tive statistics for the background variables are given in Table 5, and the model is given in
path diagram form in Figure 2.
Since the same y-items have been administered at the two occasions we may hypoth-
esize time-invariance of the thresholds and loadings. We may allow for different intercepts
at the two time points and also allow the variances of the measurement errors of e to vary
over time by specifying A as diag (A) = (1 1 1 6 x 62 63), where the unit elements
are fixed.
The above model is fitted into the general framework by using part 1, 2, and 3. Here,
a 1, a 2 , and o 3 have 6, 30, 15 elements, respectively. A chi-square value of 32.03 with 32
130 PSYCHOMETRIKA
"Y4C
FIGURE2.
A Longitudinal Model.
degrees of freedom is obtained with the GLS estimator. Hence the notion that we are
tapping the same latent variable construct at the two time points cannot be rejected. Had
a significant value been obtained, the three-part model structure makes it convenient to
separately test for fit the relations between the y's and the x's and between the y's.
The parameter estimates are given in Table 6. From Table 6 we note a significantly
larger intercept in the structural relation for the last occasion as compared to the first.
Using the sample means of N, L1, Lz, L3, and L4 we find estimated r/1 and r/a means of
1.236 and 1.356, respectively. We may also note the difference in regression slopes for N.
A chi-square test of slope equality with one degree of freedom obtains a significant value
of 11.43. This could indicate that the presumed stability over time of the liability to
neurosis does not hold true as measured by N.
We may also further constrain the model by specifying time-invariant measurement
error variances. This means that we would have time-invariant probit regressions for each
BENGT MUTHEN 131
TABLE 6
Intercept N L1 L2 L3 L4
Time-point i .486
(.087)
bFixed parameter.
of the A, D, I items on the respective ~7; see also Muth6n and Christoffersson (1981, p.
411). Still allowing for time-varying residual variances, this specification is consistent with
the restriction 61 = 62 = 63. However, this hypothesis is rejected with a chi-square of 8.06
on two degrees of freedom.
5. Conclusion
In the preceding sections developments have been presented for appropriately deal-
ing with ordered categorical indicators in structural equation models. What are the limi-
tations?
First of all, normality of underlying latent variables was specified. In a Pearsonian
spirit, the categorical variables were viewed as manifestations of continuous normal vari-
ables. The present author does not believe that underlying normality is always the most
appropriate specification. This modeling needs to be confronted with data, e.g. as was
done in Table 2. It is unknown how sensitive results are to deviations from this state of
affairs. The normality specification is perhaps less strict in Case B models, where the
132 PSYCHOMETRIKA
latent response variable distribution is to some extent generated by the x distribution and
the normality specification is on the residuals.
The GLS (as opposed to an unweighted least squares) estimator requires the creation
of a weight matrix which grows very rapidly with the number of y-variables. This limits
its practical use to about 15-20 variables, i.e. small-sized problems or the scrutinizing of
marginal parts of models. No doubt the appropriate creation of the weight matrix re-
quires large samples. Furtherresearch needs to show what the requirements actually are
(e.g., in terms of bivariate frequencies for polychorics).
Although data are often observed in the form discussed here, some researchers may
find the proposed modeling too complex. People in the scaling tradition may prefer some
optimal scaling approach, while LISREL users may choose to ignore the scale problem.
The former approach would seem to give a less powerful analysis. As we have seen, the
latter way out may be undesirable, although often it may not make that much of a differ-
ence for structural parameters. However, to quote Cox (1970, p. 18) in discussing logit
versus ordinary regression: "the use of a model, the nature of whose limitations can be
foreseen, is not wise, except for very limited purposes." Our approach is computationally
heavy, but it gives an interesting possibility for a rather detailed analysis. The GLS esti-
mator provides a general approach for analyzing any kind of statistics for which the
three-part structure is relevant. What is needed is to find the appropriate weight matrix.
REFERENCE NOTES
1. Gruveaus, G. T., & J6reskog, K. G. (1970). A computer program for minimizing a function of several vari-
ables. Research Bulletin 70-14. Princeton, N.J.: Educational Testing Service.
2. J6reskog, K. G., & S6rbom, D. (1981). LISREL V. Analysis of linear structural relationships by maximum
likelihood and least squares methods. Research Report 81-8, Department of Statistics, University of Uppsala.
REFERENCES
Browne, M. W. (1974). Generalized least squares estimates in the analysis of covariance structures. South African
Statistical Journal, 8, 1-24. (Reprinted in D. J. Aigner, & A. S. Goldberger (Eds.), (1977), Latent variables in
socio-economic models. Amsterdam: North-Holland.)
Cox, D. R. (1970). The analysis of binary data. London: Methuen.
Henderson, A. S., Byrne, D. G , & Duncan-Jones, P. (1981). Neurosis and the social environment. Sidney: Aca-
demic Press.
J6reskog, K. G. (1973). A general method for estimating a linear structural equation system. In A. S. Goldberger
and O. D. Duncan (Eds.), Structural equation models in the social sciences. New York: Seminar Press,
85-112.
J6reskog, K. G. (1977). Structural equation models in the social sciences: Specification, estimation and testing.
In P. R, Krishnaiah (Ed.), Applications of statistics. Amsterdam: North-Holland.
Muth6n, B. (1978). Contributions to factor analysis of dichotomous variables. Psychometrika, 43, 551-560.
Muth6n, B. (1979). A structural probit model with latent variables. Journal of the American Statistical Associ-
ation, 74, 807-811.
Muth6n, B. (1983). Latent variable structural equation modeling with categorical data. Journal of Econometrics,
22, 43-65.
Muth6n, B., & Christoffersson, A. (t981). Simultaneous factor analysis of dichotomous variables in several
groups. Psychometrika, 46, 407-419.
Olsson, U. (1979). Maximum likelihood estimation of the polychoric correlation coefficient. Psychometrika, 44,
443-460. (a)
Olsson, U. (1979). On the robustness of factor analysis against crude classification of the observations. Multi-
variate Behavioral Research, 14, 485-500. (b)
Olsson, U., Drasgow, F., & Dorans, N. J. (1982). The polyserial correlation coefficient. Psychometrika, 47,
337-347.
S6rbom, D. (1982). Structural equation models with structured means. In K. G. J6reskog & H. Wold (Eds.),
Systems under indirect observation: Causality, structure, prediction. Amsterdam: North-Holland Publishing
Company.
Manuscript received 12/4/81
First revision received 3/16/83
Final version received 10/27/83