Item Regression:

Multivariate Regression Models

Statistics for Psychosocial Research II:

Structural Models
November 8, 2004
 General Idea
• Y’s are all measuring the same thing or similar things.
• Want to summarize the association between an X and all of
the Y’s.
• BUT! We are not making the STRONG assumption that
there is latent variable accounting for the correlation
between the Y’s.
• First: Make model that allows each Yi to be associated
with X
• Next: Summarize/Marginalize over associations
• Sort of like STA
– But wait! I thought STA was “bad” relative to SAA!
– Not if you don’t want to make the assumption of a latent variable!
– More later…..
 Example: Vision Impairment in the
Elderly
• Salisbury Eye Evaluation (SEE, West et al.
1997).
– Community dwelling elderly population
– N = 1643 individuals who drive at night
• Want to examine which aspects of vision
(X’s) (e.g. visual acuity, contrast
sensitivity) affect performance of activities
that require seeing at a distance (Y’s).
 Variables of Interest
• Y’s: Difficulty….
– reading signs at night
– reading signs during day
– seeing steps in dim light
– seeing steps in day light
– watching TV
• X’s:
“Psychophysical” vision measures Potential confounders
-- visual acuity -- age
-- contrast sensitivity -- sex
-- glare sensitivity -- race
-- steropsis (depth perception) -- education
-- MMSE
-- central vision field
-- GHQ
-- # of reported comorbidities
 Technically…..
• The Y’s are binary, and we are using
logistic regression.
• To simplify notation, I refer to the outcomes
as “Y” but in theory, they are “logit(Y).”
• Assume N individuals, k outcomes (Y’s), p
predictors (X’s).
• For individual i:

Yi1 = β10 + β11 xi1 + β12 xi 2 + ... + β1 p xip

Yi 2 = β 20 + β 21 xi1 + β 22 xi 2 + ... + β 2 p xip
M
Yik = β k 0 + β k1 xi1 + β k 2 xi 2 + ... + β kp xip

• What is the same and what is different across

equations here?
• We are fitting k regressions and estimating
k*(p+1) coefficients
 Good or Bad approach?
• Not accounting for correlations between Y’s
from the same individual:
– e.g. may see that X Î Y1, but really X Î Y2
and Y1 is correlated with Y2.
• Simply: not summarizing!
• Alternative: Fit one “grand” model.
– Can decide if same coefficient is appropriate
across Y’s or not.
– Accounting for correlation among responses
within individuals.
 Analyze THEN Summarize, OR
Analyze AND Summarize?
• Includes all of the outcomes (Y’s) in the same model
• But, there is not an explicit assumption of a latent variable.
• Includes correlation among outcomes
– Do not assume that Y’s are independent given a latent variable
– Avoid latent variable approach and allows Y’s to be directly
correlated

“Multivariate” Model Latent Variable Approach

Y1 Y1
X1 X1
Y1 η
Y1
X2 X2
Y1 Y1
 Why Multivariate Approach?
• Latent variable approach makes stronger
assumptions
• Assumes underlying construct for which
Y’s are “symptoms”
• Multivariate model is more exploratory
• Based on findings from MV model, we may
adopt latent variable approach.
 Data Setup for Individuals 1 and 2
item (Y) ID Visual Acuity Age
y11 1 x11 x12 We have a “block”
Person 1

y12 1 x11 x12 for each individual

y13 1 x11 x12 instead of a “row”
y14 1 x11 x12 like we are used
y15 1 x11 x12 to seeing.
y21 2 x21 x22 Stack the “blocks”
y22 2 x21 x22 together to get the
Person 2

y23 2 x21 x22 whole dataset.

y24 2 x21 x22
y25 2 x21 x22

What if we entered this in standard logistic regression model?

 Model Interpretation

Yij = β0 + β1vai + β2 agei

Yi1 = β0 + β1vai + β2 agei

Yi 2 = β0 + β1vai + β2 agei
Yi 3 = β0 + β1vai + β2 agei
Yi 4 = β0 + β1vai + β2 agei
Yi 5 = β0 + β1vai + β2 agei
 Additional Parameters….
item (Y) ID Visual Acuity Age I(item=2) I(item=3) I(item=4) I(item=5)
y11 1 x11 x12 0 0 0 0
y12 1 x11 x12 1 0 0 0
y13 1 x11 x12 0 1 0 0
y14 1 x11 x12 0 0 1 0
y15 1 x11 x12 0 0 0 1
y21 2 x21 x22 0 0 0 0
y22 2 x21 x22 1 0 0 0
y23 2 x21 x22 0 1 0 0
y24 2 x21 x22 0 0 1 0
y25 2 x21 x22 0 0 0 1

Now what does regression model look like?

What are the interpretations of the coefficients?
 Model Interpretation
Yij = β 0 + β1vai + β 2 agei + α 2 I ( j = 2) +
α 3 I ( j = 3) + α 4 I ( j = 4) + α 5 I ( j = 5)

Yi1 = β 0 + β1vai + β 2 agei

Yi 2 = β 0 + α 2 + β1vai + β 2 agei
Yi 3 = β 0 + α 3 + β1vai + β 2 agei
Yi 4 = β 0 + α 4 + β1vai + β 2 agei
Yi 5 = β 0 + α 5 + β1vai + β 2 agei
 Parameter Interpretation
• β0 = intercept for item 1
• α2 = difference between intercept for item 1 and
for item 2.
• β0 + α2 = intercept for item 2
• β1 = expected difference in risk of difficulty in
any item for a one unit change in visual acuity.
• Intuitively, how does this model differ than
previous one (i.e. one without α terms)?
– Each item has its own intercept
– Accounts for differences in prevalences among
outcome items
– Still assumes that age and visual acuity all have same
association with outcomes.
 Is that enough parameters?
What if the association between visual acuity is NOT the same
for reading signs at night and for watching TV?

item (Y) Visual Acuity Age I2 I3 I4 I5 va*I2 va*I3 va*I4) va*I5

y11 x11 x12 0 0 0 0 0 0 0 0
y12 x11 x12 1 0 0 0 x11 0 0 0
y13 x11 x12 0 1 0 0 0 x11 0 0
y14 x11 x12 0 0 1 0 0 0 x11 0
y15 x11 x12 0 0 0 1 0 0 0 x11
y21 x21 x22 0 0 0 0 0 0 0 0
y22 x21 x22 1 0 0 0 x21 0 0 0
y23 x21 x22 0 1 0 0 0 x21 0 0
y24 x21 x22 0 0 1 0 0 0 x21 0
y25 x21 x22 0 0 0 1 0 0 0 x21

NOW how are regression parameters interpreted?

Note: I2 = I(item=2); va = Visual Acuity
 Model Interpretation
5
Yij = β 0 + β1vai + β 2 agei + ∑ α k I ( j = k ) +
k =2
5

∑δ
k =2
k I ( j = k ) ×vai

Yi1 = β 0 + β1vai + β 2 agei

Yi 2 = β 0 + α 2 + ( β1 + δ 2 )vai + β 2 agei
Yi 3 = β 0 + α 3 + ( β1 + δ 3 )vai + β 2 agei
Yi 4 = β 0 + α 4 + ( β1 + δ 4 )vai + β 2 agei
Yi 5 = β 0 + α 5 + ( β1 + δ 5 )vai + β 2 agei
 Parameter Interpretation
• β0 = intercept for item 1
• α2 = difference between intercept for item 1 and
for item 2.
• β1 = expected change in risk in item 1 for a one
unit change in visual acuity.
• δ2 = difference between expected change in risk in
item 2 for a unit change in visual acuity and
expected change in risk in item 1.
• β1 + δ2 = expected difference in risk in item 2 for a
one unit change in visual acuity.
 Parameter Interpretation
• β1 + δ2 = expected difference in risk in item
2 for a one unit change in visual acuity.
• The δ terms allow for the association
between visual acuity and each of the
outcomes to be different.
• We can test whether or not all the δ terms
are equal to zero or not.
• If they are equal to zero, that implies……
 Logistic Regression: Vision example
Covariate Estimate Robust SE Model SE Robust Z
Intercept (β0) ---- ----- ---- -----
Visual acuity (β1) -4.10 0.28 0.27 -14.7
Age (β2) -0.03 0.008 0.008 -3.5
I2 (α2) -1.47 0.06 0.06 -24.5
I3(α3) 0.74 0.12 0.13 6.0
I4(α4) -0.21 0.07 0.07 -3.1
I5(α5) 0.85 0.18 0.17 4.7
I2*va (δ2) 0.66 0.21 0.27 3.2
I3*va (δ3) 2.25 0.32 0.29 7.1
I4*va (δ4) 2.10 0.31 0.27 6.8
I5*va (δ5) 0.59 0.30 0.28 2.0
So far…same logistic and linear regression type stuff.
The difference:
• We need to deal with the associations!
• Items from the same individual are NOT independent
• Vision example: Odds Ratio between items is 7.69! We can’t
ignore that!
• We incorporate an “association” model into the model we
already have (the “mean” model).
• Consider an adjustment:
– mean model: used for inference
– association model: adjustment so that test statistics are
valid.
 Accounting for Correlations
Within Individuals
• “Marginal Models”
– parameters are the same as if you analyzed
separately for each item, but measures of
precision are more appropriate
– describes population average relationship
between responses and covariates as opposed to
subject-specific.
– We average (or marginalize) over the items in
our case.
 Fitting Approach #1
Post-hoc adjustment
– Idea: Ignoring violation of independence
invalidates standard errors, but not the slope
coefficients.
– So: We fit the model “näively” and then adjust
the standard errors to correctly account for the
association afterwards.
• Problem with this? Its outdated! We have
better ways of dealing with this presently.
 Related Example: drinks per week
• Suppose Yi, i = 1,…,N are independent but each is sample
mean of ni responses with equal variances, σ2. (e.g. drinks
per week, averaged over 2 or more weeks).
• Results from “usual” SLR, where y is drinks per week and
x family support.
^
se( β 1 ) = N
σ2
2
∑ ( xi − x )
i =1
• But, it is true (due to the averaging of y) that the actual s.e.
is

∑ [( xi − x ) 2 / ni ]
N
2
^ σ
se( β 1 ) = i =1
2
N 2
 ∑ ( xi − x ) 
• This is a valid analysis: We first
i =1 fit the
SLR and then

correct the standard error of the slope.
 Fitting Approach #2
• Marginal Model (GEE or ML)
– approach #1 is okay, but not as good as simultaneously
estimating the mean model and the association model
(i.e. we can iterate between the two, and update
estimates each time).
– We estimate regression coefficients using a procedure
that accounts for lack of independence, and specifically
the correlation structure that you specify.
– Correlation structure is estimated as part of the model.
 Related Example Revisited:
Drinks per week
• If Y1 is based on 2 observations (i.e. 2 weeks), and
Y2 is based on 20 observations (i.e. 20 weeks), we
want to account for that.
• We want to “weight” individuals with more
observations more heavily because they have more
“precision” in their estimate of Y.
• Results: Weight is proportional to ni .
• Resulting regression is better by accounting for
this in the estimation procedure.
 Fitting Approach #2 (continued)
• Here we use the within unit correlation to compute
the weights.
• GEE solution: “working correlation”
• If specified structure is good, the regression
coefficients are very good.
• If specified structure is bad, coefficients and
standard errors are still valid, but not as good.
• ROBUST PROCEDURE
 Fitting this for the Vision example
Approach 1: too complex to be feasible. Need to know all of the
associations and adjust many estimates.
Approach 2: account for correlation in estimation procedure
In STATA:
Logistic model:
xtgee y va age i2 i3 i4 i5 va2 va3 va4 va5, i(id)
link(logit) corr(exchangeable) robust

xi: xtgee y i.item*va age, i(id) link(logit) robust

(default corr is exc)

Linear model:
xtgee y x, i(id) corr(exchangeable) robust
 Problem with Approach #1
• Often correlation structure is more complex (our
example was very simple compared to most
situations)
• Post-hoc adjustments won’t always work because
estimating the correlation structure is not as
simple.
• In general, people don’t use approach #1
especially because many stats packages can handle
the adjustments currently (Stata, Splus, R, SAS)
 How do I know the correlation
structure?
• You don’t usually.
• Approaches commonly used for multivariate
outcome
– Exchangeable:
• individuals items are all equally correlated with each other.
• Simple and intuitive, easy to estimate and describe.
• Could be a bad assumption
– Unstructured:
• uses empirical estimates from data.
• Less prone to model mis-specification
• less powerful approach.
 Summarizing Findings
(1) Constrain equal slopes across items
(2) Constrain slopes that should be
constrained, and allow others to vary
(3) Detailed summary discussion that covers
everything
(4) Complicated: joint tests/CI’s for groups
of items
 Multiple Regression Results:
Odds Ratio between items estimated to be 8.69

Vision Variable Item Item O.R. 95% CI for OR

Visual acuity 1 0.427 (0.36,0.51)
2 0.515 (0.43,0.62)
3 0.863 (0.71,1.06)
4 0.817 (0.68,0.99)
5 0.514 (0.43,0.62)
Best contrast sensitivity 1-5 1.477 (1.26,1.73)
Diff in contrast sensitivity 1-5 0.696 (0.58,0.84)
Log(steropsis) 1-5 0.904 (0.86,0.95)
Best central vision field 1-5 0.902 (0.83,0.98)

1 = day signs; 2 = night signs; 3 = day steps; 4 = dim steps; 5 = TV

West SK, Munoz B, Rubin GS, Schein OD, Bandeen-Roche K, Zeger S, German S, Fried LP. Function
and visual impairment in a population-based study of older adults. The SEE project. Salisbury Eye
Evaluation. Invest Ophthalmol Vis Sci. 1997 Jan;38(1):72-82.
 Alternate Approach
• Use Bayesian (hierarchical) approach to model estimation
• Models correlation by assuming that ‘like’ parameters come
from a common distribution.

β j ~ N (β ,σ ) 2
β
• We estimate β and σβ as part of the model.
• If are βj’s not similar, then σβ will be large.
• Like a ‘random effects’ model, but broader.
 A New Example:
Hyper-Methylation of Genes and Breast Cancer

• Background:
– Methylation of certain genes is thought to be associated with
different prognosis for breast cancer
– Goal is to determine what risk factors are associated with
methylation of genes
– Methylation status of genes are highly correlated.
– We don’t have a very big dataset (N=111 breast cancer tissue
samples)

Mehrotra, J., Ganpat, M.M., Kanaan, Y., Fackler, M.J., McVeigh, M., Lahti-
Domenici, J.,Polyak, K., Argani, P., Naab, T., Garrett, E.S., Parmigiani, G.,
Broome, C., Sukumar, S.ER/PR-negative breat cancers of young African
American women have a higher frequency of methylation of multiple genes
than those of Caucasian women. Clinical Cancer Research, 10(6):2052-2057,
2004.
 Data
• Genes: HIN-1, Twist, Cyclin D2, RAR-beta, and
RASSF1A
• Risk factors:
– AfAm vs. Caucasian
– Age < 50 versus > 50
– Estrogen Receptor Status (+/-)
• Only 111 patients in the dataset
• Data is somewhat ‘sparse’
– For HIN-1, if we tabulate methylation by race, age, and
ER, we have empty cells.
– Can’t estimate saturated model (i.e. three-way
interaction)
 Modeling Issues
• By fitting multivariate model, we get good stuff:
– WE ACCOUNT FOR CORRELATION AMONG GENES
– WE BORROW STRENGTH ACROSS GENES
– WE CAN SUMMARIZE ASSOCIATIONS OF RISK FACTORS
WITH GENES
• Notation:
– yij = 1 if gene j in tumor i is methylated.
– racei = 1 if tumor is from AfAm patient
– ERi = 1 if tumor I is ER+
– agei = 1 if age of person i <50
• Notation is simplified from previous example.
 Started with main effects ‘hierarchical’ model:
logit( yij ) = β0 + β j + γ j racei + α j agei + δ j eri
Assume that ‘like’ parameters are from common distribution
With 5 genes and 3 covariates, we have 20 parameters* to
estimate in this model.

Gene (j) β0+ βj γj (race) αj(age) δj (ER)

1 -0.63 0.84 0.20 2.09
2 -0.74 0.72 0.03 0.15
3 -0.61 0.70 0.04 0.24
4 -1.42 0.60 -0.02 0.68
5 -0.13 0.73 0.01 2.13
Keep all? + - - +

* Note that β j’s are constrained to sum to 0.

 Next ‘hierarchical’ model: Allow for interactions
logit ( yij ) = β0 + β j + γ racei + α agei + δ j eri +
θ j racei × agei + φ j racei × eri + σ j agei × eri +
ν j racei × agei × eri

Gene β0+ βj γj (race) αj(age) δj (ER) θj φj σj νj

(j) Racexage racexer agexer racexagexer

1 or 1-5 -0.45 0.40 -1.12 2.11 2.61 -0.05 1.20 -2.20

2 -0.67 0.56 2.22 -0.21 0.85 -2.41
3 -0.43 0.54 1.72 -0.26 1.21 -2.44
4 -1.42 0.76 1.65 -0.10 1.35 -1.80
5 -0.15 2.23 1.81 -0.31 1.30 -2.30
Keep + - - +
all?
 Next ‘hierarchical’ model:
logit ( yij ) = β0 + β j + γ racei + α agei + δ j eri +
θ j racei × agei + φ racei × eri + σ agei × eri +
ν j racei × agei × eri

Gene β0+ βj γj (race) αj(age) δj (ER) θj φj σj νj

(j) Racexage racexer agexer racexagexer

1 or 1-5 -0.50 0.48 -1.04 2.26 2.51 -0.28 1.04 -2.05

2 -0.73 0.54 2.08 -2.39
3 -0.50 0.60 1.65 -2.30
4 -1.50 0.97 1.61 -1.58
5 0.07 2.33 1.73 -2.19
 Closing Remarks
• Model specification is still important here!
– Mean model
– Correlation structure
• GEE, random effects
• Bayesian
• Get robust estimates if possible (GEE)
• Fitting methods:
– Stata: xtgee
– WinBugs hierarchical model