Comparison of Two Groups

7.
Comparison of Two Groups

Goal: Use CI and/or significance test to compare
means (quantitative variable)
proportions (categorical variable)
Group 1
Population mean
Population proportion
1
1
Group 2
2
2
Estimate
y2 y1
2 1
We conduct inference about the difference between the means

or difference between the proportions (order irrelevant).
Copyright 2009 Pearson Education. Inc.
Example: Does cell phone use while

driving impair reaction times?
Article in Psych. Science (2001, p. 462) describes experiment
that randomly assigned 64 Univ. of Utah students to cell phone
group or control group (32 each). Driving simulating machine
flashed red or green at irregular periods. Instructions: Press
brake pedal as soon as possible when detect red light.
See http://www.psych.utah.edu/AppliedCognitionLab/
Cell phone group: Carried out conversation about a political

issue with someone in separate room.
Control group: Listened to radio broadcast
Outcome measure: mean response time for a

subject over a large number of trials
Purpose of study: Analyze whether (conceptual)
population mean response time differs significantly for
the two groups, and if so, by how much.
Data
Cell-phone group:
Control group:
y2
y1
= 585.2 milliseconds, s1 = 89.6
= 533.7, s2 = 65.3.
Shape? Outliers?
Types of variables and samples

The outcome variable on which comparisons are
made is the response variable.
The variable that defines the groups to be compared is
the explanatory variable.
Example: Reaction time is response variable
Experimental group is explanatory variable
-- a categorical var. with categories: (cell-phone, control)
Or, could express experimental group as

cell-phone use with categories (yes, no)
Different methods apply for

independent samples -- different samples, no
matching, as in this example and in cross-sectional
studies
dependent samples -- natural matching between each
subject in one sample and a subject in other sample,
such as in longitudinal studies, which observe
subjects repeatedly over time
Example: We later consider a separate experiment in

which the same subjects formed the control group at
one time and the cell-phone group at another time.
se for difference between two estimates

(independent samples)
The sampling distribution of the difference between two
estimates is approximately normal (large n1 and n2) and has
estimated
se ( se1 ) ( se2 )
2
Example: Data on Response times has

32 using cell phone with sample mean 585.2, s = 89.6
32 in control group with sample mean 533.7, s = 65.3
What is se for difference between sample means of
585.2 533.7 = 51.4?
se1 s1 / n1 89.6 / 32 15.84

se2 s2 / n2 65.3 / 32 11.54
se ( se1 ) 2 ( se2 ) 2 (15.84) 2 (11.54) 2 19.6
(Note
this is larger than each separate se. Why?)
So, the estimated difference of 51.4 has a margin of error of about

2(19.6) = 39.2 (more precise details later using t distribution)
95% CI for is about 51.4 39.2, or (12, 91).
Interpretation: We are 95% confident that population mean for
cell phone group is between 12 milliseconds higher and 91
milliseconds higher than population mean for control group.
(In practice, good idea to re-do analysis without outlier, to check
its influence. What do you think would happen?)
CI comparing two proportions

Recall se for a sample proportion used in a CI is
se (1 ) / n
So, the se for the difference between two sample proportions for
independent samples is
1 (1 1 ) 2 (1 2 )
se ( se1 ) ( se2 )
n1
n2
2
A CI for the difference between population proportions is
1 (1 1 ) 2 (1 2 )
( 2 1 ) z
n1
n2
As usual, z depends on confidence level, 1.96 for 95% confidence

Example: College Alcohol Study conducted by

Harvard School of Public Health
(http://www.hsph.harvard.edu/cas/)
Trends over time in percentage of binge drinking
(consumption of 5 or more drinks in a row for men and 4 or

more for women, at least once in past two weeks)
and of activities perhaps influenced by it?

Have you engaged in unplanned sexual activities
because of drinking alcohol?
1993:
2001:
19.2% yes of n = 12,708

21.3% yes of n = 8783
What is 95% CI for change saying yes?

Estimated change in proportion saying yes is

0.213 0.192 = 0.021.
1 (1 1 ) 2 (1 2 )
(.192)(.808) (.213)(.787)
se
0.0056
n1
n2
12, 708
8783
95% CI for change in population proportion is

0.021 1.96(0.0056) = 0.021 0.011, or roughly
(0.01, 0.03)
We can be 95% confident that the population
proportion saying yes was between about 0.01
larger and 0.03 larger in 2001 than in 1993.
Comments about CIs for difference between

two population proportions
If 95% CI for 2 1 is (0.01, 0.03), then 95% CI
for 1 2 is (-0.03, -0.01).
It is arbitrary what we call Group 1 and Group 2 and
what the order is for comparing the proportions.
When 0 is not in the CI, we can conclude that one
population proportion is higher than the other.
(e.g., if all positive values for Group 2 Group 1, then conclude
population proportion higher for Group 2 than Group 1)
When 0 is in the CI, it is plausible that the population

proportions are identical.
Example: Suppose 95% CI for change in population proportion
(2001 1993) is (-0.01, 0.03)
95% confident that population proportion saying yes was
between 0.01 smaller and 0.03 larger in 2001 than in 1993.
There is a significance test of H0: 1 = 2 that the population
proportions are identical
(i.e., difference 1 - 2 = 0), using test statistic
z = (difference between sample proportions)/se
For unplanned sex in 1993 and 2001,
z = diff./se = 0.021/0.0056 = 3.75
Two-sided P-value = 0.0002
This seems to be statistical significance without practical
significance!
Details about test on pp. 189-190 of text; use se0

which pools data to get better estimate of se under H0
(We study this test as a special case of chi-squared
test in next chapter, which deals with possibly many
groups, many outcome categories)
The theory behind the CI uses the fact that sample
proportions (and their differences) have approximate
normal sampling distributions for large ns, by the
Central Limit Theorem, assuming randomization)
In practice, formula works ok if at least 10 outcomes of
each type for each sample (Note: We dont use t dist. for
inference about proportions; however, there are specialized
small-sample methods, e.g., using binomial distribution)
Quantitative Responses:
Comparing Means
Parameter: 2 - 1
Estimator:
y2 y1
Estimated standard error:
se
s12 s22
n1 n2
Sampling dist.: Approximately normal (large ns, by CLT)

CI for independent random samples from two normal
population distributions has form
s12 s22
y2 y1 t ( se), which is y2 y1 t
n1 n2
Formula for df for t-score is complex (later). If both sample
sizes are at least 30, can just use z-score
Example: GSS data on number of close friends

Use gender as the explanatory variable:
486 females with mean 8.3, s = 15.6
354 males with mean 8.9, s = 15.5
se1 s1 / n1 15.6 / 486 0.708
se2 s2 / n2 15.5 / 354 0.824
se ( se1 ) 2 ( se2 ) 2 (0.708) 2 (0.824) 2 1.09
Estimated difference of 8.9 8.3 = 0.6 has a margin

of error of 1.96(1.09) = 2.1, and 95% CI is
0.6 2.1, or (-1.5, 2.7).
We can be 95% confident that the population mean number

of close friends for males is between 1.5 less and 2.7 more
than population mean number of close friends for females.
Order is arbitrary. 95% CI comparing means for females
males is (-2.7, 1.5)
When CI contains 0, it is plausible that the difference is 0 in
the population (i.e., population means equal)
Here, normal population assumption clearly violated. For
large ns, no problem because of CLT, and for small ns the
method is robust. (But, means may not be relevant for very
highly skewed data.)
Alternatively could do significance test to find strength of
evidence about whether population means differ.
Significance Tests for 2 - 1

Typically we wish to test whether the two population
means differ
(null hypothesis being no difference, no effect).
H0: 2 - 1 = 0 (1 = 2)
Ha: 2 - 1 0 (1 2)
Test Statistic:
y2 y1 0
se
y2 y1
s12 s22
n1 n2
Test statistic has usual form of

(estimate of parameter H0 value)/standard error.
P-value: 2-tail probability from t distribution
For 1-sided test (such as Ha: 2 - 1 > 0), P-value =
one-tail probability from t distribution (but, not robust)
Interpretation of P-value and conclusion using -level
same as in one-sample methods
ex. Suppose P-value = 0.58. Then, under supposition
that null hypothesis true, probability = 0.58 of getting
data like observed or even more extreme, where
more extreme determined by Ha
Example: Comparing female and male mean number of

close friends, H0: 1 = 2 Ha: 1 2
Difference between sample means = 8.9 8.3 = 0.6
se = 1.09 (same as in CI calculation)
Test statistic t = 0.6/1.09 = 0.55
P-value = 2(0.29) = 0.58 (using standard normal table)
If H0 true of equal population means, would not be
unusual to get samples such as observed.
For = 0.05 = P(Type I error), not enough evidence to
reject H0. (Plausible that population means are identical.)
For Ha: 1 < 2 (i.e., 2 - 1 > 0), P-value = 0.29
For Ha: 1 > 2 (i.e., 2 - 1 < 0), P-value = 1 0.29 = 0.71
Equivalence of CI and Significance Test

H0: 1 = 2 rejected (not rejected) at = 0.05 level in
favor of Ha: 1 2
is equivalent to
95% CI for 1 - 2 does not contain 0 (contains 0)
Example: P-value = 0.58, so We do not reject H0 of
equal population means at 0.05 level
95% CI of (-1.5, 2.7) contains 0.
(For other than 0.05, corresponds to 100(1 - )% confidence)
Alternative inference comparing means

assumes equal population standard deviations
We will not consider formulas for this approach here
(in Sec. 7.5 of text), as its a special case of analysis
of variance methods studied later in Chapter 12.
This CI and test uses t distribution with
df = n1 + n2 - 2
We will see how software displays this approach and
the one weve used that does not assume equal
population standard deviations.
Example: Exercise 7.30, p. 213. Improvement scores for

therapy A: 10, 20, 30
therapy B: 30, 45, 45
A: mean = 20, s1 = 10
B: mean = 40, s2 = 8.66
Data file, which we input into SPSS and analyze
Subject Therapy Improvement
1
A
10
2
A
20
3
A
30
4
B
30
5
B
45
6
B
45
Test of
H 0: 1 = 2
Ha: 1 2
Test statistic t = (40 20)/7.64 = 2.62

When df = 4, P-value = 2(0.0294) = 0.059.
For one-sided Ha: 1 < 2 (i.e., predict before study that
therapy B is better), P-value = 0.029
With = 0.05, insufficient evidence to reject null for twosided Ha, but can reject null for one-sided Ha and
conclude therapy B better.
(but remember, must choose Ha ahead of time!)
How does software get df for unequal

variance method?
When allow 12 22 recall that
s12 s22
n1 n2
se
The adjusted degrees of freedom for the t distribution

approximation is (Welch-Satterthwaite approximation) :
s
s

n2
n1
2
1
df
s2 2
1 n
n1 1
2
2
s
2
2

n
2
n2 1
Some comments about comparing means

If data show potentially large differences in variability
(say, the larger s being at least double the smaller s),
safer to use the unequal variances method
One-sided t tests not robust against severe violations
of normal population assumption, when n is small.
Better then to use nonparametric methods (which do
not assume a particular form of population distribution)
for one-sided inference; see text Sec. 7.7.
CI more informative than test, showing whether
plausible values are near or far from H0.
Effect Size
When groups have similar variability, a summary
measure of effect size is
mean 2 mean1
effect size =
standard deviation in each group
Example: The therapies had sample means of 20 for A
and 40 for B and standard deviations of 10 and 8.66. If
common standard deviation in each group is estimated to
be s = 9.35 (say), then
effect size = (40 20)/9.35 = 2.1.
Mean for therapy B estimated to be about two standard
deviations larger than the mean for therapy A.
This is a large effect.
This effect size measure is sometimes called Cohens

d. He considered
d = 0.2 = weak, d = 0.5 = medium, d > 0.8 large.
Example: Which study showed the largest effect?
1. y1 20, y2 30, s 10
2. y1 200, y2 300, s 100
3. y1 20, y2 25, s 2
Comparing Means with Dependent Samples

Setting: Each sample has the same subjects (as in
longitudinal studies or crossover studies) or matched
pairs of subjects
Then, it is not true that for comparing two statistics,
se ( se1 ) 2 ( se2 ) 2
Must allow for correlation between estimates (Why?)

Data: yi = difference in scores for subject (pair) i
Treat data as single sample of difference scores, with
sample mean yd and sample standard deviation sd
and parameter d = population mean difference score.
In fact, d =
Example: Cell-phone study also had experiment with

same subjects in each group
(data on p. 194 of text)
For this matched-pairs design, data file has the form
Subject Cell_no Cell_yes

1
604
636
2
556
623
3
540
615
(for 32 subjects)
Sample means are 534.6 milliseconds without cell phone

585.2 milliseconds, using cell phone
We reduce the 32 observations to 32 difference scores,

636 604 = 32
623 556 = 67
615 540 = 75
.
and analyze them with standard methods for a single sample
yd = 50.6 = 585.2 534.6,
sd = 52.5 = std dev of 32, 67, 75
se sd / n 52.5 / 32 9.28
For a 95% CI, df = n 1 = 31, t-score = 2.04
We get 50.6 2.04(9.28), or (31.7, 69.5)
We can be 95% confident that the population mean

using a cell phone is between 31.7 and 69.5
milliseconds higher than without cell phone.
For testing H0 : d = 0 against Ha : d 0, the test
statistic is
t = (yd - 0)/se = 50.6/9.28 = 5.46, df = 31,
Two-sided P-value = 0.0000058, so there is
extremely strong evidence against the null
hypothesis of no difference between the population
means.
In class, we will use SPSS to

Run the dependent-samples t analyses
Plot cell_yes against cell_no and observe a strong
positive correlation (0.814), which illustrates how an
analysis that ignores the dependence between the
observations would be inappropriate.
Note that one subject (number 28) is an outlier
(unusually high) on both variables
With outlier deleted, SPSS tell us that t = 5.26, df =
30 for comparing means (P = 0.00001) for comparing
means, 95% CI of (29.1, 66.0). The previous results
were not influenced greatly by the outlier.
SPSS output for original dependent-samples t

analysis (including the outlier)
Some comments
Dependent samples have advantages of (1) controlling
sources of potential bias (e.g., balancing samples on
variables that could affect the response), (2) having a
smaller se for the difference of means, when the pairwise
responses are highly positively correlated (in which case, the
difference scores show less variability than the separate
samples)
With dependent samples, why cant we use the se formula
for independent samples?
se
s12 s22
n1 n2
Ex. (artificial, but makes the point)

Weights before and after anorexia therapy
Subject
1
2
3
4
Before After
115
122
91
98
100
107
132
139
Difference
7
7
7
7
Lots of variability within each group of observations, but

no variability for the difference scores (so, actual se is
much smaller than independent samples formula suggests)
If you plot x = before against y = after, what do you see?

The McNemar test (pp. 201-203) compares

proportions with dependent samples
Fishers exact test (pp. 203-204) compares
proportions for small independent samples
Sometimes its more useful to compare groups using
ratios rather than differences of parameters
Example: U.S. Dept. of Justice reports that proportion of

adults in prison is about
900/100,000 for males, 60/100,000 for females
Difference: 900/100,000 60/100,000 = 840/100,000 = 0.0084
Ratio:
[900/100,000]/[60/100,000] = 900/60 = 15.0
In applications in which the proportion refers to an

undesirable outcome (e.g., most medical studies), the
ratio is called the relative risk. Inference methods (CI,
test) are available for it also.
A few summary questions

1.
Give an example of (a) independent samples, (b) dependent

samples
2. Give an example of (a) response var., (b) categorical
explanatory var., and identify whether response is quantitative or
categorical and state the appropriate analyses.
3. Suppose that a 95% CI for difference between Massachusetts and
Texas in the population proportion supporting legal same-sex
marriage is (0.15, 0.22).
a.
b.
Population proportion of support is higher in Texas

Since 0.15 and 0.22 < 0.50, less than half the population supports
legal same-sex marriage.
c. The 99% CI could be (0.17, 0.20)
d. It is plausible that population proportions are equal.
e. P-value for testing equal population proportions against two-sided
alternative could be 0.40.
f. We can be 95% confident that the population proportion of
support in MA is between 0.15 higher and 0.22 higher than in TX.
Example: Anorexia study, studying weight change for 3

groups (behavioral therapy, family therapy, control).
Patients randomly assigned to one of the three
therapies. Is this an example of independent samples
or dependent samples?

Comparison of Two Groups

Uploaded by

Copyright:

Available Formats

Comparison of Two Groups

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Comparison of Two Groups

Uploaded by

Copyright:

Available Formats

7.

Comparison of Two Groups

We conduct inference about the difference between the means

Example: Does cell phone use while

Cell phone group: Carried out conversation about a political

Outcome measure: mean response time for a

= 585.2 milliseconds, s1 = 89.6

Copyright 2009 Pearson Education. Inc.

Types of variables and samples

Or, could express experimental group as

Different methods apply for

Example: We later consider a separate experiment in

se for difference between two estimates

Example: Data on Response times has

se1 s1 / n1 89.6 / 32 15.84

this is larger than each separate se. Why?)

So, the estimated difference of 51.4 has a margin of error of about

CI comparing two proportions

A CI for the difference between population proportions is

As usual, z depends on confidence level, 1.96 for 95% confidence

Example: College Alcohol Study conducted by

(consumption of 5 or more drinks in a row for men and 4 or

and of activities perhaps influenced by it?

19.2% yes of n = 12,708

What is 95% CI for change saying yes?

Estimated change in proportion saying yes is

95% CI for change in population proportion is

Copyright 2009 Pearson Education. Inc.

Comments about CIs for difference between

When 0 is in the CI, it is plausible that the population

Details about test on pp. 189-190 of text; use se0

Sampling dist.: Approximately normal (large ns, by CLT)

Example: GSS data on number of close friends

Estimated difference of 8.9 8.3 = 0.6 has a margin

We can be 95% confident that the population mean number

Significance Tests for 2 - 1

Copyright 2009 Pearson Education. Inc.

Test statistic has usual form of

Example: Comparing female and male mean number of

Equivalence of CI and Significance Test

Alternative inference comparing means

Example: Exercise 7.30, p. 213. Improvement scores for

Copyright 2009 Pearson Education. Inc.

Test statistic t = (40 20)/7.64 = 2.62

How does software get df for unequal

The adjusted degrees of freedom for the t distribution

Copyright 2009 Pearson Education. Inc.

Some comments about comparing means

This effect size measure is sometimes called Cohens

Example: Which study showed the largest effect?

Comparing Means with Dependent Samples

Must allow for correlation between estimates (Why?)

Example: Cell-phone study also had experiment with

Subject Cell_no Cell_yes

Sample means are 534.6 milliseconds without cell phone

We reduce the 32 observations to 32 difference scores,

yd = 50.6 = 585.2 534.6,

sd = 52.5 = std dev of 32, 67, 75

We can be 95% confident that the population mean