CHAPTER 10: HYPOTHESIS TESTING WITH TWO SAMPLES

1
Lecture Notes for Introductory Statistics

Daphne Skipper, Augusta University (2016)

[]Chapter 10Hypothesis Testing with Two Samples

In this chapter our hypothesis tests allow us to compare the means (or propor-
tions) of two different populations using a sample from each population.
For example, we might compare the mean SAT score of a group of students who
have taken an SAT prep course to the mean score of a different group of students
who haven’t taken the course. In this way we could determine if there is statis-
tical evidence that the SAT prep course is effective. We will compare population
means using two “independent” samples in sections 10.1 (t distribution) and 10.2
(z distribution).
In section 10.3 we test claims about two independent proportions using “inde-
pendent” samples.
When testing with two “independent” samples, the hypotheses do not involve
comparing a mean or proportion to a specified number, as they do when testing
with one sample. Instead they involve comparing two means to each other. Our
null hypothesis (assumption) is that the two means (or proportions) of
the two underlying populations have some condition involving equality:
≤, ≥, or =. The alternative is that they are different in some way: >, <,
or 6=.

Example 1. Null and alternative hypotheses for two INDEPENDENT

samples. Write the null and alternative hypotheses that would be used to test
each claim.
(1) The mean daily high temperature in Augusta is higher than the mean daily
high temperature in Savannah. (We have to have some way to distinguish
the two population means. We do this with subscripts: µ1 and µ2 , but we
must keep track of which is “1” and which is “2”.)
(2) The percentage of children who contract the flu after using the nasal vaccine
is smaller than the percentage of children who contract the flu after using
the placebo.

In section 10.4 we return to testing claims about population means, but this
time with “paired data”. For example, we might again test the efficacy of the SAT
prep course, but this time by collecting “before” and “after” SAT scores from a
single group of students. This “paired” sample data is handled differently than
the two independent samples in sections 10.1 and 10.2. We will subtract each pair
(the two test scores from a single student, for example) to create a new data set
of “differences” (after score − before score, for example). We will treat this new
(single) data set using the methods of chapter 9 (T-Test or Z-Test).

1
These lecture notes are intended to be used with the open source textbook “Introductory
Statistics” by Barbara Illowsky and Susan Dean (OpenStax College, 2013).
1
Chapter 10 Notes Hypothesis Testing with Two Samples D. Skipper, p 2

1. Two Population Means with Unknown Standard Deviations.

In this section we use a t-test to compare the population means of two popu-
lations using two independent samples. Two samples are independent if they
come from two distinct populations.
1.1. Requirements. The requirements for using a t-test for two independent sim-
ple random samples are:
• the combined sample size is small (< 30) AND
• the data appear to be normally distributed AND
• the sample standard deviation is unknown.
1.2. Distribution. We are interested in testing if the mean of one population is
greater than the mean of the other. For example, we may test if µ1 > µ2 , which
is equivalent to µ1 − µ2 > 0. Rather than handling the two distributions X̄1
and X̄2 separately, which would be pretty complicated, we work instead with the
distribution of differences of sample means: X̄1 − X̄2 . The sample statistic in this
case is the best point estimate for µ1 − µ2 , which is x̄1 − x̄2 . In this scenario, an
outcome that supports the alternative hypothesis (contradicts the null hypothesis)
would be if x̄1 − x̄2 is large, since that would mean x̄1 is much larger than x̄2 ,
running counter to our assumption that µ1 = µ2 .
Under the requirements listed above, this new distribution of differences of sam-
ple means is also normally distributed with the following distribution:
 s 
(s ) 2 (s )2
1 2
X̄1 − X̄2 ∼ N µ1 − µ2 , + ,
n1 n2
where the subscripts indicate which population each statistic or parameter is rep-
resenting.
data value−mean
Recall that a t-score (or z-score) is has the general formula standard deviation , so
using this distribution we get:

(x̄1 − x̄2 ) − (µ1 − µ2 )

test statistic = q
(s1 )2 (s2 )2
n1 + n2
and the p-value is the area under the normal curve beyond x̄1 − x̄2 in the direction
of the alternative hypothesis.

NOTE: Our null hypothesis assumption is that µ1 = µ2 , which means that

µ1 − µ2 = 0. This observation puts “0” at the center of our normal distribution.
We conclude that we have obtained an “unexpected outcome” if the two sample
means are very different from one another.

Since we are using sample standard deviations to approximate population stan-

dard deviations with a small sample size, we must use a t-test for error correction.
The degrees of freedom calculation is quite complicated. The formula is in the
book, but understanding the derivation of the formula is beyond the scope of this
course. Instead, the degrees of freedom of the T distribution used in each test will
be an output of the calculator function that we use to run the test. You will see
that the degrees of freedom does increase with sample size, but not in an easily
predictable way.
The calculator function we will use for testing two means with independent sam-
ples when sample size is small (less than 30 combined) and population standard
deviations are uknown: 2-SampTTest.
Chapter 10 Notes Hypothesis Testing with Two Samples D. Skipper, p 3

1.3. Two Sample T-test summary and example. To summarize:

Requirements: Two independent simple random samples of quantitative data.
Sample sizes are small (less than 30 combined) and population standard deviations
are unknown.
Sample statistic: x̄1 − x̄2 = the difference in the sample means of our two actual
data sets.
Distribution: X̄1 − X̄2 = the difference in sample means of two randomly se-
lected
 q independent samples
 from the two populations of interest. X̄1 − X̄2 ∼
(s1 )2 (s2 )2
N 0, n1 + n2 . However, we must use the Student T distribution because
σ is unknown. The degrees of freedom for the t distribution has a complicated for-
mula for two independent samples. However, the degrees of freedom is calculated
as a result of running the calculator function 2-SampTTest.
Value assumed in the null hypothesis (center of the normal distribution):
µ1 − µ2 = 0. (We assume the two underlying populations have the same mean:
µ1 = µ2 .)
Hypotheses: Hypotheses have the following form:
H0 : µ1 (=, ≤, ≥) µ2
Ha : µ1 (6=, >, <) µ2 .
Outcome supporting the alternative hypothesis: When there is a big differ-
ence between x̄1 and x̄2 in the direction of the alternative hypothesis.
Test statistics: This is a t-score that tells us how many standard deviations our
sample statistic x̄1 − x̄2 is away from the assumed value of 0: t = r(x̄1 −x̄
2
2 )−0
2
.
(s1 ) (s2 )
n1 + n2

P-value: The probability of getting a difference between sample means this large or
larger (direction of the the alternative hypothesis) assuming the difference between
the population means is 0: p-value = tail area under the tdf curve beyond the test
statistic t in the direction of the alternative hypothesis.
Calculator test: 2-SampTTest

Example 2. Compare two population means: independent samples; σ

unknown. A simple random sample of 13 four-cylinder cars is obtained, and the
braking distance measured. The mean breaking distance is 137.5 ft and the standard
deviation is 5.8 ft. A simple random sample of 12 six-cylinder cars is obtained and
the braking distances have a mean of 136.3 ft with a standard deviation of 9.7 ft.
Use a 0.05 significance level to test the claim that the mean braking distance of
four-cylinder cars is greater than that of six-cylinder cars. Assume that the samples
appear to be somewhat normally distributed
(1) Write the null and alternative hypotheses. Express both using both forms.
(For example, Ha : µ1 < µ2 can also be expessed as Ha : µ1 − µ2 < 0.)
(2) Calculate the sample statistic x̄1 − x̄2 . Do you suspect that we will reject
the assumption that there is no difference between the population sample
means?
(3) Choose the distribution (t or z) and calculator function to use: Are we
testing a claim about a mean or proportion? Are σ1 and σ2 known or is
the combined sample size at least 30?
(4) Use the calculator function 2-SampTTest to find the test statistic and
p-value.
(5) Find the degrees of freedom for the t distribution in this case. (Output of
2-SampTTest.)
(6) What is your decision about the null hypothesis?
Chapter 10 Notes Hypothesis Testing with Two Samples D. Skipper, p 4

(7) State the conclusion about the braking distances of 4 cylinder versus 6
cylinder cars.

2. Two Population Means with Known Standard Deviations.

2.1. Requirements. A z-test for two samples (2-SampZTest) is appropriate under
the following conditions:
• We are comparing population means using two independent samples.
• The population standard deviations are known OR the combined sample
size is greater than 30. (In the latter case, it is safe to use the Z distribution,
treating s as σ.)
2.2. Distribution. This scenario is just like the last section, except this time we
somehow know the population standard deviation for both populations or the com-
bined sample size is at least 30. The distribution of X̄1 − X̄2 is the same, except
the sample standard deviations are replaced by population standard deviations:
 s 
(σ ) 2 (σ )2
1 2
X̄1 − X̄2 ∼ N µ1 − µ2 , + ,
n1 n2
so that the
(x̄1 − x̄2 ) − (µ1 − µ2 )
test statistic = q
(σ1 )2 (σ2 )2
n1 + n2
Of course, we don’t need the t-distribution when the population standard deviation
is known, so we use 2-SampZTest to find the test statistic and p-value.
2.3. Two Sample Z-test summary and example. To summarize:
Requirements: Two independent simple random samples of quantitative data.
Sample sizes are large (at least 30 combined) OR population standard deviations
are known.
Sample statistic: x̄1 − x̄2 = the difference in the sample means of our two actual
data sets.
Distribution: X̄1 − X̄2 = the difference in sample means of two randomly se-
lected
 q independent samples
 from the two populations of interest. X̄1 − X̄2 ∼
2 2
N 0, (σn11) + (σn22) .
Value assumed in the null hypothesis (center of the normal distribution):
µ1 − µ2 = 0.
Hypotheses: Hypotheses have the following form:
H0 : µ1 (=, ≤, ≥) µ2
Ha : µ1 (6=, >, <) µ2 .
Outcome supporting the alternative hypothesis: When there is a big differ-
ence between x̄1 and x̄2 in the direction of the alternative hypothesis.
Test statistics: This is a z-score that tells us how many standard deviations our
sample statistic x̄1 − x̄2 is away from the assumed value of 0: z = r(x̄1 −x̄
2
2 )−0
2
.
(σ1 ) (σ2 )
n1 + n2

P-value: The probability of getting a difference between sample means this large or
larger (direction of the the alternative hypothesis) assuming the difference between
the population means is 0: p-value = tail area under the Z (standard normal) curve
beyond the test statistic z in the direction of the alternative hypothesis = tail area
under the density curve for X̄1 − X̄2 beyond the sample statistic x̄1 − x̄2 .
Chapter 10 Notes Hypothesis Testing with Two Samples D. Skipper, p 5

Calculator test: 2-SampZTest

Example 3. Compare two population means: independent samples; σ

known. An interested citizen wanted to know if Democratic U. S. senators are
older than Republican U.S. senators, on average. On May 26, 2013, the mean age of
30 randomly selected Republican Senators was 61 years 247 days old (61.675 years)
with a standard deviation of 10.17 years. The mean age of 30 randomly selected
Democratic senators was 61 years 257 days old (61.704 years) with a standard
deviation of 9.55 years. Do the data indicate that Democratic senators are older
than Republican senators, on average? Test at a 5% level of significance. (Why is
this a z-test instead of a t-test?) TryIt 9.14.

3. Comparing two independent population proportions.

In this section, we compare the proportions of two populations. We use two in-
dependent samples, one from each population, and count the number of “successes”
in each to arrive at sample proportions for each population. We compare the dif-
ference in the two sample proportions p01 − p02 to test if they are different enough
to provide evidence that the underlying populations have different proportions.

3.1. Requirements. It is easy to see this is a binomial scenario, so the require-

ments are similar to those for testing a claim about a single population proportion.
The requirements for testing a claim about two population proportions using 2-
PropZTest are
• The two samples are binomially distributed independent simple random
samples.
• There are at least 5 successes (x = np) and at least 5 failures (n − x = nq)
in each of the two samples.
When comparing two population proportions we are, of course, interested in the
two sample proportions: p01 = nx11 and p02 = nx22 . Also of interest is the “pooled
proportion”, which is the proportion of successes in both samples combined:
x1 + x2
pooled proportion = pc = .
n1 + n2
The distribution for the differences of the sample proportions can be approxi-
mated by a normal distribution under the conditions stated above,
s  !
1 1
P10 − P20 ∼N p1 − p2 , pc (1 − pc ) + ,
n1 n2
which makes the test statistic the z-score,

(p0 − p02 ) − (p1 − p2 )

z=r 1  .
pc (1 − pc ) n11 + n12

As in the cases above when comparing sample means, we assume the populations
have the same proportion of successes in the null hypothesis, so the center of the
associated normal distribuiton is p1 − p2 = 0.
Chapter 10 Notes Hypothesis Testing with Two Samples D. Skipper, p 6

3.2. Two sample test of proportions summary and example. To summarize:

Requirements: Two independent simple random samples of binomially distributed
data. At least 5 successes and 5 failures in EACH sample.
Sample statistic: p01 − p02 = the difference between the sample proportions of our
two actual data sets: p01 = nx11 and p02 = nx22 .
Value assumed in the null hypothesis (center of the normal distribution):
p1 − p2 = 0. (We assume the populations have the same proportion of successes:
p1 = p2 .)
Hypotheses: Hypotheses have the following form:
H0 : p1 (=, ≤, ≥) p2
Ha : p1 (6=, >, <) p2 .
Distribution: P10 − P20 = the difference between the sample proportions of two
randomly
 selected
r independent samples from the two populations of interest. P10 −
 
P20 ∼ N 0, pc (1 − pc ) n11 + n12
Outcome that supports the null hypothesis: When there is a big difference
between p01 and p02 in the direction of the alternative hypothesis.
Test statistics: This is a z-score that tells us how many standard deviations our
(p01 −p02 )−0
sample statistic p01 −p02 is away from the assumed value of 0: z = r  
1
pc (1−pc ) n1 + n1
2

P-value: The probability of getting a difference between sample proportions this

large or larger (in the direction of the the alternative hypothesis) assuming the
populations have the same proportion of successes.
Calculator test: 2-PropZTest

Example 4. Compare two population proportions. Two types of phone

operating system are being tested to determine if there is a difference in the pro-
portions of system failures (crashes). Fifteen out of a random sample of 150 phones
with OS1 had system failures within the first eight hours of operation. Nine out of
another random sample of 150 phones with OS2 had system failures within the first
eight hours of operation. OS2 is believed to be more stable (have fewer crashes)
than OS1 . Complete a hypothesis test at the α = 0.05 significance level. Chapter 10
Practice 46-50.

4. Comparing population means: Matched or paired samples.

Here we consider two sets of data collected from a single random sample: the
test scores of a group of students before and after taking a prep course, for example.
In this case it makes sense to subtract the two test scores of a single student to find
the difference in the two scores. If we do this for every student, we create a new
set of data called the differences, d. For each student, we calculate the difference:
after score − before score = score difference,
y − x = d.
We end up with a single set of data to run a hypothesis test on, so we can simply
run a T-Test with n − 1 degrees of freedom, as in the last chapter.

4.1. Two Paired Samples T-test summary and example. To summarize:

Requirements: Two paired quantitative data sets from a single simple random
sample. If the sample size is small (less than 30) the data must be approximately
Chapter 10 Notes Hypothesis Testing with Two Samples D. Skipper, p 7

normal. Often paired tests have small sample size with unknown population stan-
dard deviation. If so, we must apply the Student T distribution.
Sample statistic: d¯ = the sample mean of the differences of the paired data
¯ subtract each data pair, then find the mean of the differences:
values. To get d,
(1) For each subject, calculate d = y − x.
¯
(2) Find the mean of the d values, d.
(Note that for independent samples, the sample statitstic is the difference of the
sample means: find the mean of the two samples, then subtract those two values:
x¯1 − x¯2 .)
Value assumed in the null hypothesis (center of the normal distribution):
µd = 0 (On average, there is no difference between the two data values collected
from each subject).
Hypotheses: Hypotheses have the following form:
H0 : µd (=, ≤, ≥) 0
Ha : µd (6=, >, <) 0.
Distribution: D̄ = the sample mean of the differences of paired data values from
samples of size n: D̄ ∼ N (0, √sdn ), but because the population standard deviations
are unknown, we use tdf = tn−1 .
Outcome that supports the alternative hypothesis: When d¯ is very different
from 0, in the direction of the alternative hypothesis.
Test statistics: This is a t-score that tells us how many standard deviations our
¯
sample statistic d¯ is away from the assumed value of 0: t = d−0s
√d
n
P-value: The probability of getting a sample mean of differences this large or
larger (direction of the the alternative hypothesis) assuming there is no difference
on average between the two data values for each subject (µd = 0).
Calculator test: TTest

Example 5. Compare two population means; paired quantitative data.

A study was conducted to investigate the effectiveness of hypnotism in reducing
pain. Results for randomly selected subjects are shown in the table below. A lower
score indicates less pain. The “before” value is matched to an “after” value and
the differences are calculated. The differences have a normal distribution. Are the
sensory measurements, on average, lower after hypnotism? Test at a 5% significance
level.
Subject Before After
A 6.6 6.8
B 6.5 2.4
C 9.0 7.4
D 10.3 8.5
E 11.3 8.1
F 8.1 6.1
G 6.3 3.4
H 11.6 2.0
Example 10.11.