STATS 200: Introduction to Statistical Inference

Lecture 1: Course introduction and polling

 U.S. presidential election projections by state

(Source: fivethirtyeight.com, 25 September 2016)

 Polling

Let’s try to understand how polling can be used to determine the

popular support of a candidate in some state (say, Iowa).

Key quantities:
I N = 3,046,355 – population of Iowa
# people who support Hillary Clinton
I p= N
I 1− p = # people who support
N
Donald Trump

We know N but we don’t know p.

 Polling

Let’s try to understand how polling can be used to determine the

popular support of a candidate in some state (say, Iowa).

Key quantities:
I N = 3,046,355 – population of Iowa
# people who support Hillary Clinton
I p= N
I 1− p = # people who support
N
Donald Trump

We know N but we don’t know p.

Question #1: What is p?

 Polling

Let’s try to understand how polling can be used to determine the

popular support of a candidate in some state (say, Iowa).

Key quantities:
I N = 3,046,355 – population of Iowa
# people who support Hillary Clinton
I p= N
I 1− p = # people who support
N
Donald Trump

We know N but we don’t know p.

Question #1: What is p?

Question #2: Is p > 0.5?
 Polling

Let’s try to understand how polling can be used to determine the

popular support of a candidate in some state (say, Iowa).

Key quantities:
I N = 3,046,355 – population of Iowa
# people who support Hillary Clinton
I p= N
I 1− p = # people who support
N
Donald Trump

We know N but we don’t know p.

Question #1: What is p?

Question #2: Is p > 0.5?
Question #3: Are you sure?
 Simple random sample

Suppose we poll a simple random sample of n = 1000 people

from the population of Iowa. This means:
I Person 1 is chosen at random (equally likely) from all N
people in Iowa. Then person 2 is chosen at random from the
remaining N − 1 people. Then person 3 is chosen at random
from the remaining N − 2 people, etc.
Or equivalently, all Nn = n!(N−n)!
N!


I possible sets of n people
are equally likely to be chosen.
Then we can estimate p by
# sampled people who support Hillary Clinton
p̂ =
n
 Simple random sample

Say 540 out of the 1000 people we surveyed support Hillary, so

p̂ = 0.54.

Does this mean p = 0.54? Does this mean p > 0.5?

 Simple random sample

Say 540 out of the 1000 people we surveyed support Hillary, so

p̂ = 0.54.

Does this mean p = 0.54? Does this mean p > 0.5?

No! Let’s call our data X1 , . . . , Xn :

(
1 if person i supports Hillary
Xi =
0 if person i supports Donald

X1 + X2 + . . . + Xn
Then p̂ = .
n

The data X1 , . . . , Xn are random, because we took a random

sample. Therefore p̂ is also random.
 Understanding the bias

p̂ is a random variable—it has a probability distribution.

We can ask: What is E[p̂]? What is Var[p̂]? What is the

distribution of p̂?
 Understanding the bias

p̂ is a random variable—it has a probability distribution.

We can ask: What is E[p̂]? What is Var[p̂]? What is the

distribution of p̂?

Each of the N people of Iowa is equally likely to be the i th person

that we sampled. So each Xi ∼ Bernoulli(p), and E[Xi ] = p.
 
X1 + . . . + Xn 1
E[p̂] = E = (E[X1 ] + . . . + E[Xn ]) = p
n n

Interpretation: The “average value” of p̂ is p.

We say that p̂ is unbiased.
 Understanding the variance

For the variance, recall that for any random variable X ,

Var[X ] = E[X 2 ] − (E[X ])2

Let’s compute E[p̂ 2 ]:

"  #
2 X1 + . . . + Xn 2
E[p̂ ] = E
n
1 
= 2 E X12 + . . . + Xn2 + 2(X1 X2 + X1 X3 + . . . + Xn−1 Xn )

n
 Understanding the variance

For the variance, recall that for any random variable X ,

Var[X ] = E[X 2 ] − (E[X ])2

Let’s compute E[p̂ 2 ]:

"  #
2 X1 + . . . + Xn 2
E[p̂ ] = E
n
1 
= 2 E X12 + . . . + Xn2 + 2(X1 X2 + X1 X3 + . . . + Xn−1 Xn )

n    
1 n
= 2 n E[X12 ] + 2 E[X1 X2 ]
n 2
1 n−1
= E[X12 ] + E[X1 X2 ]
n n
 Understanding the variance
From the previous slide:
1 n−1
E[p̂ 2 ] = E[X12 ] + E[X1 X2 ]
n n
Since X1 is 0 or 1, X1 = X12 . Then E[X12 ] = E[X1 ] = p.

Q: Are X1 and X2 independent?

 Understanding the variance
From the previous slide:
1 n−1
E[p̂ 2 ] = E[X12 ] + E[X1 X2 ]
n n
Since X1 is 0 or 1, X1 = X12 . Then E[X12 ] = E[X1 ] = p.

Q: Are X1 and X2 independent?

A: No.

E[X1 X2 ] = P[X1 = 1, X2 = 1] = P[X1 = 1] P[X2 = 1 | X1 = 1]

We have:
Np − 1
P[X1 = 1] = p, P[X2 = 1 | X1 = 1] =
N −1
 Understanding the variance

Var[p̂] = E[p̂ 2 ] − (E[p̂])2

 
1 n−1 Np − 1
= p+ p − p2
n n N −1
 Understanding the variance

Var[p̂] = E[p̂ 2 ] − (E[p̂])2

 
1 n−1 Np − 1
= p+ p − p2
n n N −1
   
1 n−1 1 n−1 N
= − p+ − 1 p2
n n N −1 n N −1
 Understanding the variance

Var[p̂] = E[p̂ 2 ] − (E[p̂])2

 
1 n−1 Np − 1
= p+ p − p2
n n N −1
   
1 n−1 1 n−1 N
= − p+ − 1 p2
n n N −1 n N −1
N −n n−N 2
= p+ p
n(N − 1) n(N − 1)
 Understanding the variance

Var[p̂] = E[p̂ 2 ] − (E[p̂])2

 
1 n−1 Np − 1
= p+ p − p2
n n N −1
   
1 n−1 1 n−1 N
= − p+ − 1 p2
n n N −1 n N −1
N −n n−N 2
= p+ p
n(N − 1) n(N − 1)
 
p(1 − p) N − n p(1 − p) n−1
= = 1−
n N −1 n N −1
 Understanding the variance

 
p(1 − p) n−1
Var[p̂] = 1−
n N −1
When N is much bigger than n, this is approximately p(1−p)
n , which
would be the variance if we sampled n people in Iowa with
replacement. (In that case p̂ would be a Binomial(n, p) random
n−1
variable divided by n.) The factor 1 − N−1 is the correction for
sampling without replacement.

For N = 3,046,355,p n = 1000, and p ≈ 0.54, the standard

deviation of p̂ is Var[p̂] ≈ 0.016.
 Understanding the sampling distribution
Finally, let’s look at the distribution of p̂. Suppose p = 0.54. We
can use simulation to randomly sample X1 , . . . , Xn from Np
people who support Hillary and N(1 − p) people who support
Donald, and then compute p̂. Doing this 500 times, here’s a
histogram of the 500 (random) values of p̂ that we obtain:
Histogram of p_hat
100
80
Frequency

60
40
20
0

0.50 0.52 0.54 0.56 0.58

p_hat
 Understanding the sampling distribution

p̂ looks like it has a normal distribution, with mean 0.54 and

standard deviation 0.016. Why?

Heuristically, if N is much larger than n, then X1 , . . . , Xn are

“almost independent”. If n is also reasonably large, then the
distribution of
√ √ (X1 − p) + . . . + (Xn − p)
n(p̂ − p) = n
n
is approximately N (0, p(1 − p)) by the Central Limit Theorem.

So p̂ is approximately N (p, p(1−p)

n ).
 A confidence statement

Recall that 95% of the probability density of a normal distribution

is within 2 standard deviations of its mean.

(0.54 − 2 × 0.016, 0.54 + 2 × 0.016) = (0.508, 0.572)

is a 95% confidence interval for p. In particular, we are more

than 95% confident that p > 0.5.
 Fundamental principle

We will assume throughout this course:

Data is a realization of a random process.

 Fundamental principle

We will assume throughout this course:

Data is a realization of a random process.

Why? Possible reasons:
1. We introduced randomness in our experimental design (for
example, polling or clinical trials)
2. We are actually studying a random phenomenon (for example,
coin tosses or dice rolls)
3. Randomness is a modeling assumption for something we don’t
understand (for example, errors in measurements)
 Statistical inference

Statistical inference = Probability−1

Probability: For a specified probability distribution, what are the
properties of data from this distribution?
iid
Example: X1 , . . . , X10 ∼ N (2.3, 1). What is P[X1 > 5]? What is
1
the distribution of 10 (X1 + . . . + X10 )?
 Statistical inference

Statistical inference = Probability−1

Probability: For a specified probability distribution, what are the
properties of data from this distribution?
iid
Example: X1 , . . . , X10 ∼ N (2.3, 1). What is P[X1 > 5]? What is
1
the distribution of 10 (X1 + . . . + X10 )?

Statistical inference: For a specified set of data, what are

properties of the distribution(s)?
iid
Example: X1 , . . . , X10 ∼ N (θ, 1) for some θ. We observe
X1 = 3.67, X2 = 2.24, etc. What is θ?
 Goals

In statistical inference, there is usually not a single right answer.

I For a given inferential question, what is a good (best?)
method of answering that question using data? How do we
compare different methods for answering the same question?
I How do we understand the error/uncertainty in our answer?
I How do we understand the dependence of our answer on our
modeling assumptions?
 Inference tasks

I Hypothesis testing: Asking a binary question about the

distribution. (Is p > 0.5?)
I Estimation: Determining the distribution, or some
characteristic of it. (What is our best guess for p?)
I Confidence intervals: Quantifying the uncertainty of our
estimate. (What is a range of values to which we’re
reasonably sure p belongs?)
 Course logistics

Webpage: stats200.stanford.edu

All course information (syllabus, office hours), lecture notes/slides,

and homeworks will be posted here.

Grades and other restricted content will be posted on Stanford

Canvas. (There’s a link in the above page.)
 Prerequisites

I Probability theory (STATS 116 or equivalent)

I Multivariable calculus (MATH 52 or equivalent)
Homework assignments will include simple computing exercises
asking you to perform small simulations, create histograms and
plots, and analyze data. You may use any language (e.g. R,
Python, Matlab) and will be graded only on your results, not on
the quality of your code.

Your TA Alex Chin will teach an Introduction to R section, time

and place TBD. The first couple homework assignments will also
walk you through how to do these things in R.
 Requirements

Homework: Due Wednesdays at the start of class. First homework

due next Wednesday, October 5.

Collaboration: You can work together on homework, but you must

submit your own write-up, in your own words and using your
own code for the programming exercises. Please indicate at
the top of your write-up the names of the students with whom you
worked.

Exams: One midterm, one final (both closed-book).

 Notes and textbook

Lectures will switch between slides and blackboard; I’ll post

slides/notes online after class. Readings are assigned from John A.
Rice, Mathematical Statistics and Data Analysis:

“Teaching two separate courses, one on theory and one on data

analysis, seems to me artificial.”—Rice
 For reference:

Morris H. DeGroot and Mark J. Schervish, Probability and Statistics

Larry Wasserman, All of Statistics: A concise course in statistical
inference
 “Students who analyze data, or who aspire to develop new
methods for analyzing data, should be well grounded in basic
probability and mathematical statistics. Using fancy tools like
neural nets, boosting, and support vector machines without
understanding basic statistics is like doing brain surgery before
knowing how to use a band-aid.”—Wasserman