Hofman Notes
Hofman Notes
Hofman Notes
Fall 2006
Heike Hofmann
October 1, 2007
Contents
1 Introduction 1
1.1 Basic Probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1.1 Examples for sample spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1.2 Examples for events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Basic Notation of Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Kolmogorov’s Axioms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.4 Counting Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.4.1 Two Basic Counting Principles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.4.2 Ordered Samples with Replacement . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.4.3 Ordered Samples without Replacement . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.4.4 Unordered Samples without Replacement . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.5 Conditional Probabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.6 Independence of Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.7 Bayes’ Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
1.8 Bernoulli Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2 Random Variables 23
2.1 Discrete Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.1.1 Expectation and Variance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.1.2 Some Properties of Expectation and Variance . . . . . . . . . . . . . . . . . . . . . . . 27
2.1.3 Probability Distribution Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.2 Special Discrete Probability Mass Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.2.1 Bernoulli pmf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.2.2 Binomial pmf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.2.3 Geometric pmf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.2.4 Poisson pmf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
2.2.5 Compound Discrete Probability Mass Functions . . . . . . . . . . . . . . . . . . . . . 34
2.3 Continuous Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
2.4 Some special continuous density functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
2.4.1 Uniform Density . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
2.4.2 Exponential distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
2.4.3 Erlang density . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
2.4.4 Gaussian or Normal density . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
2.5 Central Limit Theorem (CLT) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
3 Elementary Simulation 49
3.1 Basic Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.2 Random Number Generators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
3.2.1 A general method for discrete data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
i
0 CONTENTS
4 Stochastic Processes 59
4.1 Poisson Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
4.2 Birth & Death Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
5 Queuing systems 69
5.1 Little’s Law . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
5.2 The M/M/1 Queue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
5.3 The M/M/1/K queue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
5.4 The M/M/c queue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
5.5 Machine-Repairmen Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
6 Statistical Inference 79
6.1 Parameter Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
6.1.1 Maximum Likelihood Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
6.2 Confidence intervals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
6.2.1 Large sample C.I. for µ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
6.2.2 Large sample confidence intervals for a proportion p . . . . . . . . . . . . . . . . . . . 89
6.2.2.1 Conservative Method: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
6.2.2.2 Substitution Method: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
6.2.3 Related C.I. Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
6.3 Hypothesis Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
6.4 Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
6.4.1 Simple Linear Regression (SLR) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
6.4.1.1 The sample correlation r . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
6.4.1.2 Coefficient of determination R2 . . . . . . . . . . . . . . . . . . . . . . . . . . 100
6.4.2 Simple linear Regression Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
Introduction
Motivation
In every field of human life there are processes that cannot be described exactly (by an algorithm). For
example, how fast does a web page respond? when does the bus come? how many cars are on the parking
lot at 8.55 am?
By observation of these processes or by experiments we can detect patterns of behavior, such as: “ usually,
the first week of semester the campus network is slow”, “by 8.50 am the parking lot at the Design building
usually is full”.
Our goal is to analyze these patterns further:
1
2 CHAPTER 1. INTRODUCTION
Ω1 in tabular form:
Example 1.2.1
2. time required to retrieve and display a particular web page. Let A, B, C and D be events: A =
[100, 200), B = [150, ∞), C = [200, ∞) and D = [50, 75].
Then A ∪ B = [100, ∞) and A ∪ C = [100, ∞) and A ∪ D = [50, 75] ∪ [100, 200]
Example 1.2.2
Example 1.2.3
3. message example
Let D be the event that a message gets through.
D̄ = { ( D,D,U), (D,U,D), (U,D,D), (D,D,D) }.
4 CHAPTER 1. INTRODUCTION
A∩B =∅
Example 1.3.1
Draw a single card from a standard deck of playing cards
Ω = { red, black }
Model 1 Model 2
P (Ω) = 1 P (Ω) = 1
P ( red ) = 0.5 P ( red ) = 0.3
P ( black ) = 0.5 P ( black ) = 0.7
Beginning from the axioms of probability one can prove a number of useful theorems about how a probability
model must operate.
We start with the probability of Ω and derive others from that.
Theorem 1.3.1
Let A be an event in Ω, then
P (Ā) = 1 − P (A) for all A ⊂ Ω.
For the proof we need to consider three main facts and piece them together appropriately:
3. A and Ā are disjoint and therefore the probability of their union equals the sum of the individual
probabilities (axiom iii).
All together:
(1) (2) (3)
1 = P (Ω) = P (A ∪ Ā) = P (A) + P (Ā).
This yields the statement. 2
Example 1.3.2
3. If I believe that the probability that a message gets through is 0.9, I also must believe that it fails with
probability 0.1
Corollary 1.3.2
The probability of the empty set P (∅) is zero.
For a proof of the above statement we exploit that the empty set is the complement of Ω. Then we can
apply Theorem 1.3.1.
Thm 1.3.1
P (∅) = P (Ω̄) = 1 − P (Ω) = 1 − 1 = 0.
2
P (A ∪ B) = P (A) + P (B) − P (A ∩ B)
To see why this makes sense, think of probability as the area in the Venn diagram: By simply adding P (A)
and P (B), P (A ∩ B) gets counted twice and must be subtracted off to get P (A ∪ B).
Example 1.3.3
6 CHAPTER 1. INTRODUCTION
1. AOL dial-up:
If I judge:
P ( phone up ) = 0.9
P ( network up ) = 0.6
P ( phone up, network up ) = 0.55
then
P ( phone up or network up) = 0.9 + 0.6 − 0.55 = 0.95
diagram:
phone
up down
network up .55 .05 .60
down .35 .05 .40
.90 .10 1
C A
Ω
From the Venn diagram we see that the students who have a computer but no laptop are inside C but not
inside L, i.e. in (a) we are looking for C ∩ L̄. Since 30 students out of the total of 50 students in C have a
laptop there are 20 remaining students who have a computer but no laptop. this corresponds to a probability
of 20%: P (C ∩ L̄) = 0.2
In (b) we are looking for the intersection of C and A. We cannot compute this value exactly, but we can
give an upper and a lower limit:
1.3. KOLMOGOROV’S AXIOMS 7
and since
which can not be greater than 1, we know that P (A ∩ C) needs to be at least 0.4.
In short:
0.4 ≤ P (A ∩ C) ≤ 0.5
i.e. between 40 and 50 % of all students have both access to a lab and a computer of his/her own.
The number of students who have a laptop or a computer is just the number of students who have a computer,
since laptops are a subgroup of computers. Therefore
P (C ∪ L) = P (C) = 0.5.
Example 1.3.5
A box contains 4 chips, 1 of them is defective.
A person draws one chip at random.
What is a suitable probability that the person draws the defective chip?
Common sense tells us, that since one out of the four chips is defective, the person has a chance of 25% to
draw the defective chip.
Just for training, we will write this down in terms of probability theory:
One possible sample space Ω is: Ω = {g1 , g2 , g3 , d} (i.e. we distinguish the good chips, which may be a bit
artificial. It will become obvious, why that is a good idea anyway, later on.)
The event to draw the defective chip is then A = {d}.
We can write the probability to draw the defective chip by comparing the sizes of A and Ω:
|A| |{d}|
P (A) = = = 0.25.
|Ω| |{g1 , g2 , g3 , d}|
Be careful, though! The above method to compute probabilities is only valid in a special case:
Theorem 1.3.4
If all elementary events in a sample space are equally likely (i.e. P ({ωi }) = const for all ω ∈ Ω), the
probability of an event A is given by:
|A|
P (A) = ,
|Ω|
where |A| gives the number of elements in A.
Then
|E| 3
P (E) = = = 0.5.
|Ω| 6
Finding P (E) involves counting the number of outcomes in E. Counting by hand is sometimes not feasible
if Ω is large.
Therefore, we need some standard counting methods.
Multiplication Principle If a complex action can be broken down in a series of k components and these
components can be performed in respectively n1 , n2 , . . . , nk ways, then the complex action can be performed
in n1 · n2 · . . . · nk different ways.
Example 1.4.1
Toss a coin first, then toss a die: results in 2 · 6 = 12 possible outcomes of the experiment.
die
1
2
3
4
coin 5
6
H
T
1
2
3
4
5
6
Situation:
This is our first application of the multiplication principle: Instead of looking at the complex action, we
break it down into the k single draws. For each draw, we have n different possibilities to draw a ball.
The complex action can therefore be done in n · . . . · n} = nk different ways.
| · n {z
k times
The sample space Ω can be written as:
Example 1.4.2
(a) How many valid five digit octal numbers (with leading zeros) do exist?
In a valid octal number each digit needs to be between 0 and 7. We therefore have 8 choices for each
digit, yielding 85 different five digit octal numbers.
(b) What is the probability that a randomly chosen five digit number is a valid octal number?
One possible sample space for this experiment would be
85
P ( “randomly chosen five digit number is a valid octal number” ) = ≈ 0.328.
105
Again, we break up the complex action into k single draws and apply the multiplication principle:
Draw 1st 2nd 3rd ... kth
# of Choices n (n − 1) (n − 2) ... (n − k + 1)
total choices:
n!
n · (n − 1) · (n − 2) · . . . · (n − k + 1) =
(n − k)!
n!
The fraction (n−k)! is important enough to get a name of its own:
Example 1.4.4
(a) I only remember that a friend’s (4 digit) telephone number consists of the numbers 3,4, 8 and 9.
How many different numbers does that describe?
That’s the situation, where we take 4 objects out of a set of 4 objects and order them - that is P (4, 4)!.
4! 4! 24
P (4, 4) = = = = 24.
(4 − 4)! 0! 1
(b) In a survey, you are asked to choose from seven items on a pizza your favorite three and rank them.
How many different results will the survey have at most? - P (7, 3).
7!
P (7, 3) = = 7 · 6 · 5 = 210.
(7 − 3)!
Variation: How many different sets of “top 3” items are there? (i.e. now we do not regard the order
of the favorite three items.)
Think: The value P (7, 3) is the result of a two-step action. First, we choose 3 items out of 7. Secondly,
we order them.
Therefore (multiplication principle!):
P (7, 3) = X
|{z} · P (3, 3)
| {z } | {z }
# of ways to choose # of ways to choose # of ways to choose
3 from 7 and order them 3 out of 7 items 3 out of 3 and order them
So:
P (7, 3) 7! 7·6·5
X= = = = 35.
P (3, 3) 4!3! 3·2·1
49 49!
|Ω| = = = 1906884.
5 5!44!
The odds to win a matching five are 1: 1 906 884 - they are about the same as to die from being struck
by lightning.
(b) What is the probability to have exactly three matching numbers?
Answering this question is a bit tricky. But: since the order of the five numbers you’ve chosen doesn’t
matter, we can assume that we picked the three right numbers at first and then picked two wrong
numbers.
Do you see it? That’s again a complex action that we can split up into two simpler actions.
We need to figure out first, how many ways there are to choose 3 numbers out of the right 5 numbers.
Obviously, this can be done in 53 = 10 ways.
Secondly, the number of ways to choose the remaining 2 numbers out of the wrong 49-5 = 44 numbers
is 44
2 = 231.
In total, we have 10 · 231 = 2310 possible ways to choose three right numbers, which gives a probability
of 11/90804 ≈ 0.0001.
Note: the probability to have exactly three right numbers was given as
5 49−5
3 5−3
P ( “3 matching numbers” ) = 49
5
We will come across these probabilities quite a few times from now on.
12 CHAPTER 1. INTRODUCTION
(b) What is the probability to win? (i.e to have at least three matching numbers)
In order to have a win, we need to have exactly 3, 4 or 5 matching numbers. We already know
the probabilities for exactly 3 or 5 matching numbers. What remains, is the probability for exactly 4
matching numbers.
If we use the above formula and substitute the 3 by a 4, we get
5 49−5
4 5−4 5 · 49
P ( “4 matching numbers” ) = 49
= 49
≈ 0.000128
5 5
Please note: In the previous examples we’ve used parentheses ( ), see definition , to indicate that the order
of the elements inside matters. These constructs are called tuples.
If the order of the elements does not matter, we use { } - the usual symbol for sets.
Conclusion: The probability of an event A may change if we know (before we start the experiment for A)
the outcome of another event B.
We need to add another term to our mathematical description of probabilities:
Comments The safety system at a nuclear reactor is an example for a “parallel system”
A parallel system consists of k components c1 , . . . , ck , that are arranged as drawn in the diagram 1.1.
C1
C2
1 2
Ck
The system works as long as there is at least one unbroken path between 1 and 2 (= at least one of the
components still works).
Under the assumption that all components work independently from each other, it is fairly easy to compute
the probability that a parallel system will fail:
A similar kind of calculation can be done for a “series system”. A series system, again, consists of k
supposedly independent components c1 , . . . , ck arranged as shown in diagram 1.2.
1 C1 C2 Ck 2
This time, the system only works, if all of its components are working.
Therefore, we can compute the probability that a series system works as:
Please note that based on the above probabilities it is easy to compute the probability that a parallel system
is working and a series system fails, respectively, as:
T hm1.3.1
P ( parallel system works ) = 1 − P ( parallel system fails)
T hm1.3.1
P ( series system fails ) = 1 − P ( parallel system works)
The probability that a system works is sometimes called the system’s reliability. Note that a parallel system
is very reliable, a series system usually is very unreliable.
Warning: independence and disjointness are two very different concepts!
1.7. BAYES’ RULE 15
Disjointness: Independence:
If A and B are disjoint, their intersection is empty, If A and B are independent events, the probabil-
has therefore probability 0: ity of their intersection can be computed as the
product of their individual probabilities:
P (A ∩ B) = P (∅) = 0.
P (A ∩ B) = P (A) · P (B)
B1
1/3
1/2 gold
1/3 B2
1/2 silver
1/3
B3
1 silver
The lines are marked by the probabilities, with which each step is done:
Choosing one box (at random) means, that all boxes are equally likely to be chosen: P (Bi ) = 13 for i = 1, 2, 3.
In the first box are two gold coins: A gold coin in this box is therefore chosen with probability 1.
The second box has one golden and one silver coin. A gold coin is therefore chosen with probability 0.5.
16 CHAPTER 1. INTRODUCTION
P (E1 ) = P ( choose Box 1 and pick one of the two golden coins) =
= P ( choose Box 1 ) · P ( pick one of the two golden coins |B1 ) =
1
= · 1.
3
and
P (E2 ) = P ( choose Box 2 and pick one of the two golden coins) =
= P ( choose Box 2 ) · P ( pick one of the two golden coins |B2 ) =
1 1 1
= · = .
3 2 6
The probability to choose a golden coin is the sum of P (E1 ) and P (E2 ) (since those are the only ways to get
a golden coin, as we’ve seen in the tree diagram).
1 1
P ( golden coin ) = + = 0.5.
3 6
Bi ∩ Bj = ∅ for all i, j
What is a cover, then? – You can think of a cover as several non-overlapping pieces, which in total contain
every possible case of the sample space, like pieces of a jig-saw puzzle e.g.
Compare with diagram 1.3.
1.7. BAYES’ RULE 17
The boxes from the last example, B1 , B2 , and B3 , are a cover of the sample space.
Theorem 1.7.2 (Total Probability)
this is a for- If the set B1 , . . . , Bk is a cover of the sample space Ω, we can compute the probability for an event A by
mal way for
“Divide and (cf. fig.1.4):
Conquer”
Xk
P (A) = P (Bi ) · P (A|Bi ).
i=1
Note: Instead of writing P (Bi )·P (A|Bi ) we could have written P (A∩Bi ) - this is the definition of conditional
probability cf. def. 1.5.1.
Figure 1.4: The probability of event A is put together as sum of the probabilities of the smaller pieces (theorem
of total probability).
The challenge in using this Theorem is to identify what set of events to use as cover, i.e. to identify in which
parts to dissect the problem.
Very often, the cover B1 , B2 , . . . , Bk has only two elements, and looks like E, Ē.
Tree Diagram:
B1 P(A| B1)
A B1
B2 P(A| B2)
A B2
Bk P(A| Bk)
A Bk
The probability of each node in the tree can be calculated by mul-
tiplying all probabilities from the root to the event (1st rule of tree
diagrams).
Summing up all the probabilities in the leaves gives P (A) (2nd
rule).
18 CHAPTER 1. INTRODUCTION
The denominator is a bit more tricky. But remember the theorem of total probabilities - we just need a proper
cover to compute this probability.
The way this particular problem is posed, we find a suitable cover in the events match and no match. Using
the theorem of total probability gives us:
P ( test pos. ) = P ( match ) · P ( test pos. | match ) + P ( no match ) · P ( test pos. | no match )
We have got the numbers for all of these probabilities! Plugging them in gives:
In total this gives a probability for the man with the positive test result to be a true match of slightly less
than 10%!
P ( match | test pos. ) = 10−6 · (1.1 · 10− 5.) = 1/11.
Is that result plausible? - If you look at the probability for a false positive test result and compare it with the
overall probability for a true DNA match, you can see, that the test is ten times more likely to give a positive
result than there are true matches.This means that, if 10 Mio people are tested, we would expect 10 people
to have a true DNA match. On the other hand, the test will yield additional 100 false positive results, which
gives us a total of 110 people with positive test results.
This, by the way, is not a property limited to DNA tests - it’s a property of every test, where the overall
percentage of positives is fairly small, like e.g. tuberculosis tests, HIV tests or - in Europe - tests for mad
cow disease.
P (Bj ∩ A) P (A | Bj ) · P (Bj )
P (Bj |A) = = Pk for all j and ∅ =
6 A ⊂ Ω.
P (A) i=1 P (A | Bi ) · P (Bi )
Example 1.7.3
A given lot of chips contains 2% defective chips. Each chip is tested before delivery.
However, the tester is not wholly reliable:
If the test device says the chip is defective, what is the probability that the chip actually is defective?
Example 1.8.1
Transmit binary digits through a communication channel with success = “digit received correctly”.
Toss a coin repeatedly, success = “head”.
Sample spaces
Ω1 = {0, 1}
Ω2 = {(0, 0), (0, 1), (1, 0), (1, 1)} = { 00, 01, 10, 11 }
| {z }
all two-digit binary numbers
..
.
Ωn = {n − digit binary numbers} = {n − tuples of 0s and 1s}
Probability assignment
We will assume, that only those darts count, that actually hit the dartboard.
If a player throws a dart and hits the board at random, the probability to hit
the red zone will be directly proportional to the red area. Since out of the nine
squares in total 8 are gray and only one is red, the probabilities are:
P ( red ) = 19 P ( gray ) = 98 .
A player now throws three darts, one after the other.
What are the possible sequences of red and gray hits, and what are their probabilities?
We have, again, a step-wise setup of the problem, we can therefore draw a tree:
1.8. BERNOULLI EXPERIMENTS 21
sequence probability
r
1
r rrr 93
g
8
r rrg 93
r
8
g rgr 93
g 82
rgg 93
r 8
r
grr 93
g 82
g
grg 93
r 82
g
ggr 93
g 83
ggg 93
Most of the time, however, we are not interested in the exact sequence in which the darts are thrown - but
in the overall result, how many times a player hits the red area.
This leads us to the notion of a random variable.
22 CHAPTER 1. INTRODUCTION
Chapter 2
Random Variables
If the value of a numerical variable depends on the outcome of an experiment, we call the variable a random
variable.
X(s) = k, if s consists of k hits to the red area and 3 − k hits to the gray area.
{s|X(s) = 2}
8 8 8
P ({s|X(s) = 2}) = P (rrg ∪ rgr ∪ grr) = P (rrg) + P (rgr) + P (grr) = + 3 + 3 = 0.03.
93 9 9
23
24 CHAPTER 2. RANDOM VARIABLES
Im(x) := X(Ω).
Depending on whether or not the image of a random variable is countable, we distinguish between discrete
and continuous random variables.
Example 2.0.5
1. Put a disk drive into service, measure Y = “time till the first major failure”.
Sample space Ω = (0, ∞).
Y has uncountable image → Y is a continuous random variable.
Note: this gives us an easy method to check, whether a function is a probability mass function!
2.1. DISCRETE RANDOM VARIABLES 25
Example 2.1.1
Which of the following functions is a valid probability mass function?
x -3 -1 0 5 7
1.
pX (x) 0.1 0.45 0.15 0.25 0.05
y -1 0 1.5 3 4.5
2.
pY (y) 0.1 0.45 0.25 -0.05 0.25
z 0 1 3 5 7
3.
pZ (z) 0.22 0.18 0.24 0.17 0.18
We need to check the two properties of a pmf for pX , pY and pZ .
1st property: probabilities between 0 and 1 ?
This eliminates pY from the list of potential probability mass functions, since pY (3) is negative.
The other two functions fulfill the property.
2nd
P property: sum of all probabilities is 1?
Pi p(xi ) = 1, so pX is a valid probability mass function.
i p(zi ) = 0.99 6= 1, so pZ is not a valid probability mass function.
83
pX (0) = P (X = 0) = P (ggg) = ≈ 0.70
93
82
pX (1) = P (X = 1) = P (rgg) + P (grg) + P (ggr) = 3 ·
≈ 0.26
93
8
pX (2) = P (X = 2) = P (rrg) + P (rgr) + P (grr) = 3 · 3 ≈ 0.03
9
1
pX (3) = P (X = 3) = P (rrr) = 3 ≈ 0.01
9
2. Roll of a fair die
Let Y be the number of spots on the upturned face of a die:
Obviously, Y is a random variable with image {1, 2, 3, 4, 5, 6}.
Assuming, that the die is a fair die means, that the probability for each side is equal. The probability
mass function for Y therefore is pY (i) = 61 for all i in {1, 2, 3, 4, 5, 6}.
3. The diagram shows all six faces of a particular die. If Z denotes the number of
spots on the upturned face after toss this die, what is the probability mass function
for Z?
Assuming, that each face of the die appears with the same probability, we have 1
possibility to get a 1 or a 4, and two possibilities for a 2 or 3 to appear, which gives
a probability mass function of:
x 1 2 3 4
p(x) 1/6 1/3 1/3 1/6
26 CHAPTER 2. RANDOM VARIABLES
A second common measure for describing a random variable is a measure, how far its values are spread out.
We measure, how far we expect values to be away from the expected value:
2.1. DISCRETE RANDOM VARIABLES 27
The variance
p is measured in squared units of X.
σ := V ar[X] is called the standard deviation of X, its units are the original units of X.
Theorem 2.1.5
For two random variables X and Y and two real numbers a, b holds:
Theorem 2.1.6
For a random variable X and a real number a holds:
(i) E[X 2 ] = V ar[X] + (E[X])2
(ii) V ar[aX] = a2 V ar[X]
Note: in contrast to the probability mass function, FX is defined on R (not only on the image of X).
Example 2.1.6 Roll a fair die
X = # of spots on upturned face
Ω = {1, 2, 3, 4, 5, 6}
pX (1) = pX (2) = . . . = pX (6) = 16
P Pbtc btc
F (X)(t) = i<t pX (i) = i=1 pX (i) = 6 , where btc is the truncated value of t.
Properties of FX The following properties hold for the probability distribution function FX of a random
variable X.
• 0 ≤ FX (t) ≤ 1 for all t ∈ R
• FX is monotone increasing, (i.e. if x1 ≤ x2 then FX (x1 ) ≤ FX (x2 ).)
• limt→−∞ FX (t) = 0 and limt→∞ FX (t) = 1.
• FX (t) has a positive jump equal to pX (xi ) at {x1 , x2 , x3 , . . .}; FX is constant in the interval [xi , xi+1 ).
Whenever no confusion arises, we will omit the subscript X.
P (s) = pk (1 − p)n−k .
Now we need to know, how many possibilities there are, to have k successes in n trials: think of the n trials
as numbers from 1 to n. To have k successes, we need to choose a set of k of these numbers out of the n
possible numbers. Do you see it? - That’s the Binomial coefficient, again.
pX (k) is therefore:
n k
pX (k) = p (1 − p)n−k .
k
This probability mass function is called the Binomial mass function.
The distribution function FX is:
btc
X n
FX (t) = pi (1 − p)n−i =: Bn,p (t)
i=0
i
This function is called the Binomial distribution Bn,p , where n is the number of trials, and p is the probability
for a success.
It is a bit cumbersome to compute values for the distribution function. Therefore, those values are tabled
with respect to n and p.
Example 2.2.1
Compute the probabilities for the following events:
A box contains 15 components that each have a failure rate of 2%. What is the probability that
15
1. P (exactly two out of the fifteen components are defective) = pX (2) = 2 0.022 0.9813 = 0.0323.
2. P (at most two components are broken) = P (X ≤ 2) = B15,0.02 (2) = 0.9638.
3. P ( more than three components are broken ) = P (X > 3) = 1 − P (X ≤ 3) = 1 − 0.9945 = 0.0055.
4. P ( more than 1 but less than 4 are broken ) = P (1 < X < 4) = P (X ≤ 3) − P (X ≤ 1) = 0.9945 −
0.8290 = 0.1655.
X ∼ Bn,p
Ω = {1, 2, 3, 4, . . .}
We still need to compute the distribution function FX . Remember, FX (t) is the probability for X ≤ t.
Instead of tackling this problem directly, we use a trick and look at the complementary event X > t. If X is
greater than t, this means that the first btc trials yields failures. This is easy to compute! It’s just (1 − p)btc .
Therefore the probability distribution function is:
• Y is less than 3?
P (Y < 3) = P (Y ≤ 2) = 1 − 0.952 = 0.0975
• the first job to time out is between the third and the seventh?
P (3 ≤ Y ≤ 7) = P (Y ≤ 7) − P (Y ≤ 2) = 1 − 0.957 − (1 − 0.952 ) = 0.204
e−λ λx
p(x) = for x = 0, 1, 2, 3, . . .
x!
λ is called the rate parameter.
P oλ (t) := FX (t) is the Poisson distribution (function).
We need to check that p(x) as defined above is actually a probability mass function, i.e. we need to check
whether the two basic properties (see theorem 2.1.2) are true:
Now, we need to remember from calculus that the exponential function has the series representation
∞
X xn
ex = .
n=0
n!
Computing E[X] and V ar[X] involves some math, but as it is not too hard, we can do the computation for
E[X]:
∞ ∞
X e−λ λx X λx
E[X] = x = e−λ x = for x = 0 the expression is 0
x=0
x! x=0
x!
∞ ∞
X λx X λx
= e−λ x = e−λ =
x=1
x! x=1
(x − 1)!
∞
−λ
X λx−1
= e λ x = start at x = 0 again and change summation index
x=1
(x − 1)!
∞
X λx
= e−λ λ x = e−λ λeλ = λ
x=0
(x)!
Example 2.2.4
A manufacturer of chips produces 1% defectives.
What is the probability that in a box of 100 chips no defective is found?
Let X be the number of defective chips found in the box.
So far, we would have modelled X as a Binomial variable with distribution B100,0.01 .
Then P (X = 0) = 100 0 0.99
100
0.010 = 0.366.
On the other hand, a defective chip can be considered to be a rare event, since p is small (p = 0.01). What
else can we do?
We expect 100 · 0.01 = 1 chip out of the box to be defective. If we model X as Poisson variable, we know,
that the expected value of X is λ. In this example, therefore, λ = 1.
−1 0
Then P (X = 0) = e 0!1 = 0.3679.
No big differences between the two approaches!
For larger k, however, the binomial coefficient nk becomes hard to compute, and it is easier to use the
Poisson approximation of Binomial pmf For large n, the Binomial distribution is approximated by
the Poisson distribution, where λ is given as np:
(np)k
n k
p (1 − p)n−k ≈ e−np
k k!
Rule of thumb: use Poisson approximation if n ≥ 20 and (at the same time) p ≤ 0.05.
Why does the approximation work? - We will have a closer look at why the Poisson distribution
approximates the Binomial distribution. This also explains why the Poisson is defined as it is.
Example 2.2.5 Typos
Imagine you are supposed to proofread a paper. Let us assume that there are on average 2 typos on a page
and a page has 1000 words. This gives a probability of 0.002 for each word to contain a typo.
The number of typos on a page X is then a Binomial random variable, i.e. X ∼ B1000,0.002 .
Let’s have a closer look at a couple of probabilities:
• the probability for no typo on a page is P (X = 0). We know, that
P (X = 0) = (1 − 0.002)1000 = 0.9981000 .
We can also write this probability as
1000
2
P (X = 0) = 1− (= 0.13506).
1000
From calculus we know, that x n
lim 1− = ex .
n→∞ n
Therefore the probability for no typo on the page is approximately
P (X = 0) ≈ e−2 (= 0.13534).
• and a last one: the probability for exactly three typos on a page is
1000
P (X = 3) = 0.0023 · 0.998997 (= 0.18063),
3
which is
997
23 23 −2
1000 · 999 · 998 2
P (X = 3) = · 1− ≈ ·e (= 0.18045)
3·2 1000 · 1000 · 1000 1000 3!
Example 2.2.6
A box contains 5 unmarked PowerPC G4 processors of different speeds:
2 400 mHz
1 450 mHz
2 500 mHz
Select two processors out of the box (without replacement) and let
X = speed of the first selected processor
Y = speed of the second selected processor
For a sample space we can draw a table of all the possible combinations of processors. We will distinguish
between processors of the same speed by using the subscripts 1 or 2 .
2nd processor
400 450 500 (mHz)
400 0.1 0.1 0.2
450 0.1 0.0 0.1
500 (mHz) 0.2 0.1 0.1
1st proc.
What is the probability for X = Y ?
this might be important if we wanted to match the chips to assemble a dual processor machine:
P (X > Y ) = pX,Y (400, 450) + pX,Y (400, 500) + pX,Y (450, 500) =
= 0.1 + 0.2 + 0.1 = 0.4.
Just as we had the notion of expected value for functions with a single random variable, there’s an expected
value for functions in several random variables:
X
E[h(X, Y )] := h(x, y)pX,Y (x, y)
x,y
The most important cases for h(X, Y ) in this context are linear combinations of X and Y .
For two variables we can measure how “similar” their values are:
Note, that this definition looks very much like the definition for the variance of a single random variable. In
fact, if we set Y := X in the above definition, the Cov(X, X) = V ar(X).
Cov(X, Y )
% := p
V ar(X) · V ar(Y )
read: “rho”
Facts about %:
• % is between -1 and 1
% is a measure of linear association between X and Y . % near ±1 indicates a strong linear relationship, %
near 0 indicates lack of linear association.
% therefore is
Cov(X, Y ) −500
%= p = = −0.25,
V ar(X)V ar(Y ) 2000
% indicates a weak negative (linear) association.
Note: so far, we’ve had a definition for the independence of two events A and B: A and B are independent,
if P (A ∩ B) = P (A) · P (B).
Random variables are independent, if all events of the form X = x and Y = y are independent.
Theorem 2.2.5
For two random variables X and Y and three real numbers a, b, c holds:
Note: by comparing the two results, we see that for two independent random variables X and Y , the
covariance Cov(X, Y ) = 0.
Properties of FX The following properties hold for the probability distribution function FX for random
variable X.
• 0 ≤ FX (t) ≤ 1 for all t ∈ R
• FX is monotone increasing, (i.e. if x1 ≤ x2 then FX (x1 ) ≤ FX (x2 ).)
• limt→−∞ FX (t) = 0 and limt→∞ FX (t) = 1.
f (x) is no Now, however, the situation is slightly different from the discrete case:
probability!
f (x) may
be > 1.
Definition 2.3.2 (density function)
For a continuous variable X with distribution function FX the density function of X is defined as:
0
fX (x) := FX (x).
Relationship between fX and FX Since the density function fX is defined as the derivative of the
distribution function, we can re-gain the distribution function from the density by integrating: Then
Rt
• FX (t) = P (X ≤ t) = −∞ f (x)dx
Rb
• P (a ≤ X ≤ b) = a
f (x)dx
Therefore, Z a
P (X = a) = P (a ≤ X ≤ a) = f (x)dx = 0.
a
Example 2.3.1
Let Y be the time until the first major failure of a new disk drive.
A possible density function for Y is
−y
e y>0
f (y) =
0 otherwise
First, we need to check, that f (y) is actually a density function. Obviously, f (y) is a non-negative function
on whole of R.
The second condition, f must fulfill to be a density of Y is
Z ∞ Z ∞
f (y)dy = e−y dy = −e−y |∞
0 = 0 − (−1) = 1
−∞ 0
What is the probability that the first major disk drive failure occurs within the first year?
Z 1
P (Y ≤ 1) = e−y dy = −e−y |10 = 1 − e−1 ≈ 0.63.
0
What is the distribution function of Y ?
Z t Z t
FY (t) = f (y)dy = e−y dy = 1 − e−t for all t ≥ 0.
∞ 0
2.4. SOME SPECIAL CONTINUOUS DENSITY FUNCTIONS 39
Summary:
discrete random variable continuous random variable
image Im(X) finite or countable infinite image Im(X) uncountable
probability distribution function:
P Rt
FX (t) = P (X ≤ t) = k≤btc pX (k) FX (t) = P (X ≤ t) = ∞
f (x)dx
probability mass function: probability density function:
0
pX (x) = P (X = x) fX (x) = FX (x)
expected value:
P R
E[h(X)] = x h(x) · pX (x) E[h(X)] = x
h(x) · fX (x)dx
variance:
V ar[X] =PE[(X − E[X])2 ] = V ar[X] =RE[(X − E[X])2 ] =
∞
= x (x − E[X])2 pX (x) = −∞ (x − E[X])2 fX (x)dx
We now know how to compute expected value and variance of a continuous random variable.
40 CHAPTER 2. RANDOM VARIABLES
1/
(b-a)
a b x
b2 − a2 1
= = (a + b).
2(b − a) 2
Z b
a+b 2 1 (b − a)2
V ar[X] = (x − ) dx = . . . = .
a 2 b−a 12
Example 2.4.1
The(pseudo) random number generator on my calculator is supposed to create realizations of U (0, 1) random
variables.
Define U as the next random number the calculator produces.
What is the probability, that the next number is higher than 0.85?
1
For that, we want to compute P (U ≥ 0.85). We know the density function of U : fU (u) = 1−0 = 1. Therefore
Z 1
P (U ≥ 0.85) = 1du = 1 − 0.85 = 0.15.
0.85
λe−λx if x ≥ 0
fX (x) =
0 otherwise
Mean, variance and distribution function are easy to compute. They are:
1
E[X] =
λ
1
V ar[X] =
λ2
0 if x < 0
Expλ (t) = FX (t) =
1 − e−λx if x ≥ 0
The following example will accompany us throughout the remainder of this class:
2.4. SOME SPECIAL CONTINUOUS DENSITY FUNCTIONS 41
f2
f1
f0.5
Figure 2.3: Density functions of exponential variables for different rate parameters 0.5, 1, and 2.
How long do we have to wait at most, to observe a first hit with a probability of 0.9?
This is a very different approach to what we have looked at so far!
Here, we want to find a t, for which P (Y ≤ t) = 0.9:
P (Y ≤ t) = 0.9
⇐⇒ 1 − e−2t = 0.9
⇐⇒ e−2t = 0.1
⇐⇒ t = −0.5 ln 0.1 ≈ 1.15 (min) - that’s approx. 69 s.
Memoryless property
Example 2.4.3 Hits on a web page
In the previous example I stated that we start to observe the web page a time point 0. Does the choice of
this time point affect our analysis in any way?
Let’s assume, that during the first minute after we started to observe the page, there is no hit.
What is the probability, that we have to wait for another 40 seconds for the first hit? - this implies an answer
to the question, what would have happened, if we had started our observation of the web page a minute
later - would we still get the same results?
42 CHAPTER 2. RANDOM VARIABLES
The probability we want to compute is a conditional probability. If we think back - the conditional probability
of A given B was defined as
P (A ∩ B)
P (A|B) :=
P (B)
Now, we have to identify, what the events A and B are in our case. The information we have is, that during
the first minute, we did not observe a hit =: B, i.e. B = (Y > 1). The probability we want to know, is that
we have to wait another 40 s for the first hit: A = wait for 1 min and 40 s for the first hit (= Y ≤ 5/3).
P (A ∩ B) P (Y ≤ 5/3 ∩ Y > 1)
P ( first hit within 5/3 min | no hit during 1st min) = P (A|B) = = =
P (B) P (Y > 1)
P (1 < Y ≤ 5/3) e−2 − e−10/3
= = = 0.736.
1 − P (Y < 1) e−2
P (Y ≤ t + s|Y ≥ s) = 1 − e−λt = P (Y ≤ t)
This means: a random variable with an exponential distribution “forgets” about its past. This is called the
memoryless property of the exponential distribution.
An electrical or mechanical device whose lifetime we model as an exponential variable therefore “stays as
good as new” until it suddenly breaks, i.e. we assume that there’s no aging process.
Expected value and variance of an Erlang distributed variable X can be computed using the properties of
expected value and variance for sums of independent random variables:
k k
X X 1
E[X] = E[ Yi ] = E[Yi ] = k ·
i=1 i=1
λ
k k
X X 1
V ar[X] = V ar[ Yi ] = V ar[Yi ] = k ·
i=1 i=1
λ2
In order to compute the distribution function, we need another result about the relationship between P oλ
and Expλ .
Theorem 2.4.3
If X1 , X2 , X3 , . . . are independent exponential random variables with parameter λ and (cf. fig. 2.4)
j
X
W := largest index j such that Xi ≤ T
i=1
<- occurrence
* * * * * times
0
X1 X2 X3 T
With this theorem, we can derive an expression for the Erlang distribution function. Let X be an Erlangk,λ
variable:
1st trick
Erlangk,λ (x) = P (X ≤ x) =
above theorem
X
= 1 − P (X > x) = 1 − P( Yi > x) =
i
| {z }
less than k hits observed
= 1 − P o( a Poisson r.v. with rate xλ ≤ k − 1) =
= 1 − P oλx (k − 1).
If we wait for the third hit, what is the probability that we have to wait more than 1 min?
Z := waiting time until the third hit has an Erlang(3,2) distribution.
Note:
The exponential distribution is a special case of an Erlang distribution:
Expλ = Erlang(k=1,λ)
Erlang distributions are used to model waiting times of components that are exposed to peak stresses. It is
assumed that they can withstand k − 1 peaks and fail with the kth peak.
We will come across the Erlang distribution again, when modelling the waiting times in queueing systems,
where customers arrive with a Poisson rate and need exponential time to be served.
Note: the parameters µ and σ 2 are actually mean and variance of X - and that’s what they are called.
f0,0.5
f0,1
f0,2
Figure 2.5: Normal densities for several parameters. µ determines the location of the peak on the x−axis,
σ 2 determines the “width” of the bell.
2.4. SOME SPECIAL CONTINUOUS DENSITY FUNCTIONS 45
Unfortunately, there does not exist a closed form for this integral - fµ,σ2 does not have a simple anti-
derivative. However, to get probabilities means we need to evaluate this integral. This leaves us with several
choices:
1. personal numerical integration uuuh, bad,
bad, idea
Basic facts about the normal distribution that allow the use of tables
(i) for X ∼ N (µ, σ 2 ) holds:
X −µ
Z := ∼ N (0, 1)
σ
This process is called standardizing X.
(this is at least plausible, since
1
E[Z] = (E[X] − µ) = 0
σ
1
V ar[Z] = V ar[X] = 1
σ2
(ii) Φ(−z) = 1 − Φ(z) since f0,1 is symmetric in 0 (see fig. 2.6 for an explanation).
f0,1
x
-z +z
Figure 2.6: standard normal density. Remember, the area below the graph up to a specified vertical line
represents the probability that the random variable Z is less than this value. It’s easy to see, that the areas
in the tails are equal: P (Z ≤ −z) = P (Z ≥ +z). And we already know, that P (Z ≥ +z) = 1 − P (Z ≤ z),
which proves the above statement.
Example 2.4.6
Suppose Z is a standard normal random variable.
46 CHAPTER 2. RANDOM VARIABLES
1. P (Z < 1) = ?
straight look-up
P (Z < 1) = Φ(1) = 0.8413.
2. P (0 < Z < 1) = ?
look-up
P (0 < Z < 1) = P (Z < 1) − P (Z < 0) = Φ(1) − Φ(0) = 0.8413 − 0.5 = 0.3413.
3. P (Z < −2.31) = ?
look-up
P (Z < −2.31) = 1 − Φ(2.31) = 1 − 0.9896 = 0.0204.
4. P (|Z| > 2) = ?
look-up
P (|Z| > 2) = P (Z < −2) + P (Z > 2) = 2(1 − Φ(2)) = 2(1 − 0.9772) = 0.0456.
(1) (2)
f0,1 f0,1
(3) (4)
f0,1 f0,1
Example 2.4.7
Suppose, X ∼ N (1, 2)
P (1 < X < 2) =?
X−1
A standardization of X gives Z := √ .
2
1−1 X −1 2−1
P (1 < X < 2) = P( √ < √ < √ )=
2 2 2
√
= P (0 < Z < 0.5 2) = Φ(0.71) − Φ(0) = 0.7611 − 0.5 = 0.2611.
Note that the standard normal table only shows probabilities for z < 3.99. This is all we need, though, since
P (Z ≥ 4) ≤ 0.0001.
Example 2.4.8
Suppose the battery life of a laptop is normally distributed with σ = 20 min.
Engineering design requires, that only 1% of batteries fail to last 300 min.
What mean battery life is required to ensure this condition?
Let X denote the battery life in minutes, then X has a normal distribution with unknown mean µ and
standard deviation σ = 20 min.
What is µ?
The condition, that only 1% of batteries is allowed to fail the 300 min limit translates to:
Normal distributions have a “reproductive property”, i.e. if X and Y are normal variables, then W :=
aX + bY is also a normal variable, with:
E[W ] = aE[X] + bE[Y ]
V ar[W ] = a2 V ar[X] + b2 V ar[Y ] + 2abCov(X, Y )
The normal distribution is extremely common/ useful, for one reason: the normal distribution approximates
a lot of other distributions. This is the result of one of the most fundamental theorems in Math:
Corollary 2.5.2
(a) for large n the binomial distribution Bn,p is approximately normal Nnp,np(1−p) .
(b) for large λ the Poisson distribution P oλ is approximately normal Nλ,λ .
(c) for large k the Erlang distribution Erlangk,λ is approximately normal N k , k
λ λ2
Why?
(a) Let X be a variable with a Bn,p distribution.
We know, that X is the result from repeating the same Bernoulli experiment n times and looking at
the overall number of successes. We can therefor, write X as the sum of n B1,p variables Xi :
X := X1 + X2 + . . . + Xn
X is then the sum of n independent, identically distributed random variables. Then, the Central
Limit Theorem states, that X has an approximate normal distribution with E[X] = nE[Xi ] = np and
V ar[X] = nV ar[Xi ] = np(1 − p).
48 CHAPTER 2. RANDOM VARIABLES
(b) it is enough to show the statement for the case that λ is a large integer:
Let Y be a Poisson variable with rate λ. Then we can think of Y as the number of occurrences in
an experiment that runs for time λ - that is the same as to observe λ experiments that each run
independently for time 1 and add their results:
Y = Y1 + Y2 + . . . + Yλ , with Yi ∼ P o1 .
Again, Y is the sum of n independent, identically distributed random variables. Then, the Central
Limit Theorem states, that X has an approximate normal distribution with E[Y ] = λ · 1 and V ar[Y ] =
λV ar[Yi ] = λ.
(c) this statement is the easiest to prove, since an Erlangk,λ distributed variable Z is by definition the sum
of k independently distributed exponential variables Z1 , . . . , Zk .
k
For Z the CLT holds, and we get, that Z is approximately normal distributed with E[Z] = kE[Zi ] = λ
and V ar[Z] = kV ar[Zi ] = λk2 .
Why do we need the central limit theorem at all? - first of all, the CLT gives us the distribution of the
sample mean in a very general setting: the only thing we need to know, is that all the observed values come
from the same distribution, and the variance for this distribution is not infinite.
A second reason is, that most tables only contain the probabilities up to a certain limit - the Poisson table
e.g. only has values for λ ≤ 10, the Binomial distribution is tabled only for n ≤ 20. After that, we can use
the Normal approximation to get probabilities.
Elementary Simulation
unless g is simple, k is small, and we are very lucky, we may not be able to solve these problems analytically.
Using simulation, we can do the following:
steps of Simulation:
1. Simulate some large number (say n) of values for each of the k variables X, Y, . . . , Z.
we then have a set of n k-tuples of the form
(Xi , Yi , . . . , Zi ) for i = 1, . . . , n
Vi = g(Xi , Yi , . . . , Zi ) for i = 1, . . . , n
3. then approximate
(a) P (a ≤ V ≤ b) by
#Vi : Vi ∈ [a, b]
n
49
50 CHAPTER 3. ELEMENTARY SIMULATION
(b) E[h(V )] by
n n
1X 1X
h(Vi ) i.e. E[V ] = Vi = V̄
n i=1 n i=1
(c) V ar[V ] by
n
1X
(Vi − V̄ )2
n i=1
Note: this sequence still depends on the choice of x0 , the so-called seed of the sequence. Choosing different
seeds yields different sequences.
That way, we get a sequence with elements in [0, m − 1].
We define ui := xmi .
The choice of the parameters a, c and m is crucial!
obviously, we want to get as many different numbers as possible - therefore m needs to be as large as possible
and preferably prime (that way we get rid of small cycles).
c = 2
a = 16807
m = 231 − 1
c = 0
a = 65539
m = 231
3.2. RANDOM NUMBER GENERATORS 51
It was widely used, before people discovered how bad it actually is: Knowing two successive random numbers
gives you the possibility to predict the next number pretty well. . . . that’s not, how rng s are supposed to
work.
For more information about random number generators and different techniques, how to produce and check
them, look at
http://crypto.mat.sbg.ac.at/results/karl/server/
State of the art at the moment is the Marsaglia-Multicarry-RNG:
/*
Whenever you need random integers or random reals in your
C program, just insert those six lines at (near?) the beginning
of the program. In every expression where you want a random real
in [0,1) use UNI, or use IUNI for a random 32-bit integer.
No need to mess with ranf() or ranf(lastI), etc, with their
requisite overheads. Choices for replacing the two multipliers
36969 and 18000 are given below. Thus you can tailor your own
in-line multiply-with-carry random number generator.
Armed with a Uniform rng, all kinds of other distributions can be generated:
52 CHAPTER 3. ELEMENTARY SIMULATION
This is less complicated than it looks. Have a look at figure 3.1. Getting the right x-value for a specific u
is done by drawing a horizontal line from the y-axis to the graph of F and following the graph down to the
x-axis. - This is, how we get the inverse of a function, graphically.
u1 x
u2 x
0 x
x1 x2 x3 x4 ... xn
Figure 3.1: Getting the value corresponding to ui is done by drawing a straight line to the right, until we hit
the graph of F , and following the graph down to xj .
FX
xo x
Figure 3.2: Starting at some value x0 any continuous distribution function has an inverse. In this example,
x0 = 1.
General Method:
For a given standard uniform variable U ∼ U(0,1) we define
−1
X := FX (U )
Why? For a proof of the above statement, we must compute the distribution function X has. Remember,
the distribution function of X, FX at value x is the probability that X is x or less:
trick
P (X ≤ x) = apply FX to both sides of the inequality
dfn of U
= P (FX (X) ≤ FX (x)) =
= P (U ≤ FX (x)) = U is a standard uniform variable,P (U ≤ t) = t
= F (x).
Therefore, X has exactly the distribution, we wanted it to have. 2
For specific densities there are a lot of different special tricks for simulating observations:
For all of the next sections, let’s assume that we have a sequence of independent standard uniform variables
U1 , U2 , U3 , . . .
Then
n
X
X := Xi ∼ Bn,p
i=1
and W := # of Xi until the first is 1
W ∼ Geometricp
then Y ∼ P oλ
To simulate a normal distribution, we need two sequences of standard uniform variables. Let U1 and U2 be
two independent standard uniform variables.
Define
−1/2
Z1 := [−2 ln U1 ] cos(2πU2 )
−1/2
Z2 := [−2 ln U1 ] sin(2πU2 )
Then both Z1 and Z2 have a standard normal distribution and are independent,
Z1 , Z2 ∼ N (0, 1)
and
X := µ + σZi ∼ N (µ, σ 2 )
3.3. EXAMPLES 55
3.3 Examples
Example 3.3.1 Simple electric circuit
#
# compute R:
R <- R1 + R2*R3/(R2 + R3)
#
# now get the estimates:
mean(R)
> [1] 149.9741
sd(R)
> [1] 2.134474
#
# ... the probability that R is less than 146 is given by the number of values
# that are less than 146 divided by 1000:
sum(R<146)/1000
> [1] 0.04
Question: What is the probability that at the end of the day you have inventory left in the stock.
Let I be the number of Blueberry IMacs in stock at the end of the day.
I = 20 − X + Y
# Example: MacMall
#
# Goal: generate 1000 Poisson values with lambda = 30
#
# Remember: 1 Poisson value needs several exponential values
# step 1: produce exponential values
u1 <- runif(33000)
e1 <- -1/30*log(u1)
sum(e1)
[1] 1099.096
#
# sum of the exponential values is > 1000, therefore we have enough values
# to produce 1000 Poisson values
#
# step 2:
# add the exponential values (cumsum is cumulative sum)
E1 <- cumsum(e1)
E1[25:35]
[1] 0.7834028 0.7926534 0.7929962 0.7959631 0.8060001 0.8572329 0.8670336
[8] 0.8947401 1.0182220 1.0831698 1.1001983
E1 <- floor(E1)
E1[25:35]
[1] 0 0 0 0 0 0 0 0 1 1 1
#
# Each time we step over the next integer, we get another Poisson value
# by counting how many exponential values we needed to get there.
#
# step 3:
# The ’table’ command counts, how many values of each integer we have
X <- table(E1)
X[1:10]
0 1 2 3 4 5 6 7 8 9
32 26 31 32 17 27 31 33 32 31
#
# we have 1099 values, we only need 1000
X <- X[1:1000]
#
# check, whether X is a Poisson variable (then, e.g. mean and variance
# must be equal to lambda, which is 30 in our example)
#
mean(X)
[1] 30.013
var(X)
[1] 29.84067
3.3. EXAMPLES 57
#
# generate another 1000 Poisson values, this time lambda is 15
Y <- rpois(1000,15)
# looks a lot easier!
#
# now compute the variable of interest: I is the number of Blueberry IMacs
# we have in store at the end of the day
I <- 20 - X + Y
#
# and, finally,
# the result we were looking for;
# the (empirical) probability, that at the end of the day there are still
# computers in the store:
sum(I > 0)/1000
[1] 0.753
Using simulation gives us the answer, that with an estimated probability of 0.753 there will be Blueberry
IMacs in stock at the end of the day.
i.e. we have the picture that for large values of n, p̂ has a density centered at the “true” value for P (V ∈ [a, b])
with small spread.
i.e. for large n p̂ is close to p with high probability.
Similarly, for Vi i.i.d, h(Vi ) are also i.i.d.
Then
n
1X
E[h̄] = E[h(Vi )] = E[h(V )]
n i=1
and
n
1 X
V ar[h̄] = V ar[h(Vi )] = V ar[h(V )]/n → 0 for n → ∞.
n2 i=1
Once again we have that picture for h̄, that the density for h̄ is centered at E[h(V )] for large n and has small
spread.
58 CHAPTER 3. ELEMENTARY SIMULATION
Chapter 4
Stochastic Processes
59
60 CHAPTER 4. STOCHASTIC PROCESSES
Note:
• X(t) can be thought of as the number of occurrences until time t.
• Similarly, X(t2 ) − X(t1 ) is the number of occurrences in the interval (t1 , t2 ].
• With the same argument, X(0) = 0 - ALWAYS!
• The distribution of X(t) is Poisson with rate λt, since:
O0 = 0
Oj = time of the jth occurrence =
= the first t for which X(t) ≥ j
Ij = Oj − Oj−1 for j = 1, 2, . . .
The time until the kth hit Ok is therefore given as the sum of inter-arrival times Ok = I1 + . . . + Ik .
Theorem 4.1.2
X(t) is a Poisson process with rate λ
⇐⇒ The inter-arrival times I1 , I2 , . . . are i.i.d. Expλ .
Further: the time until the kth hit Ok is an Erlangk,λ distributed variable, ⇐⇒ X(t) is a Poisson process
with rate λ.
This theorem is very important! - it links the Poisson, Exponential, and Erlang
distributions tightly together.
2. Evaluate the probability that the time till the first hit exceeds 10 seconds. Let Y be the time until the
first hit - then Y has an Exponential distribution with parameter λ = 10 per minute or λ = 1/6 per
second.
P (Y ≥ 10) = 1 − P (Y ≤ 10) = 1 − (1 − e−10·1/6 ) = e−5/3 = 0.1889.
3. Evaluate the mean and the variance of the time till the 4th hit. Let Z be the time till the 4th hit. Then
Z has an Erlang distribution with stage parameter k = 4 and λ = 10 per minute.
k 4
E[Z] = = = 0.4 minutes
λ 10
k 4
V ar[Z] = = = 0.04 minutes2 .
λ2 100
4.2. BIRTH & DEATH PROCESSES 61
4. Evaluate the probability that the time till the 4th hit exceeds 24 seconds.
P (Z > 24) = 1 − P (Z ≤ 24) = 1 − Erlang4,1/6 (24) =
table,p.786
= 1 − (1 − P o1/6·24 (4 − 1)) = P o4 (3) = 0.433
5. The number of hits in the first hour is Poisson with mean 600. You would like to know the probability
of more than 650 hits. Exact calculation isn’t really feasible. So approximate this probability and
justify your approximation. A Poisson distribution with large rate λ can be approximated by a normal
distribution (corollary from the Central Limit Theorem) with mean µ = λ and variance σ 2 = λ.
approx X−600 approx
Then X ∼ N (600, 600) → Z := √
600
∼ N (0, 1).
Then:
650 − 600
P (X > 650) = 1 − P (X ≤ 650) = 1 − P Z≤ √ ≈
600
table, p.789
≈ 1 − Φ(2.05) = 1 − 0.9798 = 0.0202.
Another interesting property of the Poisson process model that’s consistent with thinking of it as “random
occurrences” in time t, is
Theorem 4.1.3
Let X(t) be a Poisson process. Given that X(T ) = k, the conditional distribution of the time of the k
occurrences O1 , . . . , Ok is the same as the distribution of k ordered independent standard uniform variables
U(1) , U(2) , . . . , U(k) .
This tells us a way to simulate a Poisson process with rate λ on the interval (0, T ):
- first, draw a Poisson value w from P oλT . - This tells us, how many uniform values Ui we need to
simulate.
- secondly, generate w many standard uniform values u1 , . . . , uw
- define oi = T · u(i) , where u(i) is the ith smallest value among u1 , . . . , uw .
The above theorem tells us, that, if we pick k values at random from an interval (0, t), we can assume, that if
we order them, the distance between two successive values has an exponential distribution with rate λ = k/t.
So far, we are looking only at arrivals of events. Besides that, we could, for example, look at the number of
surfers that are on our web site at the same time.
There, we have departures as well and, related to that, the time each surfer stays - which we will call service
time (from the perspective of the web server).
This leads us to another model:
0 1 2 3 ...
job i 1 2 3 4 5 6 7 8 9 10
arrival time 0.10 0.40 0.78 1.06 1.36 1.84 1.87 2.04 3.10 4.42
job i 11 12 13 14 15 16 17 18 19 20
arrival time 4.46 4.66 4.68 4.89 5.01 5.56 5.56 5.85 6.32 6.99
job i 1 2 3 4 5 6 7 8 9 10
finishing time 0.22 0.63 1.61 1.71 1.76 1.90 2.32 2.68 3.42 4.67
job i 11 12 13 14 15 16 17 18 19 20
finishing time 5.31 5.54 5.59 5.62 5.84 6.04 6.83 7.10 7.23 7.39
Let X(t) be the number of jobs in the printer and its queue at time t. X(t) is a Birth & Death process.
2
1
0
0 2 4 6
Time (in h)
(b) What is the (empirical) probability that there are 5 jobs in the printer and its queue at some time t?
The empirical probability for 5 jobs in the printer is the time, X(t) is in state 5 divided by the total
time:
This implies, that, given the process is in state k, the probability to move to state
λk
k+1 is
µk + λk
µk
k−1 is .
µk + λk
i.e. Y itself is again an exponential variable, its rate is the the sum of the rates of B and D.
Knowing the distribution of Y , the staying time in state k, gives us, e.g. the possibility to compute the
mean staying time in state k. The mean staying time in state k is the expected value of an exponential
distribution with rate λk + µk . The mean staying time therefore is 1/(λk + µk ). We will mark this result by this is re-
sult (*)
(*) and use it below.
Note: A Poisson process with rate λ is a special case of a Birth & Death process, where the birth rates and
death rates are constant, λk = λ and µk = 0 for all k.
The analysis of this model for small t is mathematically difficult because of “start-up” effects - but in some
cases, we can compute the “large t” behaviour.
15
X(t)
5
0
20
0
200
0
In the picture, three different simulations of Birth & Death processes are shown. Only in the first case, the
process is stable (birth rate < death rate). The other two processes are unstable (birth rate = death rate
(2nd process) and birth rate > death rate (3rd process)).
Only if the B+D process is stable, it will find an equilibrium after some time - this is called the steady state
of the B+D process.
Mathematically, the notion of a steady state state translates to
lim P (X(t) = k) = pk for all k,
t→∞
P
where the pk are numbers between 0 and 1, with k pk = 1. The pk probabilities are called the steady state
probabilities of the B+D process, they form a density function for X.
At the moment it is not clear why the steady state probabilities need to exist at all - in fact, for some systems
they do not.
For the moment, though, we will assume, that they exist and try to compute them. On the way to the result
we will come across conditions under which they will actually exist.
We can figure out what the pk must be as follows:
is the long run rate of transitions from state k to k + 1 and, similarly, µk pk is the long run rate of transitions
from state k to state k − 1.
From the very simple principle, that overall everything that flows into state k has to flow out again, we get
the so-called balance equations for the steady state probabilities:
Balance equations
The Flow-In = Flow-Out Principle provides us with the means to derive equations between the steady state
probabilities.
1. For state 0
µ1 p1 = λ0 p0
0
0 1
λ0 1
i.e. p1 = µ1 p0 .
2. For state 1
µ1 p1 + λ1 p1 = λ0 p0 + µ2 p2
0 1
0 1 2
λ1 λ0 λ1 1 2
i.e. p2 = µ2 p1 = µ1 µ2 p0 .
3. For state 2
µ2 p2 + λ2 p2 = λ0 p0 + µ3 p3
0 1
0 1 2
λ2 λ0 λ1 λ2 1 2
i.e. p3 = µ3 p2 = µ1 µ2 µ3 p0 .
ok, so now we know all the steady state probabilities depending on p0 . But what use has that, if we don’t
know p0 ?
Here, we need another trick: we know, that the steady state probabilities are the density function for the
state X. Their sum must therefore be 1!
Then
1 = p0 + p1 + p2 + . . .
λ0 λ0 λ1
= p0 1 + + + ...
µ1 µ1 µ2
| {z }
:=S
If this sum S converges, we get p0 = S −1 . If it doesn’t converge, we know that we don’t have any steady
state probabilities, i.e. the B+D process never reaches an equilibrium. The analysis of S is crucial!
66 CHAPTER 4. STOCHASTIC PROCESSES
If S exists, p0 does, and with p0 all pk , which implies, that the Birth & Death process is stable.
If S does not exist, then the B & D process is unstable, i.e. it does not have an equilibrium and no steady
state probabilities.
Special case:
Birth & Death process with constant birth and death rates
If all birth rates λk = λ a constant birth rate and µk = µ for all k, the ratio between birth and death rates
is constant, too:
λ
a :=
µ
a is called the traffic intensity.
In order to decide, whether a specific B&D process is stable or not, we have to look at S. For constant traffic
intensities, S can be written as:
∞
λ0 λ0 λ1 X
S =1+ + + . . . = 1 + a + a2 + a3 + ... = ak
µ1 µ1 µ2
k=0
0 1 2 3
4 4 4 4
(b) What is the (true) probability that at some time t the printer is idle?
3
P (X(t) = 0) = p0 = 1 − = 0.25.
4
(c) What is the probability that there arrive more than 7 jobs during one hour?
Let Y be the number of arrivals. Y is a Poisson Process with arrival rate λ = 3.
Y (t) ∼ P oλ·t .
(d) What is the probability that the printer is idle for more than 1 hour at a time? (Hint: this is the
probability that X(t) = 0 and - at the same time - no job arrives for more than one hour.)
Let Z be the time until the next arrival, then Z ∼ Exp3 .
X(t),Zindependent
P (X(t) = 0 ∩ Z > 1) = P (X(t) = 0) · P (Z > 1) = p0 · (1 − Exp3 (1)) = 0.25 · e−3 = 0.0124
(d) What is the probability that there are 3 jobs in the printer queue at time t (including the job printed
at the moment)?
(e) What is the difference between the true and the empirical probability of exactly 5 jobs in th printer
system?
1 1 1 1
0 1 2 3 4
0.5 1 1 1
(b) Find the steady state probability that there are no messages in the system.
Since p0 is S −1 , we need to compute S first:
λ0 λ0 λ1 λ0 λ1 λ2 λ0 λ1 λ2 λ3
S = 1+ + + + =
µ1 µ1 µ2 µ1 µ2 µ3 µ1 µ2 µ3 µ4
= 1 + 2 + 2 + 2 + 2 = 9.
Therefore p0 = 19 .
68 CHAPTER 4. STOCHASTIC PROCESSES
(c) Find the steady state probability mass function of X(t) (i.e. find the other pk s)
(d) How many messages are in the system on average once it has reached its stable state?
1 1 1 1 1
0 1 2 3 4
(b) As it turns out, the large t probability that there are no customers in the system is p0 = 1/9.
What is the probability that a customer entering the bank must enter the queue and wait for service?
A person entering the bank must queue for service, if at least three people are in the bank (not including
the one who enters at the moment). We are therefore looking for the large t probability, that X(t) is
at least 3:
Queuing systems
Queueing system
server 1
enter the system server 2
exit the
some population of according to some
individuals random mechanism system
server c
Depending upon the specifics of the application there are many varieties of queuing systems corresponding
combinations like
• etc.
69
70 CHAPTER 5. QUEUING SYSTEMS
L = λ̄ · W
where L is the average number of individuals in the system, W is the average time spent in the system, and
λ̄ is the average rate at which individuals enter the system.
Lq = λ¯q · Wq
L = E[X(t)]
W = L/λ
WS = E[S] Wq = W - WS Lq = Wq λ
We’ve already seen that the ratio λ/µ is very important for the analysis of the B&D process. This ratio is
called the traffic intensity a. For a M/M/1 queuing system, the traffic intensity is constant for all k.
The previous problem of finding the steady state probabilities of the B&D process is equivalent to finding
the steady state probabilities for the number of individuals in the queuing system,
1 = p0 (1 + a + a2 + . . .)
p0 = 1−a
p1 = a(1 − a)
p2 = a2 (1 − a)
p3 = a3 (1 − a)
...
pk = ak (1 − a) N (t) has a geometric distribution for large t!
The mean number of individuals in the queuing system L is limt→∞ E[N (t)]: mean num-
ber in sys-
tem
a
L = lim E[N (t)] = .
t→∞ 1−a
The closer the service rate is to the arrival rate, the larger is the expected number of people in the system. mean time
in system
The mean time spent in the system W is then, using Little’s Law:
1 1
W = L/λ = ·
µ 1−a
72 CHAPTER 5. QUEUING SYSTEMS
average time
The overall time spent in the system is a sum of the time spent in the queue Wq and the average time spent in the queue
in service Ws . Since we know that service times are exponentially distributed with rate µ, Ws = µ1 . For the
time spent waiting in the queue we therefore get
1 1 1 a
Wq = W − Ws = −1 = .
µ 1−a µ1−a
average length The average length of the queue is, using Little’s Law again, given as
of the queue
a2
Lq = Wq λ =
1−a
server uti- Further we see, that the long run probability that the server is busy is given as:
lization rate
distribution Denote by q(t) the time that an individual entering the system at time t has to spend waiting in the queue.
of time in
queue Clearly, the distribution of the waiting times depends on the number of individuals already in the queue at
time t.
Assume, that the individual entering the system doesn’t have to wait at all in the queue - that happens
exactly when the system at time t is empty. For large t we therefore get:
lim P (q(t) = 0) = p0 = 1 − a.
t→∞
Think: If there are k individuals in the queue, the waiting time q(t) is Erlangk,µ (we’re waiting for k
departures, departures occur with a rate of µ). This is a conditional distribution for q(t), since it is based
on the assumption about the number of people in the queue:
We can put those pieces together in order to get the large t distribution for q(t) using the theorem of total
probability. For large t and x ≥ 0:
Let X(t) be the number of jobs in the printer and its queue at time t.
We know already: X(t) is a Birth & Death Process with constant arrival rate λ = 3 and constant death rate
µ = 4.
The properties of interest for this printer system then are:
a 0.75
L = E[X(t)] = = =3
1−a 0.25
1
Ws = = 0.25 hours = 15 min
µ
L 3
W = = = 1 hour
λ 3
Wq = W − Ws = 0.75 hours = 45 minutes
Lq = Wq λq = 0.75 · 3 = 2.25
On average, a job has to spend 45 min in the queue. What is the probability that a job has to spend less
than 20 min in the queue?
We denoted the waiting time in the queue by q(t). q(t) has distribution function 1 − aey(µ−λ) . The probability
asked for is
P (q(t) < 2/6) = 1 − 0.75 · e−2/6·(4−3) = 0.4626.
λ λ λ λ
0 1 2 K
µ µ µ µ
Since X(t) has only a finite number of states, it’s a stable process independently from the values of λ and µ.
The steady state probabilities pk are:
pk = = ak p0
1−a
p0 = S−1 =
1 − aK+1
1−aK+1
where a = µλ , the traffic intensity and S = 1 + a + a2 + ... + aK = 1−a .
The mean number of individuals in the queue L then is:
L = E[X(t)] = 0 · p0 + 1 · p1 + 2 · p2 + ... + K · pK =
K K
X X a (K + 1)aK+1
= kpk = kak · p0 = ... = −
1−a 1 − aK+1
k=0 k=0
Another interesting property of a queuing system with limited size is the number of individuals that get
turned away. From a marketing perspective they are the ”expensive” ones - they are most likely annoyed
and less inclined to return. It’s therefore a good strategy to try and minimize this number.
74 CHAPTER 5. QUEUING SYSTEMS
Since an incoming individual is turned away, when the system is full, the probability for being turned away
is pK . The rate of individuals being turned away therefore is pK · λ.
For the expected total waiting time W , we used Little’s theorem:
L
W =
λ̄
where λ̄ is the average arrival rate into the system.
At this point we have to be careful when dealing with limited systems: λ̄ is NOT equal to the arrival rate
λ. We have to adjust λ by the rate of individuals who are turned away. The adjusted rate λa of individuals
entering the system is:
λa = λ − pK λ = (1 − pK )λ.
The expected total waiting time is then W = L/λa and the expected length of the queue Lq = Wq · λa .
Example 5.3.1 Convenience Store
In a small convenience store there’s room for only 4 customers. The owner himself deals with all the customers
- he likes chatting a bit. On average it takes a customer 4 minutes to pay for his/her purchase. Customers
arrive at an average of 1 per 5 minutes. If a customer finds the shop full, he/she will go away immediately.
1. What fraction of time will the owner be in the shop on his own?
The number of customers in the shop can be modelled as a Birth& death Process with arrival rate
λ = 0.2 per minute and µ = 0.25 per minute and upper size K = 4.
1−a 0.2
The probability (or fraction of time) that the owner will be alone is p0 = 1−aK+1
= 1−0.85 = 0.2975.
2. What is the mean number of customers in the store?
a (K + 1)aK+1
L= − = 1.56.
1−a 1 − aK+1
3. What fraction of customers is turned away per hour?
p4 λ = 0.84 · 0.2975 · 0.2 per minute = 0.0243 per minute = 1.46 per hour
L
W = = 1.56/(0.2 − 0.0243) = 8.88 minutes .
λa
For limited queueing systems the adjusted arrival rate λa must be considered for applying Little’s Law.
λ λ λ λ λ λ
0 1 2 K
c-1 1c
µ 2µ 3µ (c-1)µ cµ cµ
5.4. THE M/M/C QUEUE 75
Clearly, the critical thing here in terms of whether or not a steady state exists is whether or not λ/(cµ) < 1.
Let a = λ/µ and % = a/c = λ/(cµ).
The balance equations for steady state are:
p1 = ap0 ac
pc+1 = %· c! p0
a2
p2 = 2·1 p0 ...
a3 ac
p3 = 3! p0 pn = %n−c · c! p0 for n ≥ c.
...
ac
pc = c! p0
In order to get an expression for p0 , we use the condition, that the overall sum of probabilities must be 1.
This gives:
∞ ∞
c−1 k
! c−1 k
ac X k−c ac 1
X X a X a
1= pk = p0 + % = p0 + .
k! c! k=0 k!
c! 1 − %
k=0 k=0 k=c
| {z }
=:S
p0 = S −1 .
The steady state mean number of individuals in the queue Lq is average num-
ber in queue
∞ ∞
X X ak
Lq = (k − c)pk = (k − c) p0 =
c!ck−c
k=c k=c
c ∞
a X ac %
= p0 k%k = p0
c! c! (1 − %)2
k=1
| {z }
%( ∞ k 0
k=1 % )
P
%
= C(c, a).
1−%
76 CHAPTER 5. QUEUING SYSTEMS
%
L=W ·λ=a+ C(c, a).
1−%
c−1 k
!−1 −1
ac 1 4 23
X a 1 1
p0 = + = 1+2+ + · = .
k! c! 1 − % 2 3! 1 − % 9
k=0
ac %
Lq = p0 = 8/9.
c! (1 − %)2
Wq = Lq /λ = 8/9 minutes .
1
Ws = = 2 minutes.
µ
W = Ws + Wq = 26/9 minutes
L = W λ = 26/9
(c-1) c c c c
This system, though there are only K + 1 possible states, is computationally tricky.
5.5. MACHINE-REPAIRMEN PROBLEMS 77
Denote by a = λ/µ the traffic intensity. The steady state probabilities are given as
p1 = Kap0 K−c K
pc+1 = c apc = (K − c)% c ac p0
K(K−1) 2
p2 = 2·1 a p0 ...
K(K−1)(K−2) 3 k! k−c K
p3 = 3! a p0 pk = c! % k ac p0 for k ≥ c
...
K
pk = k ak p0 for k ≤ c
and
c−1 K !−1
X K k
X k! k−c K c
p0 = a + % a
k c! k
k=0 k=c
Once we have the steady state probabilities, we can compute the average number of machines waiting for
services, Lq , as:
XK
Lq = (k − c)pk
k=c+1
Example 5.5.1
A company has 3 machines that each break down at exponentially distributed intervals of mean 1 hour.
When one of these machines is down, only one repairman can work on it at a time, and while it is down the
company loses $50/hour in profit. Repairs require 1 hour on average to complete and the repair times are
exponentially distributed. Repairmen can be hired for $30/hour (and must be paid regardless of whether
there is a broken machine to work on). Do you recommend that the company employs 1 repairman or 2
repairmen? Show your whole analysis.
Let X(t) be the number of broken machines at time t. Then X(t) can be modelled by a B&D process for a)
a single repairman and b) two repairmen.
The two state diagrams are:
a) b)
0 1 2 3 0 1 2 3
1 = p0 (1 + 3 + 6 + 6) → p0 = 0.0625
and p1 = 0.1875, p2 = p3 = 0.375
2
1 = p0 (1 + 3 + 3 + 1.5) → p0 =
17
6 6 3
and p1 = , p2 = , p3 =
17 17 17
If hiring one repairman the company loses $30 each hour. On top of that each hour down time of each
machine costs another $50. Let L be the loss per hour.
a)
30 + 240 + 780 + 1080
E[L] = 30p0 + 80p1 + 130p2 + 180p3 = = 133.125.
16
b)
1
E[L] = (60 · 2 + 110 · 6 + 160 · 6 + 210 · 3) = 139.412
17
With only one repairman the overall costs for the company are slightly less than with two repairmen.
78 CHAPTER 5. QUEUING SYSTEMS
Chapter 6
Statistical Inference
From now on, we will use probability theory only to find answers to the questions arising from specific
problems we are working on.
In this chapter we want to draw inferences about some characteristic of an underlying population - e.g. the
average height of a person. Instead of measuring this characteristic of each individual, we will draw a sample,
i.e. choose a “suitable” subset of the population and measure the characteristic only for those individuals.
Using some probabilistic arguments we can then extend the information we got from that sample and make
an estimate of the characteristic for the whole population. Probability theory will give us the means to find
those estimates and measure, how “probable” our estimates are.
Of course, choosing the sample, is crucial. We will demand two properties from a sample:
• the sample should be representative - taking only basketball players into the sample would change our
estimate about a person’s height drastically.
• if there’s a large number in the sample we should come close to the “true” value of the characteristic
• estimation of parameters:
point or interval estimates: “my best guess for value x is . . . ”, “my guess is that value x is in interval
(a, b)”
79
80 CHAPTER 6. STATISTICAL INFERENCE
X̄ = n1 i Xi
P
Mean (Average)
Minimum X(1) - Parentheses indicate that the values are sorted
Maximum X(n)
Range X(n) − X(1)
Mode value(s) that appear(s) most often
Median “middle value” - that value, for which one half of the data is larger,
the other half is smaller. If n is odd the median is X(n/2) , if n is
even, the median is the average of the two middle values: 0.5 ·
X((n−1)/2) + 0.5 · X((n+1)/2)
For this section it is important to distinguish between xi and Xi properly. If not stated otherwise, any capital
letter denotes some random variable, a small letter describes a realization of this random variable, i.e.
what we have observed. xi therefore is a real number, Xi is a function, that assigns a real number to an
event from the sample space.
Example 6.1.1
Let X1 , . . . , Xn be n i.i.d. random variables with E[Xi ] = µ.
Pn
Then X̄ = n1 i=1 Xi is an unbiased estimator of µ, because
n
1X 1
E[X̄] = E[Xi ] = n · µ = µ.
n i=1 n
ok, so, once we have an estimator, we can decide, whether it has the properties. But how do we find
estimators?
6.1. PARAMETER ESTIMATION 81
f with =0
f with = -1.8
f with =1
By changing the value for θ we can “move the density function fθ around” - in the diagram, the third density
function fits the data best.
Principle: since we do not know the true value θ of the distribution, we take that value θ̂ that most likely
produced the observed values, i.e.
maximize something like
Xi are independent!
P (X1 = x1 ∩ X2 = x2 ∩ . . . ∩ Xn = xn ) =
= P (X1 = x1 ) · P (X2 = x2 ) · . . . · P (Xn = xn ) =
n
Y
= P (Xi = xi ) (*)
i=1
This is not quite the right way to write the probability, if X1 , . . . , Xn are continuous variables. (Remember:
P (X = x) = 0 for a continuous variable X; this is still valid)
We use the above “probability” just as a plausibility argument. To come around the problem that P (X =
x) = 0 for a continuous variable, we will write (*) as:
n
Y n
Y
pθ (xi ) and fθ (xi )
i=1 i=1
| {z } | {z }
for discreteXi for continuousXi
where pθ ) is the probability mass function of discrete Xi (all Xi have the same, since they are i.d) and fθ is
the density function of continuous Xi .
Both these functions depend on θ. In fact, we can write the above expressions as a function in θ. This
function, which we will denote by L(θ), is called the Likelihood function of X1 , . . . , Xn .
The goal is now, to find a value θ̂ that maximizes the Likelihood function. (this is what “moves” the density
to the right spot, so it fits the observed values well)
How do we get a maximum of L(θ)? - by the usual way, we maximize a function! - Differentiate it and
set it to zero! (After that, we ought to check with the second derivative, whether we’ve actually found a
maximum, but we won’t do that unless we’ve found more than one possible value for θ̂.)
Most of the time, it is difficult to find a derivative of L(θ) - instead we use another trick, and find a maximum
for log L(θ), the Log-Likelihood function.
Note: though its name is “log”, we use the natural logarithm ln.
The plan to find an ML-estimator is:
1. Find Likelihood function L(θ).
2. Get natural log of Likelihood function log L(θ).
3. Differentiate log-Likelihood function with respect to θ.
82 CHAPTER 6. STATISTICAL INFERENCE
5. Solve for θ.
1 2 3 4 5 6 7 8 9 11 14 16 20 27 29
k 1 2 3 4 5 6 7 8 9 11 14 15 16 17 20 21 27 29
# trials 18 20 8 9 9 5 8 3 5 3 3 3 1 1 1 1 1 1
We know, that k the number of rolls until a 6 shows up has a geometric distribution Geop . For a fair die, p
is 1/6.
The Geometric distribution has probability mass function p(k) = (1 − p)k−1 · p.
What is the ML-estimate p̂ for p?
100
Y 100
Y P100 P100
L(p) = (1 − p)ki −1 p = p100 · (1 − p)ki −1 = p100 · (1 − p) i=1 (ki −1) = p100 · (1 − p) i=1 ki −100
.
i=1 i=1
P100
log L(p) = log p100 · (1 − p) i=1 ki −100 =
P100
= log p100 + log (1 − p) i=1 ki −100 =
100
!
X
= 100 log p + ki − 100 log(1 − p).
i=1
6.1. PARAMETER ESTIMATION 83
d
log L(p̂) = 0
dp
100
!
1 X
⇐⇒ 100 − p̂ ki = 0
p̂(1 − p̂) i=1
100
!
1 X
100 − p̂ ki = 0
p̂(1 − p̂) i=1
100
X
100 − p̂ ki = 0
i=1
100 1
p̂ = P100 = 1
P100 .
i=1 ki 100 i=1 ki
100
In total, we have an estimate p̂ = 568 = 0.1710.
Setting it to zero:
10
1 X
· Xi = 10
λ̂ i=1
10
1 X
⇐⇒ λ̂ = Xi
10 i=1
29
⇐⇒ λ̂ = = 2.9
10
This gives us an estimate for λ - and since λ is also the expected value of the Poisson distribution, we can
say, that on average the number of red cars pulling into the parking lot each morning between 8:30 and 8:40
pm is 2.9.
i=1 2πσ 2
Log-Likelihood:
n
n 1 X
log L(µ, σ 2 ) = − ln(2πσ 2 ) − 2 (xi − µ)2
2 2σ i=1
Since we have now two parameters, µ and σ 2 , we need to get 2 partial derivatives of the log-Likelihood:
n n
d −1 X 1 X
log L(µ, σ 2 ) = 0−2· · (x i − µ)2
· (−1) = (xi − µ)2
dµ 2σ 2 i=1 σ 2 i=1
n
d n 1 1 X
log L(µ, σ 2 ) = − + (xi − µ)2
dσ 2 2 σ2 2(σ 2 )2 i=1
We know, must find values for µ and σ 2 , that yield zeros for both derivatives at the same time.
d
Setting dµ log L(µ, σ 2 ) = 0 gives
n
1X
µ̂ = xi ,
n i=1
d
plugging this value into the derivative for σ 2 and setting dσ 2 log L(µ̂, σ 2 ) = 0 gives
n
1X
σˆ2 = (xi − µ̂)2
n i=1
6.2. CONFIDENCE INTERVALS 85
• for any given set of values x1 , . . . , xn the value or θ̂ is fixed, as well as the interval (θ̂ − e, θ̂ + e).
x-e x x+e
-e e
prob £ 1 -
confidence interval for
Figure 6.1: The probability that x̄ falls into an e interval around µ is α. Vice versa, we know, that for all of
those x̄ µ is within an e interval around x̄. That’s the idea of a confidence interval.
!!DON’T DO!!
we can therefore say, that θ̂ has a probability of at least α to fall into an e- interval around θ. Unfortunately,
that doesn’t help at all, since we do not know θ!
How do we compute confidence intervals, then? - that’s different for each estimator.
First, we look at estimates of a mean of a distribution:
86 CHAPTER 6. STATISTICAL INFERENCE
How do we read Φ−1 (0.975) from the standard normal table? - We look for which z the probability N(0,1) (z) ≥
0.975!
6.2. CONFIDENCE INTERVALS 87
21543 ± 588,
i.e. if we repeat this study 100 times (with 100 different employees each time), we can say: in 95 out of 100
studies, the true parameter µ falls into a $588 range around x̄.
Example 6.2.2
Suppose, we want to analyze some complicated queueing system, for which we have no formulas and theory.
We are interested in the mean queue length of the system after reaching steady state.
The only thing possible for us is to run simulations of this system and look at the queue length at some large
time t, e.g. t = 1000 hrs.
After 50 simulations, we have got data:
X1 = number in queue at time 1000 hrs in 1st simulation
X2 = number in queue at time 1000 hrs in 2nd simulation
...
X50 = number in queue at time 1000 hrs in 50th simulation
q Pn
1 2
Our observations yield an average queue length of x̄ = 21.5 and s = n−1 i=1 (xi − x̄) = 15.
A 90% confidence interval is given as
s s 15 15
x̄ − z · √ , x̄ + z · √ = 21.5 − 1.65 · √ , 21.5 + 1.65 · √ =
n n 50 50
= (17.9998, 25.0002)
Example 6.2.3
The graphs show a set of 80 experiments. The values from each experiment are shown in one of the green
framed boxes. Each experiment consists of simulating 20 values from a standard normal distributions (these
are drawn as the small blue lines). For each of the experiments, the average from the 20 value is computed
(that’s x̄) as well as a confidence interval for µ- for parts a) and b) it’s the 95% confidence interval, for part
c) it is the 90% confidence interval, for part d) it is the 99% confidence interval. The upper and the lower
confidence bound together with the sample mean are drawn in red next to the sampled observations.
88 CHAPTER 6. STATISTICAL INFERENCE
a) 95 % confidence intervals
b) 95 % confidence intervals
c) 90 % confidence intervals
d) 99 % confidence intervals
There are several things to see from this diagram. First of all, we know in this example the “true” value of
the parameter µ - since the observations are sampled from a standard normal distribution, µ = 0. The true
parameter is represented by the straight horizontal line through 0.
6.2. CONFIDENCE INTERVALS 89
We see, that each sample yields a different confidence interval, all of the are centered around the sample
mean. The different sizes of the intervals tells us another thing: in computing these confidence intervals, we
had to use the estimate s instead of the true standard deviation σ = 1. Each sample gave a slightly different
standard deviation. Overall, though, the intervals are not very different in lengths between parts a) and b).
The intervals in c) tend to be slightly smaller, though - these are 90% confidence intervals, whereas the
intervals in part d) are on average larger than the first ones, they are 99% confidence intervals.
Almost all the confidence intervals contain 0 - but not all. And that is, what we expect. For a 90% confidence
interval we expect, that in 10 out of 100 times, the confidence interval does not contain the true parameter.
When we check that - we see, that in part c) 4 out of the 20 confidence intervals don’t contain the true
parameter for µ - that’s 20%, on average we would expect 10% of the conficence intervals not to contain µ.
Official use of Confidence Intervals:
In an average of 90 out of 100 times the 90% confidence interval of θ does contain
the true value of θ.
Besides that, conservative confidence intervals (as the name says) are larger than confidence intervals found
by substitution. However, they are at the same time easier to compute.
The substitution method gives a slightly smaller confidence interval, but both intervals contain 0.3. There is
not enough evidence to allow the conclusion that the true average is not 0.3.
Confidence intervals give a way to measure the precision we get from simulations intended to evaluate
probabilities. But besides that it also gives as a way to plan how large a sample size has to be to get a
desired precision.
Example 6.2.6
Suppose, we want to estimate the fraction of records in the 2000 IRS data base that have a taxable income
over $35 K.
We want to get a 98% confidence interval and wish to estimate the quantity to within 0.01.
this means that our boundaries e need to be smaller than 0.01 (we’ll choose a conservative confidence interval
for ease of computation):
e ≤ 0.01
1
⇐⇒ z · √ ≤ 0.01 z is 2.33
2 n
1
⇐⇒ 2.33 · √ ≤ 0.01
2 n
√ 2.33
⇐⇒ n ≥ = 116.5
2 · 0.01
⇒ n ≥ 13573
Why? The argumentation in both cases is very similar - we will only discuss the confidence interval for
the difference between means.
X̄1 − X̄2 is approximately normal, since X̄1 and X̄2 are approximately normal, with (X̄1 , X̄2 are independent)
Then we can use the same arguments as before and get a C.I. for µ1 − µ2 as shown above. 2
Example 6.2.7
Assume, we have two parts of the IRS database: East Coast and West Coast.
We want to compare the mean taxable income between reported from the two regions in 2000.
East Coast West Coast
# of sampled records: n1 = 1000 n2 = 2000
mean taxable income: x̄1 = $37200 x̄2 = $42000
standard deviation: s1 = $10100 s2 = $15600
We can, for example, compute a 2 sided 95% confidence interval for µ1 − µ2 = difference in mean taxable
income as reported from 2000 tax return between East and West Coast:
r
101002 156002
37000 − 42000 ± + = −5000 ± 927
1000 2000
Note: this shows pretty conclusively that the mean West Coast taxable income is higher than the mean East
Coast taxable income (in the report from 2000). The interval contains only negative numbers - if it contained
the 0, the message wouldn’t be so clear.
One-sided intervals
idea: use only one of the end points x̄ ± z √sn
This yields confidence intervals for µ of the form
(−∞, #) (##, ∞)
| {z } | {z }
upper bound lower bound
However, now we need to adjust z to the new situation. Instead of worrying about two tails of the normal
distribution, we use for a one sided confidence interval only one tail.
P( x < + e) <
x e
x+e
prob ≤ 1 -
confidence interval for
Figure 6.2: One sided (upper bounded) confidence interval for µ (in red).
Critical values z = Φ−1 (α) for the one sided confidence interval are
6.3. HYPOTHESIS TESTING 93
α z = Φ−1 (α)
0.90 1.29
0.95 1.65
0.98 2.06
0.99 2.33
Example 6.2.9
Two different digital communication systems send 100 large messages via each system and determine how
many are corrupted in transmission.
p̂1 = 0.05 and pˆ2 = 0.10.
What’s the difference in the corruption rates? Find a 98% confidence interval:
Use: r
0.05 · 0.95 0.10 · 0.90
0.05 − 0.1 ± 2.33 · + = −0.05 ± 0.086
100 100
This calculation tells us, that based on these sample sizes, we don’t even have a solid idea about the sign of
p1 − p2 , i.e. we can’t tell which of the pi s is larger.
So far, we have only considered large sample confidence intervals. The problem with smaller sample sizes is,
that the normal approximation in the CLT doesn’t work, if the standard deviation σ 2 is unknown.
What you need to know is, that there exist different methods to compute C.I. for smaller sample sizes.
p(x)
observed x
x
94 CHAPTER 6. STATISTICAL INFERENCE
Events that are as unlikely or less likely are, that the lady got all 10 cups right or - very different, but
nevertheless very rare - that she only got 1 cup or none right (note, this would be evidence of some “anti-
skill”, but it would certainly be evidence against her guessing).
The total probability for these events is (remember, the binomial probability mass function is p(x) = nx px (1−
p)n−x )
p(0) + p(1) + p(9) + p(10) = 0.510 + 10 · 0.510 + 10 · 0.510 + 0.510 = 0.021
i.e. what we have just observed is a fairly rare event under the assumption, that the lady is only guessing.
This suggests, that the lady may have some skill in detecting which was poured first into the cup.
Jargon: 0.021 is called the p-value for testing the hypothesis p = 0.5.
The fact that the p-value is small is evidence against the hypothesis.
Hypothesis testing is a formal procedure to check whether or not some - previously made - assumption can
be rejected based on the data.
We are going to abstract the main elements of the previous example and cook up a standard series of steps
for hypothesis testing:
Example 6.3.2
University CC administrators have historical records that indicate that between August and Oct 2002 the
mean time between hits on the ISU homepage was 2 per min.
They suspect that in fact the mean time between hits has decreased (i.e. traffic is up) - sampling 50
inter-arrival times from records for November 2002 gives: X̄ = 1.7 min and s = 1.9 min.
Is this strong evidence for an increase in traffic?
Formal Procedure Application to Example
1 State a “null hypothesis” of the form H0 : func- H0 : µ = 2.0 min between hits
tion of parameter(s) = #
meant to embody a status quo/ pre data view
2 State an “alternative hypothesis” of the form Ha : µ < 2 (traffic is down)
>
Ha : function of parameter(s) 6= #
<
meant to identify departure from H0
3 State test criteria - consists of a test statistic, test statistic will be Z = X̄−2.0
√
s/ n
a “reference distribution” giving the behavior The reference density will be standard normal,
of the test statistic if H0 is true and the kinds large negative values for Z count as evidence
of values of the test statistic that count as ev- against H0 in favor of Ha
idence against H0 .
1.7−2.0
4 show computations sample gives z = √
1.9/ 50
= −1.12
5 Report and interpret a p-value = “observed The p-value is P (Z ≤ −1.12) = Φ(−1.12) =
level of significance, with which H0 can be re- 0.1314
jected”. This is the probability of an observed This value is not terribly small - the evidence of
value of the test statistic at least as extreme as a decrease in mean time between hits is some-
the one at hand. The smaller this value is, the what weak.
less likely it is that H0 is true.
Note aside: a 90% confidence interval for µ is
s
x̄ ± 1.65 √ = 1.7 ± 0.44
n
There are four basic hypothesis tests of this form, testing a mean, a proportion or differences between two
means or two proportions. Depending on the hypothesis, the test statistic will be different. Here’s an
overview of the tests, we are going to use:
Hypothesis Statistic Reference Distribution
X̄−#
H0 : µ = # Z = s/√n Z is standard normal
3. test statistic:
p̂ − 0.05
Z=p
0.05 · 0.95/n
Z has under the null hypothesis a standard normal distribution, any large values of Z - positive and
negative values - will count as evidence against H0 .
p
4. computation: z = (0.061 − 0.05)/ 0.05 · 0.95/1000 = 1.59
5. p-value: P (|Z| ≥ 1.59) = P (Z ≤ −1.59) + P (Z ≥ 1.59) = 0.11 This is not a very small value, we
therefore have only very weak evidence against H0 .
Does this provide conclusive evidence that the new design has a larger mean time to failure under “acceler-
ated” stress conditions?
How strong is the evidence of a difference between the t = 2000 availability of a server for the two systems?
1. state null hypothesis: H0 : p1 = p2 (p1 − p2 = 0)
2. alternative hypothesis: Ha : p1 6= p2 (p1 − p2 6= 0)
3. Preliminary: note that, if there was no difference between the two systems, a plausible estimate of the
availability of a server would be
np̂1 + np̂2 551 + 303
p̂ = = = 0.569
n1 + n2 1000 + 500
a test statistic is:
p̂1 − p̂2 − 0
Z=p q
p̂(1 − p̂) · n11 + 1
n2
Z has under the null hypothesis a standard normal distribution, we will consider large values of Z as
evidence against H0 .
p p
4. computation: z = (0.551 − 0.606)/( 0.569 · (1 − 0.569) 1/1000 + 1/500) = −2.03
5. p-value: P (|Z| > 2.03) = 0.04 This is fairly strong evidence of a real difference in t=2000 availabilities
of a server between the two systems.
6.4 Regression
A statistical investigation only rarely focusses on the distribution of a single variable. We are often interested
in comparisons among several variables, in changes in a variable over time, or in relationships among several
variables.
The idea of regression is that we have a vector X1 , . . . , Xk and try to approximate the behavior of Y by
finding a function g(X1 , . . . , Xk ) such that Y ≈ g(X1 , . . . , Xk ).
Simplest possible version is:
6.4. REGRESSION 97
• linear relationship may be true only after a transformation of X and/or Y , i.e. one needs to find the
“right” scale for the variables:
ln y ≈ b |{z}
ln x + ln c,
|{z}
=:y 0 x0
so on a log scale for both x and y-axis one gets a linear relationship.
30
M 25
P
G
20
Transform weight by 1
x to weight−1 . A scatterplot of mpg versus weight−1 reveals a linear relationship:
35
30
M 25
P
G
20
0 20 40 60 80
year
The plot shows that it is perhaps reasonable to say that
y ≈ β0 + β1 x
The first issue to be dealt with in this context is: if we accept that y ≈ β0 + β1 x, how do we derive empirical
values of β0 , β1 from n data points (x, y)? The standard answer is the “least squares” principle:
y
y=b0 + b1 x
0.75
0.50
0.25
-0.00
In comparing lines that might be drawn through the plot we look at:
n
X 2
Q(b0 , b1 ) = (yi − (b0 + b1 xi ))
i=1
i.e. we look at the sum of squared vertical distances from points to the line and attempt to minimize this
6.4. REGRESSION 99
sum of squares:
n
d X
Q(b0 , b1 ) = −2 (yi − (b0 + b1 xi ))
db0 i=1
n
d X
Q(b0 , b1 ) = −2 xi (yi − (b0 + b1 xi ))
db1 i=1
1100·175.518
9079.584 − 22
b1 = 11002
74608 − = 0.0155(in m)
22
175.518 1100
b0 = − · 0.0155 = 7.2037
22 22
It is useful for addition, to be able to judge how well the line describes the data - i.e. how “linear looking”
a plot really is.
There are a couple of means doing this:
100 CHAPTER 6. STATISTICAL INFERENCE
The numerator is the numerator of b1 , one part under the root of the denominator is the denominator of b1 .
Because of its connection to %, the sample correlation r fulfills (it’s not obvious to see, and we want prove
it):
• −1 ≤ r ≤ 1
• r = ±1 exactly, when all (x, y) data pairs fall on a single straight line.
0.50 y
0.25
-0.00
After fitting the line ŷ = b0 + b1 x, one doesn’t predict y as ȳ anymore and suffer the errors of prediction
above, but rather only the errors
ŷi − yi =: ei .
So, after fitting the line
n
X n
X
e2i = (yi − ŷ)2 = SSES um of S quares of E rrors
i=1 i=1
y
y=b0 + b1 x
0.75
0.50
0.25
-0.00
R2 = r2 if and only if ŷ = b0 + b1 x
102 CHAPTER 6. STATISTICAL INFERENCE
It is possible to go beyond simply fitting a line and summarizing the goodness of fit in terms of r and R2 to
doing inference, i.e. making confidence intervals, predictions, . . . based on the line fitting. But for that, we
need a probability model.
density of
y given x
Distribution Tables
Binomial Distribution
bxc
X n
Bn,p (x) = pi (1 − p)n−i
i=0
i
n=1 p=0.01 0.05 0.1 0.15 1/6 0.2 0.25 0.3 1/3 0.4 0.5
x=0 0.99 0.95 0.9 0.85 0.8333333 0.8 0.75 0.7 0.6666667 0.6 0.5
n=2 p=0.01 0.05 0.1 0.15 1/6 0.2 0.25 0.3 1/3 0.4 0.5
x=0 0.9801 0.9025 0.81 0.7225 0.6944444 0.64 0.5625 0.49 0.4444444 0.36 0.25
1 0.9999 0.9975 0.99 0.9775 0.9722222 0.96 0.9375 0.91 0.8888889 0.84 0.75
n=3 p=0.01 0.05 0.1 0.15 1/6 0.2 0.25 0.3 1/3 0.4 0.5
x=0 0.970299 0.857375 0.729 0.614125 0.5787037 0.512 0.421875 0.343 0.2962963 0.216 0.125
1 0.999702 0.992750 0.972 0.939250 0.9259259 0.896 0.843750 0.784 0.7407407 0.648 0.500
2 0.999999 0.999875 0.999 0.996625 0.9953704 0.992 0.984375 0.973 0.9629630 0.936 0.875
n=4 p=0.01 0.05 0.1 0.15 1/6 0.2 0.25 0.3 1/3 0.4 0.5
x=0 0.960596 0.8145062 0.6561 0.5220063 0.4822531 0.4096 0.3164063 0.2401 0.1975309 0.1296 0.0625
1 0.999408 0.9859812 0.9477 0.8904813 0.8680556 0.8192 0.7382812 0.6517 0.5925926 0.4752 0.3125
2 0.999996 0.9995188 0.9963 0.9880187 0.9837963 0.9728 0.9492188 0.9163 0.8888889 0.8208 0.6875
3 1.000000 0.9999938 0.9999 0.9994937 0.9992284 0.9984 0.9960938 0.9919 0.9876543 0.9744 0.9375
n=5 p=0.01 0.05 0.1 0.15 1/6 0.2 0.25 0.3 1/3 0.4 0.5
x=0 0.9509900 0.7737809 0.59049 0.4437053 0.4018776 0.32768 0.2373047 0.16807 0.1316872 0.07776 0.03125
1 0.9990199 0.9774075 0.91854 0.8352100 0.8037551 0.73728 0.6328125 0.52822 0.4609053 0.33696 0.18750
2 0.9999901 0.9988419 0.99144 0.9733881 0.9645062 0.94208 0.8964844 0.83692 0.7901235 0.68256 0.50000
3 1.0000000 0.9999700 0.99954 0.9977725 0.9966564 0.99328 0.9843750 0.96922 0.9547325 0.91296 0.81250
4 1.0000000 0.9999997 0.99999 0.9999241 0.9998714 0.99968 0.9990234 0.99757 0.9958848 0.98976 0.96875
n=6 p=0.01 0.05 0.1 0.15 1/6 0.2 0.25 0.3 1/3 0.4 0.5
x=0 0.9414801 0.7350919 0.531441 0.3771495 0.3348980 0.262144 0.1779785 0.117649 0.0877915 0.046656 0.015625
1 0.9985396 0.9672262 0.885735 0.7764843 0.7367755 0.655360 0.5339355 0.420175 0.3511660 0.233280 0.109375
2 0.9999804 0.9977702 0.984150 0.9526614 0.9377143 0.901120 0.8305664 0.744310 0.6803841 0.544320 0.343750
3 0.9999999 0.9999136 0.998730 0.9941148 0.9912980 0.983040 0.9624023 0.929530 0.8998628 0.820800 0.656250
4 1.0000000 0.9999982 0.999945 0.9996013 0.9993356 0.998400 0.9953613 0.989065 0.9821674 0.959040 0.890625
5 1.0000000 1.0000000 0.999999 0.9999886 0.9999786 0.999936 0.9997559 0.999271 0.9986283 0.995904 0.984375
n=7 p=0.01 0.05 0.1 0.15 1/6 0.2 0.25 0.3 1/3 0.4 0.5
x=0 0.9320653 0.6983373 0.4782969 0.3205771 0.2790816 0.2097152 0.1334839 0.0823543 0.05852766 0.0279936 0.0078125
1 0.9979690 0.9556195 0.8503056 0.7165841 0.6697960 0.5767168 0.4449463 0.3294172 0.26337449 0.1586304 0.0625000
2 0.9999660 0.9962430 0.9743085 0.9262348 0.9042245 0.8519680 0.7564087 0.6470695 0.57064472 0.4199040 0.2265625
3 0.9999997 0.9998064 0.9972720 0.9878968 0.9823674 0.9666560 0.9294434 0.8739640 0.82670325 0.7102080 0.5000000
4 1.0000000 0.9999940 0.9998235 0.9987784 0.9979960 0.9953280 0.9871216 0.9712045 0.95473251 0.9037440 0.7734375
5 1.0000000 0.9999999 0.9999936 0.9999305 0.9998714 0.9996288 0.9986572 0.9962092 0.99314129 0.9811584 0.9375000
6 1.0000000 1.0000000 0.9999999 0.9999983 0.9999964 0.9999872 0.9999390 0.9997813 0.99954275 0.9983616 0.9921875
103
104 APPENDIX A. DISTRIBUTION TABLES
n=8 p=0.01 0.05 0.1 0.15 1/6 0.2 0.25 0.3 1/3 0.4 0.5
x=0 0.9227447 0.6634204 0.4304672 0.2724905 0.2325680 0.1677722 0.1001129 0.05764801 0.03901844 0.01679616 0.00390625
1 0.9973099 0.9427553 0.8131047 0.6571830 0.6046769 0.5033165 0.3670807 0.25529833 0.19509221 0.10637568 0.03515625
2 0.9999461 0.9942118 0.9619082 0.8947872 0.8651531 0.7969178 0.6785431 0.55177381 0.46822131 0.31539456 0.14453125
3 0.9999993 0.9996282 0.9949756 0.9786475 0.9693436 0.9437184 0.8861847 0.80589565 0.74135040 0.59408640 0.36328125
4 1.0000000 0.9999846 0.9995683 0.9971461 0.9953912 0.9895936 0.9727020 0.94203235 0.91205609 0.82632960 0.63671875
5 1.0000000 0.9999996 0.9999766 0.9997577 0.9995588 0.9987686 0.9957733 0.98870779 0.98033836 0.95019264 0.85546875
6 1.0000000 1.0000000 0.9999993 0.9999881 0.9999756 0.9999155 0.9996185 0.99870967 0.99740893 0.99148032 0.96484375
7 1.0000000 1.0000000 1.0000000 0.9999997 0.9999994 0.9999974 0.9999847 0.99993439 0.99984758 0.99934464 0.99609375
n=9 p=0.01 0.05 0.1 0.15 1/6 0.2 0.25 0.3 1/3 0.4 0.5
x=0 0.9135172 0.6302494 0.3874205 0.2316169 0.1938067 0.1342177 0.07508469 0.04035361 0.02601229 0.01007770 0.00195313
1 0.9965643 0.9287886 0.7748410 0.5994792 0.5426588 0.4362076 0.30033875 0.19600323 0.14306762 0.07054387 0.01953125
2 0.9999197 0.9916390 0.9470279 0.8591466 0.8217404 0.7381975 0.60067749 0.46283117 0.37717828 0.23178701 0.08984375
3 0.9999988 0.9993574 0.9916689 0.9660685 0.9519785 0.9143583 0.83427429 0.72965910 0.65030737 0.48260966 0.25390625
4 1.0000000 0.9999668 0.9991091 0.9943713 0.9910499 0.9804186 0.95107269 0.90119134 0.85515419 0.73343232 0.50000000
5 1.0000000 0.9999988 0.9999358 0.9993660 0.9988642 0.9969336 0.99000549 0.97470516 0.95757761 0.90064742 0.74609375
6 1.0000000 1.0000000 0.9999970 0.9999536 0.9999061 0.9996861 0.99865723 0.99570911 0.99171874 0.97496525 0.91015625
7 1.0000000 1.0000000 0.9999999 0.9999980 0.9999954 0.9999811 0.99989319 0.99956697 0.99903470 0.99619891 0.98046875
8 1.0000000 1.0000000 1.0000000 1.0000000 0.9999999 0.9999995 0.99999619 0.99998032 0.99994919 0.99973786 0.99804688
n=10 p=0.01 0.05 0.1 0.15 1/6 0.2 0.25 0.3 1/3 0.4 0.5
x=0 0.9043821 0.5987369 0.3486784 0.1968744 0.1615056 0.1073742 0.05631351 0.02824752 0.01734153 0.00604662 0.00097656
1 0.9957338 0.9138616 0.7360989 0.5442998 0.4845167 0.3758096 0.24402523 0.14930835 0.10404918 0.04635740 0.01074219
2 0.9998862 0.9884964 0.9298092 0.8201965 0.7752268 0.6777995 0.52559280 0.38278279 0.29914139 0.16728975 0.05468750
3 0.9999980 0.9989715 0.9872048 0.9500302 0.9302722 0.8791261 0.77587509 0.64961072 0.55926434 0.38228060 0.17187500
4 1.0000000 0.9999363 0.9983651 0.9901259 0.9845380 0.9672065 0.92187309 0.84973167 0.78687192 0.63310326 0.37695313
5 1.0000000 0.9999972 0.9998531 0.9986168 0.9975618 0.9936306 0.98027229 0.95265101 0.92343647 0.83376138 0.62304688
6 1.0000000 0.9999999 0.9999909 0.9998654 0.9997325 0.9991356 0.99649429 0.98940792 0.98033836 0.94523812 0.82812500
7 1.0000000 1.0000000 0.9999996 0.9999913 0.9999806 0.9999221 0.99958420 0.99840961 0.99659605 0.98770545 0.94531250
8 1.0000000 1.0000000 1.0000000 0.9999997 0.9999992 0.9999958 0.99997044 0.99985631 0.99964436 0.99832228 0.98925781
9 1.0000000 1.0000000 1.0000000 1.0000000 1.0000000 0.9999999 0.99999905 0.99999410 0.99998306 0.99989514 0.99902344
n=11 p=0.01 0.05 0.1 0.15 1/6 0.2 0.25 0.3 1/3 0.4 0.5
x=0 0.8953383 0.5688001 0.3138106 0.1673432 0.1345880 0.08589935 0.04223514 0.01977327 0.01156102 0.00362797 0.00048828
1 0.9948203 0.8981054 0.6973569 0.4921860 0.4306816 0.32212255 0.19709730 0.11299010 0.07514663 0.03023309 0.00585938
2 0.9998446 0.9847647 0.9104381 0.7788120 0.7267751 0.61740155 0.45520091 0.31274045 0.23411065 0.11891681 0.03271484
3 0.9999969 0.9984477 0.9814652 0.9305551 0.9044313 0.83886080 0.71330452 0.56956234 0.47255669 0.29628426 0.11328125
4 1.0000000 0.9998881 0.9972490 0.9841116 0.9754937 0.94959043 0.88537359 0.78969538 0.71100273 0.53277420 0.27441406
5 1.0000000 0.9999942 0.9997043 0.9973431 0.9953912 0.98834579 0.96567249 0.92177521 0.87791495 0.75349813 0.50000000
6 1.0000000 0.9999998 0.9999771 0.9996781 0.9993707 0.99803464 0.99243879 0.97838085 0.96137106 0.90064742 0.72558594
7 1.0000000 1.0000000 0.9999988 0.9999724 0.9999392 0.99976479 0.99881172 0.99570911 0.99117682 0.97071852 0.88671875
8 1.0000000 1.0000000 1.0000000 0.9999984 0.9999961 0.99998106 0.99987388 0.99942230 0.99862826 0.99407555 0.96728516
9 1.0000000 1.0000000 1.0000000 0.9999999 0.9999998 0.99999908 0.99999189 0.99995276 0.99987016 0.99926600 0.99414063
10 1.0000000 1.0000000 1.0000000 1.0000000 1.0000000 0.99999998 0.99999976 0.99999823 0.99999435 0.99995806 0.99951172
n=12 p=0.01 0.05 0.1 0.15 1/6 0.2 0.25 0.3 1/3 0.4 0.5
x=0 0.8863849 0.5403601 0.2824295 0.1422418 0.1121567 0.06871948 0.03167635 0.01384129 0.00770735 0.00217678 0.00024414
1 0.9938255 0.8816401 0.6590023 0.4434596 0.3813326 0.27487791 0.15838176 0.08502505 0.05395143 0.01959104 0.00317383
2 0.9997944 0.9804317 0.8891300 0.7358181 0.6774262 0.55834575 0.39067501 0.25281535 0.18112265 0.08344332 0.01928711
3 0.9999954 0.9977636 0.9743625 0.9077937 0.8748219 0.79456895 0.64877862 0.49251577 0.39307468 0.22533728 0.07299805
4 0.9999999 0.9998161 0.9956707 0.9760781 0.9636500 0.92744450 0.84235632 0.72365547 0.63152071 0.43817822 0.19384766
5 1.0000000 0.9999889 0.9994588 0.9953584 0.9920750 0.98059472 0.94559777 0.88215126 0.82227754 0.66520856 0.38720703
6 1.0000000 0.9999995 0.9999498 0.9993279 0.9987075 0.99609687 0.98574722 0.96139916 0.93355236 0.84178771 0.61279297
7 1.0000000 1.0000000 0.9999966 0.9999283 0.9998445 0.99941876 0.99721849 0.99051063 0.98124157 0.94269008 0.80615234
8 1.0000000 1.0000000 0.9999998 0.9999945 0.9999866 0.99993780 0.99960834 0.99830834 0.99614445 0.98473273 0.92700195
9 1.0000000 1.0000000 1.0000000 0.9999997 0.9999992 0.99999547 0.99996239 0.99979362 0.99945620 0.99718982 0.98071289
10 1.0000000 1.0000000 1.0000000 1.0000000 1.0000000 0.99999980 0.99999779 0.99998459 0.99995296 0.99968123 0.99682617
11 1.0000000 1.0000000 1.0000000 1.0000000 1.0000000 1.00000000 0.99999994 0.99999947 0.99999812 0.99998322 0.99975586
n=13 p=0.01 0.05 0.1 0.15 1/6 0.2 0.25 0.3 1/3 0.4 0.5
x=0 0.8775210 0.5133421 0.2541866 0.1209055 0.09346388 0.05497558 0.02375726 0.00968890 0.00513823 0.00130607 0.00012207
1 0.9927511 0.8645761 0.6213450 0.3982769 0.33646996 0.23364622 0.12670541 0.06366992 0.03853673 0.01262534 0.00170898
2 0.9997347 0.9754922 0.8661172 0.6919643 0.62807727 0.50165218 0.33260170 0.20247826 0.13873224 0.05790241 0.01123047
3 0.9999933 0.9968970 0.9658393 0.8819973 0.84192262 0.74732431 0.58425272 0.42060565 0.32242400 0.16857970 0.04614258
4 0.9999999 0.9997134 0.9935398 0.9658354 0.94884530 0.90086939 0.79396190 0.65431356 0.55203870 0.35304185 0.13342285
5 1.0000000 0.9999803 0.9990800 0.9924664 0.98733746 0.96996468 0.91978741 0.83460252 0.75869193 0.57439642 0.29052734
6 1.0000000 0.9999990 0.9999007 0.9987325 0.99760204 0.99299644 0.97570986 0.93762479 0.89646076 0.77115605 0.50000000
7 1.0000000 1.0000000 0.9999919 0.9998382 0.99965496 0.99875438 0.99435067 0.98177719 0.96534517 0.90232913 0.70947266
8 1.0000000 1.0000000 0.9999995 0.9999846 0.99996289 0.99983399 0.99901088 0.99596903 0.99117682 0.96791567 0.86657715
9 1.0000000 1.0000000 1.0000000 0.9999989 0.99999711 0.99998394 0.99987388 0.99934804 0.99835228 0.99220698 0.95385742
10 1.0000000 1.0000000 1.0000000 0.9999999 0.99999985 0.99999893 0.99998894 0.99992730 0.99978737 0.99868467 0.98876953
11 1.0000000 1.0000000 1.0000000 1.0000000 0.99999999 0.99999996 0.99999940 0.99999500 0.99998307 0.99986243 0.99829102
12 1.0000000 1.0000000 1.0000000 1.0000000 1.00000000 1.00000000 0.99999999 0.99999984 0.99999937 0.99999329 0.99987793
105
n=14 p=0.01 0.05 0.1 0.15 1/6 0.2 0.25 0.3 1/3 0.4 0.5
x=0 0.8687458 0.4876750 0.2287679 0.1027697 0.07788657 0.04398047 0.01781795 0.00678223 0.00342549 0.00078364 6.1035e-05
1 0.9915988 0.8470144 0.5846291 0.3566712 0.29596895 0.19791209 0.10096837 0.04747562 0.02740390 0.00809763 9.1553e-04
2 0.9996649 0.9699464 0.8416400 0.6479112 0.57947605 0.44805099 0.28112762 0.16083576 0.10533374 0.03979158 6.4697e-03
3 0.9999908 0.9958268 0.9558671 0.8534924 0.80628173 0.69818988 0.52133996 0.35516743 0.26119341 0.12430878 2.8687e-02
4 0.9999998 0.9995726 0.9907698 0.9532597 0.93102485 0.87016037 0.74153460 0.58420119 0.47550047 0.27925699 8.9783e-02
5 1.0000000 0.9999669 0.9985259 0.9884717 0.98092210 0.95614562 0.88833103 0.78051584 0.68980752 0.48585459 2.1198e-01
6 1.0000000 0.9999980 0.9998186 0.9977925 0.99589128 0.98839009 0.96172924 0.90671811 0.85053781 0.69245220 3.9526e-01
7 1.0000000 0.9999999 0.9999828 0.9996724 0.99931280 0.99760279 0.98969047 0.96853147 0.94238370 0.84985990 6.0474e-01
8 1.0000000 1.0000000 0.9999987 0.9999626 0.99991157 0.99961807 0.99784582 0.99171148 0.98256627 0.94168106 7.8802e-01
9 1.0000000 1.0000000 0.9999999 0.9999968 0.99999141 0.99995395 0.99965813 0.99833434 0.99596046 0.98249046 9.1022e-01
10 1.0000000 1.0000000 1.0000000 0.9999998 0.99999939 0.99999594 0.99996018 0.99975352 0.99930901 0.99609359 9.7131e-01
11 1.0000000 1.0000000 1.0000000 1.0000000 0.99999997 0.99999975 0.99999679 0.99997469 0.99991783 0.99939132 9.9353e-01
12 1.0000000 1.0000000 1.0000000 1.0000000 1.00000000 0.99999999 0.99999984 0.99999839 0.99999394 0.99994094 9.9908e-01
13 1.0000000 1.0000000 1.0000000 1.0000000 1.00000000 1.00000000 1.00000000 0.99999995 0.99999979 0.99999732 9.9994e-01
n=15 p=0.01 0.05 0.1 0.15 1/6 0.2 0.25 0.3 1/3 0.4 0.5
x=0 0.8600584 0.4632912 0.2058911 0.08735422 0.06490547 0.03518437 0.01336346 0.00474756 0.00228366 0.00047019 3.0518e-05
1 0.9903702 0.8290475 0.5490430 0.31858598 0.25962189 0.16712577 0.08018077 0.03526760 0.01941110 0.00517204 4.8828e-04
2 0.9995842 0.9637998 0.8159389 0.60422520 0.53222487 0.39802321 0.23608781 0.12682772 0.07935713 0.02711400 3.6926e-03
3 0.9999875 0.9945327 0.9444444 0.82265520 0.76848078 0.64816210 0.46128688 0.29686793 0.20924019 0.09050190 1.7578e-02
4 0.9999997 0.9993853 0.9872795 0.93829461 0.91023433 0.83576628 0.68648594 0.51549106 0.40406478 0.21727771 5.9235e-02
5 1.0000000 0.9999472 0.9977503 0.98318991 0.97260589 0.93894857 0.85163192 0.72162144 0.61837184 0.40321555 1.5088e-01
6 1.0000000 0.9999965 0.9996894 0.99639441 0.99339642 0.98194119 0.94337969 0.86885743 0.79696105 0.60981316 3.0362e-01
7 1.0000000 0.9999998 0.9999664 0.99939039 0.99874255 0.99576025 0.98270016 0.94998746 0.91176840 0.78689682 5.0000e-01
8 1.0000000 1.0000000 0.9999972 0.99991910 0.99981178 0.99921501 0.99580699 0.98475747 0.96917208 0.90495259 6.9638e-01
9 1.0000000 1.0000000 0.9999998 0.99999166 0.99997810 0.99988677 0.99920505 0.99634748 0.99149573 0.96616670 8.4912e-01
10 1.0000000 1.0000000 1.0000000 0.99999935 0.99999806 0.99998754 0.99988466 0.99932777 0.99819282 0.99065234 9.4077e-01
11 1.0000000 1.0000000 1.0000000 0.99999996 0.99999987 0.99999899 0.99998764 0.99990834 0.99971489 0.99807223 9.8242e-01
12 1.0000000 1.0000000 1.0000000 1.00000000 0.99999999 0.99999994 0.99999908 0.99999128 0.99996857 0.99972110 9.9631e-01
13 1.0000000 1.0000000 1.0000000 1.00000000 1.00000000 1.00000000 0.99999996 0.99999948 0.99999784 0.99997477 9.9951e-01
14 1.0000000 1.0000000 1.0000000 1.00000000 1.00000000 1.00000000 1.00000000 0.99999999 0.99999993 0.99999893 9.9997e-01
n=16 p=0.01 0.05 0.1 0.15 1/6 0.2 0.25 0.3 1/3 0.4 0.5
x=0 0.8514578 0.4401267 0.1853020 0.07425109 0.05408789 0.0281475 0.01002260 0.00332329 0.00152244 0.00028211 1.5259e-05
1 0.9890671 0.8107597 0.5147278 0.28390121 0.22716915 0.1407375 0.06347644 0.02611159 0.01370195 0.00329130 2.5940e-04
2 0.9994921 0.9570621 0.7892493 0.56137932 0.48679104 0.3518437 0.19711105 0.09935968 0.05937512 0.01833721 2.0905e-03
3 0.9999835 0.9929961 0.9315938 0.78989070 0.72910480 0.5981343 0.40498711 0.24585586 0.16594583 0.06514674 1.0635e-02
4 0.9999996 0.9991427 0.9829960 0.92094870 0.88660874 0.7982454 0.63018618 0.44990412 0.33912325 0.16656738 3.8406e-02
5 1.0000000 0.9999191 0.9967032 0.97645562 0.96221063 0.9183121 0.81034543 0.65978233 0.54693615 0.32884041 1.0506e-01
6 1.0000000 0.9999940 0.9994955 0.99441374 0.98993133 0.9733427 0.92044275 0.82468663 0.73743131 0.52717411 2.2725e-01
7 1.0000000 0.9999997 0.9999387 0.99894100 0.99785153 0.9929964 0.97287004 0.92564845 0.87349929 0.71606335 4.0181e-01
8 1.0000000 1.0000000 0.9999941 0.99983979 0.99963357 0.9985241 0.99253028 0.97432647 0.95003752 0.85773028 5.9819e-01
9 1.0000000 1.0000000 0.9999995 0.99998078 0.99995038 0.9997524 0.99835553 0.99287048 0.98405451 0.94168106 7.7275e-01
10 1.0000000 1.0000000 1.0000000 0.99999819 0.99999473 0.9999674 0.99971476 0.99843368 0.99596046 0.98085808 8.9494e-01
11 1.0000000 1.0000000 1.0000000 0.99999987 0.99999957 0.9999967 0.99996189 0.99973417 0.99920754 0.99510427 9.6159e-01
12 1.0000000 1.0000000 1.0000000 0.99999999 0.99999997 0.9999998 0.99999622 0.99996640 0.99988401 0.99906155 9.8936e-01
13 1.0000000 1.0000000 1.0000000 1.00000000 1.00000000 1.0000000 0.99999974 0.99999702 0.99998808 0.99987330 9.9791e-01
14 1.0000000 1.0000000 1.0000000 1.00000000 1.00000000 1.0000000 0.99999999 0.99999984 0.99999923 0.99998926 9.9974e-01
15 1.0000000 1.0000000 1.0000000 1.00000000 1.00000000 1.0000000 1.00000000 1.00000000 0.99999998 0.99999957 9.9998e-01
n=17 p=0.01 0.05 0.1 0.15 1/6 0.2 0.25 0.3 1/3 0.4 0.5
x=0 0.8429432 0.4181203 0.1667718 0.06311342 0.04507324 0.0225180 0.00751695 0.00232631 0.00101496 0.00016927 7.6294e-06
1 0.9876910 0.7922280 0.4817852 0.25245369 0.19832227 0.1182195 0.05011298 0.01927510 0.00964211 0.00208762 1.3733e-04
2 0.9993878 0.9497470 0.7617972 0.51975760 0.44352072 0.3096225 0.16370240 0.07738525 0.04415073 0.01231885 1.1749e-03
3 0.9999786 0.9911994 0.9173594 0.75561400 0.68871917 0.5488762 0.35301810 0.20190701 0.13042226 0.04642293 6.3629e-03
4 0.9999994 0.9988354 0.9778558 0.90129000 0.86035808 0.7582232 0.57388641 0.38868964 0.28139745 0.12599913 2.4521e-02
5 1.0000000 0.9998803 0.9953325 0.96812958 0.94961032 0.8942988 0.76530561 0.59681886 0.47766519 0.26393120 7.1732e-02
6 1.0000000 0.9999903 0.9992162 0.99172002 0.98531121 0.9623366 0.89291842 0.77521534 0.67393293 0.44784063 1.6615e-01
7 1.0000000 0.9999994 0.9998944 0.99826191 0.99653149 0.9890657 0.95976322 0.89535990 0.82814329 0.64050766 3.1453e-01
8 1.0000000 1.0000000 0.9999885 0.99970497 0.99933656 0.9974185 0.98761522 0.95972306 0.92452477 0.80106351 5.0000e-01
9 1.0000000 1.0000000 0.9999990 0.99995963 0.99989758 0.9995068 0.99689922 0.98730728 0.97271551 0.90810075 6.8547e-01
10 1.0000000 1.0000000 0.9999999 0.99999558 0.99998734 0.9999244 0.99937495 0.99676472 0.99199181 0.96518727 8.3385e-01
11 1.0000000 1.0000000 1.0000000 0.99999962 0.99999876 0.9999908 0.99990011 0.99934402 0.99812518 0.98940580 9.2827e-01
12 1.0000000 1.0000000 1.0000000 0.99999997 0.99999991 0.9999991 0.99998764 0.99989673 0.99965852 0.99747864 9.7548e-01
13 1.0000000 1.0000000 1.0000000 1.00000000 0.99999999 0.9999999 0.99999886 0.99998784 0.99995339 0.99954860 9.9364e-01
14 1.0000000 1.0000000 1.0000000 1.00000000 1.00000000 1.0000000 0.99999993 0.99999899 0.99999552 0.99994288 9.9883e-01
15 1.0000000 1.0000000 1.0000000 1.00000000 1.00000000 1.0000000 1.00000000 0.99999995 0.99999973 0.99999545 9.9986e-01
16 1.0000000 1.0000000 1.0000000 1.00000000 1.00000000 1.0000000 1.00000000 1.00000000 0.99999999 0.99999983 9.9999e-01
106 APPENDIX A. DISTRIBUTION TABLES
n=18 p=0.01 0.05 0.1 0.15 1/6 0.2 0.25 0.3 1/3 0.4 0.5
x=0 0.8345138 0.3972143 0.1500946 0.05364641 0.03756104 0.0180144 0.00563771 0.00162841 0.00067664 0.00010156 3.8147e-06
1 0.9862435 0.7735226 0.4502839 0.22405265 0.17278077 0.0990792 0.03946397 0.01419046 0.00676639 0.00132028 7.2479e-05
2 0.9992708 0.9418711 0.7337960 0.47966202 0.40265431 0.2713419 0.13530504 0.05995221 0.03264786 0.00822636 6.5613e-04
3 0.9999726 0.9891268 0.9018032 0.72023554 0.64785276 0.5010255 0.30568917 0.16455048 0.10166508 0.03278130 3.7689e-03
4 0.9999992 0.9984536 0.9718061 0.87943860 0.83175160 0.7163538 0.51866933 0.33265485 0.23107238 0.09416865 1.5442e-02
5 1.0000000 0.9998280 0.9935848 0.95810364 0.93473495 0.8670837 0.71745081 0.53438010 0.41224261 0.20875837 4.8126e-02
6 1.0000000 0.9999848 0.9988279 0.98818146 0.97936106 0.9487290 0.86101522 0.72169640 0.60851035 0.37427686 1.1894e-01
7 1.0000000 0.9999989 0.9998265 0.99728062 0.99466145 0.9837199 0.94305202 0.85931654 0.77673984 0.56344085 2.4034e-01
8 1.0000000 0.9999999 0.9999791 0.99948851 0.99886905 0.9957480 0.98065222 0.94041412 0.89239761 0.73684117 4.0726e-01
9 1.0000000 1.0000000 0.9999980 0.99992143 0.99980408 0.9990891 0.99457822 0.97903201 0.95665193 0.86528585 5.9274e-01
10 1.0000000 1.0000000 0.9999998 0.99999019 0.99997238 0.9998409 0.99875602 0.99392749 0.98556638 0.94235266 7.5966e-01
11 1.0000000 1.0000000 1.0000000 0.99999901 0.99999686 0.9999775 0.99976882 0.99857023 0.99608072 0.97971839 8.8106e-01
12 1.0000000 1.0000000 1.0000000 0.99999992 0.99999972 0.9999975 0.99996575 0.99973092 0.99914740 0.99424950 9.5187e-01
13 1.0000000 1.0000000 1.0000000 1.00000000 0.99999998 0.9999998 0.99999605 0.99996050 0.99985510 0.99872062 9.8456e-01
14 1.0000000 1.0000000 1.0000000 1.00000000 1.00000000 1.0000000 0.99999966 0.99999565 0.99998147 0.99978517 9.9623e-01
15 1.0000000 1.0000000 1.0000000 1.00000000 1.00000000 1.0000000 0.99999998 0.99999966 0.99999832 0.99997442 9.9934e-01
16 1.0000000 1.0000000 1.0000000 1.00000000 1.00000000 1.0000000 1.00000000 0.99999998 0.99999990 0.99999808 9.9993e-01
17 1.0000000 1.0000000 1.0000000 1.00000000 1.00000000 1.0000000 1.00000000 1.00000000 1.00000000 0.99999993 1.00000000
n=19 p=0.01 0.05 0.1 0.15 1/6 0.2 0.25 0.3 1/3 0.4 0.5
x=0 0.8261686 0.3773536 0.1350852 0.04559945 0.03130086 0.01441152 0.00422828 0.00113989 0.00045109 6.0936e-05 1.9073e-06
1 0.9847262 0.7547072 0.4202650 0.19849172 0.15024415 0.08286623 0.03100741 0.01042185 0.00473648 8.3279e-04 3.8147e-05
2 0.9991406 0.9334536 0.7054448 0.44132061 0.36434206 0.23688934 0.11134478 0.04622368 0.02402070 5.4639e-03 3.6430e-04
3 0.9999656 0.9867640 0.8850024 0.68414951 0.60698635 0.45508874 0.26309314 0.13317100 0.07865934 2.2959e-02 2.2125e-03
4 0.9999990 0.9979872 0.9648058 0.85555814 0.80110179 0.67328814 0.46542429 0.28222354 0.18793662 6.9614e-02 9.6054e-03
5 1.0000000 0.9997593 0.9914070 0.94630389 0.91757105 0.83693770 0.66775544 0.47386252 0.35185253 1.6292e-01 3.1784e-02
6 1.0000000 0.9999769 0.9983036 0.98366978 0.97192338 0.93239993 0.82512412 0.66550151 0.54308777 3.0807e-01 8.3534e-02
7 1.0000000 0.9999982 0.9997267 0.99591575 0.99211138 0.97672169 0.92254282 0.81803049 0.72066334 4.8778e-01 1.7964e-01
8 1.0000000 0.9999999 0.9999639 0.99915733 0.99816778 0.99334235 0.97125217 0.91608484 0.85384502 6.6748e-01 3.2380e-01
9 1.0000000 1.0000000 0.9999961 0.99985649 0.99964824 0.99842088 0.99109672 0.96744664 0.93523383 8.1391e-01 5.0000e-01
10 1.0000000 1.0000000 0.9999996 0.99997987 0.99994433 0.99969051 0.99771157 0.98945884 0.97592823 9.1153e-01 6.7620e-01
11 1.0000000 1.0000000 1.0000000 0.99999769 0.99999278 0.99995021 0.99951562 0.99717741 0.99257594 9.6477e-01 8.2036e-01
12 1.0000000 1.0000000 1.0000000 0.99999978 0.99999924 0.99999349 0.99991652 0.99938271 0.99812518 9.8844e-01 9.1647e-01
13 1.0000000 1.0000000 1.0000000 0.99999998 0.99999994 0.99999932 0.99998848 0.99989163 0.99961920 9.9693e-01 9.6822e-01
14 1.0000000 1.0000000 1.0000000 1.00000000 1.00000000 0.99999994 0.99999876 0.99998510 0.99993935 9.9936e-01 9.9039e-01
15 1.0000000 1.0000000 1.0000000 1.00000000 1.00000000 1.00000000 0.99999990 0.99999846 0.99999271 9.9990e-01 9.9779e-01
16 1.0000000 1.0000000 1.0000000 1.00000000 1.00000000 1.00000000 0.99999999 0.99999989 0.99999938 9.9999e-01 9.9964e-01
17 1.0000000 1.0000000 1.0000000 1.00000000 1.00000000 1.00000000 1.00000000 1.00000000 0.99999997 1.00000000 9.9996e-01
18 1.0000000 1.0000000 1.0000000 1.00000000 1.00000000 1.00000000 1.00000000 1.00000000 1.00000000 1.00000000 1.00000000
n=20 p=0.01 0.05 0.1 0.15 1/6 0.2 0.25 0.3 1/3 0.4 0.5
x=0 0.8179069 0.3584859 0.1215767 0.03875953 0.02608405 0.01152922 0.00317121 0.00079792 0.00030073 3.6562e-05 9.5367e-07
1 0.9831407 0.7358395 0.3917470 0.17555788 0.13042027 0.06917529 0.02431263 0.00763726 0.00330802 5.2405e-04 2.0027e-05
2 0.9989964 0.9245163 0.6769268 0.40489628 0.32865907 0.20608472 0.09126043 0.03548313 0.01759263 3.6115e-03 2.0123e-04
3 0.9999574 0.9840985 0.8670467 0.64772517 0.56654564 0.41144886 0.22515605 0.10708680 0.06044646 1.5961e-02 1.2884e-03
4 0.9999986 0.9974261 0.9568255 0.82984685 0.76874922 0.62964826 0.41484150 0.23750778 0.15151086 5.0952e-02 5.9090e-03
5 1.0000000 0.9996707 0.9887469 0.93269203 0.89815951 0.80420779 0.61717265 0.41637083 0.29721389 1.2560e-01 2.0695e-02
6 1.0000000 0.9999661 0.9976139 0.97806490 0.96286466 0.91330749 0.78578195 0.60800981 0.47934269 2.5001e-01 5.7659e-02
7 1.0000000 0.9999971 0.9995844 0.99407885 0.98874672 0.96785734 0.89818814 0.77227180 0.66147148 4.1589e-01 1.3159e-01
8 1.0000000 0.9999998 0.9999401 0.99867109 0.99715838 0.99001821 0.95907483 0.88666854 0.80945113 5.9560e-01 2.5172e-01
9 1.0000000 1.0000000 0.9999928 0.99975162 0.99940150 0.99740517 0.98613558 0.95203810 0.90810423 7.5534e-01 4.1190e-01
10 1.0000000 1.0000000 0.9999993 0.99996137 0.99989498 0.99943659 0.99605786 0.98285518 0.96236343 8.7248e-01 5.8810e-01
11 1.0000000 1.0000000 0.9999999 0.99999502 0.99998471 0.99989827 0.99906461 0.99486184 0.98702670 9.4347e-01 7.4828e-01
12 1.0000000 1.0000000 1.0000000 0.99999947 0.99999816 0.99998484 0.99981630 0.99872112 0.99627543 9.7897e-01 8.6841e-01
13 1.0000000 1.0000000 1.0000000 0.99999995 0.99999982 0.99999815 0.99997049 0.99973895 0.99912119 9.9353e-01 9.4234e-01
14 1.0000000 1.0000000 1.0000000 1.00000000 0.99999999 0.99999982 0.99999619 0.99995706 0.99983263 9.9839e-01 9.7931e-01
15 1.0000000 1.0000000 1.0000000 1.00000000 1.00000000 0.99999999 0.99999961 0.99999445 0.99997492 9.9968e-01 9.9409e-01
16 1.0000000 1.0000000 1.0000000 1.00000000 1.00000000 1.00000000 0.99999997 0.99999946 0.99999715 9.9995e-01 9.9871e-01
17 1.0000000 1.0000000 1.0000000 1.00000000 1.00000000 1.00000000 1.00000000 0.99999996 0.99999977 1.00000000 9.9980e-01
18 1.0000000 1.0000000 1.0000000 1.00000000 1.00000000 1.00000000 1.00000000 1.00000000 0.99999999 1.00000000 9.9998e-01
19 1.0000000 1.0000000 1.0000000 1.00000000 1.00000000 1.00000000 1.00000000 1.00000000 1.00000000 1.00000000 1.00000000
n<-1
probs<-c(0.01,0.05,0.1,0.15,1/6,0.2,0.25,0.3,1/3,0.4,0.5)
ptabline<-function(x){
return(pbinom(x,prob=probs,size=n))
107
ptab<-function(x){
n<<-x
t(sapply(0:(x-1),FUN=ptabline))
}
lapply(1:20,FUN=ptab)
108 APPENDIX A. DISTRIBUTION TABLES
bxc
X λk
P oλ (x) = e−λ
k!
k=0
x λ =0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
0 0.9048374 0.8187308 0.7408182 0.6703200 0.6065307 0.5488116 0.4965853 0.4493290 0.4065697 0.3678794
1 0.9953212 0.9824769 0.9630637 0.9384481 0.9097960 0.8780986 0.8441950 0.8087921 0.7724824 0.7357589
2 0.9998453 0.9988515 0.9964005 0.9920737 0.9856123 0.9768847 0.9658584 0.9525774 0.9371431 0.9196986
3 0.9999962 0.9999432 0.9997342 0.9992237 0.9982484 0.9966419 0.9942465 0.9909201 0.9865413 0.9810118
4 0.9999999 0.9999977 0.9999842 0.9999388 0.9998279 0.9996055 0.9992145 0.9985887 0.9976559 0.9963402
5 1.0000000 0.9999999 0.9999992 0.9999960 0.9999858 0.9999611 0.9999100 0.9998157 0.9996565 0.9994058
6 1.0000000 1.0000000 0.9999998 0.9999990 0.9999967 0.9999911 0.9999793 0.9999566 0.9999168
7 1.0000000 0.9999999 0.9999998 0.9999992 0.9999979 0.9999952 0.9999898
8 1.0000000 1.0000000 0.9999999 0.9999998 0.9999995 0.9999989
9 1.0000000 1.0000000 1.0000000 0.9999999
10 1.0000000
x λ =1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2.0
0 0.3328711 0.3011942 0.2725318 0.2465970 0.2231302 0.2018965 0.1826835 0.1652989 0.1495686 0.1353353
1 0.6990293 0.6626273 0.6268231 0.5918327 0.5578254 0.5249309 0.4932455 0.4628369 0.4337490 0.4060058
2 0.9004163 0.8794871 0.8571125 0.8334977 0.8088468 0.7833585 0.7572232 0.7306211 0.7037204 0.6766764
3 0.9742582 0.9662310 0.9569045 0.9462747 0.9343575 0.9211865 0.9068106 0.8912916 0.8747022 0.8571235
4 0.9945647 0.9922542 0.9893370 0.9857467 0.9814241 0.9763177 0.9703852 0.9635933 0.9559186 0.9473470
5 0.9990321 0.9984998 0.9977694 0.9967989 0.9955440 0.9939597 0.9920006 0.9896220 0.9867808 0.9834364
6 0.9998512 0.9997489 0.9995964 0.9993777 0.9990740 0.9986642 0.9981249 0.9974306 0.9965539 0.9954662
7 0.9999799 0.9999630 0.9999357 0.9998935 0.9998304 0.9997396 0.9996123 0.9994385 0.9992065 0.9989033
8 0.9999976 0.9999951 0.9999909 0.9999837 0.9999723 0.9999546 0.9999283 0.9998903 0.9998366 0.9997626
9 0.9999997 0.9999994 0.9999988 0.9999978 0.9999959 0.9999929 0.9999880 0.9999806 0.9999696 0.9999535
10 1.0000000 0.9999999 0.9999999 0.9999997 0.9999994 0.9999990 0.9999982 0.9999969 0.9999948 0.9999917
11 1.0000000 1.0000000 1.0000000 0.9999999 0.9999999 0.9999997 0.9999995 0.9999992 0.9999986
12 1.0000000 1.0000000 1.0000000 0.9999999 0.9999999 0.9999998
13 1.0000000 1.0000000 1.0000000
x λ =2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 2.9 3.0
0 0.1224564 0.1108032 0.1002588 0.09071795 0.0820850 0.07427358 0.06720551 0.06081006 0.05502322 0.04978707
1 0.3796149 0.3545701 0.3308542 0.30844104 0.2872975 0.26738488 0.24866040 0.23107824 0.21459056 0.19914827
2 0.6496314 0.6227137 0.5960388 0.56970875 0.5438131 0.51842958 0.49362449 0.46945368 0.44596320 0.42319008
3 0.8386428 0.8193524 0.7993471 0.77872291 0.7575761 0.73600164 0.71409218 0.69193743 0.66962342 0.64723189
4 0.9378739 0.9275037 0.9162493 0.90413141 0.8911780 0.87742349 0.86290786 0.84767606 0.83177708 0.81526324
5 0.9795509 0.9750902 0.9700243 0.96432749 0.9579790 0.95096285 0.94326833 0.93488969 0.92582620 0.91608206
6 0.9941379 0.9925387 0.9906381 0.98840592 0.9858127 0.98282990 0.97943055 0.97558938 0.97128327 0.96649146
7 0.9985140 0.9980224 0.9974112 0.99666138 0.9957533 0.99466624 0.99337883 0.99186926 0.99011549 0.98809550
8 0.9996627 0.9995305 0.9993584 0.99913802 0.9988597 0.99851305 0.99808637 0.99756722 0.99694217 0.99619701
9 0.9999307 0.9998991 0.9998561 0.99979846 0.9997226 0.99962435 0.99949864 0.99933991 0.99914188 0.99889751
10 0.9999870 0.9999802 0.9999705 0.99995696 0.9999384 0.99991329 0.99987995 0.99983627 0.99977979 0.99970766
11 0.9999978 0.9999964 0.9999944 0.99999155 0.9999874 0.99998158 0.99997354 0.99996261 0.99994797 0.99992861
12 0.9999996 0.9999994 0.9999990 0.99999846 0.9999976 0.99999638 0.99999460 0.99999209 0.99998861 0.99998385
13 0.9999999 0.9999999 0.9999998 0.99999974 0.9999996 0.99999934 0.99999897 0.99999844 0.99999768 0.99999660
14 1.00000000 1.0000000 1.0000000 0.99999996 0.9999999 0.99999989 0.99999982 0.99999971 0.99999956 0.99999933
15 1.00000000 1.0000000 0.99999998 0.99999997 0.99999995 0.99999992 0.99999988
16 1.00000000 1.00000000 0.99999999 0.99999999 0.99999998
17 1.00000000 1.00000000 1.00000000
A.1. POISSON DISTRIBUTION 109
x λ =3.5 4.0 4.5 5.0 5.5 6.0 6.5 7.0 7.5 8.0
0 0.03019738 0.01831564 0.01110900 0.00673795 0.00408677 0.00247875 0.00150344 0.00091188 0.00055308 0.00033546
1 0.13588823 0.09157819 0.06109948 0.04042768 0.02656401 0.01735127 0.01127579 0.00729506 0.00470122 0.00301916
2 0.32084720 0.23810331 0.17357807 0.12465202 0.08837643 0.06196880 0.04303595 0.02963616 0.02025672 0.01375397
3 0.53663267 0.43347012 0.34229596 0.26502592 0.20169920 0.15120388 0.11184961 0.08176542 0.05914546 0.04238011
4 0.72544495 0.62883694 0.53210358 0.44049329 0.35751800 0.28505650 0.22367182 0.17299161 0.13206186 0.09963240
5 0.85761355 0.78513039 0.70293043 0.61596066 0.52891869 0.44567964 0.36904068 0.30070828 0.24143645 0.19123606
6 0.93471190 0.88932602 0.83105058 0.76218346 0.68603598 0.60630278 0.52652362 0.44971106 0.37815469 0.31337428
7 0.97326108 0.94886638 0.91341353 0.86662833 0.80948528 0.74397976 0.67275778 0.59871384 0.52463853 0.45296081
8 0.99012634 0.97863657 0.95974269 0.93190637 0.89435668 0.84723749 0.79157303 0.72909127 0.66196712 0.59254734
9 0.99668506 0.99186776 0.98290727 0.96817194 0.94622253 0.91607598 0.87738405 0.83049594 0.77640761 0.71662426
10 0.99898061 0.99716023 0.99333133 0.98630473 0.97474875 0.95737908 0.93316121 0.90147921 0.86223798 0.81588579
11 0.99971101 0.99908477 0.99759572 0.99454691 0.98901186 0.97990804 0.96612044 0.94665038 0.92075869 0.88807600
12 0.99992404 0.99972628 0.99919486 0.99798115 0.99554912 0.99117252 0.98397336 0.97300023 0.95733413 0.93620280
13 0.99998140 0.99992367 0.99974841 0.99930201 0.99831488 0.99637151 0.99289982 0.98718861 0.97843535 0.96581930
14 0.99999574 0.99998007 0.99992634 0.99977375 0.99940143 0.99859965 0.99704424 0.99428280 0.98973957 0.98274301
15 0.99999908 0.99999511 0.99997972 0.99993099 0.99979983 0.99949090 0.99884016 0.99759342 0.99539168 0.99176900
16 0.99999981 0.99999887 0.99999473 0.99998013 0.99993678 0.99982512 0.99956975 0.99904182 0.99804111 0.99628200
17 0.99999996 0.99999975 0.99999870 0.99999458 0.99998109 0.99994308 0.99984872 0.99963822 0.99921000 0.99840574
18 0.99999999 0.99999995 0.99999970 0.99999860 0.99999463 0.99998240 0.99994945 0.99987015 0.99969700 0.99934963
19 1.00000000 0.99999999 0.99999993 0.99999966 0.99999855 0.99999482 0.99998391 0.99995560 0.99988925 0.99974706
20 1.00000000 0.99999999 0.99999992 0.99999963 0.99999855 0.99999511 0.99998551 0.99996134 0.99990603
21 1.00000000 0.99999998 0.99999991 0.99999961 0.99999858 0.99999547 0.99998709 0.99996659
22 1.00000000 0.99999998 0.99999990 0.99999961 0.99999865 0.99999587 0.99998861
23 1.00000000 0.99999998 0.99999990 0.99999961 0.99999873 0.99999627
24 0.99999999 0.99999997 0.99999989 0.99999963 0.99999883
25 1.00000000 0.99999999 0.99999997 0.99999989 0.99999964
26 1.00000000 0.99999999 0.99999997 0.99999990
27 1.00000000 0.99999999 0.99999997
28 1.00000000 0.99999999
29 1.00000000
x λ=9 10 11 12 13 14 15 20 25 30
0 0.00012341 0.00004540 0.00001670 0.00000614 0.00000226 0.00000083 0.00000031 0.00000000 0.00000000 0.00000000
1 0.00123410 0.00049940 0.00020042 0.00007987 0.00003164 0.00001247 0.00000489 0.00000004 0.00000000 0.00000000
2 0.00623220 0.00276940 0.00121087 0.00052226 0.00022264 0.00009396 0.00003931 0.00000046 0.00000000 0.00000000
3 0.02122649 0.01033605 0.00491587 0.00229179 0.00105030 0.00047425 0.00021138 0.00000320 0.00000004 0.00000000
4 0.05496364 0.02925269 0.01510460 0.00760039 0.00374019 0.00180525 0.00085664 0.00001694 0.00000027 0.00000000
5 0.11569052 0.06708596 0.03751981 0.02034103 0.01073389 0.00553205 0.00279243 0.00007191 0.00000140 0.00000002
6 0.20678084 0.13014140 0.07861437 0.04582231 0.02588692 0.01422792 0.00763190 0.00025512 0.00000611 0.00000012
7 0.32389696 0.22022060 0.14319153 0.08950450 0.05402825 0.03161966 0.01800219 0.00077859 0.00002292 0.00000052
8 0.45565260 0.33281970 0.23198513 0.15502780 0.09975791 0.06205520 0.03744649 0.00208726 0.00007548 0.00000205
9 0.58740824 0.45792970 0.34051064 0.24239220 0.16581190 0.10939940 0.06985366 0.00499541 0.00022148 0.00000712
10 0.70598832 0.58303980 0.45988870 0.34722940 0.25168200 0.17568120 0.11846440 0.01081172 0.00058646 0.00002235
11 0.80300838 0.69677610 0.57926676 0.46159730 0.35316490 0.26003990 0.18475180 0.02138682 0.00141597 0.00006388
12 0.87577343 0.79155650 0.68869665 0.57596520 0.46310470 0.35845840 0.26761100 0.03901199 0.00314412 0.00016770
13 0.92614923 0.86446440 0.78129117 0.68153560 0.57304460 0.46444760 0.36321780 0.06612764 0.00646748 0.00040728
14 0.95853367 0.91654150 0.85404401 0.77202450 0.67513150 0.57043670 0.46565370 0.10486430 0.01240206 0.00092068
15 0.97796434 0.95125960 0.90739609 0.84441570 0.76360690 0.66935990 0.56808960 0.15651310 0.02229302 0.00194748
16 0.98889409 0.97295840 0.94407565 0.89870900 0.83549310 0.75591770 0.66412320 0.22107420 0.03774765 0.00387273
17 0.99468043 0.98572240 0.96780948 0.93703370 0.89046500 0.82720060 0.74885880 0.29702840 0.06047504 0.00727022
18 0.99757360 0.99281350 0.98231349 0.96258350 0.93016690 0.88264290 0.81947170 0.38142190 0.09204086 0.01293270
19 0.99894405 0.99654570 0.99071054 0.97872020 0.95733130 0.92349510 0.87521880 0.47025730 0.13357480 0.02187347
20 0.99956075 0.99841170 0.99532892 0.98840230 0.97498820 0.95209160 0.91702910 0.55909260 0.18549230 0.03528462
21 0.99982505 0.99930030 0.99774808 0.99393490 0.98591860 0.97115590 0.94689360 0.64369760 0.24729880 0.05444340
22 0.99993317 0.99970430 0.99895765 0.99695260 0.99237750 0.98328780 0.96725580 0.72061130 0.31753350 0.08056902
23 0.99997548 0.99987990 0.99953614 0.99852710 0.99602820 0.99067240 0.98053540 0.78749280 0.39387550 0.11464590
24 0.99999135 0.99995310 0.99980129 0.99931440 0.99800570 0.99498010 0.98883520 0.84322740 0.47339850 0.15724200
25 0.99999706 0.99998230 0.99991795 0.99969220 0.99903400 0.99739240 0.99381510 0.88781500 0.55292140 0.20835740
26 0.99999904 0.99999360 0.99996731 0.99986670 0.99954810 0.99869130 0.99668810 0.92211320 0.62938580 0.26733660
27 0.99999969 0.99999770 0.99998742 0.99994420 0.99979570 0.99936490 0.99828420 0.94751930 0.70018610 0.33286910
28 0.99999991 0.99999920 0.99999532 0.99997740 0.99991060 0.99970160 0.99913930 0.96566650 0.76340070 0.40308250
29 0.99999997 0.99999970 0.99999831 0.99999110 0.99996210 0.99986420 0.99958160 0.97818180 0.81789610 0.47571700
30 0.99999999 0.99999990 0.99999941 0.99999660 0.99998440 0.99994010 0.99980270 0.98652530 0.86330890 0.54835150
31 1.00000000 1.00000000 0.99999980 0.99999880 0.99999380 0.99997430 0.99990970 0.99190820 0.89993210 0.61864300
110 APPENDIX A. DISTRIBUTION TABLES
x λ=9 10 11 12 13 14 15 20 25 30
32 1.00000000 1.00000000 0.99999993 0.99999960 0.99999760 0.99998930 0.99995980 0.99527260 0.92854400 0.68454120
33 0.99999998 0.99999980 0.99999910 0.99999570 0.99998260 0.99731160 0.95021960 0.74444880
34 0.99999999 0.99999990 0.99999970 0.99999830 0.99999270 0.99851100 0.96615760 0.79730830
35 1.00000000 1.00000000 0.99999990 0.99999940 0.99999700 0.99919630 0.97754190 0.84261650
36 1.00000000 0.99999980 0.99999880 0.99957710 0.98544770 0.88037340
37 0.99999990 0.99999950 0.99978290 0.99078940 0.91098700
38 1.00000000 0.99999980 0.99989120 0.99430370 0.93515570
39 0.99999990 0.99994680 0.99655640 0.95374700
40 1.00000000 0.99997460 0.99796440 0.96769040
41 0.99998810 0.99882290 0.97789300
42 0.99999460 0.99933390 0.98518050
43 0.99999760 0.99963100 0.99026480
44 0.99999890 0.99979990 0.99373140
45 0.99999950 0.99989360 0.99604240
46 0.99999980 0.99994460 0.99754960
47 0.99999990 0.99997170 0.99851170
48 1.00000000 0.99998580 0.99911300
49 0.99999300 0.99948110
50 0.99999660 0.99970200
51 0.99999840 0.99983190
52 0.99999930 0.99990690
53 0.99999970 0.99994930
54 0.99999980 0.99997290
55 0.99999990 0.99998570
56 1.00000000 0.99999260
57 0.99999620
58 0.99999810
59 0.99999910
60 0.99999960
ll <- seq(0.1:1,by=0.1)
poistab<-function(x){
return(ppois(x,lambda=ll))
}
t(sapply(0:10,FUN=poistab))
ll <- ll+1
t(sapply(0:13,FUN=poistab))
ll <- ll+1
t(sapply(0:17,FUN=poistab))
ll <- seq(3.5,8,by=0.5)
t(sapply(0:32,FUN=poistab))
ll <- c(9,10,11,12,13,14,15,20,25,30)
t(sapply(0:60,FUN=poistab))
A.2. STANDARD NORMAL DISTRIBUTION 111
0.3
0.1
N0,1(x)
-4 -2 0 2 4
Z x
1 2
− t2
Nµ=0,σ2 =1 (x) = √ ·e dt
−∞ 2π
x 0.00 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09
0.0 0.5 0.5039894 0.5079783 0.5119665 0.5159534 0.5199388 0.5239222 0.5279032 0.5318814 0.5358564
0.1 0.5398278 0.5437953 0.5477584 0.5517168 0.55567 0.5596177 0.5635595 0.5674949 0.5714237 0.5753454
0.2 0.5792597 0.5831662 0.5870644 0.5909541 0.5948349 0.5987063 0.6025681 0.6064199 0.6102612 0.6140919
0.3 0.6179114 0.6217195 0.6255158 0.6293 0.6330717 0.6368307 0.6405764 0.6443088 0.6480273 0.6517317
0.4 0.6554217 0.659097 0.6627573 0.6664022 0.6700314 0.6736448 0.6772419 0.6808225 0.6843863 0.6879331
0.5 0.6914625 0.6949743 0.6984682 0.701944 0.7054015 0.7088403 0.7122603 0.7156612 0.7190427 0.7224047
0.6 0.7257469 0.7290691 0.7323711 0.7356527 0.7389137 0.7421539 0.7453731 0.7485711 0.7517478 0.7549029
0.7 0.7580363 0.7611479 0.7642375 0.7673049 0.77035 0.7733726 0.7763727 0.7793501 0.7823046 0.7852361
0.8 0.7881446 0.7910299 0.7938919 0.7967306 0.7995458 0.8023375 0.8051055 0.8078498 0.8105703 0.8132671
0.9 0.8159399 0.8185887 0.8212136 0.8238145 0.8263912 0.8289439 0.8314724 0.8339768 0.8364569 0.8389129
1.0 0.8413447 0.8437524 0.8461358 0.848495 0.85083 0.8531409 0.8554277 0.8576903 0.8599289 0.8621434
1.1 0.8643339 0.8665005 0.8686431 0.8707619 0.8728568 0.8749281 0.8769756 0.8789995 0.8809999 0.8829768
1.2 0.8849303 0.8868606 0.8887676 0.8906514 0.8925123 0.8943502 0.8961653 0.8979577 0.8997274 0.9014747
1.3 0.9031995 0.9049021 0.9065825 0.9082409 0.9098773 0.911492 0.913085 0.9146565 0.9162067 0.9177356
1.4 0.9192433 0.9207302 0.9221962 0.9236415 0.9250663 0.9264707 0.927855 0.9292191 0.9305634 0.9318879
1.5 0.9331928 0.9344783 0.9357445 0.9369916 0.9382198 0.9394292 0.9406201 0.9417924 0.9429466 0.9440826
1.6 0.9452007 0.9463011 0.9473839 0.9484493 0.9494974 0.9505285 0.9515428 0.9525403 0.9535213 0.954486
1.7 0.9554345 0.9563671 0.9572838 0.9581849 0.9590705 0.9599408 0.9607961 0.9616364 0.962462 0.963273
1.8 0.9640697 0.9648521 0.9656205 0.966375 0.9671159 0.9678432 0.9685572 0.9692581 0.969946 0.970621
1.9 0.9712834 0.9719334 0.9725711 0.9731966 0.9738102 0.9744119 0.9750021 0.9755808 0.9761482 0.9767045
2.0 0.9772499 0.9777844 0.9783083 0.9788217 0.9793248 0.9798178 0.9803007 0.9807738 0.9812372 0.9816911
2.1 0.9821356 0.9825708 0.982997 0.9834142 0.9838226 0.9842224 0.9846137 0.9849966 0.9853713 0.9857379
2.2 0.9860966 0.9864474 0.9867906 0.9871263 0.9874545 0.9877755 0.9880894 0.9883962 0.9886962 0.9889893
2.3 0.9892759 0.9895559 0.9898296 0.9900969 0.9903581 0.9906133 0.9908625 0.991106 0.9913437 0.9915758
2.4 0.9918025 0.9920237 0.9922397 0.9924506 0.9926564 0.9928572 0.9930531 0.9932443 0.9934309 0.9936128
2.5 0.9937903 0.9939634 0.9941323 0.9942969 0.9944574 0.9946139 0.9947664 0.9949151 0.99506 0.9952012
2.6 0.9953388 0.9954729 0.9956035 0.9957308 0.9958547 0.9959754 0.996093 0.9962074 0.9963189 0.9964274
2.7 0.996533 0.9966358 0.9967359 0.9968333 0.996928 0.9970202 0.9971099 0.9971972 0.9972821 0.9973646
2.8 0.9974449 0.9975229 0.9975988 0.9976726 0.9977443 0.997814 0.9978818 0.9979476 0.9980116 0.9980738
2.9 0.9981342 0.9981929 0.9982498 0.9983052 0.9983589 0.9984111 0.9984618 0.998511 0.9985588 0.9986051
3.0 0.9986501 0.9986938 0.9987361 0.9987772 0.9988171 0.9988558 0.9988933 0.9989297 0.998965 0.9989992
3.1 0.9990324 0.9990646 0.9990957 0.999126 0.9991553 0.9991836 0.9992112 0.9992378 0.9992636 0.9992886
3.2 0.9993129 0.9993363 0.999359 0.999381 0.9994024 0.999423 0.9994429 0.9994623 0.999481 0.9994991
3.3 0.9995166 0.9995335 0.9995499 0.9995658 0.9995811 0.9995959 0.9996103 0.9996242 0.9996376 0.9996505
3.4 0.9996631 0.9996752 0.9996869 0.9996982 0.9997091 0.9997197 0.9997299 0.9997398 0.9997493 0.9997585
3.5 0.9997674 0.9997759 0.9997842 0.9997922 0.9997999 0.9998074 0.9998146 0.9998215 0.9998282 0.9998347
3.6 0.9998409 0.9998469 0.9998527 0.9998583 0.9998637 0.9998689 0.9998739 0.9998787 0.9998834 0.9998879
3.7 0.9998922 0.9998964 0.9999004 0.9999043 0.999908 0.9999116 0.999915 0.9999184 0.9999216 0.9999247
3.8 0.9999277 0.9999305 0.9999333 0.9999359 0.9999385 0.9999409 0.9999433 0.9999456 0.9999478 0.9999499
3.9 0.9999519 0.9999539 0.9999557 0.9999575 0.9999593 0.9999609 0.9999625 0.9999641 0.9999655 0.999967
4.0 0.9999683 0.9999696 0.9999709 0.9999721 0.9999733 0.9999744 0.9999755 0.9999765 0.9999775 0.9999784
normtab<-function(x){
return(pnorm(x+dec2))
}
t(sapply(dec1,FUN=normtab))