StatIdea Slides 1

1.
Probability
J. CaballÈ (UAB - MOVE - BSE) Probability and Statistics IDEA 1 / 48

1.1. Random experiments and sample spaces
Random experiment: experiment, trial, or observation that can be
repeated under the same conditions and whose outcomes are
uncertain but observable.
⼀
Sample space: The set W of possible outcomes of a random

⼀
experiment.
Sample point (or "state of the nature"): An outcome of a
random experiment, w 2 W.
Example 1: Random experiment: Roll a dice. W1 = f1, 2, 3, 4, 5, 6g .
Example 2: Random experiment: Flip a coin. W2 = fH, T g .
Example 3: Random experiment: Roll a dice and áip a coin.
W3 = W1 $ W2 = f(1, H ) , (2, H ) , (3, H ) , (4, H ) , (5, H ) , (6, H ) ,
(1, T ) , (2, T ) , (3, T ) , (4, T ) , (5, T ) , (6, T )g .

Example 4: Random experiment: You pick an individual from a large
population and observe his/her wealth measured in cents.
! "
W4 = ⑥
the set of all integers comprised between
the minimum and the maximum wealth
.
A set W is discrete or countable if there exists an injective (or

⼀
one-to-one) function from W to the set of natural numbers N.

⻛ rfy
—
x=
y
A set W is continuous if it is not discrete.
Sample spaces can be either discrete (or countable) or continuous.
Discrete sample spaces can be either Önite or (countable) inÖnite.

⼀
µ m

1.2. Composition of experiments and combinatorics
Fundamental theorem of combinatorics. Consider an experiment

(or operation) that consists of K sub-experiments (or steps) and each
step j can be made in nj di§erent ways, j = 1, 2, ..., K , then the
O
K
-
How may sleps ,
whole operation can be made in ’ nj = n1 !n2 !...!nK di§erent ways.

j =1
Sometimes the experiment consists of selecting permutations (or ⾤
orders or arrangements) of n objects.
The number of permutations Pn of n di§erent objects is given by
Pn = n! " n ! (n # 1) ! (n # 2) ! ... ! 3 ! 2 ! 1,
where n is a natural number (n = 0, 1, 2, ...) and the symbol "!"

denotes the factorial.

Variations (or permutations) of r objects taken from a set of n
di§erent objects (n $ r ): Vnr .
Here only r objects are taken from the set of n di§erent objects (with
n $ r ) and, thus, two variations may di§er because their objects are
di§erent or because they are arranged in a di§erent order.
The number of variations of r objects taken from a set of n di§erent

objects is given by
n!
排列
Vnr = n ! (n # 1) ! (n # 2) ! ... ! (n # r + 1) = ,
| {z } (n # r ) !
r elements
where n and r are natural numbers with n $ r .
Example: A club has 24 members. Its executive committee is formed

by 4 members of the club: the president, the vice-president, the
treasurer and the secretary. Then, this executive committee can be
formed in 24 ! 23 ! 22 ! 21 = 255 024 di§erent ways.
Permutations with repeated objects.
The number of permutations of n objects with n1 of type 1, n2 of
K
type 2,..., nK of type K , with Â nj = n, is given by
j =1
balls
havve
may
0
n! n!
= . the same color
n1 !n2 !...nK ! K
’ nj !
j =1
Example: With the letters appearing in the word ëTallahasseeí we

can form
11! 39 916 800
= = 831 600
品
1!3!2!1!2!2! 48
di§erent words.

Number of combinations of r objects taken from a set of n di§erent
objects (n $ r ), Cnr :
number of ways we can select r objects from a set of n di§erent objects
pick); or
— ←
(we do not care here about the order but only about the objects we
number of subsets of r elements from a set of n elements.
The number of combinations is given by the following combinatorial

Theorder
number: ' (
δ
n n! Vr
Cnr = " = n,
r r ! (n # r ) ! r!
where n and r are natural numbers with n $ r .
' (
n
The combinatorial number also gives us the number of
r
permutations of n objects with r of type 1 and (n # r ) of type 2.
可 !
Number of ways of partitioning a set with n elements into K subsets
K
of n1, n2 , ..., nK elements, with Â nj = n:
j =1
' (
n n!
" ,
n1, n2 , ..., nK n1 !n2 !...nK !
which coincides with the formula of the number of permutations with

repeated objects.

Properties of combinatorial numbers:
1. ' ( ' (
n n
= .
r n#r
Proof: ' ( ' (

n n! n! n
= = = .
r r ! (n # r ) ! (n # r )!r ! n#r
[n (n
- -
γ ) } =γ .
2.
0! = 1.
Proof: ' (
n n! 1
=1= = =) 0! = 1.
n n!0! 0!

' (
n
The combinatorial number is also a coe¢cient in the polynomial
r
expansion of a binomial power (Newtonís binomial theorem):
n ' (
n n #r r
(x + y ) = Â
n
x y .
r =0 r
Moreover, Cxxxs , .
txr + + …
"
!n
K
Â xj =
j =1
n n n ' ( K
n
Â Â ... Â r1, r2 , ..., rK
(x1r1 !x2r2 !...!xKrK ) , with Â rj = n.
r 1 =0 r 2 =0 r K =0 j =1
Example. Let (x + y + z )6 . Then, the value of the coe¢cient c in

the term cx 3 yz 2 of the expansion is
' (
6 6!
c= = = 60.
3, 1, 2 3!1!2!

1.3. sigma-algebras, events, and measurable spaces.
B c denotes the complement of B relative to W, that is,

B c = fw 2 W jw 2 / Bg
A [ B = fw 2 W jw 2 A, or w 2 B, or both g .
A \ B = fw 2 W jw 2 A and w 2 B g .
AnB = fw 2 W jw 2 A and w 2
/ B g . Therefore, B c = WnB.

DeÖnition. A s-algebra (or s-Öeld) on W is a collection F of subsets
of W, such that
1. ∆ 2 F , W 2 F .
2. If B 2 F , then B c 2 F .
3. If fB1 , B2 , ...g is a countable (Önite or inÖnite) collection of
S
elements of F , then Bi 2 F .
i

Note: If fB1 , B2 , ...g is a countable (Önite or inÖnite) collection of
T
elements of the s-algebra F , then Bi 2 F .
i
Proof. Note that, if fB1 , B2 , ...g is a countable collection of elements

of the s-algebra F , then fB1c , B2c , ...g is also a countable collection of
elements of F . Therefore,
!c
[ [ \
Bic 2 F =) Bic 2 F =) Bi 2 F . Q.E.D. (?)
i i i
Moreover, part 3 of the deÖnition of s-algebra could be replaced by

the following:
⇌ 3í. If fB1 , B2 , ...g is a countable (Önite or inÖnite) collection of

T
elements of F , then Bi 2 F .
i
This replacement is justiÖed by just interchanging the set operations
S T
λ
and in (?). Λ
i i
In the deÖnition of algebra on W we replace the word "countable" in
(3) by "Önite".
Therefore, a s-algebra is an algebra, but the converse is not true.

⼀
The elements of a s-algebra are called measurable sets, B 2 F .

≈
If W is a sample space and we deÖne a s-algebra F on it, the

0
elements of F are called "events".
B is an event if the question "Does w 2 B?" has a deÖnite "yes" or

"not" answer after the experiment has been performed and delivered
the outcome w, for all outcomes w 2 W.
←

“
Events
lHow mamy subsels
in 2 .
「
The smallest s-algebra on W is f∆, Wg , whereas the largest is the
collection of all subsets of W (the power set, denoted by 2W ).
0
Let the number of elements (or cardinality) of W be #W = n. The
number of elements of the power set 2W is 、
所有⼦集数 ,
' ( ' ( ' (

W n n n
#2 = + + ... + = (1 + 1)n = 2n .
0 1 n ≡
Neweon BinomralThere rem .
←… cnbn
xnoy
”
别
Example: We roll a dice and áip a coin. This random experiment has
12 sample points (or outcomes):
W = f(1, H ) , (2, H ) , (3, H ) , (4, H ) , (5, H ) , (6, H ) ,
(1, T ) , (2, T ) , (3, T ) , (4, T ) , (5, T ) , (6, T )g .
Examples of events in the power set 2W :

A = f(2, H ) , (4, H ) , (6, H )g Even number of points and Head
B = f(1, H ) , (2, H ) , (3, H ) , (4, H ) , (5, H ) , (6, H )g Head
C = f(2, H ) , (4, H )g Number of points even no larger than 4 and Head
The power set 2W contains 212 = 4096 di§erent events.

DeÖnition. Let S = fA1 , A2 , ...g be a collection of arbitrary subsets
of W. The s-algebra s(S ) generated by S is the smallest s-algebra
containing S.
⼀
Note: f∆, Wg / s (S ) / 2W .
Example. Let W = f1, 2, 3, 4, 5, 6g and S = ff1g , f1, 3, 4gg . Then,
De
s(S ) = fW, ∆, f1g, f1, 3, 4g, f2, 3, 4, 5, 6g, f2, 5, 6g, f1, 2, 5, 6g, f3, 4gg.
~~

Usually, we consider s-algebras generated by partitions of W.
DeÖnition. A partition of the set W is a collection of subsets of W,

fB1 , B2 , ...g such that
(a) Bi 6= ∆, for all i,
T
(b) Bi Bj = ∆, for all pairs (i, j ) with i 6= j,
S
(c) Bi = W.
i

Example. Let W = f1, 2, 3, 4, 5, 6g and S = ff1, 2g , f3, 4g , f5, 6gg
is a partition of W. Then,
s(S ) = fW, ∆, f1, 2g, f3, 4g, f5, 6g, f1, 2, 3, 4g, f1, 2, 5, 6g, f3, 4, 5, 6gg.
If F1 / F2 , then we say that the s-algebra F2 is Öner than F1 (or

that F1 is coarser than F2 ).
f∆, Wg is the coarsest s-algebra on W, whereas the power set 2W is

⼀
the Önest s-algebra on W.

⼀
DeÖnition. The pair (W, F ) , where F is a s-algebra on W is called

a measurable space.
⼀

1.4. Measure, probability, and probability spaces.
DeÖnition. A measure µ on the measurable space (W, F ) is a set

function on F taking values on the set of non-negative extended real
→
numbers, * Indude o and ∞

µ : F #! [0, •] " R + , O ⇌
such that ⼀函数

1. If fB1 , B2 , ...g is a countable (Önite or inÖnite) collection of
mutually disjoint measurable sets (elements of F ), then
!
0
[
µ Bi = Â µ (Bi ) (countable additivity).
i i
algebramake
1
must be the t -
poperty of
2. µ (∆) = 0. sense of
o
Property 2 is redundant if at least one of the measurable sets has

Önite measure.
A signed measure on the measurable space (W, F ) is a countably
additive set function on F taking values on the set of extended real
numbers R " [#•, •] such that µ (∆) = 0.
0
Therefore, a measure is a signed measure that takes values on R + .

DeÖnition. The triple (W, F , µ), where (W, F ) is a measurable
→
space and µ is a measure on it, is called a measure space.

⼀
DeÖnition. A probability measure (or "probability") P on (W, F ),

where W is a sample space and F is a s-algebra of events, is a
measure such that 冖
P (W) = 1.
⼀
⼀
DeÖnition. The triple (W, F , P ) , where P is a probability on (W, F )

is called a probability space.
⼀
0
⼀
Note: We only assign probability to events.
µ
Measume
Space
-
P H2 + P( ∞
) = 1
Probabitey Space
-

Usually, if the sample space is discrete and the s-algebra of events is
the power set 2W , we assign probabilities to all the sample points and,
using the countable additivity 0property, we construct a probability on
Oz
/
the measurable space W, 2W .
.
/ 0
Proposition. Consider the probability space W, 2W , P , where
W = fw 1 , w 2 , ...g is a discrete sample space, then
P (B ) = Â P fw i g .
w i 2B

/ 0
Proposition (Laplace). Consider the probability space W, 2W , P . If
#W = N, #B = n, and all the sample points are equally likely, then
n
P (B ) = (Laplace formula).
N
⼀
N
S
Proof: Observe that W = fw i g . Therefore,
i =1
N
P (W) = Â P fw i g . Since all the sample points are equally likely,
i =1
P fw i g = p for all i. Then,
N
1 = P (W) = Â p = Np,
i =1
1
which implies that p = . Moreover,
N
1 n
P (B ) = Â P fw i g = Â
| {z } w 2B N
=
N
w i 2B i
=p
since the event B has n elements. Q.E .D.
Pierre Simon de Laplace (1749 - 1827)

Example: We roll a dice. Let W = f1, 2, 3, 4, 5, 6g , F = 2W ,
A = f1, 2, 3g , B = f3, 4g , C = f4, 5g , and D = f4g .
Then, P (A) = 3/6 = 1/2, P (B ) = 2/6 = 1/3, P (C ) = 2/6 = 1/3

and P (D ) = 1/6.
A [ C = f1, 2, 3, 4, 5g so that P (A [ C ) = 5/6.
Note also that A \ C = ∆ so that

3 2 5
P (A [ C ) = P (A) + P (C ) = + = .
6 6 6

Note on the relationship between probability and frequency: If
we randomly pick an object (or individual) from a population having a
variable X distributed according to the relative frequency fX , then
P fX 2 B g = Â fX ( x ) ,
x 2B
where fX 2 B g is the event where the variable X of the object we

pick takes a value belonging to the set B.
Notation:
P (A \ B ) = P (A, B ).

1.5. Properties of probability measures.
1 P (Ac ) = 1 # P (A). P ∞) = P ( AUAc ) = 1
2 If A / B, then P (A) 2 P (B ).
←
3 0 2 P (A) 2 1. ⇒ d ←
Aca .
4 P (A [ B ) = P (A) + P (B ) # P (A \ B ).
「
5 ⼀
P (A [ B [ C ) = P (A) + P (B ) + P (C ) # P (A \ B ) # P (A \ C )
D #P (B \ C ) + P (A \ B \ C ).
Prof
.
∞
AVB =
Properties 2, 4, and 5 also hold for a general measure µ and Property

1 then becomes µ(Ac ) = µ(W) # µ(A), while Property 3 becomes
0
0 2 µ (A) 2 µ ( W ).
All the previous rules of probability have to be satisÖed both when the
e
probability is objective (i.e., based on scientiÖc predictions) and when
e (i.e., based on beliefs).
the probability is subjective
— —
—
Example: We roll a dice. Let W = f1, 2, 3, 4, 5, 6g , F = 2W ,

A = f1, 2, 3g and B = f3, 4g .
Then, P (A) = 3/6 = 1/2 and P (B ) = 2/6 = 1/3.
A [ B = f1, 2, 3, 4g and A \ B = f3g.
Then, P (A [ B ) = 4/6 = 2/3 and P (A \ B ) = 1/6.
Note that
4 3 2 1 4
P (A [ B ) = = P (A) + P (B ) # P (A \ B ) = + # = .
6 6 6 6 6

1.6. Conditional probability.
DeÖnition. If A and B are events in the sample space W and

P (A) 6= (>)0, the conditional probability of B given A is
≈
P (A \ B )
P (B jA ) = .
P (A)
Eg P (x 51 x )= (x 7) 0
1
> 0
0
.
P :0 .
=
.
7
. .
Therefore, if A and B are events in the sample space W and

P (A) 6= (>)0, then
P (A \ B ) = P (A) ! P (B jA ) .

Proposition. If P (A1 \ A2 \ ... \ An #1 ) > 0, then
P (A1 \ A2 \ ... \ An ) =
P (A1 ) ! P (A2 jA1 ) ! P (A3 jA1 \ A2 ) ! ... ! P (An jA1 \ A2 \ ... \ An #1 ) .
Proof.
P (A1 ) ! P (A2 jA1 ) ! P (A3 jA1 \ A2 ) ! ... ! P (An jA1 \ A2 \ ... \ An #1 )
P (A1 \ A2 ) P (A1 \ A2 \ A3 ) P (A1 \ A2 \ ... \ An )

= P (A1 ) ! ! ! ... !
P (A1 ) P (A1 \ A2 ) P (A1 \ A2 \ ... \ An #1 )
= P (A1 \ A2 \ ... \ An ). Q.E.D.

Note: If we randomly pick an object (or individual) from a population
having the variables X and Y distributed according to the relative joint
frequency fX ,Y and, thus, with the relative marginal frequencies fX and
fY , and the conditional frequencies fX jY and fY jX , then
P fX 2 B, Y 2 C g = Â Â fX ,Y (x, y ) =)
x 2B y 2C
P fX = x, Y = y g = fX ,Y (x, y ) ,
P fX 2 B g = Â fX ( x ) =) P fX = x g = fX (x ) ,
x 2B
P fY 2 C g = Â fY ( y ) =) P fY = y g = fY (y ) ,
y 2C

Â Â fX ,Y (x, y )
P fX 2 B, Y 2 C g x 2B y 2C
P fX 2 B jY 2 C g = =
P fY 2 C g Â fY ( y )
y 2C
and
Â Â fX ,Y (x, y )
P fX 2 B, Y 2 C g x 2B y 2C
P fY 2 C jX 2 B g = =
P fX 2 B g Â fX ( x )
x 2B
=)
P f X 2 B j Y = y g = Â fX j Y ( x j y ) , P f Y 2 C j X = x g = Â fY j X ( y j x )
x 2B y 2C
and
P f X = x j Y = y g = fX j Y ( x j y ) , P f Y = y j X = x g = fY j X ( y j x ) ,
where fX 2 B g is the event where the variable X of the object we pick
takes a value belonging to the set B and fY 2 C g is the event where the
variable Y of the object we pick takes a value belonging to the set C .
Example. We extract two cards with "no replacement" from a deck
of poker cards. The probability that the two cards will be aces is
4 3
P (A1 \ A2 ) = P (A1 ) ! P (A2 jA1 ) = ! = 0.0045.
52 51
Assume now that the extractions are made "with replacement", that
is, the cards are introduced back in the deck after each extraction.
The probability that the two cards so extracted will be aces is
4 4
P (A1 \ A2 ) = P (A1 ) ! P (A2 jA1 ) = ! = 0.0059.
52 52
Example: We extract three cards with "no replacement" from a deck

of poker cards. The probability that all the three cards will be aces is
P (A1 \ A2 \ A3 ) = P (A1 ) ! P (A2 jA1 ) ! P (A3 jA1 \ A2 )

4 3 2
= ! ! = 0.000181.
52 51 50

1.7. Independent events.
DeÖnition. The events A and B are independent if
P (A \ B ) = P (A) ! P (B ) .
fd ,
it
Proposition. Let P (A) > 0. The events A and B are independent if

depends
always ouother
sobset
and only if
P (B ) = P (B jA ) .
Proof. Since P (A \ B ) = P (A) ! P (B jA ) if P (A) > 0 and the

events A and B are independent (i.e., P (A \ B ) = P (A) ! P (B )) ,
we immediately obtain the desired result. Q.E .D.

Example. We toss 3 coins (or the same coin three times). This random
experiment has 8 equally likely outcomes,
W = f(HHH ) , (HHT ) , (HTH ) , (THH ) , (TTH ) , (THT ) , (HTT ) , (TTT )g
so that the probability of each outcome is 1/8.
Consider the following events when F = 2W :

A = f(HHH ) , (HHT )g , fHead in the Örst two coinsg
B = f(HHT ) , (HTT ) , (THT ) , (TTT )g , fTail in the third coing
C = f(HTT ) , (THT ) , (TTH )g fExactly two tailsg
A \ B = f(HHT )g and B \ C = f(HTT ) , (THT )g .

P (A) = 2/8 = 1/4, P (B ) = 4/8 = 1/2, P (C ) = 3/8,
P (A \ B ) = 1/8, and P (B \ C ) = 2/8 = 1/4.
The events A and B are independent since

1 1 1 1
P (A \ B ) = = P (A) ! P (B ) = ! = .
8 4 2 8
The events B and C are not independent since

1 1 3 3
P (B \ C ) = 6 = P (B ) ! P (C ) = ! = .
4 2 8 16

Proposition. If A and B are independent events, then
(a) Ac and B are independent.
(b) Ac and B c are independent.

Proof. (a) Observe that
B = W \ B = (A [ Ac ) \ B = (A \ B ) [ (Ac \ B ) .
Since (A \ B ) and (Ac \ B ) are disjoint events,
P (B ) = P [(A \ B ) [ (Ac \ B )] = P (A \ B ) + P (Ac \ B ) ,
which is equivalent to
P (Ac \ B ) = P (B ) # P (A \ B ) .
Therefore, since P (Ac ) = 1 # P (A) and the events A and B are

independent (i.e., P (A \ B ) = P (A) ! P (B )) , we have
P (Ac \ B ) = P (B ) # P (A \ B ) = P (B ) # P (A) ! P (B )
= P (B ) [1 # P (A)] = P (B ) ! P (Ac ),
which proves the independency between Ac and B.

(b) Obvious from (a). Q.E .D.
DeÖnition. The events in the collection S = fA1 , A2 , ...g are

independent
~
if the probability of every Önite intersection of events in
⇌
S equals the product of their respective probabilities.

1.8. Theorem of total probability.
←
Theorem of Total Probability. Let fB1 , B2 , ...g be a countable
(Önite or inÖnite) collection of events that constitutes a partition of
the sample space W and assume that P (Bi ) > 0 for all i. Then,
P (A) = Â P (Bi ) ! P (A jBi ) , for every event A.
i
Proof. Since
!
[ [
A = A\W = A\ Bi = (A \ Bi )
i i
and the events in the countable collection fA \ B1 , A \ B2 , ...g are
0disjoint, we get
!
→
[
P (A) = P (A \ Bi ) = Â P (A \ Bi ) = Â P (Bi ) ! P (A jBi ) ,
i i i
where the last equality follows since P (A \ Bi ) = P (Bi ) ! P (A jBi ) if

P (Bi ) > 0. Q.E.D.
Example. We extract two cards with "no replacement" from a deck
of poker cards. The probability that the second card is an ace is
P (A2 ) = P (A1 ) ! P (A2 jA1 ) + P (Ac1 ) ! P (A2 jAc1 )

' ( ' (
4 3 48 4 4
= ! + ! = .
52 51 52 51 52
Assume now that the extractions are made "with replacement", that
is, the cards are introduced back in the deck after each extraction.
The probability that the second card is an ace is, obviously,
P (A2 ) = P (A1 ) ! P (A2 jA1 ) + P (Ac1 ) ! P (A2 jAc1 )

' ( ' (
4 4 48 4 4
= ! + ! = .
52 52 52 52 52

Note that, if the extractions are made with "no replacement", then
the events A1 and A2 are not independent since
4 3 4 4
P (A1 \ A2 ) = ! 6= P (A1 ) ! P (A2 ) = ! .
52 51 52 52
If the extractions are made "with replacement", then the events A1

and A2 are independent since
4 4 4 4
P (A1 \ A2 ) = ! = P (A1 ) ! P (A2 ) = ! .
52 52 52 52

The theorem of total probability can be modiÖed to apply for
intersections of events as follows:
Proposition. Let fB1 , B2 , ...g be a countable (Önite or inÖnite)
collection of events that constitutes a partition of the event B and
assume that P (Bi ) > 0 for all i. Then,
P (A \ B ) = Â P (Bi ) ! P (A jBi ) , for every event A.
i
S S
Proof. Since Bi = B, we have that (A \ Bi ) = A \ B, where the
i i
events in the countable collection fA \ B1 , A \ B2 , ...g are disjoint.
Thus, !
[
P (A \ B ) = P (A \ Bi ) = Â P (A \ Bi )
i i
= Â P (Bi ) ! P (A jBi ) . Q.E.D.

i
If we make B = W in the previous proposition, then we recover the

original theorem of total probability since P (A \ W) = P (A).
1.9. Bayesí theorem.
Thomas Bayes (1702 - 1761)

Bayesí theorem. Let fB1 , B2 , ...g be a countable (Önite or inÖnite)
collection of events that constitutes a partition of the sample space W
and assume that P (Bi ) > 0 for all i. Then, for every event A such
that P (A) > 0,
P (Bj ) ! P (A jBj )
P (Bj jA ) = , for all j = 1, 2, ...
Â P (Bi ) ! P (A jBi )
i
Proof. Obvious since

P (A \ Bj )
P (Bj jA ) = when P (A) > 0,
P (A)
and P (A \ Bj ) = P (Bj ) ! P (A jBj ) if P (Bj ) > 0, while

P (A) = Â P (Bi ) ! P (A jBi ) if P (Bi ) > 0 for all i, as follows from the
i
theorem of total probability. Q.E.D.

Note: P (Bj ) is the prior probability of the event Bj , P (A jBj ) is the
conditional probability of the event A given Bj (also called the
⼀
"likelihood" of A when Bj occurs), and P (Bj jA ) is the posterior

⼀⼀
probability of the event Bj given A, for j = 1, 2, ...

节
Example: We want to select a candidate for a job and we observe

the result of a test undertaken by the candidate.
fg , b g " fcandidate is good for a job, candidate is bad for a jobg ,

constitutes a partition of W. Thus, b = g c .
⼀
fp, f g " fcandidate has passed a test, candidate has failed a testg .
Prior probabilities: P (g ) = 0.25 so that P (b ) = 0.75.
Likelihoods: P (p jg ) = 0.99 so that P (f jg ) = 1 # P (p jg ) = 0.01,

and
P (f jb ) = 0.83 so that P (p jb ) = 1 # P (f jb ) = 0.17.

Then,
P (p \g )
z }| {
P (g ) ! P (p jg )
P (g jp ) =
P (g ) ! P (p jg ) + P (b ) ! P (p jb )
| {z }
P (p )
0.2475
z }| {
0.25 ! 0.99
= = 0.66.
(0.25 ! 0.99) + (0.75 ! 0.17)
| {z }
=0.375
Note that P (b jp ) = 1 # P (g jp ) = 0.34.
Moreover, P (p ) = 0.375 so that P (f ) = 0.625.
Finally, P (p \ g ) = P fpass and goodg = 0.2475.

Exercises. Probability and Statistics. IDEA.
1. Probability
1. Prove that, for any positive integer (or natural number) n larger or equal than
2 and r = 1; 2; :::; n ! 1;
! " ! " ! "
n n!1 n!1
= + :
r r r!1
「
Hint: Use the fact that
(1 + y)n = (1 + y) (1 + y)n!1 = (1 + y)n!1 + y (1 + y)n!1 :

1
2. Prove that
Xk ! "! " ! "
m n m+n
= :
r=0
r k!r k
Hint: Use the fact that
(1 + y)m+n = (1 + y)m (1 + y)n :
⼝
3. Suppose that we are concerned with the completion of a highway construction
job, which may be delayed because of a strike. Suppose, furthermore, that
the probabilities are 0:60 that there will be a strike, 0:85 that the job will be
completed on time if there is no strike, and 0:35 that the job will be completed on
time if there is a strike. What is the probability that the job will be completed
D
on time?
scisD
035 =
=! "n
1
4. A strictly positive integer I is selected, with P fI = ng = , n = 1; 2; ::: If
2
I takes the value n, a coin with probability e!n of heads is tossed once.
-
n
(a) Find the probability that the resulting toss is a head. e
(b) Find the conditional probability of fI = 5g given that we know that the
resulting toss has been a tail.
⼝
5. The members of a consulting Örm rent cars from three rental agencies: 60%
from agency 1, 30% from agency 2, and 10% from agency 3. If 9% of the cars
from agency 1 need a tune-up, 20% of the cars from agency 2 need a tune-up,
and 6% of the cars from agency 3 need a tune-up,
(a) what is the probability that a rental car delivered to the Örm will need a
tune-up?
(b) if a rental car delivered to the Örm needs a tune-up, what is the probability
that it came from the rental agency 2?
6. Let us deÖne a generalized combinatorial number as
Q
州
x!1
+
r ) ( re )
…
! " (r ! i)
r i=0
= ; x 1
x x! ,
for any real number r and any natural number x: Moreover, by deÖnition, we
have ! "
r
= 1;
0
for any real number r:
Prove that,
(a) If n is natural number such that n $ x; then we obtain the traditional

formula for a combinatorial number,
! "
n n!
= :
x x!(n ! x)!
(b) If n is a natural number such that n < x;

! "
n
= 0:
x
(c) If r and x are two natural numbers,

! " ! "
!r x r+x!1
= (!1) ;
x x
and, moreover, if r is a strictly positive natural number,
! " ! "
!r x r+x!1
= (!1) :
x r!1
(d) For any natural number x;

! " ! "
!1 !2
= (!1)x and = (!1)x (1 + x):
x x
(e) If r and z are two real numbers with jzj < 1;

X1 ! "
r r x
(1 + z) = z :
x=0
x
(f ) For any real number z and any natural number n;

Xn ! "
n n x
(1 + z) = z :
x=0
x
E 60 E&F -
-
-
10
C 10 下 30 -
⼝7. In a scientiÖc meeting of 100 people, 60 of them speak English only, 30

speak French only, and the remaining 10 speak both languages. Compute the
probability that two randomly selected participants will be able to understand
each other.
8. Ten individuals are randomly ordered and all possible orderings are equally
likely. Find the probability that two given individuals be contiguous if the 10
individuals are ordered: a) in a row, b) in a circle.
□
9. We have 11 urns, which are numbered from 2 to 12. The composition of the
urns is the following:
number of the urn 2 3 4 5 6 7 8 9 10 11 12

number of white balls 0 1 2 3 4 5 4 3 2 1 0
number of black balls 1 1 1 1 1 1 1 1 1 1 1
We roll two dices simultaneously. If the sum of points is k; we make an

extraction with replacement of a ball from the urn k: If the extracted ball
is black, the game is over. If the ball is white, we roll again the dices and repeat
the whole process. Compute the probability of making at least 3 successive
extractions.
⼝
10. There are n balls in a box. Each ball is either black or red and we assume that
the n + 1 di§erent possible compositions of the box are equally likely.
(a) We randomly pick one ball and it turns out to be red. What is the
probability that k out of the n balls in the box were red? N
Assume now that there are 7 balls in the box, n = 7; and that we extract two
balls with no replacement from the box.
(b) What is the probability that both balls will be black?
(c) What is the probability that the Örst extracted ball will be black and the
second red?
11. We have two decks of Spanish cards (with 48 cards where 12 of them are Ögures).
We randomly extract a card from the Örst deck and we insert it in the second
deck. Then, we roll a die. If we get one dot, we randomly extract a card from
the Örst deck; if we get two dots, we randomly extract a card from the second
deck; otherwise, we extract a card from one of the two decks, which is chosen
randomly. Find the probability of extracting a Ögure in this second extraction.
12. The Chicago Bulls and the Golden State Warriors are playing each other in the
NBA (National Basketball Association) Önal playo§s. In this set of games, the
Örst team to have won four games is declared to be the (world) champion.
(a) Assume that the Bulls are slightly better than the Warriors, such that
P fBulls beat the Warriors in the Örst gameg = 0:6
前 Pr 1
R
B -
-
nk
k
0
0
幽
, 。
⑩ ⼀
, α
「⼀
n
and that the outcome of a game a§ects slightly the probability of the outcome
⼯ on the following game so that
⼀⼀
n
P fBulls win a game if they have won the previous gameg = 0:7
i but
P fBulls win a game if they have lost the previous gameg = 0:5:
先
(i) What is the probability that the Warriors will win a game if they have lost
the previous game?
(ii) What is the probability that the Bulls will win the playo§s in only four
games?
(iii) What is the probability that the Warriors will win the playo§s in only four
games?
(iv) What is the probability that the Warriors have won the Örst game if we
know that they have lost the second game?
(b) If the two teams were evenly matched, that is,
P fBulls beat the Warriors in the Örst gameg = 0:5
and the probability of winning a game did not depend upon the outcome of the
previous game, would it be more likely or less likely that the series would end
in just four games than under the conditions speciÖed in (a)?
13. In a group of 20 students of the IDEA program, 12 of them have passed the
Micro exam only, 6 have passed the Macro exam only, and the remaining two
students have passed both the Micro and the Macro exams. We pick randomly
three students from this group.
(a) Compute the probability that all three students have passed the same exam.
(b) If we know that the three students we have picked have passed the same
exam, what is the probability that the three have passed the Micro exam.
⼝14. Consider the events A; B; and C on the probability space (0; F; P ) : Assume
that P (A) = 1=2; P (B) = 1=2; P (C) = 1=2; P (A \ B) = 1=4; P (A \ C) = 1=4;
P (B \ C) = 1=4; and P (A \ B \ C) = 1=4:
(a) Show that A and B are independent, A and C are independent, and B and C
are independent, but A; B, and C are not independent. Note: this is an example
of three events that are pairwise independent without being independent.
(b) Prove that P (A [ B [ C) = 1:
(c) Find P ((A \ B) \ C c ) ; where C c is the complement of the event C: Note
that the event (A \ B) \ C c contains all the sample points belonging to both A
and B but not belonging to C: That is, (A \ B) \ C c = (A \ B) nC:
Consider now the events D; E; and F on the probability space (0; F; P ) :
Assume that P (D) = 0:6; P (E) = 0:8; P (F ) = 0:5; P (D \ E) = 0:48;
P (D \ F ) = 0:3; P (E \ F ) = 0:38; and P (D \ E \ F ) = 0:24:
(d) Are D and E are independent? Are D and F independent? Are E and
F independent? Are D; E, and F independent? Show that P (D \ E \ F ) =
P (D) * P (E) * P (F ): Note: this is an example of three events whose probability
of their intersection is equal to the product of their respective probabilities.
However, they are not independent since they are not all pairwise independent.
(e) Find P ((D \ E) \ F c ) ; where F c is the complement of the event F: Note
that (D \ E) \ F c = (D \ E) nF:
(f ) Find P ((D [ E [ F )c ), where (D [ E [ F )c is the complement of the event
D [ E [ F:
⇔
15. Prove the following generalization of the theorem of total probability:
Theorem of total conditional probability. If A and C are events of
the measurable space (0; F) and fB1 ; B2 ; :::g is a discrete collection of events
that constitutes a partition of the sample space 0, with P (Bi \ C) > 0 for
i = 1; 2; :::, then
X
P (A jC ) = P (Bi jC ) * P (A jBi \ C ) for all A 2 F and C 2 F.
i
16. Consider a group of n individuals and a task that has to be done. Nobody wants
to do this task. The "drawing straws" method is used to select the individual
who is going to do it. Thus, the group leader takes n straws and ensures that
one of them is shorter than the others. The leader then grabs all of the straws
in her Öst, such that all of them appear to be of the same length. The group
leader o§ers the clenched Öst to the group. Each member of the group draws
sequentially a straw from the Öst of the group leader. When an individual draws
-
-0
the shortest straw he must perform the task and the game Önishes. Obviously,
。
if nobody has picked the shortest straw, then this straw is the one left in the
leaderís Öst and, then, the leader has to do the task. Does the order in which
individuals pick the straw a§ect their probability of picking the shortest straw?
Hint: Use the theorem of Exercise 15.
恐
⼼ "
1
⽇

StatIdea Slides 1

Uploaded by

Copyright:

Available Formats

StatIdea Slides 1

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

StatIdea Slides 1

Uploaded by

Copyright:

Available Formats

1.

J. CaballÈ (UAB - MOVE - BSE) Probability and Statistics IDEA 1 / 48

Sample space: The set W of possible outcomes of a random

J. CaballÈ (UAB - MOVE - BSE) Probability and Statistics IDEA 2 / 48

A set W is discrete or countable if there exists an injective (or

one-to-one) function from W to the set of natural numbers N.

Sample spaces can be either discrete (or countable) or continuous.

Discrete sample spaces can be either Önite or (countable) inÖnite.

J. CaballÈ (UAB - MOVE - BSE) Probability and Statistics IDEA 3 / 48

Fundamental theorem of combinatorics. Consider an experiment

How may sleps ,

whole operation can be made in ’ nj = n1 !n2 !...!nK di§erent ways.

Sometimes the experiment consists of selecting permutations (or ⾤

orders or arrangements) of n objects.

The number of permutations Pn of n di§erent objects is given by

where n is a natural number (n = 0, 1, 2, ...) and the symbol "!"

J. CaballÈ (UAB - MOVE - BSE) Probability and Statistics IDEA 4 / 48

The number of variations of r objects taken from a set of n di§erent

where n and r are natural numbers with n $ r .

Example: A club has 24 members. Its executive committee is formed

Example: With the letters appearing in the word ëTallahasseeí we

J. CaballÈ (UAB - MOVE - BSE) Probability and Statistics IDEA 6 / 48

number of ways we can select r objects from a set of n di§erent objects

number of subsets of r elements from a set of n elements.

The number of combinations is given by the following combinatorial

which coincides with the formula of the number of permutations with

J. CaballÈ (UAB - MOVE - BSE) Probability and Statistics IDEA 8 / 48

Proof: ' ( ' (

J. CaballÈ (UAB - MOVE - BSE) Probability and Statistics IDEA 9 / 48

Example. Let (x + y + z )6 . Then, the value of the coe¢cient c in

J. CaballÈ (UAB - MOVE - BSE) Probability and Statistics IDEA 10 / 48

B c denotes the complement of B relative to W, that is,

J. CaballÈ (UAB - MOVE - BSE) Probability and Statistics IDEA 11 / 48

J. CaballÈ (UAB - MOVE - BSE) Probability and Statistics IDEA 12 / 48

Proof. Note that, if fB1 , B2 , ...g is a countable collection of elements

Moreover, part 3 of the deÖnition of s-algebra could be replaced by

⇌ 3í. If fB1 , B2 , ...g is a countable (Önite or inÖnite) collection of

Therefore, a s-algebra is an algebra, but the converse is not true.

The elements of a s-algebra are called measurable sets, B 2 F .

If W is a sample space and we deÖne a s-algebra F on it, the

B is an event if the question "Does w 2 B?" has a deÖnite "yes" or

J. CaballÈ (UAB - MOVE - BSE) Probability and Statistics IDEA 14 / 48

' ( ' ( ' (

Neweon BinomralThere rem .

W = f(1, H ) , (2, H ) , (3, H ) , (4, H ) , (5, H ) , (6, H ) ,

(1, T ) , (2, T ) , (3, T ) , (4, T ) , (5, T ) , (6, T )g .

Examples of events in the power set 2W :

The power set 2W contains 212 = 4096 di§erent events.

J. CaballÈ (UAB - MOVE - BSE) Probability and Statistics IDEA 16 / 48

Example. Let W = f1, 2, 3, 4, 5, 6g and S = ff1g , f1, 3, 4gg . Then,

J. CaballÈ (UAB - MOVE - BSE) Probability and Statistics IDEA 17 / 48

DeÖnition. A partition of the set W is a collection of subsets of W,

J. CaballÈ (UAB - MOVE - BSE) Probability and Statistics IDEA 18 / 48

If F1 / F2 , then we say that the s-algebra F2 is Öner than F1 (or

f∆, Wg is the coarsest s-algebra on W, whereas the power set 2W is

the Önest s-algebra on W.

DeÖnition. The pair (W, F ) , where F is a s-algebra on W is called

J. CaballÈ (UAB - MOVE - BSE) Probability and Statistics IDEA 19 / 48

DeÖnition. A measure µ on the measurable space (W, F ) is a set