0, 
Basics (Set theory, Axioms of Probability, Total Probability, Baye’s Theorem)


  

 

  … 
  , 

|" 
#$%

  


,  & 


  #%

 
 
   ' 
 
  
|
 

   ' 
|
 

 
|
( 

( 


( | 

|
 

   ' 
|
 

 


)  
 

*  ! !
-0 ,, 1 ,
,  . & 
-
 -  ,  . & 
 
  ,!   ,! ,!

23  
4 5  23 ∞  1, 23 ∞  0, 23     ,  &   8
 9 4 5 *   23 *   23   &23 
Distribution and Density Functions (Single Random Variable)
:3  ;
&
1 PQR, :3   S >TB ,  U 0
= B
:3   ?
( @  ( 
B>MN
< :3   1 23   < :3 A&A CDEFFGDH, :3  
>
ON
>= >= √2KL *
1 1  >B N >B N K 4K *
PQR, 23   1  >TB ,  U 0, V   ,   * WDXYPGCZ, :3   * *ON , 23   1  *ON ,  U 0, V   L[ ,   L
S S L 2 2
1  ^GH_`GDY, :3   - - 1>- , V   ,   1  
EHG], :3   ,  5  5 ^, 23   ,  5  5 ^, 1,  U ^
^ ^
S -
CP_`PaWGb H_ _] cPWH_EYYG aWGDYF EHaGY ]GWFa FEbbPFF,
  ,  :3 
R_GFF_H,
  ,  :3   >T , V     S 1 1
,!  1 -> , V   ,   ,
 *
:B |
 =
2B   2B g :B g


|4   



  <

|4  :3  & 23 d4 5 e4 f gh  , Ri] 
:B  >= 1  2B g 1  2B g

:3   :3   8 :3 , 8 2k ., m  2k> 23 

One Function of Random Variable
.  8   '  8*  j :k .  l  ' l
|8  | |8  |

= =
no4  ^mp  no4p  ^nomp
Expectations

V   no4p  < :3 &  ?

4  4( 4(  no4|"p  < :3 |"&  ?
4  4( |"4(  Linearity of Expectation
>= ( >= (
=
qo4p  L3*  no4  no4p* =
p
no8p  < 8:3 &  ?
4  4( 84(   no4 * p  no4p* V  no4  <   :3 &
>= ( >=
r  no4  no4p no4p L* =
`DWs_t,
4 U g 5 bZPuXbZPt,
|4  no4p| U v 5 * w3 x  no yz3 p  < yzB :3 &
g v
>=
Φ}~ €‚ƒ„…~ƒ„ 0  no4  p  V
= =
1
Φ  no |3 p  < |B :3 & :3   < >yzB w3 x&
2K
>= >=

23k ∞, .  23k , ∞=0 = = =

Joint Distribution, Density and other relations(Functions of two random variables, Two functions of two random variables)
23   23k , ∞  :3   < :, .&. :† ‡  < :†ˆ ‡, x&x  < :3k d4‡, x, m‡, xh &x
>= >= >=

 9 4 5 * , . 9 m 5 .*   2* , .*   2 , .*   2* , .   2 , .  :, . 2, .

‰ * 2, . B Š
  < < :, .&&.
‰‰. >= >=
‹HiPR: :, .  :3 :k ., 2, .  23 2k .
4 f , m f .  1  23   2kŠ  2, .
Š‘’= BŠ, no4  ^mp  no4p  ^nomp ” &  j • , • 
  CŽ,  j 2 ‡  < < :, .&&. no4mp “ no4pnomp     – ” & 
Š0>Š‘’> = >BŠ,‘’>=
‰‡, x ‰‡, x
  CŽ, , —  ZŽ,  j 2‡, x  ˜ :3k , . &&. j :‡, x  :3k d4‡, x, m‡, xh š ‰‡ ‰x š  :3k 4‡, x, m‡, x
™ ‰.‡, x ‰.‡, x ‰8, . ‰8, .
‰‡ ‰x ‰ ‰.
š š
‰›, . ‰›, .
‰ ‰.
dx

œPGuHGa: 2 ‡  ž  :, ‡& j : ‡ 

Ÿ ¡ ¢£ 

¡Ÿ
:^‡, ‡ 
¡ 
:‡, ‡  ž 
Ÿ ¤¥B,Š
&. wx , x* 
¡ ¡ ¡ ¤
 w3 x wk x* ,  & 
3k  n¦4  §B dm  §Š h¨ BŠ = = = =
©BŠ  wx , x*   < < :, . yzªB«zNŠ &&. < < :, .&&.  1
 no4mp  no4pnomp LB LŠ >= >= >= >=
:.|: :.|: w3k x , x*   no yzª 3 yzN k p 1
:|.   = :3k , .  * ˜ >yzª B«yzNŠ w3k x , x* &&
:. ž>= :.|:& 4K
˜ yzª B«yzN Š ,
:3k .&&. ™
™
 :3k , . =
nomp  n3 onk om|4pp
Φ , *   no |ª3«|Nk p  ˜ |ªB«|NŠ :3k , .& :k .|4   
n84|"  < 8:|"&
:3  >=
™
n¬no84, m|4p­  no84, mp no8 48* m|p  no8 8* m|p no®4  no4p  m  nomp¯* p U 0 :3k† , ., ‡
:, .|‡ 
:† ‡

°± ²Fa. uX b_HFaDHa, ‰ ‰ 3k

Sequences of Random Variables

& œGHPDW °± ²Fa,  no®m  4  ^¯* p,  min x›  0 &  0 j   * ,^

 nom  * p,  min x› 0 ‰ ‰^ L3
& 3k
j   nomp  rk  r
L3* 3
4  4  '  4 2 , ¸   2 , ∞, ¸ , ∞ = =
º^ " ›&
j 23ª,…,3·  , … ,   : , ¸   < < : , * , ¸ , ¹ &* &¹ ” &  & 
>= >=
! 4, m  &   & ¾ 
. »  ’ » 5 ' . »  ’ » j , &   j :- . 
¼½
2 -> .o1  2B .p>- :B .
,  1!   ,! B 4m j
L†* =L3*  Lk*

PtPWX¿ZPWP DY`_Fa PtPWX¿ZPWP _W ¿GaZ Å0Æ

à ∞ j |4  4| 9 v, : 
Stochastic Convergence
4 ÀÁÁÁÁÁÁÁÂ 4,  f Ä 4 ÀÁÁÁÁÁÁÁÁÁÁÁÁÁÁÁÁÁÁÁÁÁÁÁÁÂ 4,  Ã ∞ j
®|4  4| 9 v¯  1
°± ±PHFP ÅW_uDuGYGaX
4 ÀÁÁÁÁÁ 4,  à ∞ j no|4  4| p à 0,  *
Ã∞ 4 ÀÁÁÁÁÁÁÁ 4,  à ∞ j
®|4  4| f v¯ à 0,  Ã∞
ÇGFaWGuEaG_H
4 ÀÁÁÁÁÁÁÁÁÂ 4,  Ã ∞ j 2  Ã 23 ,  Ã ∞
ÇPHFGaX
4 ÀÁÁÁÁÂ 4,  Ã ∞ j :  Ã :3 ,  Ã ∞ " j
^
1
È4ÈÈÈ  4  '  4 , x›  4(  & x› V  r, ¿PDs YD¿ j4È ÀÁÁÁÁÁÁÁ r, FaW_HC YD¿, j4È à r,  à ∞
ÅW_uDuGYGaX DP

1
bPHaWDY YG`Ga, ¾  4  '  4 , x› V   r,   L * 9 ∞ j ¾ à ¾~Êr, L * 
Stochastic Processes
¤· ¢Žda h…Ža  BÆ ,…,B· ;¼Æ ,…,¼·
HaZ _WiPW , 2ŽaÆ …ŽaH  Æ , … ,  ; Æ , … ,   
®4  5  , … , 4  5  , :ŽaÆ …ŽaH  Æ , … ,  ; Æ , … ,    Æ H
¤Bª …¤B·
2 ;    2 , ∞;  , *  = =
no4 * p  Ì, 
: ;    < : , * ;  , *  &* r3   n4p  < :; &
>= >=
= =
DEa_b_tDW,  , *   Ì , *   r3  r3 * 
DEa_b_WW, Ìxx, *  no44*p  < < *:, *; , *&*
>= >=
 , *  bW_FF b_WW, Ì3k  , *   no4 m* p bW_FF b_tDW,  , * 
b_WW b_P]],  , *  
Í,  *, *   Ì3k  , * 
 r3  rk * 
¿ZGaP H_GFP, ˆˆ  , *   0,  “ * , r  0, Î  1@Î ±±±, :ŽaÆ …ŽaH  Æ , … ,  ; Æ , … ,    :ŽaÆ …ŽaH  Æ , … ,  ; Æ  b, … ,   
Î
—±±, V    , no4 4* p  Ì  *   ÌÎ, no4 * p  Ì0, Î  ÌÎ  |r|* , Î  , ÌÎ  ÌÎ, Ì0 U ÌÎ
0
Ï_GHaYX —±±, no4 4* p  Ì3k   *   Ì3k Î, 3k Î  Ì3k Î  r3 rk

m  , 4    `P`_WXYPFF, m &  &  4 )    V V.    j   
Systems with Stochastic Inputs

=
YGHPDW, m  Ðo4p  4 Ñ ›  < 4  g›g&g, x›  ›  Ðo@p, n¬Ðo4p­  Ьno4p­, 4  m  
>=
=
YGHPDW, 4Ò › mÒ, œÓ‹, 48 m8 , YGHPDW, nomp  no4p Ñ ›  r3 < ›g&g
>=
= =
1 =
YGHPDW, Ì3k  , *   < Ì33  , *  g›g&g , Ìkk ? R_¿PW FRPb, Õ  ÖÌÎ < ÌÎ >y×Ø
&Î j ÌÎ  < Õ y×¼ &Õ
>= >= 2K >=
ÙÚ Û ±Ü, @Î Û 1,1 Û 2K@Õ, yÝØ Û 2K@Õ  Þ

1  B 1 1 >B«Š  «
Integration (all integrals are w.r.t x)
< ß B  <   || < ß B«Š  <   < sin  cos  < cos   
ß  ß 1
1 ß B«Ÿ 1 1
< tan  ln || < ß B«Ÿ  ß <    > « <  l    < A
ß    1

& &  l  A &. &. & &  & ß B &   
Differentiation
 
   l  A æ ç      >    ß ß B
æ Bç   B
& &  * & & & & & & *
& &
sin   cos  cos    
& &

 ^ 1  *
è è  &  ^
Matrix Operations

 &
>
 é ê ,     :   ë
|
| * ** (y
Prepared by: Hammad Munawar (Institute of Avionics and Aeronautics, Islamabad, Pakistan) hammadmunawar@gmail.com

(COPYRIGHT STATEMENT: May be used / distributed / edited freely, as long as the name of the original author is included)
 Table of Integrals∗
Basic Forms Integrals with Logarithms
√ √
Z
2
Z x ax + bdx = (−2b2 + abx + 3a2 x2 ) ax + b (26)
1
Z
x dx = n
xn+1 (1) 15a2
ln axdx = x ln ax − x (42)
n+1
Z
1
Z
ln ax 1
dx = ln |x| (2) Z p
1 dx = (ln ax)2 (43)
x
h
x(ax + b)dx =
p
(2ax + b) ax(ax + b) x 2
Z Z 4a3/2
 √
udv = uv − vdu (3) p i
−b2 ln a x + a(ax + b)
Z  
 
(27) b
ln(ax + b)dx = x + ln(ax + b) − x, a 6= 0 (44)
Z a
1 1
dx = ln |ax + b| (4)
ax + b a Z
x
b2 ln(x2 + a2 ) dx = x ln(x2 + a2 ) + 2a tan−1
Z p  
Integrals of Rational Functions b x p 3 − 2x (45)
x3 (ax + b)dx = − 2 + x (ax + b) a
12a 8a x 3
b3
 √ p 
+ 5/2 ln a x + a(ax + b) (28)
Z  
1 1
Z
8a x+a
dx = − (5) ln(x2 − a2 ) dx = x ln(x2 − a2 ) + a ln − 2x (46)
(x + a)2 x+a x−a
(x + a)n+1
Z
(x + a)n dx = , n 6= −1 (6) Z
1p 2ax + b
Z p
n+1 1 p 1
 
ln ax2 + bx + c dx = 4ac − b2 tan−1 √
p
= x x2 ± a2 ± a2 ln x + x2 ± a2 


x2 ± a2 dx
 
2 2 a 4ac − b2
Z
(x + a)n+1 ((n + 1)x − a) (29) 
b

x(x + a)n dx = + x ln ax2 + bx + c


(7) − 2x + (47)
(n + 1)(n + 2) 2a
Z
1 Z p
1 p 2 1 x
dx = tan−1 x (8) a2 − x2 dx = x a − x2 + a2 tan−1 √
Z
bx 1
1 + x2 2 2 a2 − x2 x ln(ax + b)dx = − x2
2a 4
Z
1 1 x (30)
dx = tan−1 b2
 
(9) 1
a2 + x2 a a + x2 − 2 ln(ax + b) (48)
Z 2 a
Z
x 1
p 1 2
3/2
dx = ln |a2 + x2 | (10) x x2 ± a2 dx = x ± a2 (31)
a2 + x2 2 3 Z
1
x ln a2 − b2 x2 dx = − x2 +


Z 2
x x Z 2
dx = x − a tan−1 1
 
(11)
p
√ dx = ln x + x2 ± a2  (32)
 
a2 + x2 a x2 ± a2 1

a2

x − 2 ln a2 − b2 x2
2

(49)
Z
x3 1 1 2 b
dx = x2 − a2 ln |a2 + x2 | (12) Z
1 x
a + x2
2 2 2 √ dx = sin−1 (33)
a2 − x2 a Integrals with Exponentials
Z
1 2 2ax + b
dx = √ tan−1 √ (13) Z
x
ax2 + bx + c 4ac − b 2 4ac − b2
p Z
√ dx = x2 ± a2 (34) 1 ax
x ±a
2 2 eax dx = e (50)
a
Z
1 1 a+x
, a 6= b √
Z
dx = ln (14) x p
√ 1 √ ax √

Z
(x + a)(x + b) b−a b+x √ dx = − a2 − x2 (35) i π
a −x
2 2 xeax dx = xe + 3/2 erf i ax ,
Z a 2a
x a 2
Z x
dx = + ln |a + x| (15) where erf(x) = √
2
e−t dt (51)
(x + a)2 a+x π 0
x2
Z
1 p 1
 p 
√ dx = x x2 ± a2 ∓ a2 ln x + x2 ± a2 
 
Z
x 1 x ±a
2 2 2 2 Z
dx = ln |ax2 + bx + c| (36) xex dx = (x − 1)ex (52)
ax2 + bx + c 2a
b 2ax + b
tan−1 √
 
− √
Z
(16) x 1
a 4ac − b2 4ac − b2 xeax dx = − 2 eax (53)
Z p
b + 2ax p 2 a a
Integrals with Roots ax2 + bx + cdx = ax + bx + c
4a
Z
x2 ex dx = x2 − 2x + 2 ex


2
(54)
4ac − b
 p 
+ ln

+ b + 2 a(ax2 + bx+ c) (37)
√
Z
2 8a3/2
2ax 
x − adx = (x − a)3/2 (17) x2
Z  
2x 2
3 x2 eax dx = − 2 + 3 eax (55)
a a a
√
Z
1
√ dx = 2 x ± a (18)
Z
1  √ p 2 x3 ex dx = x3 − 3x2 + 6x − 6 ex
Z

x±a p (56)
x ax2 + bx + c = 2 a ax + bx + c
Z
1 √ 48a5/2
√ dx = −2 a − x (19) xn eax
Z Z
× −3b2 + 2abx + 8a(c + ax2 ) n


a−x xn eax dx = − xn−1 eax dx (57)
 √ p  a a
+3(b3 − 4abc) ln b + 2ax + 2 a ax2 + bx + c (38)
 
√
Z
2 2
x x − adx = a(x − a)3/2 + (x − a)5/2 (20) Z
(−1)n
3 5 xn eax dx = Γ[1 + n, −ax],
an+1
Z
√
 
√ Z ∞ (58)
2b 2x Z
1 1 ta−1 e−t dt
 
ax + bdx = + ax + b (21) where Γ(a, x) =
p
√ dx = √ ln 2ax + b + 2 a(ax2 + bx + c)
 
3a 3 ax2 + bx + c a x
Z
2 (39) √
√

Z
(ax + b)3/2 dx = (ax + b)5/2 (22) 2 i π
eax dx = − √ erf ix a (59)
5a 2 a
√ √
Z
x 2 Z
√

√ dx = (x ∓ 2a) x ± a (23) π
Z 2
x 1p 2 e−ax dx = √ erf x a (60)
x±a 3 √ dx = ax + bx + c
ax2 + bx + c a 2 a
b
  Z
p p 2 1 2
x(a − x) − 3/2 ln 2ax + b + 2 a(ax2 + bx + c) xe−ax dx = − e−ax
Z r
(40)
 
x (61)
dx = − x(a − x) − a tan−1
p
(24) 2a 2a
a−x x−a

√
Z Z r
dx x 2 −ax2 1 π x −ax2
√ = √ (41) x e dx = erf(x a) − e (62)
Z r
x p √ 
(a2 + x2 )3/2 4 a3 2a
dx = x(a + x) − a ln x + x + a (25) a2 a2 + x2
a+x
∗
« 2014. From http://integral-table.com, last revised June 14, 2014. This material is provided as is without warranty or representation about the accuracy, correctness or
suitability of the material for any purpose, and is licensed under the Creative Commons Attribution-Noncommercial-ShareAlike 3.0 United States License. To view a copy of this
license, visit http://creativecommons.org/licenses/by-nc-sa/3.0/ or send a letter to Creative Commons, 171 Second Street, Suite 300, San Francisco, California, 94105, USA.
 www.mathportal.org

Math Formulas: Definite integrals of

exponential functions
Z ∞
a
1. e−ax cos bx dx =
0 a2 + b2
Z ∞
b
2. e−ax sin bx dx =
0 a2 + b2
∞
e−ax sin bx
Z
b
3. dx = arctan
0 x a
∞
e−ax − e−bx
Z
b
4. dx = ln dy
0 x a
Z ∞ r
−ax2 1 π
5. e dx =
0 2 a
Z ∞ r
−ax2 1 π − b2
6. e cos bx dx = e 4a
0 2 a
Z ∞ r
−(ax2 +bx+c) π b2 −4ac
7. e dx = e 4a
−∞ 2
Z ∞
Γ(n + 1)
8. xn e−ax dx =
0 an+1
Z ∞
Γ m+1


m −ax2 2
9. x e dx = (m+1)/2
0 2a
Z ∞
1 π −2√ab
r
−(ax2 +b/x2 )
10. e dx = e
0 2 a
Z ∞
x dx π2
11. =
0 ex − 1 6
Z ∞ n−1  
x 1 1 1
12. dx = Γ(n) + n + n + ···
0 ex − 1 1n 2 3
Z ∞
x dx π2
13. x
=
0 e +1 12
Z ∞ n−1  
x 1 1 1
14. dx = Γ(n) − n + n − ···
0 ex + 1 1n 2 3
Z ∞
sin mx 1 m 1
=
DEa_b_WW, Ìxx ,    no 4 4 p  <  {t1,A}{t2,A}.fa(a)&a
15. 2πx − 1
dx = coth −
e 4 2 2m
 *  *
0
Z ∞ 
1 −x dx
16. −e =γ
0 1+x x
Z ∞ −x2
e − e−x 1
17. dx = γ
0 x 2
Z ∞ −x

1 e
18. x
− dx = γ
0 e −1 x
Z ∞ −ax
− e−bx
 2
b + p2

e 1
19. dx = ln
0 x sec(px) 2 a2 + p2
Z ∞ −ax
e − e−bx b a
20. dx = arctan − arctan
0 x csc(px) p p
∞
e−ax (1 − cos x)
Z
a
21. 2
dx = arccot a − ln(a2 + 1)
0 x 2

1
Table of Distributions

Distribution PMF/PDF and Support Expected Value Variance MGF

Bernoulli P (X = 1) = p
Bern(p) P (X = 0) = q = 1 − p p pq q + pet

n
k n−k
Binomial P (X = k) = k
p q
Bin(n, p) k ∈ {0, 1, 2, . . . n} np npq (q + pet )n

Geometric P (X = k) = q k p
p
Geom(p) k ∈ {0, 1, 2, . . . } q/p q/p2 1−qet
, qet < 1

r+n−1
r n
Negative Binomial P (X = n) = r−1
p q
p
NBin(r, p) n ∈ {0, 1, 2, . . . } rq/p rq/p2 ( 1−qe r t
t ) , qe < 1

  
w+b
 
P (X = k) = w b /
Hypergeometric k n−k n  
nw w+b−n µ µ
HGeom(w, b, n) k ∈ {0, 1, 2, . . . , n} µ= b+w w+b−1
nn (1 − n
) messy

e−λ λk
Poisson P (X = k) = k!
t
Pois(λ) k ∈ {0, 1, 2, . . . } λ λ eλ(e −1)

1
Uniform f (x) = b−a
a+b (b−a)2 etb −eta
Unif(a, b) x ∈ (a, b) 2 12 t(b−a)

2 2
f (x) = √1 e−(x − µ) /(2σ )
Normal σ 2π
σ 2 t2
N (µ, σ 2 ) x ∈ (−∞, ∞) µ σ2 etµ+ 2

Exponential f (x) = λe−λx

1 1 λ
Expo(λ) x ∈ (0, ∞) λ λ2 λ−t
, t<λ

1
f (x) = Γ(a)
(λx)a e−λx x1
Gamma  a
a a λ
Gamma(a, λ) x ∈ (0, ∞) λ λ2 λ−t
,t<λ

Γ(a+b) a−1
f (x) = Γ(a)Γ(b)
x (1 − x)b−1
Beta
a µ(1−µ)
Beta(a, b) x ∈ (0, 1) µ= a+b (a+b+1)
messy

2
1
√ e−(log x−µ) /(2σ 2 )
Log-Normal xσ 2π
2 2
LN (µ, σ 2 ) x ∈ (0, ∞) θ = eµ+σ /2 θ2 (eσ − 1) doesn’t exist

1
Chi-Square xn/2−1 e−x/2
2n/2 Γ(n/2)
χ2n x ∈ (0, ∞) n 2n (1 − 2t)−n/2 , t < 1/2

Γ((n+1)/2)
√
nπΓ(n/2)
(1 + x2 /n)−(n+1)/2
Student-t
n
tn x ∈ (−∞, ∞) 0 if n > 1 n−2
if n > 2 doesn’t exist
 Table of Laplace Transforms
f ( t ) = L -1 {F ( s )} F ( s ) = L { f ( t )} f ( t ) = L -1 {F ( s )} F ( s ) = L { f ( t )}
1 1
1. 1 2. e at
s s-a
n! G ( p + 1)
3. t n , n = 1, 2,3,K 4. t p , p > -1
s n +1 s p +1
p 1 × 3 × 5L ( 2n - 1) p
, n = 1, 2,3,K
n - 12
5. t 3 6. t n+ 1
2s 2
2n s 2
a s
7. sin ( at ) 8. cos ( at )
s + a2
2
s + a2
2

2as s2 - a2
9. t sin ( at ) 10. t cos ( at )
(s + a2 ) (s + a2 )
2 2 2
2

2a 3 2as 2
11. sin ( at ) - at cos ( at ) 12. sin ( at ) + at cos ( at )
(s + a )
2 2 2
(s + a )
2 2 2

s(s - a ) 2 2
s ( s + 3a )
2 2

13. cos ( at ) - at sin ( at ) 14. cos ( at ) + at sin ( at )

(s + a )
2 2 2
(s + a )
2 2 2

s sin ( b ) + a cos ( b ) s cos ( b ) - a sin ( b )

15. sin ( at + b ) 16. cos ( at + b )
s2 + a2 s2 + a2
a s
17. sinh ( at ) 18. cosh ( at )
s - a2
2
s - a2
2

b s-a
19. e at sin ( bt ) 20. e at cos ( bt )
(s - a) (s - a)
2 2
+ b2 + b2
b s-a
21. e at sinh ( bt ) 22. e at cosh ( bt )
(s - a) (s - a)
2 2
-b 2
- b2
n! 1 æsö
23. t ne at , n = 1, 2,3,K 24. f ( ct ) Fç ÷
(s - a)
n +1
c ècø
uc ( t ) = u ( t - c ) e - cs d (t - c )
25. 26. e - cs
Heaviside Function s Dirac Delta Function
27. uc ( t ) f ( t - c ) e F (s)
- cs
28. uc ( t ) g ( t ) e - cs L { g ( t + c )}
ect f ( t ) F ( s - c) t n f ( t ) , n = 1, 2,3,K ( -1) F ( n) ( s )
n
29. 30.
1 ¥ t F (s)
31. f (t ) ò F ( u ) du 32. ò f ( v ) dv
t s 0
s
T
t
ò e - st f ( t ) dt
33. ò f ( t - t ) g (t ) dt F (s)G (s) 34. f (t + T ) = f (t ) 0
0
1 - e - sT
35. f ¢ (t ) sF ( s ) - f ( 0 ) 36. f ¢¢ ( t ) s 2 F ( s ) - sf ( 0 ) - f ¢ ( 0 )
37. f ( n) ( t ) s n F ( s ) - s n-1 f ( 0 ) - s n- 2 f ¢ ( 0 )L - sf ( n- 2) ( 0 ) - f ( n-1) ( 0 )
8 Probability and Moment Generating Functions Conditional mean and variance
σX
E [X | Y ] = E [X] + ρ (Y − E [Y ])
 
• GX (t) = E tX |t| < 1
"∞ σY
∞
#  
X (Xt)i X E Xi
· ti
t
 Xt

• MX (t) = GX (e ) = E e =E =
p
i! i! V [X | Y ] = σX 1 − ρ2
i=0 i=0
• P [X = 0] = GX (0)
• P [X = 1] = G0X (0) 9.3 Multivariate Normal
(i)
GX (0) Covariance Matrix Σ (Precision Matrix Σ−1 )
• P [X = i] =
i!  
V [X1 ] · · · Cov [X1 , Xk ]
• E [X] = G0X (1− )
Σ=
 .. .. .. 
  (k)
• E X k = MX (0) . . . 

X!
 Cov [Xk , X1 ] · · · V [Xk ]
(k)
• E = GX (1− )
(X − k)! If X ∼ N (µ, Σ),
2
• V [X] = G00X (1− ) + G0X (1− ) − (G0X (1− ))  
−1/2 1
• GX (t) = GY (t) =⇒ X = Y
d
fX (x) = (2π)−n/2 |Σ| exp − (x − µ)T Σ−1 (x − µ)
2
Properties
9 Multivariate Distributions
• Z ∼ N (0, 1) ∧ X = µ + Σ1/2 Z =⇒ X ∼ N (µ, Σ)
9.1 Standard Bivariate Normal • X ∼ N (µ, Σ) =⇒ Σ−1/2 (X − µ) ∼ N (0, 1)


p • X ∼ N (µ, Σ) =⇒ AX ∼ N Aµ, AΣAT
Let X, Y ∼ N (0, 1) ∧ X ⊥
⊥ Z with Y = ρX + 1 − ρ2 Z

• X ∼ N (µ, Σ) ∧ a is vector of length k =⇒ aT X ∼ N aT µ, aT Σa
Joint density  2
x + y 2 − 2ρxy

1
f (x, y) = exp − 10 Convergence
2(1 − ρ2 )
p
2π 1 − ρ2
Conditionals Let {X1 , X2 , . . .} be a sequence of rv’s and let X be another rv. Let Fn denote
the cdf of Xn and let F denote the cdf of X.
(Y | X = x) ∼ N ρx, 1 − ρ2 (X | Y = y) ∼ N ρy, 1 − ρ2



and
Types of Convergence
Independence D
X⊥
⊥ Y ⇐⇒ ρ = 0 1. In distribution (weakly, in law): Xn → X

lim Fn (t) = F (t) ∀t where F continuous

n→∞
9.2 Bivariate Normal


P
Let X ∼ N µx , σx2 and Y ∼ N µy , σy2 . 2. In probability: Xn → X

1

z
 (∀ε > 0) lim P [|Xn − X| > ε] = 0
n→∞
f (x, y) = exp −
2(1 − ρ2 )
p
2πσx σy 1−ρ 2
as
3. Almost surely (strongly): Xn → X
" 2  2   #
x − µx y − µy x − µx y − µy h i h i
z= + − 2ρ P lim Xn = X = P ω ∈ Ω : lim Xn (ω) = X(ω) = 1
σx σy σx σy n→∞ n→∞
9
 Fourier Transform Table

x(t ) X (f ) X (ω )
δ (t ) 1 1
1 δ (f ) 2πδ (ω )
δ (t − t0 ) e − j 2π ft0 e − jω t0
e j 2π f0t δ (f − f0 ) 2πδ (ω − ω0 )
cos(2π f0t ) 1 π δ (ω − ω0 ) + δ (ω + ω0 )
δ (f − f0 ) + δ (f + f0 )
2
sin(2π f0t ) 1 − jπ δ (ω − ω0 ) − δ (ω + ω0 )
δ (f − f0 ) − δ (f + f0 )
2j 
rect (t ) sin c(f )  ω 
sin c  
 2π 
sin c(t ) rect (f )  ω 
rect  
 2π 
Λ(t ) sin c 2 (f )  ω 
sin c 2  
 2π 
sin c 2 (t ) Λ (f )  ω 
Λ 
 2π 
e −α t u(t ), α > 0 1 1
α + j 2π f α + jω
te −α t u(t ), α > 0 1 1
2
(α + j 2π f ) (α + jω )2
e −α |t |, α > 0 2α 2α
2 2
(α + (2π f ) (α + (ω )2
2

e −π t e −α f e −α f
2 2 2

sgn(t ) 1 2
jπ f jω
u(t ) 1 1 1
δ (f ) + πδ (ω ) +
2 j 2π f jω
d j 2π f jω
δ (t )
dt
∞ ∞ ∞
1  n  1  2π n 
∑ δ (t − nT0 )
T0 ∑ δ f −


T0  T0 ∑ δ  ω − T0 

n =−∞ n =−∞ n =−∞