APA Chapter2 T21
APA Chapter2 T21
y = 0 + 1 x
y1 x1 1 x1
= + = 0 y = X
0 1
yN xN 1 xN 1
y = 0 + 1 x + 2 x 2 + +p xp
0
y1 1 x1 x
p
1
1
= y = X
y N 1 xN xNp
p
d=2
d=2
x2
x1
yi = h (xi ) i = 1,..., N
ˆ = arg min L ( y1 ,..., yN , x1 ,..., x N ; )
1
L ( y1 ,..., y N , x1 ,..., x N ; ) = ( yi − h (xi ) )
2
N i =1
N
( y − x )
1
L ( y1 ,..., yN , x1 ,..., xN ; ) = T 2
i i
N i =1
1
= ( y − X ) ( y − X )
T
( y − X ) ( y − X ) = ( y T y − y T X − T XT y + T XT X )
1 1
L =
T
N N
2 T 2 T
= − X y + X X =
N N
ˆ = ( XT X )−1 XT y
Pseudoinverse of X
k +1 = k − h L =
k
.
h
h h
L
2
1
N N
( y − h (x )) h (x )
1 2
L = ( yi − h ( xi ) ) =−
2
i i i
N i =1
N i =1
L −2 ( yi − h ( xi ) ) h ( xi )
test test
Learning
Model
algorithm
a
N p
( yi − h ( xi ) )
1
L ( y1 ,..., yN , x1 ,..., xN ; ) = + i2
2
N i =1 2 i =1
h ( xi ) = xTi
ˆ = ( XT X + a I )−1 XT y
a
N p
( y − h (x ))
1
L ( y1 ,..., yN , x1 ,..., xN ; ) = +a
2
i i i
N i =1 i =1
Generates a 3rd order model
2 2
1 1
(1 − r )
N p p
( yi − h ( xi ) )
1
L ( y1 ,..., yN , x1 ,..., xN ; ) = + ra i + a i2
2
N i =1 i =1 2 i =1
a
N
yi − h ( xi )
1
M=
N i =1
yi
2
R 2 = 1 − 2 = y − h ( x)
y
2
R = 1− 2
2
= y − h ( x)
y
ŷ = y ŷyˆ = h ( x)
y y
N
( y − h ( x) )
N
( y − y )
1 1
=
2
y2 =
2 2
i i
N i =1
N i =1
cov ( y, yˆ )
r r= yˆ = h ( x)
y yˆ
ŷ
y and yˆ , R 2 = r 2 = 1
MIC = I ( X ; Y ) = H ( X ) − H ( X | Y )
f ( x, y )
= f ( x, y ) log
f ( x) f ( y)
dxdy
Quiz