Uni~ .

1
'~r·------.!O~~~v~,°e~"'~'~o~f:..!.;M~O~C:!:'h~ine~ ,(::e~o:.::w,,:.:':.:i._,.,......,,_,.,,.._--"".'.~.-"".": ~-; ,'.'T'
i :-:-
. ,7-",
. . r. -:::::-::-:-;-;~

~· t ~. . I• •
r,,.,, (_Ji.. ,. ' ~ : :~ _ :,

·<.'•' I'' .~t-·1

, .
,, .) 10 ·:.11. .1,.\,;
1••, •• •.,.
•,. ,'
if,0 1t), • 1- , I

,,rl!• •' ,,
.~ Lv 2 , 19 . ,i .l n ,., ,f..l 1 iliH

f• t, 1 "j I l I ~ 1; " • i' , • • ...,

' '' J ,.,' ·._ l , •• I .,! ' •
- I
'
I
I

: \. I

'( .
t '-"'o ' . ,..1o,i . .. , ,
Usina otcao-a i ~m.s ~~l,o·ct.''~do~-;(f,c~ ·, cblo.. .
nelwo~ks aie one. tt'/ .Y~ . ~f -~?chine ole.o~n1n9.
A -fundarnenlol neu(QI t\el~l~ - is\ a Nod~.
·\ r · ...-\''' -\ ~\ 1 ~
,...,( .
\' '
.\ \ ( ,\

,; \ \ . \" t \ . . .
~. . ,. t. ' ·, r "· COO
\
To ,Qe'.fine·
·, .
' ~ W ', N"lClchine.s . cl:io
, ·. ' ' ,_.
lea-an . tohQt
\ -·i I...
we.
-~ti ·c::1efin1n~.\ .:., (,- · , '·' · 1, · ' ·,"

,l,~ Bu ey~eij~-~~~,-:-- \•,· ,, ;, ,,' \'

o~
J --r, : ,• 1 ~_1\
W~ , ~n ·~h data exar<lp1e:s #i~ sbuclu,e dolo. ·,l. eornpule~ leoilns
Some~i9 ihe '}JlW dokl.,
.9JlJdoie:+ ~det~ ,id~Cl_q,arns o1ineo1, te91essi'oo_ , ,, , J
__ ,.. ,; ' . oe.c,eion +re.e
~.9 -tt,e' tie1f' ,'~r ~~lhh'l ~' rnachi~ '" eon e1e0,
11 y'
-,. each mocfeJ..,' has 'CI-~{ d.~renl tSule.s ., ~ 1 predic.l dab,
pa,an, e le. v .:'._,. hidde ,- -· , ·.
.,/
~e ,
-,. l1eu2n1 :ri~t ~-. k · "ha\e. o 1e _ .. n ;' ou~Cned pa~s
,-\.

Al/flot Ros~~ · .
I ,

\
' " ' I
• I
 'f , ' . .i
t,Jhy deep lieo'&nin9
,, ' • J. '

I ,• : 'J i I ], I( , 1 / ; . f i
1
1 , O/ j ·-' 'I;I L 1 ·• f ,..., ' J 1,~' '- I
:, ' ,
· I l ti j, .I ', I 1; I / , '.
I I , ' I, ' , ' f' ' ' '' ·I ,J
. , ,, I J
I
, ,1, J '' . f I'' 1' ' ' , , 1,i ,11.fi,'., f 1,, , 1 ,, , 11,, ,:.' 1 • • ... i/· t 1 1,,p 1 ;_-,:c.

· ·II , ll , I I' ' / j , ,, · t· ':, I,'! I1 ,, I I f, I i11 I t j,j. d , )f d f 1 1 ! l' ·; J \/. - .1

p
, ., , ' ' 1· I ,, i 1, ,, ,
.I . -· ,·' ,-,:or,·,i,,,
i:;;en ,.
q ,.., • ."" ,.I n I f, II f' !]
;
h 'J ./ ':
C ..-· ,·I. ' , II l Il '' ,,:, " I I' I I . J I ·_,, I I
') I

,,
(')

(1 1
'11 , 1''
',.-,,
i: ,
(I

(
('
,,
I 1f ;

Arrounl of dolCl. le: ifl~ /:

\;1i•.;jJ 1; ' I_) ,_,: I
l'~ododion:• -, , "d __,·);,;, l ::,1, ,, , H J j .::,::·
I · S l ' ) 1l ui])_i\ , 11 I JV:.J) •(lf"),Jl{}, l ~·) J; L)~I 1u 1{II _, , ' .•

:A I :-.simu~olion of huma.n inlelh~nce in machines ihot a,e pfC910mei:l~•,

:-lo -!hink II l<e.. hum o ns and -the.ill oc,lion s , , oi ..J 1 _, " i· 1cc , ., " , 11 ,_>'.'
1
M irnic
L :.- ~e, 1tfie.td:·.df..nswd9 dfh'Q~•1·[il'~e§)
\. iJ
ab'ililu -to l_
I ' .s .:i A
e oin . ,.
.• - i1 J,:-
'c.~'r/rtpu!fiis') '~ -
\\:houl be1n9 eix:.ptid~:~Pl!>'~•~mmedl.i /ujo ..:x.1.i oq uJc,, ,.., - "' ,_;, I ;c,tJ ·

~
L:- broode<f fom'il~ Of m~chine «ealSn'irg me.lha:S based Or) 06~ificiot
_ r
l . eo'lSOI nelwoisks u,\\:h Jepfesenlolico Le.~-2Sn_i~ q ._, . _, , ,,1
1

980 de~nilion:- , , ., 1 .) ., . I\1. l

1 i ,

L :-
nelwmsk rnc~e -lho.n -tw:::i Loye~s.
neu~o\
,. -- ,
.; t. • aH cl l· r- ,,,. Ls:J JC·l
...
U.J. __ -

change.s/evoluHon oF neu~o\ neh.uoc11<s:-

1> m0Je neuions :4hon\ ,preV(OU6'.,. ir>1•by.>p~,~ - ,, ,.,_ ,,,·J Ii,,;/ ,.J L ·_ ,,11 , ·i Ar:
.:)
7
H-no,.e complex, wo~s of connecting oeayers t~r)~eu~cns in : ne.urat oeh.1.JQ'r~

3> e.1plosin9 in ..\he amount Of C:.Ornpulln9 P?O."~?j -~__':\Oila~_teL\_~ \a'oin .

, 1 ;1), 1 j r- ~-
+>o.ulom<lUc fe.Cllu1e e~poosron - , - . ~;.,\! I ,-_1
rid
• de~:- i'..J i1, •. ,1'>l ,,11 .. 111 ,: _; { LJ, i,.~,i- "- :.i)- ,..d ,i- N 1 1. J-
'

Neu~at new.>0'6kS toilh · «atge num~~ of ~av~ e~~e 1 hi. ola~~ 'i,r')
1
One. oF fou'O' fordamenlot neu.o~k o~chtleclu-we~- ·
• uns ¥~i~ r> p,eb<tl,ned 1neh.uoiks, t. 1ili J 11 •ll(,'·J·J t \..J J_; ,_ •t_,. _, s' JI 11

• C!\nVO\ollonal •"· ,f:1~U<1P,1) .,n~t~o?S)~S,:':t: - "'' ii· , , , ,;. ; , .

h')b,H~d
'i ' '. .: d .

• Re.cU'<l'.5\Ve n e u~al rie lt.U)iks \ ,:, , I

1 o?Jch,tec.lure -1
I.I
' ·Rec:vjtenf _ r;eu,al \ n el h.ua,~ks ,_ , 1
• ' ' -. I -
I ~- I - t

'·
••
I
• l • \
I \ •• - •
\ -·-.:; 1- . l ,~·;
L - · •I
·, ~} .: J f j ,.) t(
,~
'

RelevQnl rnathemalico.t
1
I

~Cbt p1'Cduct (.Seatoi plSCduc.t, /inne~ p~cl)~-

ffiID ~. :, '·t·· I
J ., I · 1I '
11 •
fL. ' , . ' !:_, JI l ! l
X

I Gilli] [4 I'" I •~ 1. • / !
4 •
,,, I ' '
,o m, • a,.
:, ·\' ' I ,, ' '
" .... , ' , , ,,
I - , ... 1. ·

1:rl is
I
Used -lo -!ind Cosine slmilo,if~ · bfu, ,,;i,ou,s/.crotl,nns •'"' -'· '
' •. ~ ' Ele Me hltc 1'se p•oduct ( HadeTo 'o::I I piociJ,.1) :- :, '., '.C i ' ': ! ' ,.• ·.,

J, ,.
'1 ·

'" l ,'I , 'J l

1· ..
qi', , i It
·j'
,I 1,)/ I 1
/ •'

p "<>cluc[ al .1 Column w ill, 1..>ou>.,.'tr, , <l ; tral•i:i l!lr,,d; , 9"lll"':l . ,1 · ,

Vee lo• is added as ihe •o..; Qi. · -lhe: 'eril af •-fhe.'•rinolllia : :,, . · •
k> .:).() ao
• ·, ' 1' I ' f II ., I • I • • ' t
I
y • ' , j

.:>.

·fillJJJ--t ·Go
3
4 5 c; S '~ 1
I 4 )
l'I
Con"e" ~" --- ,, •~ ...... ~ --. : • ,. l •

cp . ,Ol~p~\(J-.m'3 , ~nn\ , ·I~ ~io~o ~~P' <'r-it,'ll'-'d ,, o~ , a , tnau,j-,.;··

' ,. ... fnou,\-,.)
SPo"\i,~ ' t . . 1 , , •
( j ' • •• , ' I I J ', ' • I , ., I ,, f I~ ' 1 , , • i , ' ' ·"

10 urde~sland ~ho onsWcll ,, {~" o~e. , 'l{~fl .l ~r, , ,f,g l~ ,. di';; f tfls , _bo ~~c:s
of so\vi~ syslcm9 ol ~uollcns.

so\virig S~sleMS of ~uolion s :.

Jn l\nea11 al9e.b110 ,we o~e 'rnhe.sled in .so1v·1n3 sysle.ms of line.all'
equot1on9 of 'ihG.? fo~m : A~=-b 11
1: ·
11 1 _
I
whete
A-= malihc. of· cui, :,el o~ 'tnpol P-ow vectors
II
b = +he. C:Olllmn ve.cloc1 of l0be1s fuu- each veclo~ (n J:ft;e-, A :
rnoh1\::c
• "' .. J .. '1 J-_., ' J • 'I : J. 1 , ,. /~' 'I
x. =COiumn ,re.clo~ (pociamele:ll vectoci) whic.h ~eprese.nt,s
' • .. ' ( J • J

coefficie.nls. ·
I
_; : i
~:- 1tie. below ..fi9u1e .shoUJS vi suaf, zing -the e<polion A'}(.. :•.,1~b ·'.cf_
·
: f l .J

TiQini"'9 1ec0,dsc,-.) pa~~:r-·

ec - -, -oolpot ·
. . ..
.
. ~, ' -
>
- -~ -

Jnput
&

C, 1!300 ! (', - '

,,eco1 o·I o, 4-IGE, O, 10:11
2. Ir): j )) - · 1,0
~J

O•o65:J.
1 - , • I

I
I
.,_ o,GGGG C,5 .' 2 • , ,.)
--
Input o,q14s 0, G956 et ~ ,' J f ·., . ,. rJ .-,1 ~•-0
,e.colld-
1
I
. - . ,-
:11·k •:i . . ~~fJ· (CI J IH - J~

i
:rnp.,t Q, l.1583 o, 3333 o:soal5 O• "13QI · . 1. \
- ,,,3 , 0 .',-,...
1,ecco 3_,, I o:Ju} :.1dl· rn :;-::1 • I • • -,

I _·_,JJ J17 ...,,-, ! ..,-....,: Jo,:,; L. ~• 1 - _jt 1J

'
t 11 r• , 1) )." l J".;i/'1 1 '/ J.,
\-, •/~·.1 1·, L.'. 1-1,
\ \._ - ' ; ·"' . ' •
,:_( _' _u '.i

· 5olulion 't6 ·-4re above ollon Con be., w?lillen d'tre. ell o.s l'oltows:

:. 1, t • ~:•11l!
' . , ._,_,.,r;:,, . nr
, ,,
,1 _,
-

gol -fl-ie above .so1ol\on bj doin3 " t'OQb/11- 'rnve.1rsion• .. . , _

tJJe-
' d
.second me~u._,J '
,s ,, ma L?S'"- 'l' ,,
_ecor<'lpo.sl ,on. . ,
l' :. l, ,_J.,,, ,7
.._--·· ·
Beyond mo.l~i;( de.Col"f\fX)SH.ion I -thete. 'Q f~·. '. .S~e;j ,ge-n'7ro.-l.;meHhods'
I iO'll eo\V1"9 ,sets of tlnecn ~ualions. fli 1i.i, i.L ·,1 :;_.,;:;•) ,u•i
Me~hcds fo-a 90\vil"lg 5Jslems
I
o~ linea~ equQlion.s: - _,, I-J 11 J,i \ ' ,. 1
1
'> Divecl: mell-cd ·,
;~_, :, c '''~ 1'. ,, l '.J t ·_,JY -... ,,.1.·

I~> :neisouve f'()e~.

I
I ' ---1
. t k.,.
n .
Oiled mettQ.J:• I I, I I L

Well Kf"leil>n ecjs of -tna dlve.cl rnelhod of s:,tvin9 .se~ of

I
1
fmeo.-«1 ~olion~ o,e •· (:joussiDn et,rn'm llon'' ," N0b!'t'I0I ~uollon~ •
I
Ileialive rneltods·.·
, slcc.haslic:. 9,adienl d e.se,enl
I :.... con·1uqC)le. q,adie nl desc.enl
 ?Ai ,f :I JI ' • •' i
'•
I I '

1Jie moB, bchir,d ML : .shllisl1c s

'. • I ·' I I 'I I '
," I, ' I ,,,

1'4 · ; ' -'

.'
: Disb\oollon •'

I
13> U~e.t.\•food ,. . • , ., .; . i,., ) ; _..; ,, '' ,
j Oe.sclri~l~~~ ~ . ,i,~~~!°'l~at , , ' I 1 11
I l j t f ~:. J., j

Oe.sc~\plive S~ol.~hcs '" ~lude

Jf h .'• ..... , , .._.! -c, J
'> \-tisl°9b'?r~ :-.
1
> 8o-:tp\OlS

r ,. 3~ ~c.o.lle~ plot:~ , -.: 0 , . .- J

4> Me.an
;_,,•,I-,.)•:: - I ;_)

coe ff.'cie.r,{
=~:: ::; .. .J ::, -:., '.:· !I . J
·,:u,fcb'enliat ."5l:al:-islic.; :. ra-orJ )E¾ J do.f~'(',-~-;;l -find OUt .so~ p:llle~ns ·;,,.,

I .. ~ .usinij. sl:ali~'ti~~,/ .!~~~~~7iu-ih~.;~0 ~

: --· •-
11
r:.i 'J,.,-i :-, _, i!JJ ·-, -fi'~r'.n '~ ~'~xi~tirx{ d~~~~ ~s'•n<u
1 1
stofisllcs
,,, p-\/olue.5 ,· d ' , X

; :.> oecf1b'1li,lu _infeJrVQIS , , ,, . , ,_j _,___{_!( '',,-' •,1· ~··••

1 JL , . . "5 ' ,1 1J. 11 t l '5 , t., 1I j u i' l i .A ) 1
,,

1
« p~otobilil:y , 1 ;,, 1 : 1. ·~ , ,,., -~, ~,·,1 1,.) ., ;,
:•c<Ionl::lilionat " p~obobiliUe.s ' ,,, , ,1 -I J·., ,.,iJi.•.--'1- 1 ,c-·_ ,_. I, ) . '
" 1 -' .. 1 · 1
I
'

I
ill' Posle<Sio?S p?Jol:xlbili½J
< ·1
• _· ,,c., i r ·' \ 1 ~n -.J ,'l l i
'!
,.:, :•;'. l..,1,~_1 /

ft Oish,1bullon.s
I
-t ..SOmp\QS \le\SUS popt.>lo{ion I , ,ri 1· -'l 11

-it f?e.5Qrnp Ii ng rne{hcds

r•~
* Seledlon bias
... ,.
I ', :: , ~,·.,,,,. '' , ~- 1,,_..;J.!«~• ~i, ,:. 1
1 '1 J f -. I\ f I .. ,, ~, ,
}' .
' !
I •t

, !f l , • 1
.
r f •
 f pttob00III~ ,-
probobift~ ot w o9s belw'2en o ond .1
b onc:Jillon01 p1'0bobHWes: • j ' _./ I , , , l
/ ,r,~ pr1ob0bilr½;j of a given ~v. n l bo9ed on -fhQ e111lslin9 p rQ.Senc.e
i ' ..;-1 ·, , -; ,;, 1: .- ... t , ,
'of anol'-,~
/ l. occoiilng.
e\fel"I p(£fF)
I,

I Ir: ' • 1;• : - • 1'! ' ' )r I 1 'J) IJ • ;'" •: ,; I f'

p(Bf A'), p(Al

, , I • "' •

P(Afe) ""' ' ,,_, i 'r . _j ,,, 1: I l _,

PCS>
,' I . ,, ' , •' I Jf j
' '.

.,

(' ··•• ,J u d,• )' 1 I

.,; I ,.- ..JI ) 1I _"". _,l '-· ,..J." J
:rnfo~r,,ol t~m of pesobabi \it~ ts Li \<Q t, hood •

I•
·1·J , , n.11'
... I,
•-J } I
 O&cJ do.a ""Oc.....,._ leoen,ns -
. ··-•--· !
!> RIP9•.,,.oi,,. ior, . ' • • I
oto ~s;kolktr,
I '
a> c tu<;>k-1tl"9

""> un~•~UJ"9 I . ~ • -f1Wng

l , ' I f
opUMiz:oUon This
l \
'°> Conve.1t 0plirv,iza(;o,.,

'\/isu.c

••~. ' 4, r' ,·, I 1!

~:- Cosl ::, f: (C:o.)_'.7~~~, ;; ,

.
"~- ,. ,, ' ., . ....
,•I' ·,,,
1Y;!, ;, 1I
1
' ' 1.. l ' :;
IS numea-:iCQts, . !J .Ir '' " I
! I (
\ • I
) J'' , ._,, :
'. ,f \
; 11 I 1 I
I
'
op the ~ode1:- , ., ,, .1 ,.1., .,,_,,,,. , , ,·,, ·<.~- , , . .; ,, , , ·
7Fie p~edicl/9r;,_ ~1~-o pf. ,~.. .~eft~sto~ mod~\ i_~, ~e
COrnbirJQUOr) cf <:.oeffi~ie.nt:s ( ftom po.<o.rne~.- ¥.) -the
of o clirea11 ~•c.ssior, mo de1 ond +hQn 1/p \/Q,labltas.
/,,Je can build #,is l'Ylcdc I bl_j llS!in9 -fullowin<j 6,ualion.
 ' , /.,, • I ~-.·• ,. , 1,

i I

I , ,I
I ',, I . ,• I, I ll •

Tf,is f I I ' fl ,'

.
• ) -.

-tt,Ol
piol shown in
I 15
I
lo
. ' ,'

,',1) .,· ,r l
I

~: ,' . · . , ,.
':) 0 1-•f ... .....
I • 0.J.·. ,,, ,..) I,, , .2D
. :::__,_; j ~) JJ: ,_.ir;...:: '..J.· _, c, , 1-:.,
I I ' " , - ' I." ... , ,/") f 1f ). J • 'J ; ' I ·· I
r ., , 'U _r.1 1 ~· Jr . /Y.C i ) -:.,--i .· ;;;.:-,il':cz c .,, <..;, )

lf;tr,°tl. iS!,_..de.fih.io81 a ··.J®~clicn Y-=- H')l) -tho.l pc1oduces \la\ues LJ C21os~: ~ :

~i ,meot5ure(A-Q~ -) 2feo.t rtiJo~ld J Y-\0\ues. "'tne line p?rc:cluc:ed 'I= Fl~)
omes ~close ~ ·.,~ -: £ooll~,eid coo'l>dinote.s -\he p;l-'YS af ~ndent and
\rdeperdenl vo~l'abtes.
Re\Qui -lhe lioe.cui ~eg,ess10n model:-
We co.n lfe.lQ\e "ihig --fur-cl,cn +o eo~lie'5 ~oUcr\ Ax.. ~b whe~e. A-:;.
-feo\:u1e.~ i CONH'1'luk:?., he19..,l, 59ua,e. +eet:} ..foi,- al\ i/p e.9 -+ha~ 1/\le (J.)or,l
--to rncdet , each I/ p
,e.c:.o~ is a ~ow in -the m~'l'~\x A .1fie Colotri')
\/edo~ b is --ihe oukoroes -fol' OIi -the. i /p ,e.co~ds \n A t1,c~b,\-x. ·
Osin~ c_n eas~oc1 --fuN:li~n op~miza.liori rne.\hcd we. con se~ Of ,t

potorne.le.lS such -\hol ~e. rn,n,m,'Z.e. .\he.. e.?Tb'~ -oc,osei C>ll Of ~e

edidions vs {voe culc.cme.s.

usi~ SqO ,l.Ue. _h Qve 3 COrnpcne.ni~\ -fol 5Q\v,e. ooi•i--

Vee.lei t. ,

'> A h8µ:,lh e s is abo.,l 1'.~ dol:o . ." _ . . . _ ,.•... _

I ,j,,.-_,.t f t ,niJ J t 6 ·) ~- ,U ., '•~ •.. 11......,I '&.-fhe
ptop~t Of
( A IL )

1ne inne'tS
ti
.1'-!--the.
• ·',;. , , j
. oo,orne\e."'·~ve,c~~
k ;-:1 ,hDl.f')
i ;'I (YO '.)~ ;
>t i/~
feohne.s
~> C'Os\: -i.>nc.lion ~- • , ' l

.£quo.Ted enoa- r '"'"or4idion- A~luot ot. c,edklion

 c. l ,0 "
-, "'~U\e ~- •
• vri

fuoc:.llon ~n lJ~olc ..lii"'dicm , ·tho

, \.Cos L -fur-ic.Ucn) l ' ; • I ,, • '-/
• \,, ·1 '"
.:i. OSS111c:ol1 en!.
OUlpul •,s ln ·~c 11Otlm I,, I. ,i ,.
.~•o~.~i(Jd ,of ,ofp" ,..i ', ,•,: ·-'
' I
or lol-;)QI!!
I ' j

Arno 2.on, C'om - RQc: .mrn~dollo'l . ' .

I S\j9\ctn Ell~ . s~rn j np1~ 1 7' P9 m l"f)Qi I

I
r , I /· 1· , , , , , , ., ,
> --

4 Unde1fiHlo~ di!3ea~ed/nol
1
d-i5€..0S'~
I< ove~fit:l,'ng •.•
~9ar,
' •,l , ,.• ' 1f •P II ' ,1'•-1 II I • •• ) , f (I l ) •J

l, I' _, J f • 11 I f

-fi3·. · un de.1, fuu n '3

rnod~I -t1'<r Q\le'l)~Hioq JYIOdel
Il pfcduc:.e.s 1.e.ss ell'~o, s
on r)CW cblo . "I.l: does Train ~e rr:odet ~1:1~
ool c.bsslfies -\-he. dole. . o!"ld ·11:. c!Qssi~es dalQ.
pe.~~,C:lty 2. ., c J1,; l1 u :.: :; :,o b:;i.;i'I
1
Jc cU ( >i )l .;,· Y, _ n VeJSUv.toel\ r Bu\i:
J
iLool i v,il
)

( 1,' } J ,v P- ~ !~•"..;;ci-.i,,'.q .:,nil .,~l' .::;:.,1Ji cJv' :..y ctoss1·~~-~ :,; neU,) _dQµ).. _Qr)d
:w//· , C' ;k :11ib~-.J~,.:; }_ ib ptcx:loce.S O,O"IIC . e.a~ozr ~- •
, ~'. ~>. , fc _,.:, C \l -·· l _•Ul -• ~
\ \ I I I I

I C

(1 ,10:r,") ,, , :·. / \ .:i'Gh ,i i

' ,;j~ 1.uH /~ 111 ';: i .,<~ .> , 1

'
.X. :<J ) J _., I P!.,• _)r,J) l, .:., :;_k l i l I '<'l
J

J,-·H· }( ) J1( ~ ,_-:.,;._--.· __, ·.; f _.· f' I,, ~-J ..}~ : -

( J( } ) _;• ' I. I /1 '

'.
, -ti; choose. ulidelS'~lttflg ri'ockl ord "',· ·
1, •'·
\' ) . ,. /. ~'ifo~~ J
I
op(i~iz.O~(") '! • I ·•!
I t.

~, ,•.'c "':

k .
' .. ,
..,. ' I ) d
1 ' _,, '. ,, , ·, 11 i
 I • ',

, •I I

P' '·eo·nvex. ()pCKT11.zo.tion

, __ o J, ;-

! In -fhis lcobn~ -,o.,<a°11Hhrns . deal wilh C.Cnv~;·· Cosl -func!i0ns •

~,
I

(9, 0,4)

I
f
I

- 8
.. . • .•- f ()
l ~ ' !I • /"ii :: ;_; ( . I _J i ,_J ----:--t-..._'....1 -· '- '1 ! _I,___,....
....... l....._.-+-'
· 1...

1-, i
 \ , 1' 1, f I
,l i,, d !( I • I,
,. l' J )

~9
8islic ·Rea~e.ssi on:-
• • .+ Logis ~iC: Regress iol" is a w e ll • known type of ctassil\cnU
In Line.a.~ mcde.lin9. rt u.,ci k9 fo-o b o~h bin?as~ c1as s i ficol\-00 as we\l
fu 71 multiple ocabel s •,n ~ t=i! -49~m ~of ,,,,nulljnor(lia, :;,et-~<31'slic:. "'e'3"cs9i
' I '

~is lic ~es•es.eion \g a ~~~ss\'on mode.\ tn ~h1Ch

d eper-cent va1iab\e. cole~9ila1 lt; toe;iRcol,on) • '.:; ,r; ,.. ;,. ... · .,
S ird,lcn to Lineo'tl' Regv c.ssior,:, We cro.rr e~p1e.ss 0.·. . 1
Lo:9lsll'c.
~e.9 es aion
1 rnode liro
(}
ptti.1:Aerr.,
11:,), . 1
1.l'.'>d·he f ctD'r:Yl of, A~ ,:, b
./l / • ~-,• • ,: • ,' / /f :, ) l ' I ,,1 ' .ff
Whe.1J e , . · ., . -
A is ihe ~~·· :•he.l9hl ,l
.L . . ~• -., , 1,,1 .1~ . ,,:, ,· _,,, 1:.· .,.; 1·., ' ,,? Ql,aucre
! ,- , -,,-.,, '
 ', Wh11,e. I A- ,l~ ' ..... ' f•oh"e of on 1t,.e · ·.r\,sul Cll(lt"Y}pl~ won\ rtt> ,, ' 1 '

mode\, fa ch tnp.,l aJecolld b cc -.ow '" O'llhl1. I\ ond -th«J. 11, i r, , .

.~all .Of ,•~- i"~ul ,.)~Ct'l•d~ ;!" 1hf1.
' ' I

colurnn ,- vec:.l~) b ,,;. . ~o. C'AllC:-~!1 1

A rnol2M'l. ,. ·.· 1, 1 "' ;

us·,~ 0 cosl"-fu~tion cY'ki ('y.ot~l ?"lion fl\rlhod , VII! C:.Ol'l -~nc:C a l of ~e 0

"- po~omet~us .soc.h ·44-·o\ w~ m',nlmi~~ ·U,e, ~t1101 Ot'IIO'S'3 01'-'0f -#le h ·,;
1

,edk.ltoris ve1s1.i~~ ~,va cttkomao .

I

Again w~ wit\ OGc. slcc.kl,l\c fiH,d,of)l OeJlcrnl ·to setup -fhig 0p(irnizoli0r
tob\e n1 ord L~e. h:lvo. '1h•~a C.ornr,on,.nlt_; ·fo !).)Ille. Oub po~orn«? ~ell'

' , I' I •' f'

I .

,, 1 f :. r ·~: r I I .
I

.! ,. ;j.,, ,~•,l;r ·,

J'f. )

r .r
1~ _) - -- ,_
cnih
V
s,As . .,. .
I
~-, s:.,·,h : !':::i.)):J (:i,.l) 1:.,}).:.,;,,

, I :...."' t)! _l. ·:,Jo·, ti clH qqu :__;,-;,. j J ) -j(J~~ ~i '1(;

!
i
I
I

',,
6

1fiis -lune.lion is Useful ~r, , ~,slic. cSe9'essic,~ bQ~~se i~ 'tokes any

 I I I I I I f 11,

l' · ' ' · "'th

Voftt
O ·na pt0Ce9S Of unde11slording how wen
models is -#"le c:j
Q.d' C~•,e.c.~ ~lossificaliOn and ~co measullinij -th~ ~?toe_ Pf: :-:I-he .
Ye •c.l,on ' • Yf::, '"' co,e how C'r
•n O Ce~loin eonlexl . .eome -!,rnes, IIJe !? .J. ·
Q l'Y'IOde1 9e~ s On~
.
pre.die.lion co,rec.t . othe?Swise.,
. 'Ls
1~
impo~~nl
, , , ../'
'"thQt
e ·rncd-e1 ls L-• r ediclton co1iisec;l . m?~e! 'l~n
o , c~~IU'~ ,~Y~ Or . ~-~·/r) 111 , 1 :rn\'.o'" ,i•n ~c;, ,.
tt-Qn C\:heys. , 11 ..... ;·1.. ,n f; _. ) If~). I 1

I ;
It,
Hete. We Cove.b' -the lopics tike bod ~sitive~_, ~Q~mles..s '
. , !,_:. ,.'I,:...,,·~< ,11 - J.;:''.) -) :,, ,,'r l ,w)_. -. l oll/ ) •O , \
e9ali\/es, Unbo1arce.d classes, uneciuo\ cos '
-J -

Confusion mohi x is -the basf c -kol fo~ e\/atuoli~ rnodets • I

./ .I

f' Jn; ! - ·. f - ·, , - . • 1'

e - c.onrusi6rf 'Mahsii1'~
· ._ ;1 .- · l
' ·J ,-1;i ~CJ '..JV,I n:.1 I 22 _! 1(:? ''>i ..)U .. 'L,..,, - l ,_

1Fie. confus,on mal11~ otso called Q --ldble 1Ccf!' e0,{~~rcor;-ifl1__'l4.

tabie o~ ((ows aod columns -th::H )'ep1~ts -the pfedic:bons
d --#he Oclto1 oolcomes (10.~l.s')Fo~ 'x\3..c\ossitfl)~l-
! . '3 l·
We use -Jhis -fable -to bellec, urdea-sland hew wen "fhe
IV'Q:te1 tO'b') clossi.h'e-g is pe?S~mina based on 9h,in8 ihe COc>aec.l
onswec, al -the opp21op~iote -lime

, p
1 N'
(p1c:diclcd) ( p1edickd)

p TR UF- ;ALSE.
( Acluo1) Posi11ve Neijo!ive

N FALSl TRUE
( /ldua1) positive --JepUve.
/
 FAlsE ' Neplive .
. '
-+ positiv e. preedic:.ll~' l , , • • - N•<_J'lllvc p ,cid.'clJcn
- IObe l UXl!I po~ilfvc: - lobt:-1 t.ua-i Po~ i live
'
't ', . • I f
' ''
.
,.
;AL9E p0sili11e9 1
,.,,•, .TRUE N' 1
'' t';J , I i':,, ,,,,.
,C . , , I , .,. ,, \
- posil\ 11 e P~edidlor,
- N~l~e p,c,d,c.l,cn
- roter NQ.<)Ql\ve.
- ,bbc.:,I l.Ul9 N~live I

I I
l .,, .,, ' " 1
I ., t . ' A l , • I • ' , ' ,1 ' I ••

:::Ji1 convenl1'0rxll ·I , , , . ' ,' ,, ,, . ·! I" ; ,', .:

· ' .stQ isllc.:s, A rAL9t. pcsi~iv,~ is ,c;.lso,, k~-~'-:01') .~:: •: ~. ,. ,
'11jpe I eb'lJO'lJ , and o FALsc t-J~l\Ve, ia known , ,?,S , " , 1;ype·JI;. ... ,,
8'3 iro~~,1,:,9 \-H,~~e , r:no~?,i <;T ,.~ , c.?p 0;<;--hj~ve 1.~ . fn<;>!1e. i ?;°llj-Sls
-»ie pe.~lo"Grronce oF -the. rncde\ be'jond -the b:)sk •/, or 9Utsscs -tt-,al
o,efe cowec~... . ' ·'. } ' r,; '.
·-•e c:.on oo\c.'..,\ate ; diFf' 1' ~
'. ' ' . ' I l I t 1' I
'
1
I( ''t• j .,.;,1,,,_ ,, ·..:.:.:._, L...-'
.
ed ' on• !,.
I/" evo.tuo. ens cP ..Jhe rn~ uus
mb'nolicns of ··fha o.bcve 4 c.ounls LTRUI: posil111es, FALSE positives f
TRUE N~U\IQ, FAL5~ Nego.ti"e), in -¼ha confusion rnohsht as shewn
below, 'l ·J"
-- -· - - - (')<_)) .• ' I '.; '·; ::

(,,.-) · :1 I· 'i ' I·_) -

p,ec::lsior, o,sG 6 1.

I ,v,,~ ~ ~ f/~:?~,,.;,c,,u \.er

.J
1;, ; ~-

5'~ t...ets d,sc.u.ss -+he bo~ic.s cf e.\/olutin'3 model -that is .se.nsilhrily

/and speci Rc:.i Ly . '·l . I - - /1,, :.) ...
j 1
-
,Sensilivil~ Vs Spe<;ifid~ ~- { I A
),i '
t
i l" ) , 1• ' I ·
' . /

Sens'il.iva~ - Re.c.011 (Ottie2f norne)

i
I
4 10 fY')easore -the Troe. posilive.

I i< • ,
c

, J. i!/,1!
L
•
,1 0ti~Ul~;·· J~e~,s,l,\f1~1:1 ~c

f ; , f )
. .

, ! <_j ·/ •il l
"Tvoe. posili_ve
T1ve
.

I)
•:· , I - ~'I ,·
pcsilive '-t Rllse 'Necaati,ie· - l
J .. I r:I •
. ,,

;/,! c 1
.

,), ' I _
' I ·•

f .·,r_,·
1 '=J f

. !J . )

I.
I
.Specif:ci l~ :-
, ..,, _' -- ~ i1o;!meas-u1e ,1ihe· 1·{:(oe#,fi)~oliv-i,.- '!lal~· , .. of')),' I ·1

, ,/·) ·J · .1JI, \'.-' f I r_,} ' 1:. .. ,· ,ll':s r.!._.,, 1,' I.'• ..•··,
"'Ttoe Ne~dl1ve
folJrti ul o. '.. pe~i fici~ - ..:..:.._ _.:~--------:-:-:--- •
N"fl~'e!.~-~:
5 I

"\tue ~~ilive ,/

:rn ~eot woald,-the rnodel rnus l bk'1ido 1.i1J~~1liv~'~A\\ ~~·1x:ks a,e dclccl-ed~
1

too •/,specifidly(ie. J
1
Ond 1
t(J6one. . who ' 1s ., ~ot ' .sic~ is c10.ss,~d o s sic "-)
 ,c

•• , 1, J l
,, I 'l '

I I I I I I jf 1 u
N

',' ' '1 I I I ~ I I . Ir

• . T,P
p~ec.1s1on
'J • - i
-(T,p+ F.p) !.. I I , I ; : ' I '

'
3
>Recon :-
1 lnis is -the .some thinr} a.s
~,
.se.ns'1\'Jvily · and olso ~~awn a.~
.... ....,,,,,, , : ., ,, ." ,, ... )

~---"'.::> ' .J : .; ,_, , -1 :

/ Tvue posilive ~le to'b') h,l ~ale ,

,.p
RecQII :::-
--- - -
( T, p-t F, N) . i'
..
-, - ~11 ,, 1 ' !~ , i

~·
, I I'

I'
. . 1 I 1 f f
,. . . •
,,
' ' ., ·, , ,· .
F1. -+ ,Jy;;> ~j.')~~~ ~lo~;'r~ll_OF) tJ.>e. ~ce,Hs1de~ 1
,-+h e :Fi ·~core (or)
1 1 I l . '

or
I I

F -SC:o'<e (o"i) F·- rn eosote -to be Q 'measure a models

oa;:uYacy-
1 •: ,,

1fie Fi sc.ov~ ig1, -¾he_,, i ~~l'Y<'}C)')i 'c ,. roe.on c,i I C>C!>~ ,·pte c isiol") al"l<
a'emll
tneosoteS ,. in lo o .7\ria,'e . ;5~~?~-,
)

•· ~,-'.p·• 1·; 11\ 'l (!

Fl =- - - - - --
,.. , ,~, :-2"f.· P.,· +, N.,: F;P) I
' I I I' ,' •' I

' I
 ' .
. l
., '

Neu1ro1 nell.1.,0'1 ... OIC. o c.ompulolfona1 rncdel {ml .shl,es some

~i~es 1 :i~ oti\r,,at t:ieoin .simptc un'tls :ate • <A.>oik~
'ir, , whic\.\ , tnOf'\tj

·,n F~~!~' ., ~'~ ~o ,,ce~l,ci\iz~d ,conb1~11_µni_b11f,e 11,JJ~i,q ~l1~ .b1~~,e".' . , !-.

uni\~ o,_e,._ ~ 1- ~~~~o, .~E:0':'S ?f, l~.~~ 1{e1Sr, ,inf~rr,0(1~, ~.~H~'J~u lr,, , _,
~eu~al nelwc~K6. Up1al\"'d ihe u.,ei ghls WI( ' ~e pilrt;>~~Y.1 ,~~y, ,;1\"~ !·., .
N \eo~ns new ,nfozroation.
1
w" , o•$C::u~ s -'if,d \-no~u ,\,8 S~t~ .:.~' ~o \., •-~.;' · -ihe - . i:>~~ '
equolioo J \~x:: b :rn l-fhe ,1c.onte-.:l ·~t? ,N~"is~\ ~: 'J;~r A:rnohl ':i'.i:\
i\l inpuJ.·,~~~.,\to~,., eo\um11'·
\J
i~ :· slilt '
• J ,. .. '
VecJ'21,
lo~IS)~~ t ·~J\.ll •"
,i . , ,ir
ou~<:,om~r _
,fol l~d"!J'2SCU> _tfli -I-he·; _.-. · ~lili :1he. we19'h1ts ·c n ' =-fhe Neuiilf · ;
"'A
' l ', ,
I , I - . • I
e.tooclSkS C.onnec.lions becomes • x' {po.lS?roele~ veclo~ · n ..u•~·:..~
1fie .behavio~ of -\hQ. Ne.u-zso \ Nelwo lSkS _i~ 1 .sl-pp=d b_;J j\ s ne tw o?J k.
archile~~~e. · A ~etwo~k•s c~Vch~l-~e c::~9 ~~ji _by }~e
· 11 'o ~J\0J .-s -, :,1 ·).~c of::) c :•.1.:r n·J ! .. •D ,ll
fo oWI 8. -~;>t(
I
. ..' _,,,
.
(; :••
__
, • I
() l •·)
.J
:)l.
. • ...
, . ,:H ).;,( 11 1

,-. -1 - :,
:.1 . - ,.
.

r J, ' ) ' , j _J...)

('11 0"..J,•,.i { ~-.::'.. '4 10 :....l '-' ..,
•
J,,.-
.

.1 ·-...~,., _
,.,.,. 1
I
J(; , -
•
i,._ -~
_,,.
;
-/•l
•
·.:' .. 1, · '' iC::.
? NO ., of Neuaons. ._ . , -., _. ,
:.:,y ,;,_
9e~s ·
· l•.)l . ... _:;J6-<l j "::O !JIJJ .::)] f..).!!2 2 ,j.J'j; L_.::. -' l l '( .1· ·, 1 'j • ,
~> No. of ki c ·. ) l..
 .i., 1 1
• I 1,·,,:v _,\.'1.t /i' ,; .l ,'u,· 1/ ,'._, ~l 'l·· , _,-"(J.·1 • t ' J ,',
-: 5_,., ,· _, ,;1 )oi::-o-:1',! - • - -' ·•

slep --fu,dion

f 1,

'
I •'-

can mcdel
CXJEpu l ,'n the

; X: Nef inptA
 OeS~ipllon
tI

Vttcb:,?J tea\ 1/aJueo Cl.)ef9hl£ On 1ne j

CCrine.cllcn

n
t\b ,cf inpols --to ihe pc~ceplon

b
Sias le2$rn