NvR 1

TemplateforparameterestimationwithMatlabOptimizationToolbox;includingdynamic
systems

1.Curvefitting
Aweightedleastsquaresfitforamodelwhichislesscomplicatedthanthesystemthatgeneratedthe
data(acaseofsocalledundermodeling).
System:

3
2
0 1
( 1) 1 2
x
e x
y
x x

< <
=

< <

(1.1)

Data(noisemodel):

(0,0.05)
data
y y N = + (1.2)

withadditivezeromeanwhitenoiseZMWNwithastandarddeviationof0.05.
Modeltofitthedata:

( ) a x b
fit
y e
+
= (1.3)

withparameterstobefittedu=[a,b].
Themodelcanbeforcedtofitthesecondpartofthedata(1<x<2)byassigningdifferentweights
tothedata.Errorfunction:

( )
( )
( )
1
2
( , ) 0 1
( , )
( , ) 1 2
data fit
data fit
data fit
w y y x x
e w y y x
w y y x x
u
u
u

< <

= =

< <

(1.4)

withw
1
=1andw
2
=10.
ToimplementandsolvetheweightedleastsquaresfittingprobleminMatlabthefunction
LSQNONLINoftheOptimizationToolboxisused.Optimizationalgorithms(infactaminimizationis
performed)requiretheusertospecifyaninitialguessu
0
fortheparameters.Hereweuseu
0
=[0.1,1
].
TheMatlabcodeintheboxbelowcanbecopiedandpasteintheMatlabeditorandthensaved(or
pushtheRunbuttonwhichwillsaveandautomaticallyrunthecode).

f unct i on myFi t
%myFi t Wei ght ed l east squar es f i t

%%cr eat e t he f i r st hal f of t he dat a
xdat a1 = 0: . 01: 1;
ydat a1 = exp( - 3*xdat a1) + r andn( si ze( xdat a1) ) *. 05;
wei ght 1 = ones( si ze( xdat a1) ) *1;

%%cr eat e t he second hal f of t he dat a
%use a di f f er ent f unct i on and wi t h hi gher wei ght s
xdat a2 = 1: . 01: 2;
ydat a2 = ( xdat a2- 1) . ^2 + r andn( si ze( xdat a2) ) *. 05;
wei ght 2 = ones( si ze( xdat a2) ) *10;

NvR 2

%%combi ne t he t wo dat a set s

xdat a = [ xdat a1 xdat a2 ] ;
ydat a = [ ydat a1 ydat a2 ] ;
wei ght = [ wei ght 1 wei ght 2 ] ;

%%cal l | LSQNONLI N|
par amet er _hat = l sqnonl i n( @mycur ve, [ . 1 - 1] , [ ] , [ ] , [ ] , xdat a, ydat a)

%%pl ot t he or i gi nal dat a and f i t t ed f unct i on
pl ot ( xdat a, ydat a, ' b. ' )
hol d on
f i t t ed = exp( par amet er _hat ( 1) . *( par amet er _hat ( 2) +xdat a) ) ;
pl ot ( xdat a, f i t t ed, ' r ' )
xl abel ( ' x' ) ; yl abel ( ' y' )
l egend( ' Dat a' , ' Fi t ' )

%%f unct i on t hat r epor t s t he er r or
f unct i on er r = mycur ve( par amet er , r eal _x, r eal _y)
f i t = exp( par amet er ( 1) . *( r eal _x + par amet er ( 2) ) ) ;
er r = f i t - r eal _y;

%wei ght t he er r or accor di ng t o t he | WEI GHT| vect or
er r _wei ght ed = er r . *wei ght ;
er r = er r _wei ght ed;
end

end

ExecutingthisMatlabprogramresultsin:

par amet er _hat =

3. 1001 - 1. 9679

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

-0.2
0
0.2
0.4
0.6
0.8
1
1.2
x
y


Data
Fit

So
3.1001( 1.9679) x
fit
y e

=

NvR 3

However,herewedonotobtainanyinformationabouttheaccuracyoftheestimated

u .For
example,wedonotknowhowcriticalthefitdependsonalldigitsintheparametervaluesreturned
byMatlab.

2.Parameterestimationforadynamicmodel
Inthesecondexampleweconsideradynamicalsystem.Ifbloodplasmaandatissueororganof
interestcanbeconsideredasconnectedcompartmentsthenthefollowingmodelcanbeusedto
describetissueperfusion:

( )
trans
e
a e
e
dC K
C C
dt u
=

t e e
C C u =

inwhichthearterialplasmaconcentrationC
a
(t)istheinputandthetissueconcentrationC
t
(t)isthe
output.
Thislineardifferentialequationmodelcanbetransformedinto2descriptionsoftenusedinsystem
andcontrolengineering.
a)Statespaceformat:
( ) ( ) ( )
( ) ( ) ( )
x t Ax t Bu t
y t Cx t Du t
= +

= +

, ,
a e t
u C x C y C = = = ,
u u
= =
trans trans
e e
K K
A B , , 0 u = =
e
C D
b)Transferfunction:
( )
( )
( )
trans
t
trans
a e
C s K
H s
C s s K u
= =
+

TheControlToolboxfromMatlabcanbeusedtoimplementandsimulatethismodel.
Theparametersu=[K
trans
,u
e
]needtobeestimatedfromclinicaldata.Tissueperfusioncanbe
measuredwithDynamicContrastEnhancedMRI(DCEMRI).Acontrastagentisinjectedintothe
bloodstream,circulatesthroughthebodyanddiffusesintotissuesandorgans.Thekineticsof
contrastenhancementismeasuredinavoxelinthetissueofinterest,whichcanbetranslatedintoa
concentration.
Thefollowingnoise(error)modelisassumed/proposed:

t
y C = + (1.5)

withZMWNandthereforethesumofsquarederrorsisusedasestimator.
Inthiscasethepurposeofthemodelisnotjusttobeabletodescribethedata,butdifferencesinthe
estimatedparametervalueshaveabiologicalmeaningandmightbeusedforclinicaldiagnosisand
decisions.Henceitiscriticallyimportanttoassesstheaccuracyoftheestimates.Forthisproblemthe
estimatoristheMaximumLikelihoodEstimator(MLE)anditispossibletocalculatethecovariance
NvR 4

matrixoftheparameterestimates,whichquantifiestheaccuracyoftheestimate.(Theinverseofthe
covariancematrixisknownastheFisherInformationMatrix.)
InthesubsequentMatlabcodeitisshownhowthecovariancematrixcanbecalculatedfromthe
outputsprovidedbytheLSQNONLINfunction.Asinitialguessesu
0
=[2e3,0.1]willbeused.

f unct i on compar t ment

%compar t ment Est i mat i ng phar macoki net i c par amet er s usi ng a dynami c 2
%compar t ment model
%Loads dat af i l e pmpat 010. t xt .

%%Load dat a
dat af i l e = ' pmpat 010. t xt ' ;
dat a = l oad( dat af i l e) ;
t =dat a( 1, : ) ; %[ s]
Ca=dat a( 2, : ) ; %[ mM] ar t er i al i nput f unct i on
Ct =dat a( 3, : ) ; %[ mM] t i ssue r esponse

f i gur e; pl ot ( t , Ca, ' . b' , t , Ct , ' . r ' ) ; hol d on
xl abel ( ' Ti me [ s] ' ) ; yl abel ( ' [ mM] ' )
l egend( ' AI F' , ' Ct ' )

N=l engt h( Ct ) ;

%%Si mul at e model wi t h i ni t i al par amet er val ues
%Si mul at e
Kt r ans = 2e- 3; ve = 0. 1;
p = [ Kt r ans, ve] ;
[ y, t ] = compar t _l si m( t , Ca, p) ;
pl ot ( t , y, ' k' ) ;

%%Est i mat e par amet er s and var i ances
opt i m_opt i ons = opt i mset ( ' Di spl ay' , ' i t er ' , . . .
. . . %' Tol Fun' , 1e- 6, . . . %def aul t : 1e- 4
. . . %' Tol X' , 1e- 6, . . . %def aul t : 1e- 4
' Levenber gMar quar dt ' , ' on' ) ; %def aul t : ' of f '
%opt i m_opt i ons = [ ] ;
p0 = [ Kt r ans, ve] ;
[ p, r esnor m, r esi dual , exi t f l ag, OUTPUT, LAMBDA, J acobi an] =
l sqnonl i n( @compar t _er r or , p0, [ ] , [ ] , opt i m_opt i ons, t , Ca, Ct ) ;
di sp( ' ' )
p

%Est i mat ed par amet er var i ance
J acobi an = f ul l ( J acobi an) ; %l sqnonl i n r et ur ns t he J acobi an as a spar se
mat r i x
var p = r esnor m*i nv( J acobi an' *J acobi an) / N
st dp = sqr t ( di ag( var p) ) ; %st andar d devi at i on i s squar e r oot of var i ance
st dp = 100*st dp' . / p; %[ %]

di sp( [ ' Kt r ans: ' , num2st r ( p( 1) ) , ' +/ - ' , num2st r ( st dp( 1) ) , ' %' ] )
di sp( [ ' ve: ' , num2st r ( p( 2) ) , ' +/ - ' , num2st r ( st dp( 2) ) , ' %' ] ) ;

%%Si mul at e est i mat ed model
[ y, t ] = compar t _l si m( t , Ca, p) ;
f i gur e; subpl ot ( 211) ; pl ot ( t , Ct , ' . r ' , t , y, ' b' ) ;
xl abel ( ' Ti me [ s] ' ) ; yl abel ( ' [ mM] ' )
l egend( ' dat a' , ' model ' )
NvR 5

xi = Ct ( : ) - y( : ) ; %same as r esi dual f r oml sqnonl i n

subpl ot ( 212) ; pl ot ( t , xi )
xl abel ( ' Ti me [ s] ' ) ; l egend( ' r esi dual s \ xi ' )
assen=axi s;

%%f unct i on t o si mul at e compar t ment model
f unct i on [ y, t ] = compar t _l si m( t , Ca, p)

Kt r ans = p( 1) ; ve = p( 2) ;
num= [ Kt r ans*ve] ;
den = [ ve, Kt r ans] ;
sys = t f ( num, den) ;
[ y, t ] = l si m( sys, Ca, t ) ;
end

%%f unct i on t o cal cul at e MLE er r or
f unct i on xi = compar t _er r or ( p, t , Ca, Ct )

[ y, t ] = compar t _l si m( t , Ca, p) ;
xi = Ct ( : ) - y( : ) ; %make sur e bot h vect or s ar e col umns

%f i gur e( 1) ; pl ot ( t , Ct , ' . ' , t , y, ' g' ) ; hol d of f ; dr awnow %uncomment t hi s l i ne
t o show
%dat af i r f or each opt i mi zat i on i t er at i on ( sl ows down t he execut i on)
end

end %mai n f unct i on

Whentheprogramisexecuted,firstafigureisreturnedwiththeinputdata(concentrationinblood
plasma,AIF,inblue),theoutputdata(tissuecompartment,Ct,inred)andthemodelsimulation
giventheinitialparametervaluesu
0
(solidblackline).
0 100 200 300 400 500 600
-5
0
5
10
15
20
25
30
35
Time [s]
[
m
M
]


AIF
Ct

Secondlytheresultsoftheparameterestimation(vectorofestimatedparameters,estimated
parametercovariancematrix,andtheparameterestimateswiththeirrelativestandarddeviation,in
(%)):

p =

NvR 6

0. 0097 0. 2018

var p =

1. 0e- 005 *

0. 0133 - 0. 0025
- 0. 0025 0. 2574

Kt r ans: 0. 0096992 +/ - 3. 764%
ve: 0. 20176 +/ - 0. 79515%

Finally,afigurewithaplotoftheidentifiedmodelandthedata(toppanel),andaplotofthe
differencebetweenmodelanddata(residuals,bottom).

0 100 200 300 400 500 600

-2
0
2
4
Time [s]
[
m
M
]


data
model
0 100 200 300 400 500 600
-1
-0.5
0
0.5
1
Time [s]


residuals

NvR 7

3.Template
Finallyatemplateisprovidedtoestimateasubsetoftheparametersinamodel(someparameters
areassumedtobeknownandthereforearefixed)andthemodeliscomposedofasetofcoupled
firstordernonlineardifferentialequations(simulatedwithoneoftheMatlabodesolvers).

Templ at e f or Nonl i near Least Squar es est i mat i on and Fi sher I nf or mat i on
Mat r i x

%%Load dat a
%f or exampl e f r omExcel
dat a = xl sr ead( . . .
%obt ai n / def i ne exper i ment al t i me vect or
t exp =

%%Si mul at i on t i me and i nput
t si m= t exp; %not necessar i l y t he same
%I nput mat r i x ( 1st col umn t i me ( can be di f f er ent f r omexper i ment al t i me) ,
2nd col umn, et c. cor r espondi ng i nput val ues
t u = [ t exp, u1, u2, . . .

%%Par amet er s and i ni t i al condi t i ons of st at e var i abl es
k1 = . . .
%assi gn par amet er s t o vect or wi t h known ( f i xed) par amet er s and a vect or of
par amet er s t o be est i mat ed
pest = [ . . .
pf i x = [ . . .
%t ot al par amet er vect or : p = [ pf i x pest ]

%t he same f or t he i ni t i al condi t i ons
x0est = [ . . .
x0f i x = [ . . .
%t ot al i ni t i al condi t i ons: x0 = [ x0f i x x0est ]

%t ot al vect or of quant i t i es t o be est i mat ed:
px0est =[ pest x0est ]

%%Opt i mi zat i on al gor i t hm
%Set t i ngs l sqnonl i n
l b = zer os( si ze( px0est ) ) ;
ub = [ ] ;
opt i ons = opt i mset ( ' Tol Fun' , 1e- 4, . . . %def aul t : 1e- 4
' Tol X' , 1e- 5, . . . %def aul t : 1e- 4
' Levenber gMar quar dt ' , ' on' , . . . %def aul t : on
' Lar geScal e' , ' on' ) ; %def aul t : on

%LSQNONLI N: obj ect i ve f unct i on shoul d r et ur n t he model er r or vect or
[ px0est , r esnor m, RESI DUAL, EXI TFLAG, OUTPUT, LAMBDA, J acobi an] =
l sqnonl i n( @obj f nc, px0est , l b, ub, opt i ons, . . .
t exp, dat a, t u, pf i x, x0f i x) ;

%%( est i mat ed) Pr eci si on:
J acobi an = f ul l ( J acobi an) ; %l sqnonl i n r et ur ns t he j acobi an as a spar se
mat r i x
%Par amet er covar i ance mat r i x ( i nver se of Fi sher I nf or mat i on Mat r i x)
var p = r esnor m*i nv( J acobi an' *J acobi an) / l engt h( t exp) ;
%St andar d devi at i on i s t he squar e r oot of t he var i ance
st dp = sqr t ( di ag( var p) ) ;
NvR 8

%%Reconst r uct par amet er and i ni t i al condi t i on vect or s
pest = px0est ( . : . )
p = [ pf i x pest ] ;
x0est = px0est ( . : . )
x0 = [ x0f i x x0est ] ;

%%Si mul at i on wi t h est i mat ed par amet er s
ode_opt i ons = [ ] ;
[ t , x] = ode15s( @odef nc, t si m, x0, ode_opt i ons, t u, p) ;

- -
%Obj ect i ve / cost f unct i on f or l east squar es
f unct i on e = obj f nc( px0est , t exp, dat a, t u, pf i x, x0f i x)
%e: vect or of model r esi dual s

%Reconst r uct par amet er and i ni t i al condi t i on vect or s
pest = px0est ( . : . )
p = [ pf i x pest ] ;
x0est = px0est ( . : . )
x0 = [ x0f i x x0est ] ;

%%Si mul at i on
ode_opt i ons = [ ] ;
[ t , x] = ode15s( @odef nc, t exp, x0, ode_opt i ons, t u, p) ;

%Model out put ( s)
y = x( : , 1) ;

%%Model r esi dual s
e = dat a- y;

- -
%Model i n st at e- space f or mat ( syst emof coupl ed 1st or der ODE' s)
f unct i on dx = odef nc( t , x, t u, p)
%t : cur r ent t i me
%x : st at e val ues at t i me t
%t u: mat r i x wi t h t i me poi nt s ( 1st col umn) and cor r espondi ng i nput si gnal
val ue ( 2nd col umn)
%p : par amet er s
%dx: new st at e der i vat i ves ( col umn vect or )

%cal cul at e i nput u( k) by i nt er pol at i on of t u
u = i nt er p1( t u( : , 1) , t u( : , 2) , t ) ;

%ode' s
dx1 = . . .

%col l ect der i vat i ves i n col umn vect or
dx = [ dx1; dx2 . . . ] ;