Manuscript submitted to Website: http://AIMsciences.

org
AIMS’ Journals
Volume X, Number 0X, XX 200X pp. X–XX

DRUG RESISTANCE IN CANCER CHEMOTHERAPY

AS AN OPTIMAL CONTROL PROBLEM

Urszula Ledzewicz
Dept. of Mathematics and Statistics
Southern Illinois University at Edwardsville
Edwardsville, Il 62026, USA

Heinz Schättler
Dept. of Electrical and Systems Engineering
Washington University
St. Louis, Mo 63130, USA

(Communicated by Aim Sciences)

Abstract. We analyze non cell-cycle specific mathematical models for drug

resistance in cancer chemotherapy. In each model developing drug resistance
is inevitable and the issue is how to prolong its onset. Distinguishing between
sensitive and resistant cells we consider a model which includes interactions of
two killing agents which generate separate resistant populations. We formu-
late an associated optimal control problem for chemotherapy and analyze the
qualitative structure of corresponding optimal controls.

1. Introduction. Mathematical modeling of cancer chemotherapy has more than

four decades of history (e.g. [6], [25]). Its contributions mostly lie the development
of qualitative ideas for chemotherapy scheduling, but practical quantitative appli-
cations are lacking. The reasons for this lie both in biomedicine and mathematics.
On the biological side important cell processes still are not well understood and
crucial parameters in the modelling are not known or simply vary too much from
case to case so that data are not readily transferrable. On the mathematical side the
only feasible approach to deal with realistic (and thus necessarily high-dimensional,
complicated and intricate) models is through numerical simulation studies relying
on computational power [23]. But if there is a high uncertainty or a great range
for relevant parameter values from patient to patient, these simulation studies are
inherently of limited quantitative practical value. “The best average treatment may
be the poorest option for a particular patient” [9]. Theoretical analysis on the other
hand is limited to small and hence overly simplified models whose results are thus
not applicable quantitatively. Nevertheless their analysis can further our under-
standing of some simplified aspects of the overall system, a necessary step towards
the goal of analyzing more medically relevant models. For example, these investi-
gations can help in determining how sensitive some protocols are with respect to
specific parameters and thus point to relevant, respectively, less relevant medical
parameters in quantitative approaches.

2000 Mathematics Subject Classification. Primary: 49J15; Secondary: 93C50.

Key words and phrases. Optimal control theory, cancer chemotherapy models, drug resistance.

1
2 URSZULA LEDZEWICZ AND HEINZ SCHÄTTLER

Of all the limiting factors in chemotherapy, and there exist many of them, “prob-
ably the most important - and certainly the most frustrating - is drug resistance”
[11, pg. 335]. The entire process of the drugs’ actions is “forbiddingly complex”
[9] and given the complicated biochemical processes that are necessary for the cy-
totoxic agents to be effective, many defense mechanisms are open to the cell. For
example, many cytotoxic agents are susceptible to ABC transport proteins which
remove molecules out of the cell. Thus an over-expression (for example, by gene-
amplification) in these molecules is an important mechanism for resistance to various
drugs. Another one is that repair systems within the cells become activated which
overcome the damage done by the drug. Malignant cancer cell populations are
highly heterogeneous - the number of genetic errors present within one cancer cell
lies in the thousands [16] - and fast duplications combined with genetic instabilities
provide just one of several mechanisms which allow for quickly developing acquired
resistance to anti-cancer drugs. In addition, so-called intrinsic resistance (i.e. the
specific drug’s activation mechanism simply doesn’t work) makes some cancer cells
not susceptible to many cytotoxic agents. “ ... the truly surprising thing is that
some malignancies can be cured even with current approaches.” [9, pp. 65]. Sev-
eral mechanisms to circumvent the problem of acquired drug resistance have been
tried, but so far without success and currently no medical solution to the prob-
lem of drug resistance in chemotherapy exists. In fact, it is acquired or intrinsic
drug resistance which eventually makes most chemotherapy fail and in view of the
manifold ways in which the cell can react to an attack by cytostatic agents [9, 31]
it appears unlikely that there ever will exist a drug whose effectiveness will not
ultimately be limited in this way. While a “cure for cancer” thus may simply be
the “holy grail” of medicine, a more realistic objective of treatment is to increase
the life expectancy of the patient. For this combination therapies in which different
types of drugs are administered still provide a valuable option. They are based on
the idea that cells may become resistant to some particular agent, but then still can
be treated through other cytotoxic agents which have an entirely different mode of
action. Since cancer cells can in fact lose acquired drug resistance, (for example, by
gene de-amplification), alternate treatments with different type of drugs therefore
may prolong the onset of resistance. While this is true to some extent, in reality,
unfortunately often multi-drug resistance and unacceptable levels of toxicity to the
patient limit these approaches as well.
Several probabilistic models for developing drug resistance exist in the literature
(e.g. [4, 8, 12, 33]). For example, in one of the early classical works by Coldman
and Goldie the tumor size is analyzed as a stochastic process and the probability
to have no resistant cells is maximized [4]. In this paper, as in numerous others
like [5], simple non cell-cycle specific two-compartment models distinguishing only
resistant and sensitive cells are considered. More recently, a probabilistic model for
the evolution of the drug sensitive cancer subpopulation from a single mutational
cell has been formulated and analyzed numerically by Westman et al. in [32, 33] in a
cell-cycle specific context distinguishing between the clonogenic (or quiescent) and
the growth (or proliferating) fraction. In these and earlier papers drug resistance is
treated as a sudden event, only distinguishing resistant and sensitive cells. While
drug resistance may be induced by a single mutational event (clinical resistance
sometimes appears so rapidly in patients that this would be a plausible explanation,)
the more common mechanism seem to be random mutations over time. “a partially
resistant clone may ... undergo further mutations and become progressively more
 DRUG RESISTANCE IN CANCER CHEMOTHERAPY 3

resistant [9, pg. 64].” A broad class of models which describe drug resistance not as
a single mutation event, but as a branching process, was developed by Harnevo and
Agur [7, 8] and Kimmel and Axelrod [12, 13]. Corresponding infinite-dimensional
deterministic models have been formulated and analyzed by Swierniak et al. [14, 28,
30]. However, due to high dimensionality these models often only allow a limited
analysis.
All these models have in common that they analyze developing drug resistance
with respect to a single cytotoxic agent or a group of drugs which can be lumped
together in their effect. In this paper we consider a mathematical model for combi-
nation cancer chemotherapy under evolving drug resistance which considers multiple
killing agents. The underlying model was formulated jointly with A. Swierniak in
[21]. It is probabilistic, a branching random walk model with a finite number of
states [12], but averaged over the populations in individual compartments. The
model for acquired drug resistance is based on the mechanism of gene amplification,
but the equations can easily be adjusted to fit other mechanisms. For simplicity
and as starting point, the models considered in this paper are not yet cell-cycle spe-
cific and treat drug resistance as a “complete event” [33] only distinguish between
resistant and sensitive compartments of cancer cells. Developing drug resistance is
unavoidable and will eventually lead to a halt of treatment. The problem therefore
is not to eliminate the cancer, but to prolong the patient’s life expectancy.
There are many non-equivalent ways of trying to formulate such a problem math-
ematically. In this paper we consider an optimal control approach. In section
2 we briefly consider the simpler model with one killing agent which falls into
a well researched class of bilinear optimal control problems mathematically (e.g.
[17, 18, 20, 27]). This is no longer the case when combination drug treatments are
considered. In this case considered in section 3 the minimization of an indefinite
quadratic function on a compact set determines the controls also leading to inter-
esting mathematical questions. We initiate the analysis of this model with tools of
modern optimal control in order to gain some qualitative insights into the structure
of optimal protocols. Section 4 gives some simulations of corresponding multi-drug
protocols comparing them with reasonable ad-hoc strategies.

2. Mathematical Models for Cancer Chemotherapy with a Single Killing

Agent under Drug Resistance. In this section, as a precursor to multi-drug
models, we briefly consider a model with a single cytotoxic agent. We only use the
simpler model to discuss the mechanism for drug resistance underlying this model
and to introduce some of the mathematical tools of optimal control.
2.1. Drug Resistance and Gene Amplification. Amplification of a gene is an
increase in the number of copies of that gene present in the cell after cell division,
deamplification corresponds to a decrease in its number of copies. As mentioned
before, cancer cells are genetically highly unstable and due to mutational events and
gene amplification during cell division, cells can acquire additional copies of genes
which as a result make the cells more resistant to certain drugs, for example by
addition of genes which aid removal or metabolization of the drug. The more copies
of such a gene will be present, the more resistant the cells become to even higher
concentrations of the drug. Gene amplification is thus well-documented as one of
the reasons for evolving drug resistance of cancer cells (see, for example, [1, 9, 24]).
Mathematical models for drug resistance based on gene amplification have been
proposed and analyzed probabilistically by Harnevo and Agur [7, 8] in connection
4 URSZULA LEDZEWICZ AND HEINZ SCHÄTTLER

with a one-copy forward gene amplification hypothesis which states that in cell di-
vision at least one of the two daughter cells will be an exact copy of the mother cell
while the second one with some positive probability undergoes gene amplifications.
These concepts form the background for models developed by Swierniak, Smieja
et al. [26, 29, 30] where various levels of drug resistance are considered. Taking
into account an increasing degree of gene amplification leads to infinite-dimensional
models [14] involving integro-differential equations which, however, are difficult to
analyze. Thus, assuming some level of simplification and staying within a finite-
dimensional structure enables a better analysis of these problems. In this paper we
keep the number of levels of drug resistance minimal, i.e. we only distinguish sensi-
tive and resistant compartments, but our aim is to analyze multi-drug treatments.
As a precursor we first briefly discuss the 2-dimensional model corresponding to a
one drug treatment.

2.2. The basic 2-compartment model. We only consider two compartments

consisting of drug sensitive and resistant cells, and we denote the numbers of cells in
the sensitive and resistant compartments by S and R, respectively. Within the one-
copy forward gene amplification model, once a sensitive cell undergoes cell division,
the mother cell dies and one of the daughters will remain sensitive. The other
daughter, with probability q, 0 < q < 1, changes into a resistant cell. Similarly,
if a resistant cell undergoes cell division, then the mother cell dies, and one of the
daughters remains resistant. However, for cancer cells (and different from viral
infections like HIV) it is possible that a resistant cell may mutate back into a
sensitive cell. This phenomenon is well-documented in the medical literature where
experiments have shown that the resistant cell population decreases in a drug free
medium [24]. We therefore include a probability r ≥ 0 that one of the daughters of a
resistant cell may become sensitive. Generally r will be small. The case r = 0 where
this is excluded is called stable gene amplification while unstable gene amplification
refers to the phenomenon r > 0. If we denote the inverses of the transit times
of cells through the sensitive and resistant compartments by a and c, respectively,
then the uncontrolled dynamics therefore takes the form

Ṡ = −aS + (2 − q)aS + rcR, (1)

Ṙ = −cR + (2 − r)cR + qaS. (2)

Here the first terms on the right hand sides account for the deaths of the mother
cells, the second terms describe the return flows into the compartments, and the
third terms give the cross-over flows. We now assume that a cytotoxic agent kills
sensitive cells, but has no effect on the resistant population. Let u denote the drug
dose, 0 ≤ u ≤ umax ≤ 1, with u = 0 corresponding to no drug being used and
u = umax corresponding to a full dose. For simplicity here it is assumed that the
dosage, the concentration and the effect of the drug are equal, i.e. pharmacokinetics
(PK) or pharmacodynamics (PD) are not modelled. It is assumed that the drug
kills a fixed proportion u of the outflow of the sensitive cells at time t, aS(t), and
therefore only the remaining fraction (1−u)aS(t) of cells undergoes cell division. Of
these new cells then (2−q)(1−u)aS(t) remain sensitive, while a fraction q(1−u)aS(t)
mutates to resistant cells. It is assumed that the drug has no effect on resistant
 DRUG RESISTANCE IN CANCER CHEMOTHERAPY 5

cells. Thus overall the controlled dynamics can be represented as

Ṡ = −aS + (1 − u)(2 − q)aS + rcR, (3)
Ṙ = −cR + (2 − r)cR + (1 − u)qaS, (4)
and the initial condition is given by positive numbers S0 and R0 at time t = 0.
Setting N = (S, R)T , the dynamics is described by a bilinear system
Ṅ = (A + uB)N (5)
where
   
(1 − q)a rc q−2 0
A= and B=a . (6)
qa (1 − r)c −q 0
We first show that the system has a well-defined response for any admissible control.
Proposition 1. For any Lebesgue measurable control u : [0, ∞) → [0, umax ] the
solution N (·) = (S(·), R(·))T exists on [0, ∞) and all its components are positive.
Proof. For any control u defined on [0, ∞) this is a linear system with bounded
coefficients and thus solutions exist over [0, ∞). Define τ as the supremum over all
times η such that both S and R are positive on [0, η],
τ = sup{η ≥ 0 : S(t) > 0 and R(t) > 0 for 0 ≤ t ≤ η}.
If τ were finite, one of S or R must vanish at τ while both are positive on [0, τ ). The
stated invariance property directly follows from the following well-known compari-
son lemma for 1-dimensional linear ODE’s: Suppose α and β are bounded Lebesgue-
measurable functions on R and consider the ODE µ̇ = αµ + β. If µ(t0 ) > 0 and
β(t) ≥ 0 for t ≥ t0 , then µ(t) > 0 for all t ≥ t0 . This is obvious from the represen-
tation
Z t  Z t Z t0  
µ(t) = exp α(s)ds µ(t0 ) + exp α(r)dr β(s)ds .
t0 t0 s
Hence neither S nor R can vanish first. Contradiction.
2.3. Asymptotic Properties. In no control is applied, i.e. for u ≡ 0, the dynamics
is a simple model of exponential growth. Roughly, sensitive cells grow at the rate a
and resistant cells grow at rate c, or, more precisely,
Ṙ + Ṡ = aS + cR. (7)
S
The quotient x = R satisfies a Riccati equation of the form
ẋ = rc + ((1 − q)a − (1 − r)c) x − qax2 (8)
with positive initial condition x(0) > 0. Clearly the interval [0, ∞) is invariant for
(8) and this equation has a unique stable equilibrium in this interval at
q
2
(1 − q)a − (1 − r)c + ((1 − q)a − (1 − r)c) + 4rqac
x̄ = > 0. (9)
2rc
Thus, left alone, cancer cells grow exponentially reaching the relative proportions
S = x̄R.
If full control is applied throughout, i.e. u ≡ 1 on [0, ∞), the sensitive cells can
be reduced with rate a, while the resistant compartment still will grow with rate c.
Again, more precisely now
Ṙ + Ṡ = −aS + cR. (10)
6 URSZULA LEDZEWICZ AND HEINZ SCHÄTTLER

Thus, in principle the cancer cells can be reduced through chemotherapy provided
aS > cR, the typical situation initially. However, since applying drugs diminishes
only the sensitive population the resistant population eventually takes over and the
total number of cancer cells then will still grow exponentially. Now the quotient
S
x= R satisfies a linear ODE,
ẋ = rc + (a + (1 − r)c) x,
with stable equilibrium at
rc
x̄ = . (11)
a + (1 − r)c
This value is small (since r is), even 0 in case of no gene de-amplification (r = 0).
Thus drug resistance takes over in the model, no matter what, and this is consistent
with medical experience. Clearly, in a specific case the actual parameters may be
favorable and it may take a very long time. These precisely are the situations when
chemotherapy will be successful.

2.4. The role of the objective in an optimal control formulation. In general

therefore the problem becomes to delay the onset of the time when drugs are no
longer effective. We consider this problem now as an optimal control problem.
Obviously there exist many, and non-equivalent, ways how to formulate this as a
mathematical model. In an optimal control problem, the general structure has the
form to minimize (or maximize) some objective given in the form
Z T
J= L(N, u)dt + ϕ(T, N (T )) (12)
0

over all admissible controls u : [0, T ] → [0, umax ] subject to the dynamics (5) and
additional constraints that may be imposed. For example, the objective might be
chosen as to maximize the total time T while restricting the overall amount of drug
RT
given, 0 u(t)dt ≤ A, and requiring not to violate an upper bound on the number
of cancer cells, S(t) + R(t) ≤ N̄ . State space constraints, besides leading to a much
more difficult problem mathematically, have the disadvantage that in principle they
require to monitor the number of cancer cells constantly which is not feasible. It
therefore is more practical, and easier to handle mathematically, to formulate the
constraints on drugs and cancer cells implicitly by including these terms in the
objective. Since chemotherapy is normally given over some specified time period,
it also seems reasonable to minimize an objective of the type (12) over a fixed time
interval [0, T ]. The objective of treatment is to kill as many of the sensitive cancer
cells possible while limiting both the size of the resistant subpopulation and the
overall toxicity to the patient. Mathematically we therefore for a moment consider
the general problem
RT
(P): minimize J = 0 L(N, u)dt + ϕ(N (T )) over all Lebesgue measurable con-
trols u : [0, T ] → [0, umax ] subject to the dynamics Ṅ = (A + uB)N with
N (0) = N0 given and with positive components.
Necessary conditions for optimality of a control u∗ : [0, T ] → [0, umax ] are given
by the Pontryagin Maximum Principle [22]. It is easily seen that for our case the ab-
normal situation is not possible and hence these conditions can be stated as follows:
if u∗ is an optimal control with corresponding trajectory N∗ : [0, T ] → P = {(S, R) :
S > 0, R > 0}, then it follows that there exists an absolutely continuous function λ,
 DRUG RESISTANCE IN CANCER CHEMOTHERAPY 7

which we write as row-vector, λ = (λ0 , λ1 ) : [0, T ] → (R2 )∗ , satisfying the adjoint

equation
∂L ∂ϕ
λ̇ = −λ(A + u∗ B) − (N∗ , u∗ ), λ(T ) = (N∗ (T )), (13)
∂N ∂N
such that the optimal control u∗ minimizes the Hamiltonian H,
H = L(N, u) + λ(A + uB)N, (14)
over the control set [0, umax ] along (λ(t), N∗ (t)).
If the partial derivatives of the Lagrangian L and penalty function ϕ along the
optimal trajectory and control are non-negative, then analogously to the positive
invariance of the states under the dynamics, it follows that the adjoint space P∗ =
{(λ0 , λ1 ) : λ0 > 0, λ1 > 0} is negatively invariant under the adjoint flow.
Proposition 2. For any Lebesgue measurable control u : [0, T ] → [0, umax ] the
∂L
solution λ(·) to (18) exists on [0, T ]. If all partial derivatives ∂Ni
(N∗ (t), u∗ (t)) are
∂ϕ
non-negative on [0, T ] and if the partial derivatives ∂Ni (N∗ (T )) are positive, then
λ1 and λ2 are both positive.
Proof. As solution to a linear ODE the adjoint variable exists over the full interval.
Let σ denote the infimum over all times η such that both λ0 and λ1 are positive on
[η, T ],
σ = inf{η ≤ T : λ0 (t) > 0 and λ1 (t) > 0 for η ≤ t ≤ T }.
The proof is exactly as for Proposition 1, but now using the reverse comparison:
Suppose α and β are bounded Lebesgue-measurable functions on R and consider
the ODE µ̇ = αµ + β. If µ(t1 ) > 0 and β(t) ≤ 0 for t ≤ t1 , then µ(t) > 0 for all
t ≤ t1 .
Corollary 1. Under the assumptions of Proposition 2 both the states R and S and
the costates λ0 and λ1 are positive over [0, T ]. 
The form of the Lagrangian L and the penalty function ϕ will largely determine
the structure of optimal controls. There exists no clear biological indication for
the choice of the mathematical objective and here some degree of freedom can be
exercised with really no clear favorite. Many researchers elect to use objectives
which are quadratic in the control, like, for example,
L(N, u) = N T QN + u2 and ϕ(N ) = N T QT N, (15)
with positive semi-definite matrices Q and QT . (If the entries of Q and QT are
chosen non-negative, then the conditions in Proposition 2 are satisfied.) This choice
has the advantage that it generates a strictly convex Hamiltonian function H which
has a unique minimum. However, this minimum is given as a function of N and λ,
u = u(λ, N ), i.e. is dependent on the multiplier. Thus a two-point boundary value
problem given by the dynamics (5) and the adjoint equation (13) needs to be solved
with u = u(λ, N ) given as the solution of the minimization problem (14). Essentially
the problem then is one of numerical analysis. Because of the bilinear dynamics
this leads to nonstandard nonlinear equations and questions related to the existence
of multiple solutions to the two-point boundary value problem can complicate the
structure of solutions. At least they raise the issue of optimality of any numerically
found solution. In [19] we have considered a general class of problems which also
matches these quadratic models and we have formulated sufficient conditions for
optimality which allow to verify at least strong local minimality of such solutions.
8 URSZULA LEDZEWICZ AND HEINZ SCHÄTTLER

From a practical point of view, choosing an objective that is quadratic in the control
tends to undermine the side effects and it favors giving partial doses. Typically
solutions will have segments when the control is given by the stationary point of
the Hamiltonian (whenever this minimum lies in the interior of the control set)
implying the use of time-varying partial doses dependent on the number of cancer
cells at the moment. Controls of this type are not yet realistically medically.
An alternative way, and this is the one we pursue in this paper, is the use of a
Lagrangian function L which is linear in the control u. There exists some biological
justification for this if one equates the numbers of cells killed with the numbers of
ineffective cell divisions (and, at least for some range, this number can reasonably be
assumed to be proportional to the overall amount of drugs given). Mathematically,
we thus consider the problem to minimize an objective of the form
Z T
J = kN (T ) + (`N (t) + u(t)) dt → min (16)
0
where k and ` are row-vectors of weights with the components of k positive and
those of ` non-negative. The penalty term kN (T ) represents a weighted average of
the total number of cancer cells at the end of an assumed fixed therapy interval [0, T ]
and the term `N (t) models cumulative effects during the therapy. The control term
u(t) in the Lagrangian models the negative side effects of the drugs, measured in the
L1 norm. The parameters k and ` can also be used to put a stronger emphasis on
the number of cancer cells since it can be argued that a duplication of the cancer or
the drug dose is more hazardous than a duplication of the objective would represent.
For the methods used below it is important that the Hamiltonian is linear in the
control u, but its structure in N can be rather arbitrary, i.e. more generally we
could consider
Z T
J = ϕ(N (T )) + (L(N (t)) + u(t)) dt → min (17)
0
with smooth penalty functions ϕ and Lagrangian L depending on the state N if such
an effect is considered important. In this paper, however, we use linear functions.
The mathematical problem therefore becomes to find a Lebesgue-measurable func-
tion u : [0, T ] → [0, umax ] which minimizes (16) subject to the dynamical equations
(3) and (4).
2.5. Analysis of optimal controls. Necessary conditions for optimality of a con-
trol u∗ : [0, T ] → [0, umax ] are given by the Pontryagin Maximum Principle [22].
As mentioned already, the abnormal situation is not possible and hence these
conditions can be stated as follows: if u∗ is an optimal control with correspond-
ing trajectory N∗ : [0, T ] → P = {(S, R) : S > 0, R > 0}, then it follows that
there exists an absolutely continuous function λ, which we write as row-vector,
λ = (λ0 , λ1 ) : [0, T ] → (R2 )∗ , satisfying the adjoint equation
λ̇ = −λ(A + u∗ B) − `, λ(T ) = k, (18)
such that the optimal control u∗ minimizes the Hamiltonian H,
H = `N + u + λ(A + u∗ B)N, (19)
over the control set [0, umax ] along (λ(t), N∗ (t)). Since the Hamiltonian is linear in
u and the control set is an interval, defining the switching function Φ by
∂H
Φ(t) = = 1 + λ(t)BN (t) (20)
∂u
 DRUG RESISTANCE IN CANCER CHEMOTHERAPY 9

we therefore have that


0 if Φ(t) > 0
u∗ (t) = . (21)
umax if Φ(t) < 0
A priori the control is not determined by the minimum condition at times where
Φ(t) = 0. However, if Φ(t) vanishes on an open interval, also all its derivatives must
vanish and this for our problem will determine the control. Controls of this kind are
called singular while we refer to piecewise constant controls as bang-bang controls.
Optimal controls then need to be synthesized from these and other, possibly more
complicated, candidates. For this purpose we need to compute the derivatives of
the switching functions using the system and adjoint equations. A simple direct
verification proves the following basic formula:
Lemma 1. Suppose G is a constant matrix and let Ψ(t) = λ(t)GN (t), where N
is a solution to the system equation (5) corresponding to the control u and λ is a
solution to the corresponding adjoint equation (45). Then
Ψ̇(t) = λ(t)[A + uB, G]N (t) − `GN (t), (22)
where [F, G] denotes the commutator of the matrices F and G defined as [F, G] =
GF − F G.
Differentiating the switching function Φ(t) = 1 + λ(t)BN (t) twice gives
Φ̇(t) = λ(t)[A, B]N (t) − `BN (t), (23)
Φ̈(t) = λ(t)[A + u(t)B, [A, B]]N (t) − `[A, B]N (t) − `B(A + uB)N (t)
= {λ(t)[A, [A, B]]N (t) − `[A, B]N (t) − `BAN } (24)
2
+ u{λ(t)[B, [A, B]]N (t) − `B N (t)}.
If the control u is singular on an open interval I, then we have
0 ≡ Φ(t) ≡ Φ̇(t) ≡ Φ̈(t) (25)
and formally the control can be computed as
λ(t)[A, [A, B]]N (t) − `[A, B]N (t) − `BAN (t)
usin (t) = − (26)
λ(t)[B, [A, B]]N (t) − `B 2 N (t)
provided the denominator doesn’t vanish. Whether the singular control will be
admissible, i.e. whether it will obey the control bounds 0 ≤ u ≤ umax , will depend
on the parameter values, but also on the region in the state space where the state
(S, R)T lies and cannot be answered in general. In this case the singular control is
called of order 1 and it is then a necessary condition for optimality of usin , the so-
called generalized Legendre-Clebsch condition [15], that this denominator actually
is negative, i.e.
∂ d2 ∂H
< 0. (27)
∂u dt2 ∂u
For this problem we have
B 2 = (q − 2)aB (28)
and therefore it follows from Φ̇(t) ≡ 0 that
`B 2 N (t) = (q − 2)a`BN (t) = (q − 2)aλ(t)[A, B]N (t). (29)
Hence
∂ d2 ∂H
= λ ([B, [A, B]] + a(2 − q)[A, B]) N. (30)
∂u dt2 ∂u
10 URSZULA LEDZEWICZ AND HEINZ SCHÄTTLER

A straightforward, but somewhat tedious computation shows that

q(2 − q) −(2 − q)2
 
[B, [A, B]] + a(2 − q)[A, B] = 2a2 rc = 2a2 rcww⊥ (31)
q2 −q(2 − q)
where
 
2−q
w= and w⊥ = (q, q − 2). (32)
q
Thus we get
∂ d2 ∂H
= 2a2 rcλww⊥ N
∂u dt2 ∂u
= 2a2 rc {(2 − q)λ0 + qλ1 } (qS − (2 − q)R) . (33)

Since (2 − q)λ0 + qλ1 > 0 the generalized Legendre-Clebsch condition therefore

implies:

Proposition 3. Singular controls are not optimal in regions of the state space where
qS > (2 − q)R.

This holds as long as the resistant population is very small and then optimal
controls will be bang-bang, i.e. correspond to sessions of full dose chemotherapy
with rest periods interlaced. However, as the portion of resistant cells R increases,
the Legendre-Clebsch condition will be satisfied and if admissible, there are mathe-
matical reasons to suspect that it indeed will be the optimal control in this case. It
is also quite intuitive that a full dose may do more harm than good once resistance
builds up and thus the optimal strategy may switch to give partial doses, i.e. use
singular controls.

3. Mathematical Models for Cancer Chemotherapy with Multiple Killing

Agents under Evolving Drug Resistance. Medical experience shows that over
time cancer cells will develop resistance to the killing drug until treatment no longer
will be effective, in fact even detrimental. For the model above, even with constant
maximum dose (which of course is unrealistic) the dynamics for the resistant group
is growing exponentially, Ṙ ≥ (1 − r)cR, and resistance will eventually take over.
The best one can therefore hope for is to keep the resistant population small as
long as feasible. It has been noted in clinical experiments that in the absence of
the drug cancer cells can lose acquired drug resistance through gene deamplification
[2, 10]. However, drug free sessions allow for unrestricted growth of the tumor and
it is important to inhibit this process of “repopulation” [31]. Thus an important
therapy strategy is to use combinations of drugs in order to delay developing ac-
quired drug resistance. We are assuming that the drugs used are different in their
operating mechanism, hence acquired drug resistance is due to different molecular
mechanisms. Otherwise there is no need to distinguish between them and they can
be grouped together. Mathematically this leads to a structurally quite different
model which we here consider for the case of two killing agents. Due to the drugs’
interactions the dynamics now will be quadratic in the controls, but with an in-
definite structure, and the corresponding Hamiltonian needs to be minimized over
a compact control set. These are non-standard, fully nonlinear problems which to
the best of our knowledge have not been considered before in this form.
 DRUG RESISTANCE IN CANCER CHEMOTHERAPY 11

3.1. Modeling Aspects. We consider two cytostatic killing agents whose dosages
are labelled u1 and u2 , both with values in intervals [0, uimax ], i = 1, 2. (As before,
the value 0 represents “no dose” and the value uimax corresponds to a “maximum
dose”). The state space now is comprised of four compartments, a compartment S
of cells sensitive to both drugs, a compartment L1 of cells sensitive to drug u1 , but
resistant to drug u2 , a compartment L2 of cells sensitive to drug u2 , but resistant
to drug u1 , and a compartment R of cells resistant to both drugs. We denote
the average numbers of cells in these compartments by the corresponding capital
Roman letters.

2-q1-q2
q1 q2
S
r2 r1
2-s1-r2 L1 L2 2-r1-s2
r1 r2

s1 R s2
2-r1-r2

Figure 1. Drug resistance with two killing agents

As above, it is assumed that the drugs kill a fixed proportion u1 respectively

u2 of the outflow of the sensitive cells at time t and therefore only the remaining
fraction of cells undergoes cell division. If we denote the mean inverse transit times
through the compartments by a, b1 , b2 and c respectively, then, for example, and
exactly as above, only a fraction
(1 − u1 )(2 − s1 − r2 )b1 L1 (34)
of cells from L1 reenters L1 . Here s1 is the probability that the second of the two
daughter cells becomes resistant also to the second drug u2 , i.e. enters R, and r2
gives the probability of gene deamplification to go from L1 into S, i.e. the drug
resistance to the second drug u2 is removed or lost. As above it is assumed that one
of the two daughter cells will reenter L1 (see Fig. 1). However, the terms involving
cross-over flows with the sensitive compartment change considerably simply since
the two drugs cannot kill the same cell twice. Since the drugs interact with large
numbers of cancer cells, it is reasonable to assume that the drugs act independently.
Other dependency relations can be postulated, but this will change the return flows
to the sensitive compartment. We limit ourselves to making this independence
assumption. In this case the return flow is given by
(1 − u1 )(1 − u2 )(2 − q1 − q2 )aS (35)
12 URSZULA LEDZEWICZ AND HEINZ SCHÄTTLER

and thus becomes quadratic in the controls. In many probabilistic models (e.g.
[4]) in order to simplify the analysis similar quadratic terms are linearized with the
reasoning that the probabilities involved are small. But for this model the validity
of such an argument is questionable and is not needed. Overall the dynamics we
consider is therefore given as follows:
Ṡ = −aS + (1 − u1 )(1 − u2 )(2 − q1 − q2 )aS + (1 − u1 )r2 b1 L1 + (1 − u2 )r1 b2 L2 ,
(36)
L̇1 = −b1 L1 + (1 − u1 )(2 − s1 − r2 )b1 L1 + (1 − u1 )(1 − u2 )q1 aS + r1 cR, (37)
L̇2 = −b2 L2 + (1 − u2 )(2 − s2 − r1 )b2 L2 + (1 − u1 )(1 − u2 )q2 aS + r2 cR, (38)
Ṙ = −cR + (2 − r1 − r2 )cR + (1 − u1 )s1 b1 L1 + (1 − u2 )s2 b2 L2 . (39)
Proposition 4. For all Lebesgue measurable controls u = (u1 , u2 ) : [0, ∞) →
[0, u1max ] × [0, u2max ] the solution N (·) = (S(·), L1 (·), L2 (·), R(·))T exists on [0, ∞)
and its components are positive.
Proof. As in Proposition 1 a contradiction arises if we assume there is a finite first
time any of the components would vanish.

The dynamical equations become more transparent if we change the control vari-
ables to vi = 1 − ui . Then the dynamical equations can be written in the form
Ṅ = (A + v1 B1 + v2 B2 + v1 v2 C)N (40)
where
   
−a 0 0 0 2 − q1 − q2 0 0 0
 0 −b1 0 r1 c   q1 0 0 0 
A=
 0
, C = a ,
0 −b2 r2 c   q2 0 0 0 
0 0 0 (1 − r1 − r2 )c 0 0 0 0
(41)
   
0 r2 0 0 0 0 r1 0
 0 2 − s1 − r2 0 0   0 0 0 0 
B1 = b1 
 0
 , B2 = b2   . (42)
0 0 0   0 0 2 − s2 − r1 0 
0 s1 0 0 0 0 s2 0
As above, the optimal control problem becomes to choose Lebesgue measurable
i i
functions vi : [0, T ] → [vmin , 1], vmin = 1 − uimax , i = 1, 2, to minimize an objective
of the form Z T
J = kN (T ) + `N (t) − mv(t)dt → min (43)
0
subject to the dynamics (40). Here m = (m1 , m2 ) also is a row-vector of positive
weights.

3.2. Analysis of optimal controls. It is again easily seen that optimal controls
are normal. Thus, if v = (v1∗ , v2∗ ) is an optimal control with corresponding trajectory
N∗ = (S∗ , L∗1 , L∗2 , R∗ ) : [0, T ] → P = {(S, L1 , L2 , R) : S > 0, L1 > 0, L2 > 0, R > 0},
(44)
then it follows that there exists an absolutely continuous function λ, which we write
as row-vector, λ = (λ0 , λ1 , λ2 , λ3 ) : [0, T ] → (R4 )∗ , satisfying the adjoint equation
λ̇ = −λ(A + v1∗ B1 + v2∗ B2 + v1∗ v2∗ C) − `, λ(T ) = k, (45)
 DRUG RESISTANCE IN CANCER CHEMOTHERAPY 13

such that the optimal control v∗ minimizes the Hamiltonian H,

H = `N − mv1∗ − mv2∗ + λ(A + v1∗ B1 + v2∗ B2 + v1∗ v2∗ C)N, (46)
1 2
over the control set V = [vmin , 1]×[vmin , 1] along (λ(t), N∗ (t)). For sake of a simpler
1 2
notation we henceforth take vmin = vmin = 0. The qualitative results do not change.
Analogously to the positive invariance of P under the dynamics, it follows that
the adjoint space P∗ = {(λ0 , λ1 , λ2 , λ3 ) : λi > 0 for i = 0, 1, 2, 3} is negatively
invariant under the adjoint flow.
Proposition 5. For any Lebesgue measurable control v : (−∞, T ] → V the solution
λ(·) to (45) exists on (−∞, T ] and all its components are positive.

Proof. The adjoint equation is of the form λ̇ = −` − λM with M is matrix with

only non-negative entries and λ(T ) = k has positive entries. Again let σ denote the
infimum over all times η such that all components of λ are positive on [η, T ],
σ = inf{η ≤ T : λi (t) > 0 for i = 0, . . . , 4, and η ≤ t ≤ T }.
But if ι denotes an index where λι (τ ) = 0, then we have λ̇ι (τ ) < 0. Contradiction.

Corollary 2. All states R, L1 , L2 and S and all costates λi , i = 0, 1, 2, 3 are

positive over [0, T ]. 
We now analyze properties of optimal controls. Define functions
Θ(t) = λ(t)CN (t), Ξ(t) = `N (t) + λ(t)AN (t), (47)
Ψ1 (t) = −m1 + λ(t)B1 N (t), Ψ2 (t) = −m2 + λ(t)B2 N (t), (48)
so that the Hamiltonian takes the form
H = Θ(t)v1 v2 + Ψ1 (t)v1 + Ψ2 (t)v2 + Ξ(t). (49)
By Corollary 2 we have that
Θ(t) = {(2 − q1 − q2 )λ0 (t) + q1 λ1 (t) + q2 λ2 (t)} aS(t) > 0 (50)
and thus the minimization of the Hamiltonian is equivalent to minimize
Ĥ = v1 v2 + Ψ̂1 (t)v1 + Ψ̂2 (t)v2 (51)
over V = [0, 1] × [0, 1] where
Ψi (t)
Ψ̂i (t) = , i = 1, 2. (52)
Θ(t)
Define switching curves C1 and C2 in (Ψ̂1 , Ψ̂2 )-space as the graphs of functions
C1 : Ψ̂1 = Γ(Ψ̂2 ) and C2 : Ψ̂2 = Γ(Ψ̂1 ) where

 −1 if x ≤ −1
Γ(x) = x if − 1 ≤ x ≤ 0 . (53)
0 if x ≥ 0


These two curves divide the (Ψ̂1 , Ψ̂2 )-space into four sectors as shown in Fig. 2
and the optimal control is given by one of the vertices of V as the vector (Ψ̂1 (t), Ψ̂2 (t))
lies in one of these regions. More precisely, we have:
14 URSZULA LEDZEWICZ AND HEINZ SCHÄTTLER

IV I

V=(1,0) V=(0,0)

II
III
V=(0,1)
V=(1,1)

Figure 2. Switching curves for optimal controls

Proposition 6. If v∗ = (v1∗ , v2∗ ) : [0, T ] → [0, 1] × [0, 1] is an optimal control, then



 (0, 0) if Ψ̂1 (t) > 0 and Ψ̂2 (t) > 0;
(1, 0) if Ψ̂1 (t) < 0, Ψ̂1 (t) < Ψ̂2 (t), and Ψ̂2 (t) > −1;

v∗ (t) = . (54)

 (0, 1) if Ψ̂2 (t) < 0, Ψ̂1 (t) > Ψ̂2 (t), and Ψ̂1 (t) > −1;

(1, 1) if Ψ̂1 (t) < −1 and Ψ̂2 (t) < −1.
On the boundary segments between these regions we have that
 
∗ 0 if Ψ̂1 (t) < Γ(Ψ̂2 (t)) ∗ 0 if Ψ̂2 (t) > Γ(Ψ̂1 (t))
v1 (t) = and v2 (t) = .
1 if Ψ̂1 (t) > Γ(Ψ̂2 (t)) 1 if Ψ̂2 (t) < Γ(Ψ̂2 (t))
(55)
If Ψ̂1 (t) = Ψ̂2 (t) ∈ (−1, 0), then the control is either v∗ (t) = (1, 0) or v∗ (t) = (0, 1).
Proof. The minimization problem is equivalent to minimizing the indefinite qua-
dratic form
(v1 + Ψ̂2 (t))(v2 + Ψ̂1 (t)) (56)
over [0, 1] × [0, 1] and the minimum is always attained at one of the vertices. The
values of Ĥ at the vertices are 0 at (0, 0), Ψ̂1 (t) at (1, 0), Ψ̂2 (t) at (0, 1) and 1 +
Ψ̂1 (t) + Ψ̂2 (t) at (1, 1). Thus the minimum value ρ is given by


 0 if Ψ̂1 (t) > 0 and Ψ̂2 (t) > 0;
Ψ̂1 (t) if Ψ̂1 (t) < 0, Ψ̂1 (t) < Ψ̂2 (t), and Ψ̂2 (t) > −1;

ρ=

 Ψ̂2 (t) if Ψ̂2 (t) < 0, Ψ̂1 (t) > Ψ̂2 (t), and Ψ̂1 (t) > −1;

1 + Ψ̂1 (t) + Ψ̂2 (t) if Ψ̂1 (t) < −1 and Ψ̂2 (t) < −1.
(57)
These inequalities define the open regions in (55). As the vector (Ψ̂1 (t), Ψ̂2 (t))
crosses one of the following four boundary segments,
∂1 = {(Ψ̂1 (t), Ψ̂2 (t)) : Ψ̂1 (t) < −1, Ψ̂2 (t) = −1}, (58)
∂2 = {(Ψ̂1 (t), Ψ̂2 (t)) : Ψ̂1 (t) = −1, Ψ̂2 (t) > −1}, (59)
∂3 = {(Ψ̂1 (t), Ψ̂2 (t)) : Ψ̂1 (t) > 0, Ψ̂2 (t) = 0}, (60)
and
∂4 = {(Ψ̂1 (t), Ψ̂2 (t)) : Ψ̂1 (t) = 0, Ψ̂2 (t) > 0}, (61)
 DRUG RESISTANCE IN CANCER CHEMOTHERAPY 15

the control v1 remains constant as ∂1 or ∂3 are crossed and v2 remains constant as

∂2 or ∂4 are crossed. Along the fifth boundary piece
∂5 = {(Ψ̂1 (t), Ψ̂2 (t)) : Ψ̂1 (t) = Ψ̂2 (t), −1 < Ψ̂1 (t) < 0}, (62)
both v∗ = (1, 0) or v∗ = (0, 1) are minimizing, but no other control is.
Proposition 6 completely determines the controls in the open regions in Fig.
2. It does not, however, exclude more degenerate cases corresponding to singular
controls. For example, if (Ψ̂1 (t), Ψ̂2 (t)) ∈ ∂1 , then the minimizing control satisfies
v1∗ = 1, but no information about v2 follows from the minimization condition and
in principle this control could be arbitrary. If (Ψ̂1 (t), Ψ̂2 (t)) ∈ ∂1 for t in some
open interval I, then v1∗ ≡ 1 on I and, as before, a possible singular control v2∗ is
determined by differentiation the defining relation, (that is Ψ2 (t) + Θ(t) ≡ 0 in case
of ∂1 ,) twice with respect to t and then solving for v2 . If this control is admissible,
it again needs to satisfy the generalized Legendre-Clebsch condition (27) in order
to be optimal. In principle this analysis needs to be carried out for all boundary
pieces ∂i , i = 1, . . . , 5. However, the main candidates will be bang-bang controls as
the vector (Ψ̂1 (t), Ψ̂2 (t)) crosses these boundary segments transversally.
One important qualitative feature of the structure of optimal controls which
follows from the geometry of the switching curves in Fig. 2 is that it is not optimal
to directly switch between the controls v∗ = (0, 0) and v∗ = (1, 1). Thus it is never
optimal to simultaneously withdraw or start administering both drugs at the same
time. This, of course, does not exclude that both drugs may be given at the same
time - and it may be beneficial to do so if the benefit of the combined cell kill
outweighs the side effects. But drugs should be initiated respectively withdrawn
one at a time. Intuitively this makes sense: in the absence of one drug one would
like to administer the other in order to prevent a strong repopulation and hope
that at the same time the resistance with respect to the first drug will decline in
its absence. Unfortunately, over time the cumulative toxic side effects will build up
and this may necessitate to withdraw them both. In fact, in all the simulations we
ran, the compartment R of cells resistant to both drugs takes over even from the
tiniest fraction of initial data and eventually forces the shut down of both drugs as
the “optimal” solution.

4. Simulations. We include some simulations for the multi-drug model. The nu-
merical values chosen are just for illustrative purposes and are not based on medical
data. The data for the objective are the same for all runs and we simply picked all
parameters arising in the objective as 1, i.e. k = ` = (1, 1, 1, 1) and m = (1, 1). Also,
for simplicity we simply take u1max = u2max = 1. As transition probabilities in the
dynamics we chose q1 = .02, q2 = .02, r1 = .005, r2 = .01, s1 = .02, and s2 = .02.
In the data for the cell-cycle parameters we fix a = .4, b1 = .35, and b2 = .35,
but for some of the runs we vary the transit time through the compartment R of
cells that are resistant to both drugs. Clearly these parameters strongly influence
the structure of controls and here we only compare briefly runs for c = .3 and for
c = .8. The value c = .3 corresponds to a situation when the transit times through
the resistant compartments are somewhat slower than for the sensitive compartment
(tumors that are responding to treatment) while c = .8 means that these doubly
resistant cells duplicate on average twice as fast, a situation more reminiscent of
malignant situations. In the simulations below the time horizon is always taken as
the interval [0, 10].
16 URSZULA LEDZEWICZ AND HEINZ SCHÄTTLER

We compare an extremal control (i.e. one that satisfies the necessary conditions
for optimality) with two reasonable ad hoc choices as reference controls. Reference
control 1 applies both drugs simultaneously at full dose over the intervals [0, 2],
[4, 6] and [8, 10] with rest periods in between while reference control 2 alternates
the drugs over these intervals with drug 1 given on [0, 2], [4, 6] and [8, 10] and drug
2 given on [2, 4] and [6, 8] so that there is no rest-period over the therapy interval.
Figs. 3 and 4 compare the response of the system for an initial condition given by
S(0) = .90, L1 (0) = .05, L2 (0) = .05, and R = 0 for the case when c = .3. In all
figures the solid graph represents the response of the sensitive cells S, the dashed
curves give the responses of L1 and L2 and the dash-dot curve gives the response
of the fully resistant compartment R. Note that the curves for L1 and L2 in Fig.
3 overlay because of the symmetries in the data. In this case it is evident that
alternating the drugs is the better strategy. The reason simply lies in the fact that
resistance is still very small and does not yet build up significantly over the therapy
interval. Side effects have not yet become an issue for this initial condition.
0.9

0.8

0.7

0.6
states S, L1, L2 and R

0.5

0.4

0.3

0.2

0.1

0
0 1 2 3 4 5 6 7 8 9 10

Figure 3. Response to reference control 1 for c = .3

0.9

0.8
states S, L1, L2 and R with alternating controls

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0
0 1 2 3 4 5 6 7 8 9 10

Figure 4. Response to reference control 2 for c = .3

On the other end of the spectrum of possibilities, Figs. 5 and 6 compare the
response of the system for the same initial condition (S(0) = .90, L1 (0) = .05,
 DRUG RESISTANCE IN CANCER CHEMOTHERAPY 17

L2 (0) = .05, R = 0,) but now for the case when c = .8. In this case the strategy of
alternating the drugs is drastically inferior. Comparing the total number of cancer
cells that had been normalized to 1 at the initial time, reference control 1 leads to
a reduction in the total number of cancer cells to 0.7348, but the fully resistant
population R has grown to R = .2826, more than one third of the overall number
of cancer cells. For the alternating strategy 2 the total number of cancer cells
actually increases to 1.4957 and the fully resistant portion makes up 1.3467 of it, a
horrendous outcome. The reason is that resistance, once established at any ever so
small proportion, rapidly takes over. Hence the more drug can be applied initially,
the better it seems to be. Fig. 8 gives the response to an extremal control that
was computed by backward integration from the same terminal condition that was
generated with the now better reference control 1. The corresponding control is
identical v2∗ = 1 over [0, 10] (i.e. the second drug is NOT used) and the first control
v1∗ switches from v = 0 to v = 1 at τ = 6.11. In this case, with the same number of
cancer cells at the end of the therapy interval, initially the number of cancer cells
is 1.0216; thus there is a comparable response in reducing the cancer cells, but it is
achieved with roughly half the dose since only the first drug is given.

0.9

0.8

0.7

0.6
states S, L1, L2 and R

0.5

0.4

0.3

0.2

0.1

0
0 1 2 3 4 5 6 7 8 9 10

Figure 5. Response to reference control 1 for c = .8

1.4

1.2
states S, L1, L2 and R with alternating controls

0.8

0.6

0.4

0.2

0
0 1 2 3 4 5 6 7 8 9 10

Figure 6. Response to reference control 2 for c = .8

18 URSZULA LEDZEWICZ AND HEINZ SCHÄTTLER

1.5

0.5

extremal control v1
0

−0.5

−1

−1.5
0 1 2 3 4 5 6 7 8 9 10
time

Figure 7. Extremal Control v1∗

0.9

0.8
states S, L1, L2 and R for extremal

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0
0 1 2 3 4 5 6 7 8 9 10

Figure 8. Response to extremal control

Figs. 9 and 10 compare the response of the system for a different initial condition
given by S(0) = .8, L1 (0) = 0, L2 (0) = .2, and R = 0, and again for c = .8. As
above the strategy of alternating the drugs is inferior. Comparing the total number
of cancer cells that had been normalized to 1 at the initial time, reference control
1 now only leads to a minimal reduction in the total number of cancer cells to
0.9883 with the fully resistant population R growing to R = .5309, more than
half of the overall number of cancer cells. The alternating strategy 2 is disastrous
with the total number of cancer cells multiplying more than five-fold to 5.3292 and
the doubly resistant portion making up 4.8843. The reason for the much shorter
lead time until resistance builds up of course is that we already have 20% resistant
cells, although only resistant to drug 1, at the beginning. Nevertheless, through the
transitions to R this causes a resistant population to develop faster and then quickly
to become dominant. Fig. 12 again gives the response to an extremal control that
was computed by backward integration from the same terminal condition that was
generated with the now better reference control 1. The corresponding control now
is identical v1∗ = 1 over [0, 10] (i.e. the first drug is NOT used consistent with the
existing resistance to drug 1 of some of the cells) and the second control v2∗ switches
from v = 0 to v = 1 at τ = 6.75. In this case, however, with the same number of
 DRUG RESISTANCE IN CANCER CHEMOTHERAPY 19

cancer cells at the end of the therapy interval, initially the number of cancer cells
is 1.7410 and thus giving a much better response in reducing the cancer cells which
again is achieved with roughly half the dose since only the second drug is given.

0.8

0.7

0.6

states S, L1, L2 and R

0.5

0.4

0.3

0.2

0.1

0
0 1 2 3 4 5 6 7 8 9 10

Figure 9. Response to reference control 1

4.5

4
states S, L1, L2 and R with alternating controls

3.5

2.5

1.5

0.5

0
0 1 2 3 4 5 6 7 8 9 10

Figure 10. Response to reference control 2

5. Conclusion. The emergence of resistant clones is a universal problem of chemo-

therapy and one of the main, although not the only obstacle to effective treatments.
The hope is that improvements in scheduling chemotherapy sessions may delay the
onset of drug resistance and thus give a higher life expectancy. In this paper we
presented a formulation and some preliminary analysis for two finite-dimensional
models for cancer chemotherapy taking into account drug resistance with respect
to single and multiple killing agents. For both systems we established invariance
properties for the states and adjoint variables which enables a further analysis of
the candidates for optimal protocols. Two different classes of controls represent-
ing such protocols result from the Maximum Principle: bang-bang controls, which
correspond to administering full doses with rest periods in between, and singular
controls representing time-varying partial doses. Conditions for optimality of sin-
gular controls in terms of the size of the resistant versus sensitive population are
20 URSZULA LEDZEWICZ AND HEINZ SCHÄTTLER

1.5

0.5

extremal control v2
0

−0.5

−1

−1.5
0 1 2 3 4 5 6 7 8 9 10
time

Figure 11. Extremal control v2

1.5
states S, L1, L2 and R for extremal

0.5

0
0 1 2 3 4 5 6 7 8 9 10

Figure 12. Response to extremal control

given for a single drug model. This condition indicates that if the resistant popula-
tion becomes too large, then bang-bang controls may no longer be optimal since in
this case the damage caused by a full dose to healthy cells outweighs the benefits
of killing the cancer cells. In this situation we cannot help the patient any more
with treatment with this drug and the natural choice become combination treat-
ments involving other drugs. An example of a model for such a treatment involving
two drugs is presented in the second part of the paper. Mathematically it has a
different structure than the previous model since the dynamics is quadratic in the
control. Our initial results show that in scheduling the therapy it is not optimal to
simultaneously withdraw or initiate the treatment of both drugs.

Acknowledgement. This research was supported by NSF grant DMS 0305965

and collaborative research grants DMS 0405827/0405848; U. Ledzewicz’s research
also was supported by SIUE Hoppe Research Scholar Award and Summer Research
Fellowship.
 DRUG RESISTANCE IN CANCER CHEMOTHERAPY 21

REFERENCES

[1] K. Alitalo, Amplification of cellular oncogenes in cancer cells, Trends Biochem. Sci., 10,
(1985), pp. 194-197
[2] P.C. Brown, S.M. Beverly and R.T. Schimke, Relationship of amplified dihydrofolate reduc-
tase genes to double minute chromosomes in unstably resistant mouse fibroblast cell lines,
Molecular Cell Biology, 1, (1981), pp. 1077 - 1083
[3] P. Calabresi and P.S. Schein, Medical Oncology, Basic Principles and Clinical Management
of Cancer, Mc Graw-Hill, New York, 1993
[4] A.J. Coldman and J.H. Goldie, A model for the resistance of tumor cells to cancer chemother-
apeutic agents, Mathematical Biosciences, 65, (1983), pp. 291-307
[5] M.I.S. Costa, J.L. Boldrini and R.C. Bassanezi, Drug kinetics and drug resistance in optimal
chemotherapy, Mathematical Biosciences, 125, (1995), pp. 191-209
[6] M. Eisen, Mathematical Models in Cell Biology and Cancer Chemotherapy, Lecture Notes in
Biomathematics, Vol. 30, Springer Verlag, (1979)
[7] L.E. Harnevo and Z. Agur, The dynamics of gene amplification described as a multitype
compartmental model and as a branching process, Mathematical Biosciences, 103, (1991),
pp. 115-138
[8] L.E. Harnevo and Z. Agur, Drug resistance as a dynamic process in a model for multistep
gene amplification under various levels of selection stringency, Cancer Chemotherapy and
Pharmacology, 30, (1992), pp. 469 - 476
[9] J.H. Goldie, Drug resistance in cancer: a perspective, Cancer and Metastasis Review, 20,
(2001), pp. 63-68
[10] R.J. Kaufman, P.C. Brown, and R.T. Schimke, Loss and stabilization of amplified dihydro-
folate reductase genes in mouse sarcoma S-180 cell lines, Molecular Cell Biology, 1, (1981),
pp. 1084 - 1093
[11] R.S. Kerbel, A cancer chemotherapy resistant to resistance, Nature, 390, (1997), pp. 335-336
[12] M. Kimmel and D.E. Axelrod, Branching Processes in Biology, Springer Verlag, New York,
NY, (2002)
[13] M. Kimmel and D.E. Axelrod, Mathematical models of gene amplification with applications
to cellular drug resistance and tumorigenicity, Genetics, 125, (1990), pp. 633-644
[14] M. Kimmel, A. Swierniak and A. Polanski, Infinite-dimensional model of evolution of drug re-
sistance of cancer cells, Journal of Mathematical Systems, Estimation and Control, 8, (1998),
pp. 1-16
[15] A. Krener, The high-order maximal principle and its application to singular controls, SIAM
J. Control and Optimization, 15, (1977), pp. 256-293
[16] L.A. Loeb, A mutator phenotype in cancer, Cancer Research, 61, (2001), pp. 3230-3239
[17] U. Ledzewicz and H. Schättler, Optimal bang-bang controls for a 2-compartment model in
cancer chemotherapy, Journal of Optimization Theory and Applications - JOTA, 114, (2002),
pp. 609-637
[18] U. Ledzewicz and H. Schättler, Analysis of a cell-cycle specific model for cancer chemotherapy,
J. of Biological Systems, 10, (2002), pp. 183-206
[19] U. Ledzewicz and H. Schättler, On optimal controls for a general mathematical model for
chemotherapy of HIV, Proceedings of the 2002 American Control Conference (ACC), An-
chorage, Alaska, (2002), pp. 3454-3459
[20] U. Ledzewicz and H. Schättler, Optimal control for a bilinear model with recruiting agent
in cancer chemotherapy, Proceedings of the 42nd IEEE Conference on Decision and Control
(CDC), Maui, Hawaii, December 2003, pp. 2762-2767
[21] U. Ledzewicz, H. Schättler and A. Swierniak, Finite-dimensional models of drug resistant and
phase specific cancer chemotherapy, J. of Medical Informatics and Technologies, 8, (2004),
pp. IP 5-13
[22] L.S. Pontryagin, V.G. Boltyanskii, R.V. Gamkrelidze and E.F. Mishchenko, The Mathematical
Theory of Optimal Processes, MacMillan, New York, (1964)
[23] K. Skomorovski, H. Harpak, A. Iovanovski, M. Vardi, T. Visser, S. Hartong, H. van Vliet,
G. Wagemaker and Z. Agur, New TPO treatment schedules of increased safety and efficacy:
pre-clinical validation of a thrombopoeisis simulation model, Brit. J. of Haematology, 123,
(No. 4), (2003), pp. 683ff
[24] R.T. Schimke, Gene amplification, drug resistance and cancer, Cancer Research, 44, (1984),
pp. 1735-1742
22 URSZULA LEDZEWICZ AND HEINZ SCHÄTTLER

[25] G.W. Swan, Role of optimal control in cancer chemotherapy, Math. Biosci., 101, (1990), pp.
237-284
[26] J. Smieja and A. Swierniak, Different models of chemotherapy taking into account drug
resistance stemming from gene amplification, Int. J. of Appl. Math. Comput. Sci., 13, (2003),
pp. 297-305
[27] A. Swierniak, U. Ledzewicz and H. Schättler, Optimal control for a class of compartmental
models in cancer chemotherapy, Int. J. of Appl. Math. Comput. Sci., 13, (2003), pp. 357-368
[28] A. Swierniak, A. Polanski and M. Kimmel, Optimal control problems arising in cell-cycle-
specific cancer chemotherapy, Cell prolif., 29, (1996), pp. 117-139
[29] A. Swierniak, A. Polanski, M. Kimmel, A. Bobrowski and J. Smieja, Qualitative analysis
of controlled drug resistance model - inverse Laplace and semigroup approach, Control and
Cybernetics 28, (1999), pp. 61-75
[30] A. Swierniak and J. Smieja, Cancer chemotherapy optimization under evolving drug resis-
tance, Nonlinear Analysis, 47, (2000), pp. 375-386
[31] I.F. Tannock, Tumor physiology and drug resistance, Cancer and Metastasis Reviews, 20,
(2001), pp. 123-132
[32] J.J. Westman, B.R. Fabijonas, D.L. Kern and F.B. Hanson, Probabilistic rate compartment
cancer model: alternate versus traditional chemotherapy scheduling, in: Stochastic Theory and
Control, Proceedings of a Workshop held in Lawrence, Kansas, October 18-20, 2001, Lecture
Notes in Control and Information Sciences, B. Pasik-Duncan (Editor), Springer-Verlag, New
York, pp. 491-506, (2002)
[33] J.J. Westman, B.R. Fabijonas, D.L. Kern and F.B. Hanson, Cancer treatment using multiple
chemotherapeutic agents subject to drug resistance, Proc. 15th Int. Symp. of Mathematical
Theory of Networks and Systems (MTNS), August 2002
E-mail address: uledzew@siue.edu; hms@wustl.edu