The general quadratic programming problem can be formulated mathematically as:

1 T T
Minimize f ( x )= x Qx+ c x
2
Subject to Ax ≤ b
x≥0
q11 q12 … q1 n a 11 a12 … a1 n

⋮
(⋮
q n 1 qn 2 … q nn
)
where Q= q21 q22 ⋯ q2 n is nxn symmetric ¿ A= a21 a22 ⋯ a2 n
⋱ ⋮ ⋮ ⋮ ⋱ ⋮
an 1 an 2 … ann
( )
x1 b1

() ()
x2 b2
⋮ b= ⋮ ∧¿
is mxn¿ x= , c=( c 1 , c 2 ,… ,c j , … , c n )
xj bi
⋮ ⋮
xn bm
Example: Minimizef ( x )=2 x 21+ x22 −2 x 1 x 2−5 x 1−2 x 2
Subject to 3 x 1+2 x 2 ≤ 20
−5 x 1+3 x 2 ≤ 4
x 1 ≥ 0, x 2 ≥ 0
2 −1 A= 3 2 , x= x 1 , b= 20
Q= (
−1 1
, ) (
−5 3 ) ( ) ( )
x2 4
and c= (−5 ,−2 )

T 1 T
Formulate Lagrangian function: L ( x , λ ) =c x+ x Qx+ λ (Ax−b)
2

The KKT necessary conditions for a local minimum:

∂L
≥ 0 , j=1 , 2, 3 , … , n c T + x T Q+ λ A ≥0
∂x j
∂L
≤ 0 ,i=1, 2 , 3 , …m Ax−b≤ 0
∂ λi
∂L
xj =0 , j=1 ,2 , 3 , … , n x T ( cT + xT Q+ λ A )=0
∂xj
∂L
λi =0 , i=1 ,2 , 3 , … ,n λ ( Ax−b)=0
∂ λi
xj≥0, j=1 ,2 , 3 , … ,n x≥0
λ i ≥ 0, i=1 , 2, 3 , … m λ≥0

Modified KKT necessary conditions:

x T Q+ λ A− y=−c T
Ax+ v=b
 x≥0; λ≥0; y≥0; v≥0
yx=0; λv =0

Solve the modified KKT necessary conditions by using Big-M method or two phase method

Definition of Wolfe’s Method

A Wolfe’s method is a typical method for solving a quadratic programming problem by means of
transforming the quadratic programming problem into a linear programming problem through
applying the Karush-Kuhn-Tucker necessary conditions to change the quadratic objective
function into linear.
To solve the quadratic programming problem by a Wolfe’s method, apply the following steps
1. Excluding the non-negativity conditions, define the Lagrangian function for the quadratic
programming problem (2.1) as:
1
L ( x , λ ) =c T x+ x T Qx+ λ (Ax−b) , where λ is an m-dimensional row vector.
2
2. Determine the KKT necessary conditions for the quadratic programming problem
The KKT necessary conditions for a local minimum:

∂L
≥ 0 , j=1 , 2, 3 , … , n c T + x T Q+ λ A ≥0
∂x j
∂L
≤ 0 ,i=1, 2 , 3 , …m Ax−b≤ 0
∂ λi
∂L
xj =0 , j=1 ,2 , 3 , … , n x T ( cT + xT Q+ λ A )=0
∂xj
∂L
λi =0 , i=1 ,2 , 3 , … ,n λ ( Ax−b)=0
∂ λi
xj≥0, j=1 ,2 , 3 , … ,n x≥0
λ i ≥ 0, i=1 , 2, 3 , … m λ≥0

3. Introduce nonnegative surplus variables y ∈ Rn to the 1st inequalities and nonnegative slack
variables v ∈ R m to the 2nd inequalities to obtain the equations: c T + x T Q+ λ A− y=0 &
Ax−b+ v=0
4. The KKT conditions can now be written with the constants moved to the right-hand-side.
x T Q+ λ A− y=−c T (3.3a)
Ax+ v=b (3.3b)
x≥0; λ≥0; y≥0; v≥0 (3.3c)
yx=0; λ v=0 (3.3d)
The first two expressions are linear equalities, the third restricts all the variables to be
nonnegative and the fourth prescribes complementary slackness.
5. Solve the result of step 4 by using the two phase method
 Since the right hand constant of equation (3.3a) is negative, multiply it by -1.
 Introduce artificial variable a i in each i th constraint with no basic variable.
 Formulate the phase 1 linear programming problem.
 For minimization linear programming problem, we have: z ' =a1 +a 2+ …+am
 Subject to the constraints of the original linear programming problem.
 In z ' row, change the entry under any basic variable into zero
 Solve the resulting linear programming problem by the Simplex method. At the optimal
Simplex tableau,
 If z ' =0 & a i=0 for all i (i.e. all a i becomes non-basic), then the original linear
programming problem is said to be feasible and thus we have obtained the optimal
solution and optimal value for the original linear programming problem.
 If a i> 0 for somei, then the original linear programming problem is said to be
infeasible. So, terminate the procedure.
The simplex algorithm can be used to solve (3.3a)-(3.3d) by treating the complementary
slackness conditions (3.3d) implicitly with a restricted basis entry rule. The goal is to find the
solution to the linear programming problem that minimizes the sum of the artificial variables
with the additional requirement that the complementarity slackness conditions must be satisfied
for each iteration. If the sum is zero, the solution will satisfy (3.3a)-(3.3d).
Example 1: Solve the following quadratic programming problem using Wolfe’s method
Minimize z=−8 x 1−16 x 2 + x12+ 4 x 22
Subject to x 1+ x2 ≤5
x1≤ 3 Answer: (x ⋆1 , x2⋆ ¿=(3 ,2)
x 1, x 2 ≥ 0

c T =(−8 ,−16 ) ,Q= 2 0 , A= 1 1 , bT =( 5 , 3 ) , x T =( x 1 , x 2 ) , y T =( y 1 , y 2) ,

( ) ( )
0 8 1 0
λ=( λ 1 , λ2 ) ¿ v T =( v 1 , v2 ).

Example 2: Solve the following quadratic programming problem by using the Wolfe’s method
Minimize z=3 x 21+ 2 x 22−20 x 1 +10 x2
Subject to 2 x1 −x2 ≤ 6
−x 1+ x2 ≤10 Answer: (x ⋆1 , x2⋆ ¿=(3 ,0)
x 1, x 2 ≥ 0

No of Basic All variables

RHS
row variables
x1 x2 λ1 λ2 y1 y2 v1 v2 a b
0 row ' 0 0 0 0 0 0 0 0 -1 0
z
1 st
row a 6 0 2 -1 -1 0 0 0 1 20
 2nd row y2 0 -4 1 -1 0 1 0 0 0 10
3rd row v1 2 -1 0 0 0 0 1 0 0 6
4rd row v2 -1 1 0 0 0 0 0 1 0 10
column1 Column2 Column3 Column4 Column5 Column6 Column7 Column8 Column9 Column10

Use the following simplex tableau

The general homogeneous linear ordinary differential equation of order n with constant
coefficients is given by:
a 0 y ( n) +a1 y (n−1) +…+ an y=0 (1)
The general a system of first order linear ordinary differential equations in normal form is given
by:
y '1 ( x )=a11 ( x ) y 1 ( x ) + a12 ( x ) y 2 ( x ) + …+a1 n ( x ) y n ( x ) + f 1 ( x)
y '2 ( x ) =a21 ( x ) y 1 ( x )+ a22 ( x ) y 2 ( x ) +…+a 2 n ( x ) y n ( x )+ f 2 ( x )
y '3 ( x ) =a31 ( x ) y 1 ( x ) +a32 ( x ) y 3 ( x ) +…+ a3 n ( x ) y n ( x ) +f 3 (x ) (2)
⋮ ⋮ ⋮ ⋮ ⋮ ⋮
'
y n ( t ) =an 1 ( x ) y 1 ( x ) +an 2 ( x ) y 2 ( x ) +…+ ann ( x ) y n ( x ) + f n (x)

In matrix and vector notations, we write it as:

y ' ( x )= A ( x ) y ( x ) +f ( x) (3)

y '1 ( x ) y 1( x) a11 ( x ) a 12 ( x ) ⋯ a 1 n ( x ) f 1 (x)

() ( ) (
'

'
⋮
yn (x )
y ( x)
where y ' ( t )= y 2 x , y ( x )= 2
( )

y n
⋮
(x)
a ( x ) a 22 ( x ) … a 2 n ( x )
, A ( x )= 21
a
⋮
n1 ( x ) a
⋮
n2 ( x )
⋮
… a
⋮
nn ( x )
The general a system of first order homogeneous linear ordinary differential equations in normal
f (x)
& f ( x )= 2
f
⋮
n (x)
) ( )
form is given by:
y '1 ( x )=a11 (x ) y 1 ( x )+ a12( x) y 2 ( x )+ …+a 1n ( x ) y n ( x )
y '2 ( x ) =a21 ( x) y 1 ( x ) +a22 (x) y 2 ( x ) +…+ a2 n ( x) y n ( x )
y '3 ( x ) =a31 ( x) y 1 ( x ) +a32 (x) y 3 ( x ) +…+ a3 n ( x) y n ( x )
⋮ ⋮ ⋮ ⋮ ⋮
'
y n ( x ) =an 1 (x) y 1 ( x ) +a n 2(x ) y 2 ( x )+ …+a nn(x ) y n ( x )

In matrix and vector notations, we write it as:

y ' ( x )= A ( x) y ( t )
 y '1 (x) a 11 (x) a12 ( x) ⋯ a 1 n( x ) y1 (x )

( )(
'

'
⋮
y n (x)
a (x) a22 ( x ) … a 2 n( x ) , y ( x ) = y2 (x )
where , y ' ( t)= y 2 (x) A= 21
a n1
⋮
( x) a
⋮
n2 (x)
⋮
… a
⋮
nn ( x) y
⋮
n (x )
) ( )
The general a system of first order homogeneous linear ordinary differential equations with
constant coefficients in normal form is given by:
y '1 ( x )=a11 y 1 ( x ) +a12 y 2 ( x ) +…+a 1 n y n ( x )
y '2 ( x ) =a21 y 1 ( x )+ a22 y 2 ( x ) +…+ a2 n y n ( x )
y '3 ( x ) =a31 y 1 ( x )+ a32 y 3 ( x ) + …+a3 n y n ( x )
⋮ ⋮ ⋮ ⋮ ⋮ ⋮
'
y n ( x ) =an 1 y 1 ( x )+ an 2 y 2 ( x )+ …+a nn y n ( x )

In matrix and vector notations, we write it as:

y ' ( x )= Ay (t )

y '1 (x) a 11 a12 ⋯ a1 n y1 (x )

( )(
'

'
⋮
y n (x)
a
where , y ' ( t)= y 2 (x) A= 21
a
⋮
n1
a22 … a2 n , ( ) y2 (x )

a
⋮
n2
⋮ ⋮
… a nn
y x=

y
⋮
n (x )
) ( )
Example 1: Given the 2 x 2 system of first order homogeneous linear ordinary differential
equations with constant coefficients:
y '1 ( x )=2 y 1 ( x ) −3 y 2 ( x )
y '2 ( x ) =− y 1 ( x )−5 y 2 ( x )
y '1 ( x ) y (x )
, A= 2 −3 and y ( x ) = 1
'
Here y ( x )= ( ) '
y2 ( x) (
−1 −5 )
y2 ( x ) ( )
Example 2: Given the 3 x 3 system of first order homogeneous linear ordinary differential
equations with constant coefficients:
y '1 ( x )=3 y 1 ( x )+ 4 y 2 ( x ) −2 y 2 ( x )
y '2 ( x ) =2 y 1 ( x ) + y 2 ( x )−4 y 2 ( x )
y '2 ( x ) = y 1 ( x ) +2 y 2 ( x )
y '1 ( x ) 3 4 −2 y 1 (x )
Here y '
( x ) =
( )
'
y2 ( x)
'
y3 ( x)
, A= 2 1 −4
1 2 0(∧ y ( x ) = y 2 (x )
y 3 (x )
) ( )
Eigenvalue and eigenvector method
Let λ be the eigenvalue and v be the corresponding eigenvector for A such that y ( x ) =v e λx is the
trial solution of y ' ( x )= Ay (t ).
Putting y ( x ) =v e λx and its first derivative with respect to x into y ' ( x )= Ay (t ), we get:
d (v e λx )
y ' ( x )= Ay (t ) ⇒ =Ay ( t )
dx
⇒ vλ e λx = Av e λx
⇒ Av=λv since e λx ≠ 0 for all x
⇒ ( A−λ I n ) v=0 , I n is an identity matrix of order n.
Here det ( A− λ I n ) =| A− λ I n|=0 is known as the characteristic polynomial of A.
Example 1: Find the solution of the following first order homogeneous linear ordinary
differential equations.
y '1 ( x )= y 1 ( x )−3 y 2 ( x) y '2 ( x ) = y 1( x)+5(x )

y '1 ( x ) y (x)
A= 1−3 ∧ y ( x )= 1
'
Solution, y ( x )= ' ( )
y2 ( x ) 15 ( ) y2 ( x ) ( )
Example 2: Solve the following first order homogeneous linear ordinary differential equations
with constant coefficients.
y '1 ( x )= y 1 ( x )−3 y 2 ( x ) +7 y 3 ( x) y '2 ( x ) =− y 1 ( x )− y 2 ( x )+ y 3 ( x )

y '3 ( x ) =− y 1 ( x ) + y 2 ( x )−3 y 3 (x )

y '1 ( x )
1 −3 7 y1 (x )
' '

( ) '
(
Here y ( x ) Here y ( x )= y ( x ) , A= −1 −1 1 ∧ y ( x ) = y2 (x )
y (x )
2
'
3
−1 1 −3 y 3 (x )
) ( )
Example 3: Find the solution of the following initial value problem.
y '1 ( x )= y 1( x )+2 y2 ( x ) y '2 ( x ) =3 y1 ( x )+ 2 y 2 ( x ), y 1 ( 0 )=0 ,
y 2 ( 0 )=−4

Case 1: The characteristic polynomial of A has distinct real roots

Example 1: Consider a 2 x 2 matrix A
Example 2: Consider a 3 x 3 matrix A

Case 2: The characteristic polynomial of A has repeated real roots

Example 3: Consider a 2 x 2 matrix A
Example 4: Consider a 3 x 3 matrix A
Case 3: The characteristic polynomial of A has complex roots
Example 5: Consider a 2 x 2 matrix A
Example 6: Consider a 3 x 3 matrix A