Constrained Optimization using Matlab's fmincon

A. Basic Calls (without any special options)

Example1              Example 2
B. Calls with Gradients Supplied
Matlab's HELP DESCRIPTION

For constrained minimization of an objective function f(x) (for maximization use -f),
Matlab provides the command fmincon. The objective function must be coded in a
function file in the same manner as for fminunc. In these notes this file will be
called objfun and saved as objfun.m  in the working directory.

A: Basic calls   top

Without any extra options, fmincon  is called as follows:

- with linear inequality constraints Ax£b only (as in linprog):

[x,fval]=fmincon('objfun',x0,A,b)

- with linear inequality constraints and linear equality

constraints Aeq·x=beq only:
[x,fval]=fmincon('objfun',x0,A,b,Aeq,beq)

- with linear inequality and equality constraints, and in addition a lower bound of

the form x³lb only:
[x,fval]=fmincon('objfun',x0,A,b,Aeq,beq,lb)
If only a subset of the variables has a lower bound, the components of lb
corresponding to variables without lower bound are -Inf. For example, if the variables
are (x,y), and x³1 but y has no lower bound, then lb=[1;-Inf].

- with linear inequality and equality constraints and lower as well as an upper

bound of the form x£ub only:
[x,fval]=fmincon('objfun',x0,A,b,Aeq,beq,lb,ub)
If only a subset of the variables has an upper bound, the components of ub
corresponding to variables without upper bound are Inf. For example, if the variables
are (x,y) and x£1 but y has no lower bound, then lb=[1;Inf].

- with linear inequality and equality constraints,  lower and upper bounds, and
nonlinear inequality and equality constraints:
[x,fval]=fmincon('objfun',x0,A,b,Aeq,beq,lb,ub,'constraint')
The last input argument in this call  is the name of a function file
(denoted  constraint in these notes and saved as constraint.m  in the working
directory), in which the nonlinear constraints are coded.

Constraint function file:

constraint.m is a function file (any name can be chosen) in which both the inequality
functions  c(x) and the equality constraints ceq(x) are coded and provided in the form
of column vectors. The function call

[c,ceq]=constraint(x)

must retrieve c(x) and ceq(x) for given input vector x. Examples of constraint function
files are given in Examples 1 and 2 below. If only inequality constraints are given,
define ceq=[]. Likewise, if only equality constraints are given, define c=[].

Interpretation:
The retrieved ceq(x) is interpreted by fmincon as equality constraint ceq(x)=0. The
inequalities associated with c(x) are interpreted as c(x)£0. Thus, if a constraint of the
form c(x)³0 is given, rewrite this as -c(x)£0 and code -c(x) in the constraint function
file.

Placeholders:
As shown above, the constraints have to passed to fmincon in the following order:
1. Linear inequality constraints
2. Linear equality constraints
3. Lower bounds
4. Upper bounds
5. Nonlinear constraints
If a certain constraint is required, all other constraints appearing before it have to be
inputted as well, even if they are not required in the problem. If this is the case, their
input argument is replaced by the placeholder [] (empty input).

Examples:
- If lb and (A,b) are given, but there are no other constraints, the syntax is:
[x,fval]=fmincon('objfun',x0,A,b,[],[],lb)

- If ub and (Aeq,beq) are the only constraints:

[x,fval]=fmincon('objfun',x0,[],[],Aeq,beq,[],ub)

- If only nonlinear constraints are given:

[x,fval]=fmincon('objfun',x0,[],[],[],[],[],[],'constraint')
and function file constraint.m must be provided.
 
Example 1:    top

Find the minimum of

f(x,y)=x4-x2+y2-2x+y
subject to
  linear inequalities linear equalities lower bounds upper bounds nonlinear constraints
(a) -- -- x³0 y£0 --
(b) -- x+y=0 -- x£1,  y£10 --
(c) x+y£0 -- -- -- x +y2£1
2

(d) -- -- -- -- x2+y2=1
(e) -- -- -- --  x2+y2=1,  x2-y2³1
(f) -- -- -- --  x2+y2£1,  x2-y2³1 

Solution: The objective function is coded as for unconstrained minimization:

function f=objfun(x)

f=x(1)^4-x(1)^2+x(2)^2-2*x(1)+x(2);

For (a), (b) we don't need a constraint function file. The calls are
(assuming x0=[value1;value2] is already defined):
(a):   [x,fval]=fmincon('objfun',x0,[],[],[],[],[0;-Inf],[Inf;0])
(b):   [x,fval]=fmincon('objfun',x0,[],[],[1,1],0,[],[1;10])

For (c)-(f)  we need a constraint function file. In each case the first line of the
file constraint.m  is:
function [c,ceq]=constraint(x)

followed by an empty line. The commands below the 2nd line are:
 

(c) (d) (e) (f)

c=[]; c=1-x(1)^2+x(2)^2; c1=x(1)^2+x(2)^2-1;
c=x(1)^2+x(2)^2-1;
ceq=x(1)^2+x(2)^2- ceq=x(1)^2+x(2)^2- c2=1-x(1)^2+x(2)^2;
ceq=[];
1; 1; c=[c1;c2];ceq=[];

For example, for (f) the full constraint function file is:
function [c,ceq]=constraint(x)

c1=x(1)^2+x(2)^2-1;
c2=1-x(1)^2+x(2)^2;
c=[c1;c2];ceq=[];
Function calls for (c)-(f):
(c):       [x,fval]=fmincon('objfun',x0,[1,1],0,[],[],[],[],'constraint')
(d)-(f):  [x,fval]=fmincon('objfun',x0,[],[],[],[],[],[],'constraint')

Approximate solutions found by fmincon:

 

x0 x y fval
- -
1.000000061313
(a) [1;-1] 80
0.500000141648 2.249999999999
75 96
- -
0.908524172193
(b) [1;-1] 45
0.908524172193 2.044260660473
45 01
- -
0.707106781187
(c) [0;0] 46
0.707106781187 1.871320343561
46 09
- -
0.928948444375
(d) [1;0] 17
0.370209120757 2.209321989279
12 09
- -
[.5;.1 1.000000000032
(e) ] 78
0.000008101068 2.000008101003
72 10
-
[.1;.1 1.000000000000 0.000000017925
(f) ] 09 12
1.999999982074
88

Example 2:    top

Minimize and maximize the objective function

f(x,y,z)=x3+y3+z3
subject to
x³0,      z£0,     x2+y2+z2=1,     y2³2z2.
Objective function file:
For Minimization:
function f=objfun(x)

f=x(1)^3+x(2)^3+x(3)^3;

For Maximization:
function f=objfun(x)

f=x(1)^3+x(2)^3+x(3)^3;f=-f;

Constraint function file:

function [c,ceq]=constraint(x)
c=2*x(3)^2-x(2)^2;
ceq=x(1)^2+x(2)^2+x(3)^2-1;

Function calls (in command window) and answers:

Minimization:
>> x0=[0;1;2];
>> [x,fval]=fmincon('objfun',x0,[],[],[],[],[0;-Inf;-Inf],
[Inf;Inf;0],'constraint')

Warning: Large-scale (trust region) method does not currently solve this type
of problem,
switching to medium-scale (line search).
> In C:\MATLABR12\toolbox\optim\fmincon.m at line 213
Optimization terminated successfully:
 Magnitude of directional derivative in search direction
  less than 2*options.TolFun and maximum constraint violation
  is less than options.TolCon
Active Constraints:
     1
     3
x =
   0.92898366078939
  -0.37012154351899
                  0
fval =
  -2.20932218190572

Answer for Maximization (same call, only objective function file was changed):
Warning: Large-scale (trust region) method does not currently solve this type
of problem,
switching to medium-scale (line search).
> In C:\MATLABR12\toolbox\optim\fmincon.m at line 213
Optimization terminated successfully:
 Search direction less than 2*options.TolX and
  maximum constraint violation is less than options.TolCon
Active Constraints:
     1
x =
     0
     1
     0
fval =
    -1
 

B: Call of fmincon with gradient information provided    top

As for fminunc the performance of fmincon can be improved if gradient information is

supplied. This information can be provided for the objective function, the nonlinear
constraint functions, or both. Let's consider Example 1(f) again. The objective
function file is extended as:
function [f,gradf]=objfun(x)

f=x(1)^4-x(1)^2+x(2)^2-2*x(1)+x(2);
gradf=[4*x(1)^3-2*x(1)-2;2*x(2)+1];

For providing the gradients of the nonlinear constraints, the constraint function file is
extended as:
function [c,ceq,gradc,gradceq]=constraint(x)

c1=x(1)^2+x(2)^2-1;
c2=1-x(1)^2+x(2)^2;
c=[c1;c2];ceq=[];
gradc=[2*x(1),-2*x(1);2*x(2),2*x(2)];
gradceq=[];

Note that the the first column of gradc is the gradient-vector of the first constraint, and
the second column of gradc is the gradient vector of the second constraint.

As in the unconstrained case we have to set the gradient option. We want to supply 
the gradient of the objective function as well as the nonlinear constraints. The
follwoing command sets this option:
>> options = optimset('GradObj','on','GradConstr','on');

In the function call these options are passed to fmincon as input argument after the
name of the constraint file:
>> x0=[.1;.1];[x,fval]=fmincon('objfun',x0,[],[],[],[],[],
[],'constraint',options)

Warning: Large-scale (trust region) method does not currently solve this type
of problem,
switching to medium-scale (line search).
> In C:\MATLABR12\toolbox\optim\fmincon.m at line 213
Optimization terminated successfully:
 Search direction less than 2*options.TolX and
  maximum constraint violation is less than options.TolCon
Active Constraints:
     1
     2
x =
   1.00000000000000
  -0.00000171875724
fval =
  -2.00000171875428
 
 

Matlab's HELP DESCRIPTION   top

fmincon finds a constrained minimum of a function of several variables.
fmincon attempts to solve problems of the form:
min F(X) subject to: A*X <= B, Aeq*X = Beq (linear constraints)
X C(X) <= 0, Ceq(X) = 0 (nonlinear constraints)
LB <= X <= UB (bounds)

fmincon implements four different algorithms: interior point, SQP,

active set, and trust region reflective. Choose one via the option
Algorithm: for instance, to choose SQP, set
OPTIONS = optimoptions('fmincon','Algorithm','sqp'),
and then pass OPTIONS to fmincon.

X = fmincon(FUN,X0,A,B) starts at X0 and finds a minimum X to the

function FUN, subject to the linear inequalities A*X <= B. FUN accepts
input X and returns a scalar function value F evaluated at X. X0 may be
a scalar, vector, or matrix.

X = fmincon(FUN,X0,A,B,Aeq,Beq) minimizes FUN subject to the linear

equalities Aeq*X = Beq as well as A*X <= B. (Set A=[] and B=[] if no
inequalities exist.)

X = fmincon(FUN,X0,A,B,Aeq,Beq,LB,UB) defines a set of lower and

upper
bounds on the design variables, X, so that a solution is found in
the range LB <= X <= UB. Use empty matrices for LB and UB
if no bounds exist. Set LB(i) = -Inf if X(i) is unbounded below;
set UB(i) = Inf if X(i) is unbounded above.

X = fmincon(FUN,X0,A,B,Aeq,Beq,LB,UB,NONLCON) subjects the
minimization
to the constraints defined in NONLCON. The function NONLCON accepts X
and returns the vectors C and Ceq, representing the nonlinear
inequalities and equalities respectively. fmincon minimizes FUN such
that C(X) <= 0 and Ceq(X) = 0. (Set LB = [] and/or UB = [] if
no bounds
exist.)

X =
fmincon(FUN,X0,A,B,Aeq,Beq,LB,UB,NONLCON,OPTIONS) minimizes
with
the default optimization parameters replaced by values in OPTIONS, an
argument created with the OPTIMOPTIONS function. See OPTIMOPTIONS for
details. For a list of options accepted by fmincon refer to the
documentation.

X = fmincon(PROBLEM) finds the minimum for PROBLEM. PROBLEM is a

structure with the function FUN in PROBLEM.objective, the start point
in PROBLEM.x0, the linear inequality constraints in PROBLEM.Aineq
and PROBLEM.bineq, the linear equality constraints in PROBLEM.Aeq and
PROBLEM.beq, the lower bounds in PROBLEM.lb, the upper bounds in
PROBLEM.ub, the nonlinear constraint function in PROBLEM.nonlcon, the
options structure in PROBLEM.options, and solver name 'fmincon' in
PROBLEM.solver. Use this syntax to solve at the command line a problem
exported from OPTIMTOOL.

[X,FVAL] = fmincon(FUN,X0,...) returns the value of the objective

function FUN at the solution X.

[X,FVAL,EXITFLAG] = fmincon(FUN,X0,...) returns an EXITFLAG that

describes the exit condition. Possible values of EXITFLAG and the
corresponding exit conditions are listed below. See the documentation
for a complete description.

All algorithms:

1     First order optimality conditions satisfied.

0     Too many function evaluations or iterations.
-1    Stopped by output/plot function.
-2    No feasible point found.
    Trust-region-reflective, interior-point, and sqp:
2     Change in X too small.
    Trust-region-reflective:
3     Change in objective function too small.
    Active-set only:
4     Computed search direction too small.
5     Predicted change in objective function too small.
    Interior-point and sqp:
-3    Problem seems unbounded.

[X,FVAL,EXITFLAG,OUTPUT] = fmincon(FUN,X0,...) returns a
structure
OUTPUT with information such as total number of iterations, and final
objective function value. See the documentation for a complete list.

[X,FVAL,EXITFLAG,OUTPUT,LAMBDA] =
fmincon(FUN,X0,...) returns the
Lagrange multipliers at the solution X: LAMBDA.lower for LB,
LAMBDA.upper for UB, LAMBDA.ineqlin is for the linear inequalities,
LAMBDA.eqlin is for the linear equalities, LAMBDA.ineqnonlin is for the
nonlinear inequalities, and LAMBDA.eqnonlin is for the nonlinear
equalities.

[X,FVAL,EXITFLAG,OUTPUT,LAMBDA,GRAD] =
fmincon(FUN,X0,...) returns the
value of the gradient of FUN at the solution X.

[X,FVAL,EXITFLAG,OUTPUT,LAMBDA,GRAD,HESSIAN] =
fmincon(FUN,X0,...)
returns the value of the exact or approximate Hessian of the Lagrangian at X.

Examples:

FUN can be specified using @:

X = fmincon(@humps,...)
In this case, F = humps(X) returns the scalar function value F of
the HUMPS function evaluated at X.

FUN can also be an anonymous function:

X = fmincon(@(x) 3*sin(x(1))+exp(x(2)),[1;1],[],[],[],[],
[0 0])
returns X = [0;0].

If FUN or NONLCON are parameterized, you can use anonymous functions to

capture the problem-dependent parameters. Suppose you want to minimize
the objective given in the function myfun, subject to the nonlinear
constraint mycon, where these two functions are parameterized by their
second argument a1 and a2, respectively. Here myfun and mycon are
MATLAB file functions such as

function f = myfun(x,a1)
f = x(1)^2 + a1*x(2)^2;
function [c,ceq] = mycon(x,a2)
c = a2/x(1) - x(2);
ceq = [];

To optimize for specific values of a1 and a2, first assign the values
to these two parameters. Then create two one-argument anonymous
functions that capture the values of a1 and a2, and call myfun and
mycon with two arguments. Finally, pass these anonymous functions to
fmincon:

a1 = 2; a2 = 1.5; % define parameters first

options = optimoptions('fmincon','Algorithm','interior-
point'); % run interior-point algorithm
x = fmincon(@(x) myfun(x,a1),[1;2],[],[],[],[],[],[],@(x)
mycon(x,a2),options)

See also optimoptions, optimtool, fminunc, fminbnd, fminsearch, @,

function_handle.      top