Coding using MATLAB

Prof. Dany Abou Jaoude

Dany Abou Jaoude Coding using MATLAB 1

 Useful Links
o MATLAB: https://www.mathworks.com/products/matlab.html
o CVX: http://web.cvxr.com/cvx/doc/
o YALMIP: https://yalmip.github.io/
o SDPT3: https://github.com/SQLP/SDPT3
o SEDUMI: https://github.com/sqlp/sedumi
o MOSEK: https://www.mosek.com/

o Examples #1-#5 in this lecture build on the examples from the quickstart CVX
webpage: http://web.cvxr.com/cvx/doc/quickstart.html

o The problem solved in Example #6 is from the lecture notes in

https://users.math.msu.edu/users/iwenmark/Teaching/MTH995/Papers/SDP_notes_
Marina_Epelman_UM.pdf

o Note: Examples #1-#5 are randomly generated for each test run; so it is expected to
obtain different answers than the sample ones shown on these slides.

Dany Abou Jaoude Coding using MATLAB 2

 Example #1

o Assume that we are seeking 𝑥 ∈ ℝ𝑛 that minimizes 𝐴𝑥 − 𝑏 2 for given matrix 𝐴 ∈

ℝ𝑚×𝑛 and vector 𝑏 ∈ ℝ𝑚 .

o The following MATLAB code generates the problem data and the analytic problem
solution:

clc
clear all

%least-squares problem

m = 16; n = 8;
A = randn(m,n);
b = randn(m,1);
−1
x_ls = A\b; o 𝑥 = 𝐴𝑇 𝐴 𝐴𝑇 𝑏 is the analytic solution of the
unconstrained least-squares problem.

Dany Abou Jaoude Coding using MATLAB 3

 Example #1

o Disciplined convex optimization using CVX: (solver is SDPT3)

cvx_begin
variable x(n) o Declare optimization variable
minimize( norm(A*x-b) ) o Define objective function to minimize
cvx_end

check1=max(abs(x-x_ls)) o Compare CVX solution with analytic solution

o check1 = 5.1285e-11
o Very small discrepancy!

Dany Abou Jaoude Coding using MATLAB 4

 Example #1

o Disciplined convex optimization using CVX: (solver is SDPT3)

o We cannot maximize a convex function in

cvx_begin
disciplined convex optimization!
variable y(n)
maximize( norm(A*y-b) )
o The following error output is returned:
cvx_end
Error using cvxprob/newobj (line 57)
Disciplined convex programming error:
Cannot maximize a(n) convex expression.

Dany Abou Jaoude Coding using MATLAB 5

 Example #2
o Assume we want to solve the following optimization problem:
min 𝐴𝑥 − 𝑏 2
subject to 𝑙 ≼ 𝑥 ≼ 𝑢

o That is, upper and lower bounds on the optimization variable are added to the
optimization problem in Example #1.

o The following lines are added to the code when defining the problem parameters:

bnds = randn(n,2);
l = min( bnds, [], 2 );
u = max( bnds, [], 2 );

Dany Abou Jaoude Coding using MATLAB 6

 Example #2

o Disciplined convex optimization using CVX: (solver is SDPT3)

cvx_begin
variable x(n) o Declare optimization variable
minimize( norm(A*x-b) ) o Define objective function to minimize
subject to o Add problem constraints
l <= x <= u
cvx_end

Dany Abou Jaoude Coding using MATLAB 7

 Example #2

o Verify that all constraints are satisfied by the obtained solution:

check2 =
4.4170e-10
check2=min(x-l)
check3=min(u-x)
check3 =
2.8622e-10

Dany Abou Jaoude Coding using MATLAB 8

 Example #2

o The analytic solution (to the original unconstrained least-squares problem) is no

longer valid.
o That is, the analytic solution is not the constrained problem solution.
o It is also not guaranteed that this analytic solution satisfies the imposed constraints:

check1 =
check1=max(abs(x-x_ls)) 1.3874
check4=min(x_ls-l)
check5=min(u-x_ls) check4 =
-0.6463

check5 =
-1.2353

Dany Abou Jaoude Coding using MATLAB 9

 Example #2
o The same example is also coded using YALMIP: (solver is SDPT3)

xyalmip=sdpvar(n,1); o Define optimization variable and

obj=norm(A*xyalmip-b); objective function

csts=[]; o Define problem constraints

csts=csts + [xyalmip >=l];
csts=csts + [xyalmip <=u]; o Specify solver and solver options

ops = sdpsettings('solver','sdpt3','shift',1e-6,'verbose',1);

optimize(csts,obj,ops) o Call on solver

xyalmip=value(xyalmip); o Extract problem solution

obj=value(obj);

Dany Abou Jaoude Coding using MATLAB 10

 Example #2
o Since the ‘verbose’ option is set equal to ‘1’, the following sample output is
obtained:

num. of constraints = 9
dim. of socp var = 17, num. of socp blk = 1
dim. of linear var = 16
*******************************************************************
SDPT3: Infeasible path-following algorithms
*******************************************************************
version predcorr gam expon scale_data
HKM 1 0.000 1 0
it pstep dstep pinfeas dinfeas gap prim-obj dual-obj cputime
-------------------------------------------------------------------
0|0.000|0.000|1.6e+00|6.6e+00|1.6e+03| 7.958975e+01 0.000000e+00| 0:0:00| chol 1 1
1|0.856|0.944|2.2e-01|4.4e-01|1.6e+02| 6.037779e+01 -4.245208e+00| 0:0:00| chol 1 1
2|1.000|0.990|2.2e-06|1.2e-02|3.0e+01| 2.139144e+01 -6.874668e+00| 0:0:00| chol 1 1
3|0.862|0.952|1.4e-07|1.2e-03|4.0e+00|-1.815029e+00 -5.813998e+00| 0:0:00| chol 1 1
4|0.430|0.866|2.3e-07|2.3e-04|2.4e+00|-2.723919e+00 -5.147645e+00| 0:0:00| chol 1 1

Dany Abou Jaoude Coding using MATLAB 11

 Example #2
o Sample output continued:

5|1.000|1.000|1.4e-08|7.1e-06|5.0e-01|-4.349089e+00 -4.849174e+00| 0:0:00| chol 1 1

6|0.889|0.994|3.7e-09|7.5e-07|6.3e-02|-4.680291e+00 -4.743453e+00| 0:0:00| chol 1 1
7|1.000|1.000|3.3e-09|7.2e-08|8.4e-03|-4.727309e+00 -4.735728e+00| 0:0:00| chol 1 1
8|0.987|0.988|2.6e-10|8.5e-09|1.1e-04|-4.733082e+00 -4.733187e+00| 0:0:00| chol 1 1
9|0.985|0.988|3.3e-10|1.6e-10|1.5e-06|-4.733154e+00 -4.733156e+00| 0:0:00| chol 1 1
10|0.989|1.000|3.7e-12|6.7e-11|5.0e-08|-4.733155e+00 -4.733156e+00| 0:0:00|
stop: max(relative gap, infeasibilities) < 1.00e-07
-------------------------------------------------------------------

Dany Abou Jaoude Coding using MATLAB 12

 Example #2
o Sample output continued:

number of iterations = 10
primal objective value = -4.73315546e+00
dual objective value = -4.73315551e+00
gap := trace(XZ) = 5.01e-08
relative gap = 4.79e-09
actual relative gap = 4.72e-09
rel. primal infeas (scaled problem) = 3.69e-12
rel. dual " " " = 6.70e-11
rel. primal infeas (unscaled problem) = 0.00e+00
rel. dual " " " = 0.00e+00
norm(X), norm(y), norm(Z) = 4.1e+00, 4.9e+00, 7.3e+00
norm(A), norm(b), norm(C) = 1.2e+01, 2.0e+00, 5.8e+00
Total CPU time (secs) = 0.34
CPU time per iteration = 0.03
termination code = 0
DIMACS: 3.7e-12 0.0e+00 1.3e-10 0.0e+00 4.7e-09 4.8e-09
-------------------------------------------------------------------
Dany Abou Jaoude Coding using MATLAB 13
 Example #2
o Sample output continued:

yalmipversion: '20190425'
yalmiptime: 0.5992
solvertime: 0.5208
info: 'Successfully solved (SDPT3-4)'
problem: 0

o SEDUMI can also be chosen as the solver instead of SDPT3, and other solver options
may be specified. For example:

ops = sdpsettings('solver','sedumi','shift',1e-6,'verbose',0);

o The solutions obtained using CVX and YALMIP with the various solvers are very close.

Dany Abou Jaoude Coding using MATLAB 14

 Example #3

o Assume that we are seeking 𝑥 that minimizes 𝐴𝑥 − 𝑏 ∞ instead.

o That is, we are interested in minimizing the Chebyshev norm of 𝐴𝑥 − 𝑏 instead of its
Euclidean norm:
tic
cvx_begin
variable x(n)
minimize( norm(A*x-b,Inf) )
cvx_end
toc

o Note that, while an equivalent LP formulation to this problem is possible, we here solve for the convex
optimization problem as given.
o The tic/toc commands are added to time the solvers. (E.g., Elapsed time is 1.494506 seconds.)
o It is a insightful exercise to try to solve problems for increasing values of 𝑛 and 𝑚, and to see the
resulting effect on the total solution time.

Dany Abou Jaoude Coding using MATLAB 15

 Example #3

o The coding of this optimization problem in YALMIP is given by:

tic
xyalmip=sdpvar(n,1);
obj=norm(A*xyalmip-b,Inf);

ops = sdpsettings('solver','sdpt3','verbose',0);

optimize([],obj,ops) o is used because the

problem is unconstrained.
xyalmip=value(xyalmip);
obj=value(obj);

toc

Dany Abou Jaoude Coding using MATLAB 16

 Example #4
o In this example, an equality constraint and a nonlinear inequality constraint are added
to the original least-squares problem.
o Namely, the constraints added are 𝐶 𝑇 𝑥 = 𝑑 and 𝑥 ∞ ≤ 1.
o The CVX code is:

m = 16; n = 8; cvx_begin
A = randn(m,n); variable x(n);
b = randn(m,1); minimize( norm(A*x-b) );
subject to
p = 4; C*x == d;
C = randn(p,n); norm(x,Inf) <= 1;
d = randn(p,1); cvx_end

Dany Abou Jaoude Coding using MATLAB 17

 Example #4
o The YALMIP code is:
xyalmip=sdpvar(n,1);
obj=norm(A*xyalmip-b);

Csts=[];

Csts = Csts + [C*xyalmip == d];

Csts = Csts + [norm(xyalmip,Inf) <= 1];

ops = sdpsettings('solver','sdpt3','verbose',1);

optimize(Csts,obj,ops)

xyalmip=value(xyalmip);
obj=value(obj);

o To obtain the minimum value of the objective function using CVX, check2=abs(cvx_optval-obj)
the command ‘cvx_optval’ is used. check2 =
6.7031e-08

Dany Abou Jaoude Coding using MATLAB 18

 Example #4
o Can the nonlinear equality constraint 𝑥 ∞ = 1 be imposed?
o No, disciplined convex optimization only allows for linear equality constraints.
o CVX gives the following error:

cvx_begin
variable y(n);
minimize( norm(A*y-b) ); Error using cvxprob/newcnstr (line 192)
subject to Disciplined convex programming error:
C*y == d; Invalid constraint: {convex} == {real constant}
norm(y,Inf) == 1;
cvx_end

Dany Abou Jaoude Coding using MATLAB 19

 Example #4
o Can the nonlinear equality constraint 𝑥 ∞ = 1 be imposed?
o No, disciplined convex optimization only allows for linear equality constraints.
o YALMIP gives the following error:

yyalmip=sdpvar(n,1);
obj=norm(A*yyalmip-b);

Csts=[];
Warning: Solver not applicable (sdpt3)
solvertime: 0
Csts = Csts + [C*yyalmip == d];
info: 'Solver not applicable (sdpt3)'
Csts = Csts + [norm(yyalmip,Inf) == 1];
problem: -4
yalmiptime: 0.3380
ops = sdpsettings('solver','sdpt3','verbose',1);

optimize(Csts,obj,ops)

Dany Abou Jaoude Coding using MATLAB 20

 Example #5
o We now consider the regularized least-squares problem: minimize 𝐴𝑥 − 𝑏 2 + 𝛾 𝑥 1 for a given value of 𝛾.
o To trace the tradeoff Pareto optimal curve between the competing objectives 𝐴𝑥 − 𝑏 2 and 𝑥 1 , the value
of 𝛾 is varied, and the convex optimization problem above is solved at each considered value of 𝛾.
gamma = logspace( -2, 2, 20 );
l2norm = zeros(size(gamma));
l1norm = zeros(size(gamma));

for k = 1:length(gamma)
cvx_begin
variable x(n);
minimize( norm(A*x-b)+gamma(k)*norm(x,1) );
cvx_end
l1norm(k) = norm(x,1);
l2norm(k) = norm(A*x-b);
end
plot( l1norm, l2norm );
xlabel( 'norm(x,1)' );
ylabel( 'norm(A*x-b)' );
grid on

Dany Abou Jaoude Coding using MATLAB 21

 Example #5

Dany Abou Jaoude Coding using MATLAB 22

 Example #6
o This example considers a matrix-valued decision variable with an imposed constraint that the said matrix is
positive semidefinite.
o Namely, the following optimization problem is considered:

minimize 𝑥11 + 4𝑥12 + 6𝑥13 + 9𝑥22 + 0𝑥23 + 7𝑥33

subject to

𝑥11 + 0𝑥12 + 2𝑥13 + 3𝑥22 + 14𝑥23 + 5𝑥33 = 11

0𝑥11 + 4𝑥12 + 16𝑥13 + 6𝑥22 + 0𝑥23 + 4𝑥33 = 19

𝑥11 𝑥12 𝑥13

𝑋 = 𝑥21 𝑥22 𝑥23 ≽ 0
𝑥31 𝑥32 𝑥33

Dany Abou Jaoude Coding using MATLAB 23

 Example #6
o The CVX code is:

n=3;

cvx_begin sdp o This problem is an SDP.

variable x(n,n) symmetric; o The matrix variable 𝑥 is symmetric of dimensions 𝑛 × 𝑛.

minimize( x(1,1) + 4*x(1,2) + 6*x(1,3) + 9*x(2,2) + 0*x(2,3) + 7*x(3,3) );

subject to
x(1,1) + 0*x(1,2) + 2*x(1,3) + 3*x(2,2) + 14*x(2,3) + 5*x(3,3)==11;
0*x(1,1) + 4*x(1,2) + 16*x(1,3) + 6*x(2,2) + 0*x(2,3) + 4*x(3,3)==19;
x>=0; o The matrix 𝑥 is positive semidefinite.
cvx_end

Dany Abou Jaoude Coding using MATLAB 24

 Example #6
o To verify the solution:

check1 =
11
check1=x(1,1) + 0*x(1,2) + 2*x(1,3) + 3*x(2,2) + 14*x(2,3) + 5*x(3,3)
check2 =
check2=0*x(1,1) + 4*x(1,2) + 16*x(1,3) + 6*x(2,2) + 0*x(2,3) + 4*x(3,3) 19.0000

check3 =
check3=eig(x) 0.0000
0.0000
1.8990

Dany Abou Jaoude Coding using MATLAB 25

 Example #6
o The YALMIP code is:
xyalmip=sdpvar(n); o This command defines a symmetric matrix variable of dimensions 𝑛 × 𝑛.

obj=xyalmip(1,1) + 4*xyalmip(1,2) + 6*xyalmip(1,3) + 9*xyalmip(2,2) ...

+ 0*xyalmip(2,3) + 7*xyalmip(3,3);

Csts=[];
Csts = Csts + [xyalmip(1,1) + 0*xyalmip(1,2) + 2*xyalmip(1,3) + 3*xyalmip(2,2) ...
+ 14*xyalmip(2,3) + 5*xyalmip(3,3)==11];
Csts = Csts + [0*xyalmip(1,1) + 4*xyalmip(1,2) + 16*xyalmip(1,3) + 6*xyalmip(2,2) ...
+ 0*xyalmip(2,3) + 4*xyalmip(3,3)==19];
Csts = Csts + [xyalmip>=0]; o The matrix 𝑥𝑦𝑎𝑙𝑚𝑖𝑝 is positive semidefinite.

ops = sdpsettings('solver','sdpt3','shift',1e-6,'verbose',1);

optimize(Csts,obj,ops)
xyalmip=value(xyalmip);
obj=value(obj);

Dany Abou Jaoude Coding using MATLAB 26

 Example #6
o The YALMIP/SDPT3 answer is xyalmip =
1.0559 0.3692 0.8683
0.3692 0.1291 0.3036
0.8683 0.3036 0.7140

obj =
13.9022

o If the matrix inequality is made strict, i.e., the following is used in the YALMIP code: Csts = Csts + [xyalmip>0];
then, the following warning line is received before the solution output:

Warning: Strict inequalities are not supported. A non-strict has been added instead')

Dany Abou Jaoude Coding using MATLAB 27