Optimization

Prof. A. Goswami
Department of Mathematics
Indian Institute of Technology Kharagpur

Lecture - 1
Optimization-Introduction

(Refer Slide Time: 00:23)

In todays topics optimization this one, introduction to optimization on this one, today
we are going to introduce some basic concepts of mathematical applications. Like matrix
algebra, vector space, linear algebra which are required for us. Those tools are required
for solving the problems on the optimization techniques. The first question, what comes
to us is that, what is optimization? Basically, we say that optimization is a technique, is a
mathematical application through which we can maximize or minimize a function. The
function may be of one variable or may be of several variables. And when we are
optimizing it; that is when we are maximizing or minimizing the function that may have
subject to certain constraints.
(Refer Slide Time: 01:33)

Just like if you see, this particular table; if you see go through this one, in this table what
we have done, some machines are there, if you see M 1, M 2, M 3 machines are there.
These machines are producing some products P 1, P 2, P 3 and P 4. And per hour how
much a production rate that we are telling that for machine 1 for producing P 1, what is
the time required in terms of hours these are given. And total time available per week
also is given. Unit profit for producing one product is also given, like for P 1 unit profit
is 7.5. For P 2 it is 4.6 like this way. If you see, if we assume that x j be the number of
units produced for product j. In this case; of course, it is per week we are, I am talking
about the per week, x j be the number of units produced for product j. In that case; what
actually I have to do? For product ne basically, I am producing x 1 quantity. For product
2 I am producing x 2 like this way. So, obviously; our aim is to maximize our profit
always.

So, I can develop or formulate a mathematical model for this table like this way.
Maximize Z equals 7.5 x 1 plus 4.62 x 2 plus 9.2 x 3 plus 5.25 x 4. So, this is actually
our profit function. Because I am producing x 1 quantity of product 1 whose profit is 7.5.
Similarly, for product 2, I am producing x 2 quantity, whose profit is 4.6 for unit, like
this way. So, I want to maximize the value of this subject to certain constraints. What are
the constraints? Subject to my constraint is; for each quantity of food item, how much
time it takes? And how much total time available to hours or in other sense, you can tell
2.7 x 1 plus 3 x 2 plus 4.6 x 3 plus 3 x 4 this should be less than equals 3000. Similarly,
for machine 2; 2 x 1 plus 7 x 2 plus 2 x 3 plus 5.1 x 4 should be less then equals 9500.
And 7.5 x 1 sorry, it should not be 7.5 x 1, 2.4 x 1 plus 4 x 2 plus 6.1 x 3 plus 3 x 4 less
than equals 6300.

So, if you see, our basic aim is; I want to find out how much quantity of each product
should I produce per week? So, that my profit will be maxima. So, for that one I am
formulating the model like this way. My profit function is this and subject to this
constraints. So, once I am writing this basically, what is the job of optimization
technique is that, how to find out the solution of this types of model. So, basically
through optimization technique, what we will try to learn is that how to maximize or how
to minimize a function subject to certain constraints. If you see instead of unit profit I
have written here unit profit. If I write down this portion as unit cost as unit cost in that
case instead of the maximization problem; obviously, I try to minimize my function and
in that case your maximize will be reduced will be replaced by minimize.

So, the problem may be maximization problem, the problem may be minimization
problem. Depending up on what the datas are available. So, we want to learn the
techniques through which we can solve this kind of problems. Basically, if you see
George Dantzig, he developed this optimization; first optimization technique in 1940s to
solve some problems related to military applications. That was the first thing he
developed which we call as the linear programming problem. And that was the first
optimization technique which was developed at the beginning. After that, lot of
optimization models techniques had been developed for solving different types of
problems. And these are widely used now a days in operational research, artificial
intelligence, computer science and obviously, needless to say in industry it is being
widely used. And which is helping a lot to the industry people.

Through these optimization techniques, we try to solve various kinds of problems. I can
list the problems; one we call as the linear problems. Well, in the linear problems what
we say, the optimization function basically should be linear. Whenever, the optimization
or function or the constraints are linear functions then those types of problems we call it
as the linear functions. Whereas, there is other type of problems, which I call non-linear
problems, in non-linear problems whenever the optimization function or the constraints
are non-linear in nature then those problems we call it as the non-linear problems.
We have the other type of problem that is we call it as discrete, in discrete case; what
happens, the value of the variable will take some discrete integer values. Just like if you
see whenever, I try to find out or made the class time table for teachers and I m locating
subject to different rooms always they will takes some integer values. Or, in a hospital
whenever, I am giving scheduling duties of the nurses those are taking the discrete
values.

This kind of problems we call it as the discrete problems. There is another type which we
call it as continuous problems, in continuous it can take any value any real value can be
taken over there. And in this case; anything whatever you do, it can take 1.5, 1.10
something like this way, any real values it can take. So, we can categorize our problems
linear, non-linear, discrete and continuous. And for each type of problem we have
several different optimization techniques. We will go through one by one all these things.
Of course, we will start with the linear problems initially and after that we will go to the
non-linear and other type of problems also. Now, let us give some introduction as I told
initially to the basic mathematical tools which will be required to us. You may know
everything, but in brief we will tell the things. What is required or which we will be used
afterwards in this lecture.

(Refer Slide Time: 10:41)

The first one is matrix algebra. In matrix algebra actually if you see, if you have m cross
n matrix. This m cross n matrix is we say, is an arrangement of (m n) objects. This
objects may be distinct may not be distinct in and you are placing this (m n) objects in m
rows and n columns in a particular form. Then we call it as a matrix. So, whenever we
are defining this m cross n matrix is basically nothing but; a arrangement of (m n)
objects in m rows and n columns and we write it in the form like this. a 1 1 a 1 2 a 1 n, a
2 1 a 2 2 a 2 n, a m 1 a m 2 a m n. So, we have total (m n) objects. We say that the matrix
is of order m cross n since, it has m rows and n columns. So, it is order is m cross n. And
whenever m is not equals n we tell that the matrix as rectangular matrix. Now, each
object a i j basically, these objects a i j is we call it as an element of the matrix. That is
the element on the i th row and on the j th column. This element we call it as the element
of the matrix. Whereas, if you see a 1 1 a 2 2 like this way a n n that is this diagonals a 1
1 a 2 2 a m m sorry, a m n. This one we call it as the diagonal elements.

Now, Row matrix, just some definitions; a matrix of order1 cross n, if I write down a
matrix of order 1 cross n, that is 1 row and n columns. Then we call it as the row matrix.
And we write it as a 1 1 a 1 2 like this way a 1 n. So, this one is a row matrix. Similarly,
you can define the column matrix; in case of column matrix what happens, where we
have a matrix which takes the form of m cross 1 that is m rows and 1 column. Then we
call it as the column matrix and it takes the form of this one; a 1 1 a 2 1 like this way it
will go up to a m 1. So, a 1 1 a 2 1 a m 1 this one we are calling as the column matrix.

(Refer Slide Time: 14:37)

Next comes, square matrix; if you have a matrix of order m cross n, where m equals n.
That is number of rows and numbers of columns are equal. Then we call that matrix as a
square matrix of order n. When ever, you are having the number of rows and numbers of
columns are same then we call that matrix as the square matrix. We can write down like
this. So, this matrix is the square matrix which has n rows and n columns. Next here
comes, null matrix; you have the matrix A equals a i j, the order is m cross n. If a i j
equals 0 for all a i and j then that matrix we call it as the null matrix. I can write down
something like this 0 0 0, 0 0 0 and all are 0.So, this matrix whatever I written is a null
matrix of order 3 or it has 3 rows and 3 columns.

Next diagonal matrix, in a square matrix please note that, in a square matrix not
rectangular matrix; that is square matrix means I am writing a i j n cross n, if a i j equals
0 for i not equals to j. That is, except the diagonal elements if all the elements are 0 then
that matrix we call it as the diagonal matrix.

(Refer Slide Time: 17:15)

Just like you can; for example, you can write down this matrix if you see, this one is the
diagonal matrix except the diagonals all other elements are 0. So, this is a diagonal
matrix. Next one is the identity matrix or sometimes we call it as the unit matrix also. In
a diagonal matrix; if diagonal elements are 1 then that matrix we call it as the identity
matrix. That is, these diagonal, values of these diagonal element if this is equals to 1 then
we call it as the diagonal matrix. And this we may write it like this. This matrix where all
the elements are 0 except the diagonal elements. And this identity matrix we usually
denote it by I which is the diagonal matrix or sometimes we call it as the identity matrix.
The next one is upper triangular matrix; this upper triangular matrix is a square matrix
where all the elements above and right of the diagonal matrix are non zero. Or, in other
sense, you can tell all the elements below the diagonal elements are 0. In any way you
can represent it either, all the elements above the diagonal elements and the right of the
diagonal element are non 0.

So, for an example, you may say something like this. A matrix you consider this one. So,
if you see this, in this case; all the elements above the diagonal elements are non zero
and this side all the elements below the diagonal elements are 0. So, this one we call it as
the upper triangular matrix.

(Refer Slide Time: 20:19)

And, similarly you can define the lower triangular matrix also. The lower triangular
matrix can be defined like this. I am directly giving one example, it will be clear to you I
think. If you see this then all the elements above the diagonal elements are 0. And all the
elements below these diagonal elements are non zeros. So, this kind of matrices we call it
as the lower triangular matrices. So, basic matrix operations like addition subtraction. So,
first come to the addition and subtraction. In addition and subtraction what happens, you
first consider a matrix A a i j, the size is of course, m cross n. There is another matrix B b
i j, whose size is also m cross n. I am considering the third matrix C, elements are c i j
and this is also will be order this. Now, whenever you are writing C equals A plus B. It
means one element i j th element, c i j of the resultant matrix this should be equals to a i j
plus b i j. Similarly, whenever you are writing C equals A minus B, it implies c i j this is
equals to a i j minus b i j.

So, basically you are making term wise addition or term wise subtraction for performing
the addition or subtraction operation. One thing should be noted that whenever, you are
performing the addition or subtraction operation, the order of both the matrices should
remain same it should not change over there.

(Refer Slide Time: 22:49)

Next comes matrix multiplication; in matrix multiplication what happens whenever, you
try to multiply 2 matrices; the first thing required is that either the numbers of the
columns of the first matrix should be equals to numbers of rows of the second matrix.
Then only we can multiply 2 matrices otherwise we cannot multiply the matrices. So,
again if you consider the matrices A equals a i j say, whose order is m cross n. Let us
take another matrix B which is equals to b i j of order n cross p.

And, there is a third matrices C, C equals c i j. If this is the resultant matrix; the order of
this one will be m cross p. That is number of rows of the first matrix into number of
columns of the second matrix. This will be the order of the resultant matrix. So, if you
write down C equals this c i j m cross p which is nothing but; again, here product of 2
matrices A and B. I am just writing this too. So, in details you have a 1 1 a 1 2 a 1 n like
this way, you are having in middle i th row, i 1 i 2 i n and at last you are having a m 1 a
m 2 a m n this is the matrix A. And you are having the matrix B similarly, b 1 1 here, I
am writing b 1 j b 1 p, b 2 1 b 2 j b 2 p.

Similarly, b m 1 b sorry, this should not be n cross p. So, it will be n, b n j and b n p. So,
whenever you are having 2 matrices like this. Sorry, just you see this one whenever your
matrices, you have the matrices like this way; in that case what happens, you are c i j is
nothing but; the element wise product and then some of these 2 rows and columns
whatever I am marking. That is a i 1 into b 1 j plus a i 2 into b 2 j dot dot like this way
plus a i n into b n j. I am writing, we can write down a i 1 b 1 j plus a i 2 b 2 j like this
way, it will go and a i n into b n j. Or, in other sense in summation form k equals 1 to n a
i k into b k j. So, this is the product of elements. So, like this way I can find out each
element of the resultant matrix C, c i j using this formula.

(Refer Slide Time: 26:47)

If you take one example here, suppose you have the matrix A; small matrix very 2 cross
2 matrix 4 0, 6 0. You have the matrix B 0 0 and 5 7. If I find out the product using our
procedure then first term 4 into 0, 0 into 5 it plus 0, so it is 0. Like this way, you will find
A B is 0 0. Similarly, your B A if you calculate, B A also you can calculate because
number of rows and number of columns of both the matrices are same. From your B A
you can calculate as, 0 0, 62 and 0. So, once I am calculating A B and B A one thing is
very clear that is, it is not necessary that A B will be always will be equals to B A. It may
be equals it may not be equals. Another thing is that; if product if you see here, although
A and B 2 non zero matrices, the product of these 2 are 0.

So, if product of 2 matrices A B equals 0, it does not necessarily imply that either A will
be equals to 0 or B equals to 0. We should note this one. Similarly, if you are having A B
equals A C, it does not necessarily imply that B will be equals to C. It may hold it may
not hold. So, these things we have to keep in mind. The next one is transpose of a matrix.

The transpose of a matrix is basically, whenever you are interchanging the rows and
columns of a matrix and the resultant matrix we call it as the transpose of the matrix.
That is, if you have a matrix A which you are telling as a i j m cross n matrix, suppose
you are having a 1 1 a 1 2 a 1 n, a 2 1 a 2 2 a 2 n and a m 1 a m 2 a m n, this matrix is
there.

So, if I write down A transpose; then A transpose will be by simply by interchanging the
number of rows and number of columns of this matrix. Means, here it will be a 1 1 a 2 1
and then a m 1. This column is being changed as row. The next one will be a 1 2 a 2 2
into a m 2 and the last one will be a 1 n a 2 n like this way a m n. So, transpose of a
matrix is a nothing but; whenever, you are interchanging the rows and columns of 2
matrices then we call it as the transpose of a matrix.

Transpose of a matrix has some properties like A transpose whole transpose if you take
you will get back the original matrix itself. A plus B transpose this is equals to A
transpose plus B transpose and A B transpose this is equals the opposite way it will go,
that is B transpose into A transpose. So, we will use these things for finding the solutions
you will see how we are actually using all these properties afterwards. How you are
representing, formulating a model converting in terms of matrices, you are trying find
out the solutions. So, we are going to that part.
(Refer Slide Time: 31:16)

The next one is determinant; consider a matrix A, A equals a i j of order n cross n. That
is it is a square matrix. Then det A we write denoted d e t A or this notion whatever, you
want you can use. Det A will be we write this way, a 2 1 and lastly so det A this one. For
example, if you have a 2 determinant of order 2 only that is a 1 1 a 1 2, a 2 1 a 2 2. I can
evaluate it; I can obtain the value that is this element into this element minus this element
into this element.

So, a 1 1 a 2 2 minus a 1 2 into a 2 1 like this way you can evaluate 1 determinant and
you can find a value. The next one is minor; the minor of an element a i j obviously, of a
determinate A, I am talking about with respect to A is the determinant of order n minus 1
by omitting the i th row and j th column. So, basically what is the minor? You have this
determinant, if I want to find out the minor of a 2 2 in that case; I will omit the second
row and second column and I will get another determinant whose order will be 1 less.

Since, I have deleted 1 row and 1 column; order will be n minus 1. And that we call as
the minor of the element a i j. The next term is cofactor; the cofactor of an element again
is defined as minus 1 to the power i plus j into M i j. Where, M i j is the minor of a i j.
So, once I know the minor, I can find out the cofactor of any element that is the minor
into minus 1 to the power 1 plus i plus j.
(Refer Slide Time: 34:15)

Next one is singular and non singular matrix; you have a matrix A which again, I am
writing as n cross n matrix of square matrix of order n. Now, we say that A is a singular
matrix if determinant of A is equals to 0. The matrix A is to be nonsingular matrix if
determinant of A is not equals to 0. So, if determinant of A is not equals to 0 then the
matrix is nonsingular, if it is equals to 0 then it is singular matrix. The next one is rank of
a matrix; you have a matrix A obviously, A equals a i j of order m cross n.

We say that, the rank of A these is r. If all square minors of order r plus 1 or above are
singular and there exist at least 1 minor of order r which is nonsingular. Then the rank of
that matrix is r that is all the minors of order r plus 1 whose are singular. And at least 1
minor will be there order of r which is nonsingular or whose determinant does not vanish
that we call as the rank of a matrix. Just let us take 1 example; you take a matrix A little
bigger, this one for this matrix A.

Let us perform some row operations like this, 1 operation I will do R 3 minus 2 R 1
another 1 we will do R 4 minus 3 R 1. If I do these 2 operations R 3 minus 2 R 1 that is
the first half first element of the third row will become 0. Similarly, the second element
of the third row will be become 0 like this way you do the operations. If we go through
this operations and after this I am not writing all of them because; it will take time I am
just writing here, the operations you will perform 1 after another. That is first these 2
then you operate R 3 plus R 2. Then you interchange the third row fourth row and at last
you do this operations R 3 plus 5 by sorry, R 3 plus it should be 5 by 2 R 2 3 plus 5 by 2
R 2 these operations you do in order one after another in this order. And ultimately you
will find 1 matrix of this particular form 1 3 2 4 1, 0 0 2 2 0, 0 0 0 3 3 and the last row
has become 0.

So, if you see these; what is the rank of this matrix now? The original matrix was this,
using row operations we transform this matrix into this one. You have the last row is
this. So, if you take the rank of this matrix will be 3, the rank will become 3 because; if
you take any matrix of order 4, that rank will be 0 over here because; 1 row is 0. So, if
you evaluate the determinant value will be 0, but if you take 1 matrix of size is 1
determinant of order 3 which will be non zero. So, therefore; rank is 3. So, rank is the
value for which the minor has non zero determinant or the minor is nonsingular this we
call as the rank of the matrix.

(Refer Slide Time: 39:24)

The next one is Adjoint of a matrix; the Adjoint of a matrix, you have a matrix A, a i j be
n square matrix of order n. Then Adjoint of A simply, we write as the transpose of just
transpose of the matrix of cofactors of A. So, you are first finding out the cofactor of
each element of the matrix A already, we have told how to find out the cofactor. And
then make transpose of the matrix of the cofactors that matrix resultant matrix is called
as Adjoint matrix. If you see, consider 1 example; A equals this, for this matrix cofactor
of 2 just I will calculate 1 2 cofactor of the first element 2 is you delete the
corresponding row and column. So, it is minus 1 to the power i plus j that is 1 plus 1 into
this matrix. If I delete this row and column, so it will be 0 1 and minus 1 4 if you
evaluate it will become 1.

Similarly, you are cofactor of 3 also, cofactor of minus 3 this element this will be equals
to minus 1 to the power 1 plus 2 into take the corresponding determinant after omitting
the corresponding row and the corresponding column and you will get some value. Like
this way I can find out the cofactors. So, Adjoint of a simply I can write down like this.
You find out the cofactors 1 minus 4 minus 1 you can calculate of your own. And this
one will come as this and transpose of this matrix that is you just write down this one as
1 8 minus 3 minus 4 8 2 and minus 1 2 and 3. So, this is the Adjoint of a matrix.

(Refer Slide Time: 42:09)

Next comes to the inverse of a matrix; the inverse of a matrix again, we are assuming
that this matrix A is square matrix and it is a nonsingular matrix of order n. So, we say
that matrix A is invertible, if there exist a nonsingular square matrix B their exist a
nonsingular square matrix B. Such that B; off course, the order of the B also will be n.
Such that A B equals B A equals I. Then and we denote this A inverse by Adjoint of A
by determinant of A.

So, actually what is happening; if the inverse of A matrix exist, if it is nonsingular, if it is

square. Now, if we can find a matrix B of order n, square matrix B of order n. Such that,
A B equals B A equals I. Where; off course, I is the identity matrix which we have told
earlier. Then A inverse we are defining as Adjoint A by determinant A. For the earlier
example whatever we told, A inverse can be calculated as this one, 1 by 10 into Adjoint
A which already Adjoint A, we have calculated in the earlier example as this one. This is
the Adjoint and the value of this determinant of this matrix A is 10. So, therefore;
basically a inverse becomes this one. So, this is preliminaries about the matrix.

(Refer Slide Time: 44:11)

Now, let us come to the next one; that is vector space. Basically, if you say matrices
having single row or column or often referred to as vectors; that is, if I write down
something like this. This we call as a vector of size n equals 5 or number of components
of this vector is 5. If you write something like this then; obviously, this is a column
vector, this one was the row vector, this is the column vector with of size 2 since, it has 2
elements. If you see, the vectors whenever I am writing this a 1, a 2 this actually
geometrically what I can say this represents a point in 2 dimensional space, when I am
writing a 1 a 2 and a 3 this a point in 3 dimensional space. Similarly, if you are writing a
1, a 2 like this way a n then we can imagine it as a point on the n dimensional space.

Now, come to the null vector; null vector is a vector whose all the components and
elements are 0 we denote it something like this way. So, null vector is the vector whose
all components are 0. Next one is the unit vector; unit vector usually we denote it by the
e i is the e I and what happens in the i th component the value will be 1 and all other
component value will be 0 then we call it as the unit vector. So, when you are writing e
1; that means, first component will be 1, all other components will be 0. Similarly, if I
am writing e 2 the second component will be 1 all the components will be 0. And if I am
writing e n then the last component will be 1 all other components will be 0 in this case.
So, this is your unit vector.

(Refer Slide Time: 46:56)

Now, come to the vector space; a set of vectors are there you assume a set of vectors x 1,
x 2, x n is there. This set of vectors x 1, x 2, x n is closed under addition and scalar
multiplication then; we say that this set of vectors form a vector space. So, if these
vectors x 1, x 2, x n are closed under addition and scalar multiplication then this forms a
vector space. So, next question; obviously, comes what do you means by closed under
addition and scalar multiplication? A vector closed under addition means that, if you take
any vector or 2 vectors of the set and if you add these2 vectors the resultant vector
should also be a member of the original set, Then we call it that it is your closed with
respect to addition. Similarly, vector is closed or a set is closed with respect to scalar
multiplication means, whenever you take a scalar multiply any vector the resultant vector
should also be member of the original matrix, original vector. Then we say that it is form
a vector space.

For example; if you try to take the examples, you take the set of all real numbers say;
you take any 2 real numbers you form the addition, you will get the another number
which is in the again a real number. Similarly, any 2 real numbers if I take and multiply I
will get another real number. So, set of real numbers are always forms a vector space.
Similarly, if I take set of all polynomials whose degree is less than or equals to n will
also form a vector space. But; consider the case set of all polynomials whose degree is
exactly equals to n. Let us take a set of polynomial whose degree is equals to n; that
means, I can write down P n equals something like this 1 polynomial a 1 x to the power n
minus 1 like this way a n. And Q n equals say, minus x to the power n b 1 x to the power
n minus 1 plus b n.

Now, if you make the addition of these two polynomials P n plus Q n. In that case; you
will find that, you are not you are getting 1 polynomial whose degree is n minus 1 but;
my set was set of all polynomials of degree n. So, we can conclude that set of all
polynomials will degree n will not form a vector space. So, like this way we can
calculate the vector space and check whether a set is a vector space or not.

(Refer Slide Time: 50:07)

The next one is linear combination; a linear combination suppose, you have a vector a,
which is in R n, if I write down a vector a is R n. It is a linear combination of vectors a 1,
a 2, a n. This a 1, a 2, a n also belongs to R n, if I can write down a equals lambda 1 a 1
plus lambda 2 a 2 plus lambda n a n for some scalars lambda. Sorry, for some lambda 2
and this will be lambda n for some scalars lambda 1 lambda 2 and lambda n. So, a vector
a is there; that is another set of vectors a 1 a 2 a n all of them belongs to R n. Then if I
can write down a equal lambda 1 a 1 lambda 2 a 2 plus lambda n a n in this form. Where,
lambda i is our scalars then we say that a is a linear combination of the vectors a 1 a 2 a
n. In particular, if summation i equals 1 to n lambda i this is equals to n then this
combination we call it as convex combination. Whenever, this summation i equals 1 to n
lambda i equals to 1 this we call it as the convex combination.

For example, if you take a vector a 1 (1, 2, 3), a 2 as (minus 1, 1, minus 1) and a 3 this
equals to say (0, 3, 2) always I can write down a 3 equals 1 into a 1 plus 1 into a 2. So,
your a 3 sorry, this will be a 1. a 3 will be a linear combination of a 1 and a 2. Here,
lambda 1 is 1 and lambda 2 also 1. Next comes linear dependence again, you had a set of
vectors a 1 a 2 a n which belongs to, this set of vectors a 1 a 2 an which is belongs to R n
is said to be linearly dependent. If it said to be linearly dependent, if there exist scalars
lambda i. Such that, lambda 1 a 1plus lambda 2 a 2 plus lambda n a n this is equals to 0.
But not all lambda i is a 0.

Then, we say the vectors are linearly dependent. So, a set of vectors a 1 a 2 a n are
linearly dependent. Whenever, lambda 1 a 1 plus lambda 2 a 2 plus lambda n a n this is
equals to 0 where, all lambda is are not equals to 0. So, for the earlier example, this
example a 1 a 2 a 3 these are linearly independent. Sorry, linearly dependent. Since, a 1
plus a 2 minus a 3 equals 0 and all lambda is not equals 0. But on the other way, if
lambda 1 a 1 plus lambda 2 a 2 plus lambda n a n this is equals 0. This holds when all
lambda i is a 0. That is lambda 1 equals lambda 2 equals lambda n equals 0. Then we say
that the vectors a 1 a 2 a n are they are linearly independent. I am just writing this, they
are linearly independent.

So, these two concepts are very independent. one is linearly dependent and another one
is linearly independent. So, linearly independent means, whenever you are make the
linear combination of this that is equals to 0, but all the lambda is are equals to 0. If you
consider; if I am just writing here, one vector a 1 and another vector equals this. So, if I
take lambda a 1 plus lambda 1 a 1 plus lambda 2 a 2 this is equals 0. And this is possible
only when lambda 1 lambda 2 equals 0. So, I can tell that the vectors lambda 1 sorry, the
vectors a 1 and a 2 are linearly independent.
(Refer Slide Time: 55:10)

Next there comes spanning set; again take a set of vectors a 1 a 2 a n which belongs to R
n. A set of vectors a 1 a 2 R n is set to span or generate R n if every vector of R n can be
written as a linear combination of a 1 a 2 a n. That is you are having a set of vectors from
this set of vectors you are generating the all the vectors means, any vector you take an R
n that will be a linear combination of the vectors a 1 a 2 a n. Then this set through which
you are generating we call it as the spanning set. And obviously; this set of vectors are
linearly independent vectors, this a 1 a 2 a n this spanning set should be linearly
independent. For example, if you want to take the example in that case; if you take the
set of vectors e 1 e 2 e n. If you take these vectors any vector on R n can be generated
from this set e 1 e 2 e n. So, your e n is 1 0 0 unit vectors on unity it is 1 on the first
component, all others are zeros.

So, any vector a can be represented in terms of this one that is you can write down a
equals a 1 e 1 plus a 2 e 2 plus a n e n. So, this is the spanning set of this. Next there
comes basis of a vector; a basis we call it linearly independent of set of vectors which
spans the R n, which spans the vector space. In our earlier examples; e 1 e 2 e n these are
the basis of the R n. So, basically the linearly independent of set of vectors which the
spanning all the elements all the vectors of set is called the basis of that. For this case; e
1 e 2 e n is the basis of this case. And along with this there is another one is which we
call as the dimension of a set. Dimension means, number of linearly independent vectors
whatever is there that number we call it as the basis. Now, next comes to convex set; you
have a set X, the set X as we call as convex set. If for any points say x 1 x 2 which
belongs to x, if you join the line segment of the x 1 to x 2 the line segment is also lies on
x then we say that the set is a convex set.

So, you have any 2 points; x 1 x 2 of X. If you join the line segment x 1 x 2, if inter line
segment lies on the set X then X is called the convex set. Mathematically, if I have to say
then we will say that, if x 1 x 2 belongs to X. If I can find a point a x that x equals
lambda 1 lambda x 1 plus 1 minus lambda into x 2 lambda lies between 0 to 1. If it is
satisfied then the set is a convex set. For example, if you consider; if you take a triangle
that triangle and its interior will form 1 convex set because if you take any point, any 2
points inside these if you join that will lie also inside the triangle.

Similarly, if I take a point something like this, in that case; also this will form a convex
set this entire rectangle place. But suppose, I have taken something like this, some figure
like this, this is not a convex set because; if I take a point here, if I take a point here, if I
join these line segment does not lie on this set. Therefore, this one is not a convex set.

Similarly, there is just another point that is extreme point. And extreme point of a convex
set X. This extreme point of a convex set will be that point such that, we cannot find 2
other points x 1 and x 2. Such that, if I join them I will get the point or in other sense, if I
have to say this point if you consider this point, this point these are the extreme points,
because I cannot find out any other 2 points which will lie over here. But if you consider
this x 1 say x 2 if you take another point x 3 although it is on the exterior. But I am
finding 2 points x 1 x 2 and if I joining them then x 3 lie on 8. Therefore, x 3 is not a
extreme point whereas, x 1 and x 2 is the extreme point.

Thank you.