Basic Linear Algebra

Prof. Inder K. Rana

Department of Mathematics
Indian Institute of Technology - Bombay

Lecture – 07
Reduced Row Echelon Form and Rank of a Matrix I

Okay, so let us begin today's lecture by recalling the main aspects of our row echelon form.
(Refer Slide Time: 00:29)

Because that is one of the important things. So let me just recall. Row echelon form of a matrix,
M means that there exists a number r between and the minimum of the row and the columns of the
matrix. M is m*n and there are natural numbers p1 p2 pr with the following properties. One, the
first r rows of the matrix are non-0, that means there is a at least 1 non-0 entry in that matrix, in
the rows. And the bottom rows are all 0. So remaining m-r rows are all equal to 0. So that is what
r means. There are first r rows, top r rows are non-0 and the remaining bottom are 0.
(Refer Slide Time: 01:25)
And the numbers pi indicate the place in ith row, pith place, the pith column, the entry is non-0
and that should be the first non-0 entry and the property is that the column numbers where the non-
0 entries come, for the first column should be strictly on the left side of column number 2 and so
on. So and these entries are called pivots that we saw.
(Refer Slide Time: 01:51)

So this is what the matrix looks like. So a matrix in the row echelon form will look like, there will
be some 0 in some column. The first non-0 entry at the first row comes at a place a1p1 and the
next comes for the second row, it comes at a2p2 where p2 is strictly bigger then p1. That means it
is on the right side of p2 and so on. And the bottom m-r rows are 0, right.
(Refer Slide Time: 02:21)
So we stated a theorem that every non-0 m*n matrix can be reduced, so it should be m*n, reduced
to a row echelon form.
(Refer Slide Time: 02:31)

So let me just go through 1 example again, how it is done so that it is clear to everybody. We are
given this matrix A. So the process is we scan the columns of the matrix, start with scanning the
columns. So this is the first column, okay. In the first column, the first non-0 entry comes at the
place. So the operation should be interchanged R1 with R2 so that the first entry in the first row
becomes non-0.

So let us do that. So once we interchange the second row with the first row, you get these 2 goes
up. So 2 1 0, this row becomes and 0 0 0 comes at the bottom. So the first entry in the first row has
become non-0. So the first row is a non-0 row, okay. Now once again what we do is, in that column,
everything else should be made 0.

So this is what the pivot is. Below the pivot, everything else should be made as 0. So you can
multiply this row R1 with 3, that will give you 6 here. Subtract it from the third row, that will make
this column, this entry 6 as 0. So multiply the first row with -3 and add it to the third row, it will
make it 0 and similarly for the next one.
(Refer Slide Time: 03:57)

So for that, so the operations are, so this R3 goes to, this is how we write it, R3 goes to R3-2R1.
R4 goes to R4+-3/2R1. So that will make everything below that column =0 by elementary row
operations. So once the first column is done, we look at the second column. So in this column now,
this row is already taken care of. So we should look at this submatrix, okay. So in this, here the
number 0 is coming, right.

So the first non-0 entry comes at the place -1 which is row number 3. So what we should be doing?
Interchange R3 with R2. So let us do that. So once you do that, you will get -1 up, -2 up and 3 up.
So this will become. So you will get a pivot in row 2 also. On the left, it is 0. That does not change
at all. And now once again, the operation should be make everything below the pivot to be equal
to 0 by row operations. So this already is 0. So multiply this by 1/2 and add, so make that as 0.
(Refer Slide Time: 05:11)

So once you do that, below the pivot -1, everything becomes 0. Now go on doing this. So go to the
next one. So next one again, this row is equal to 0. So you should interchange these 2, right. So
interchange R4 with R3 and so that will give you, so interchange will give you this, okay. So now
bottom row is all 0's.

So next column, we do not have to go to the next column at all now, right. Because we have gotten
the pivots 2 -1 3/2, okay. This will not make it a pivot, right. Even if you look at the last column,
this will not be pivot because the first non-0 entry in that row has already come, okay. So this
operation is over. So that means, okay.
(Refer Slide Time: 06:04)
So the last time we have started looking at how do you make this row operations operatable via
matrices? Because to put it on the machine, we cannot give an oral command or written command
that interchange R1 with R3. We have to give it executable command. So that is done by matrix
multiplications. So we are describing now the elementary matrices which are going to do the same
operations as you say interchange, add one row to another or multiply a row with another one.

So to do that, we start with an identity matrix, okay, m*m, any identity matrix of order m*m. What
you do? On this identity matrix, do the operation that you want to do. You want to interchange the
ith row with the jth row. So for identity matrix, interchange its ith row the jth row, that will give
you a new matrix, identity matrix will change, right. So the change matrix is called Eij, okay.

Or if you want to multiply the ith row by scalar alpha, right, then what you do? Take identity
matrix, multiply its ith row by alpha, you will get the matrix E alpha i, right. So this will be in the
ith column, ith row. It was 1 here in the identity, now I multiply it with alpha, right. So this is the
second type of operation, E alpha I, okay.
(Refer Slide Time: 07:41)
Similarly, you can do the next one. That is multiplying a row by a scalar and adding it to another
row, right. So that also can be done. What does this, so this is for the E interchanging of i and j.
(Refer Slide Time: 07:58)

So take the ith row. So what does this matrix means? We have taken the lower indicator, i means
you take the ith row to it, add alpha*the jth row. So identity matrix will change, right. That matrix
is denoted by Ei alpha*j. So operation goes, take i, to it add alpha*j. So you get 3 types of
elementary matrices, okay. These are called elementary matrices. What is the first one? Eij. What
does Eij means?

Take identity matrix, interchange its ith row with the jth row. So Eij simply means interchange i
with jth row. E alpha i multiply the ith row by the scalar alpha and that alpha should be non-0,
right. We want non-0. And the third one, take the ith row to the jth row multiply by alpha and add
it to the ith row, right, identity matrix. So this will change identity matrix into the third type of
matrix.

So these 3 are called elementary matrices. Why are they important? Because if I want in a matrix
A, I want to change its ith row to jth row, it is obtained by premultiplying the matrix A by the
corresponding elementary matrix. So suppose we have a got a matrix A which is m*n. You want
to interchange its ith row with the jth row. So take the corresponding m*m matrix Eij, premultiply
A with this and the resulting thing would be interchanging the rows of ith row of A with ith row
of j, right. So that one can write down the proofs of this.
(Refer Slide Time: 09:44)

So let us just look at examples of what E12 means what in 3*3. So what is E12, interchanging
identity matrix row 1 with row 2, right. So in identity matrix, it is 1 here, the left most corner, 1 0
0. So that has gone to row 2. Row 2 has come up. So this is E12. Similarly, 3+2c, E3, that means
what? In the third row, add 2 times second row multiplied by c.

So when the second row you multiply by c and added here, this was 0 earlier. So now it becomes
c here, right. So that is E lambda i, that means what? E lambda 1. E lambda 1 means multiplying
the first row by the scalar lambda. So these are illustrations of the 3 elementary matrices, right,
okay.
(Refer Slide Time: 10:43)

So the theorem says, we will not write down a formal proof of this. One can write down, multiply
and look at the ijth entry of every matrix. So there is only matter of writing. We will see how do
we use it. So it says the product of Ejk with A, okay. A is a matrix, it is m*n, m rows n columns.
So what should be the order of that matrix Ejk you should be taking? m*n. Because you are going
to premultiply, right.

A is m*n, so you should be taking the identity matrix m*m, changing, doing the operation there
and premultiply. So take the matrix Ejk which is, right, m*m, multiply it with A. So what will be
your resulting matrix? Eij is m*m. A is m*n. So the product will be m*n matrix. And will be same
as, as if you have interchanged the jth and the kth row of A, right.

So is it clear? So doing a row operation on a matrix is multiplying, premultiplying by the

appropriate order matrix, elementary matrix, with that operation done on the identity matrix.
Similarly, Ei alpha that will do, so Ei+alpha j, what this will do? It will take the jth row of A,
multiply it with c and add it to the ith row of A, right. So I think this is clearly written. So this is
type. And similarly the lambda one.
(Refer Slide Time: 12:22)
Let us see some examples of this. Let us take the matrix A, okay. This is the matrix and we want
to interchange its first row with its third row. So what I should be doing? This matrix is of order
of 3*4. So I should be looking at it elementary matrix of order 3*3 which is obtained from identity
matrix of order. So A is a matrix, we want to interchange its first row with the third row. Pick up
the corresponding elementary matrix, right.

So what should be the corresponding elementary matrix? It should be 1/3, right. That means in the
identity matrix of order 3*3, 3 is the number of rows, it was interchanged, first with the third one.
So when I interchange. This is what, E13 will look like, the elementary matrix. And now what
should I do? Take this matrix, premultiplying A with this matrix.

So let us do. So E13*A, so that is E13, that is A. Now if you multiply, right. So let us, this is 0,
this is 6 and this is 3. So 3 will come here, right. Matrix multiplication. You can check that. Clear
to everybody? Right? So pickup the corresponding elementary matrix, corresponding to the
elementary row operation that you want to do and premultiply, okay. So that is.
(Refer Slide Time: 13:49)
So the resulting matrix will be transformed accordingly. So the theorem says that we said every
matrix can be reduced to row echelon form, right, by row operations. In terms of matrices, what it
will mean? It will mean that if A is m*n matrix, then there exists elementary matrices, some
number of them, E1 E2 En of order m. They are all square matrices of order m such that if I
premultiply A with those, so these are the row operations we are doing via matrix multiplication.

This will be in the row echelon form, right. So we have just restated the theorem that every matrix
can be reduced to row echelon form by elementary row operations in terms of matrices, elementary
matrices, right. Now let us observe one thing. Suppose once you have reduced a matrix to row
echelon form, what is the row echelon form means? The first non-0 entry, right at the various rows
which is coming is in the increasing order, right.

But it does not say what is that non-0 entry. It says it is non-0. The pivot p1 for the first row is a
non-0 number, right. Let us make it 1. How do I make it 1? I will multiply that row by 1/that entry,
right. So by elementary row operation, the pivot value can be changed from the arbitrary number
whatever it is in the matrix to the number 1 by elementary row operation again. Once the pivot is
1, below the pivot, everything is 0 in the row echelon form.

That is what pivot means. Only possibility is above it, there could be non-0 entry sitting, right. But
if I make elementary row operations on that again, I can make those entries 0. Because on the left
of the pivot, everything is 0. So when I make row operations, nothing is going to change on the
left of that, right in the above rows. In the rows above the pivot. Clear to everybody? And I can
make suitable multiplications and adding subtracting the rows, the elementary row operations, so
that everything above the pivot also is equal to 0.

So this is a special form of row echelon form. Once you have gotten the row echelon form, I can
make the pivot value equal to 1 by elementary row operations and every other entry in that column
equal to 0. Below the pivot, everything is 0 already anyway. Above it also, you can make it 0 by
elementary row operations, by multiplying with suitable constant and adding in the rows above,
right.

So that is a special form. So we write this as that the row echelon form of the matrix can be further
changed, transformed by elementary row operations so that each pivot becomes 1 and all the entries
above each pivot also vanishes. Vanish means what? They become 0, okay. One should,
appropriate word should be they are made 0. So in the column where pivot is coming, right, above
the pivot everything is 0, below the pivot everything is 0, right.

So this form of the matrix is called the reduced row echelon form of the matrix. So that means
what? Every matrix by elementary row operations can be reduced to row echelon form as well as
reduced to, can be changed to reduced row echelon form where the pivots are 1 and every other
entry in that pivot column is equal to 0. We will see advantage of this form of the matrix, okay.
The advantage, one advantage is, we said that the row echelon form of matrix need not be unique,
right.

It only says where are the pivots coming and what order they are coming. They are coming like
columns are increasing order. Does it say anything more than that? For example, if a row is
multiplied with something, if a matrix is in row echelon form and you multiply a row with
something, it still remains in the row echelon form. It does not change, right. If you multiply it,
any row by a non-0 scalar, numbers will change but the positions and the ordering will not change
of the pivots, right.
But the reduced row echelon form of a matrix is always unique. Once you have done this,
normalizing, that means every pivot is made as 1 and every other entry is made 0, one can prove
the theorem that the reduced row echelon form of a matrix is unique. There is only 1 possibility,
right. Unique meaning what? Entries are same, whichever way you do it, does not matter,
whichever you do it, okay, right.
(Refer Slide Time: 18:55)

So this is what reduced row echelon form will look like. Just I picture for general one. There may
be some columns which are all 0, does not matter. The first row will have a non-0 entry at a place
p1, the first pivot is 1 and coming at a place p1. There may be something in between, 0 or non-0,
we do not bother about that. But the non-0 entry in the second row is again 1, comes at the column
number p2, right.

So p2 is on the right side of p1 and in that column, everything is 0. Around the left of that also,
everything is 0 because that is a pivot. On the left of the pivot, everything, right, in that column,
every row below that, it should be 0. So this is all 0. In the pivot column, 0 1 0, this is a second
row and goes so on. At the bottom, there will be some rows which are all 0 in the row echelon
form.

So what we are saying is the pivots has in the row echelon form, they are in the increasing order.
All the pivots, here is a pivot 1, here is pivot 1, here is pivot 1 and here is pivot 1 and everything
in that column, in the pivot column is equal to 0. So that is the general form of reduced row echelon
form of a matrix. So the theorem we had was every matrix can be reduced to.
(Refer Slide Time: 20:24)

So let us look at an example. Remember just now we looked at the matrix is beginning of the
lecture and said this is the echelon form of the matrix. So echelon form, here is the pivot 2, here is
the pivot -1, here is the pivot 3/2, right. There are 3 pivots, p1 column, p2 p3, okay. This is the
entry. These are the pivots. I want to change it to reduced row echelon form. So what should be
the first step?

This pivot should be made as 1. In the first row, the pivot entry is 2, that should be made as 1. So
how do I make it 1? Divide the first row by 2, right. So how the operation should be? Divide the
first row by 2, so that will make it 1. And then how do I make this as 1? Divide it by -1 and this
one, divide it by 3/2. So all the pivots will become 1. Once the pivots have become 1, I can use
that to make this entry 0, above the pivot.

This entry is 1, right. I can make that entry as 0 by doing a row operation in the first one and the
second one. That will not change this entry, right. That will stay as it is. So on the left side, the
entries will not change. So this pivot will remain as 1 by that row operation when I tried to make
this entry, the second entry in the row 1 equal to 0. I will, as such I can do it here itself also. I can
just add this row to this.
That will make it 0, right. So it is not, the way of attaining a reduced row echelon form is not
unique but the form is unique. You can first multiply every column 0 and then make a pivot 1 or
make the pivot 1 and then make every other entry in that column equal to 0. So once you do that,
so this is what will become. I have written those operations here. But you should try to do it
yourself, okay.

I might have made an arithmetical mistake, that is a possibility. But the entry idea should be, this
should be 1. The first pivot which was 2 earlier, should be 1 now. Here it was -1, that is 1 again.
And 3/2, that has been made as 1, okay. But these 2 are still not 0, right. So what should I do?
Multiply this row by -2 and add here, then this will become 0. So I will have reduced row echelon
form.

This will be some numbers. We do not bother about them, right. That is non-pivotal column. In
the reduced row echelon form, what matters is the pivotal columns, right. Pivotal columns are in
the increasing order, one and second, each entry, the pivot is 1 and each entry in that pivot is equal
to 0. Everything else is of irrelevance to us, right. And that form is unique. Clear to everybody
how to reduce a matrix?
(Refer Slide Time: 23:36)

So once you do those operations, this will become 1 and these are the entries which on the side
remained which are 0 non-0, we do not bother about them. So this is I have reduced a matrix to
the reduced row echelon form. So this is the reduced row echelon form of the matrix 𝐴, right. So
it is clear what we are doing?

You can reduce a matrix to a row echelon form, one by elementary row operations and you can go
on further refining that row echelon form, change it to what it is called reduced row echelon form
again by still elementary row operations only, right. The pivots are made 1 and every other entry
in that pivot column is made equal to 0, right. It is not order of doing operations. It is just property
here to finally achieve, right. That is reduced row echelon form of the matrix.