Matrices and Linear System of Equations

Solutions of Linear System of Equations

Dr. Sukanta Deb

Department of Physics, Cotton University

Panbazar, Guwahati 781001, Assam

May 25, 2021

Dr. Sukanta Deb Solutions of Linear System of Equations

 Matrices and Linear System of Equations

Outline

1 Matrices and Linear System of Equations

Dr. Sukanta Deb Solutions of Linear System of Equations

 Matrices and Linear System of Equations

Outline

1 Matrices and Linear System of Equations

Dr. Sukanta Deb Solutions of Linear System of Equations

 Matrices and Linear System of Equations

Solution of Linear Equations

Gauss Elimination Method
Gauss-Jordan Method
Gauss-Seidel Iterative Method
Jacobi Iterative Method

Dr. Sukanta Deb Solutions of Linear System of Equations

 Matrices and Linear System of Equations

Gauss Elimination Method

The Gaussian elimination method consists of two steps:

1 Reducing the matrix to its upper triangular form using
elementary row operations
2 Back Substitution
First Step: Let the system of n linear equations with n unknowns
are given by
Ax = b
Row operations consist of the following steps:
aik
aij ← aij − p ∗ akj and bi ← bi − p ∗ bk where p =
akk
for k = 1, 2, . . . n − 1, followed by i = k + 1, k + 2, . . . , n and
j = 1, 2, . . . , n. The above step reduces A|b to U|v .

Dr. Sukanta Deb Solutions of Linear System of Equations

 Matrices and Linear System of Equations

Gauss Elimination Method

Second Step: Backward substitutions are performed on U|v to
Pn
vn vi − k=i+1 uik xk
xn = and xi = , for i = n−1, n−2, . . . , 1.
unn uii
Solve the system:

2x1 + x2 + x3 =10
3x1 + 2x2 + 3x3 =18
x1 + 4x2 + 9x3 =16

Let
     
2 1 1 10 x1
A = 3 2 3 , b = 18 ,
 X = x2 

1 4 9 16 x3

Dr. Sukanta Deb Solutions of Linear System of Equations

 Matrices and Linear System of Equations

Gauss Elimination Method

First Step:
Operations with respect to the first row (k = 1): The row
operation is
Ri ← Ri − pR1 ,
for the rows i = 2, 3. The pivot element of the first row is
a11 = 2. Therefore, p = aakkik . For i = 2, p = aa21
11
= 23 and for
i = 3, p = aa31
11
= 12 . Therefore, for i = 2

R2 ←R2 − pR1
= 3 2 3 − 23 × 2 32 × 1 3
   
2 ×1
= 3 2 3 − 3 32 23
   

= 3 − 3 2 − 32 3 − 23
 

= 0 12 32
 

Dr. Sukanta Deb Solutions of Linear System of Equations

 Matrices and Linear System of Equations

Gauss Elimination Method

For i = 3,

R3 ←R3 − pR1
= 1 4 9 − 21 × 2 12 × 1 1
   
2 ×1
= 1 4 9 − 1 12 12
   

= 1 − 1 4 − 12 9 − 12 )
 

= 0 72 17
 
2

Dr. Sukanta Deb Solutions of Linear System of Equations

 Matrices and Linear System of Equations

Gauss Elimination Method

Also
bi ← bi − pbk
Therefore, for i = 2, 3
1
b2 ← b2 − 32 b1 ,
b3 ← b3 − b1
2
1
= (18 − 32 × 10), = (16 − × 10)
2
= 3, = 11

Therefore, after first row operation, the matrix A and b becomes

   
2 1 1 10
A = 0 21 32  , b =  3 
0 72 172 11

Dr. Sukanta Deb Solutions of Linear System of Equations

 Matrices and Linear System of Equations

Gauss Elimination Method

Operations with respect to the second row (k = 2): The

row operation is
Ri ← Ri − pR2 ,
for the row i = 3. The pivot element of the second row is
a22 = 12 . Therefore, p = aakkik . For i = 2, p = aa32
22
= 72 × 2 = 7.
Therefore, for i = 2

R3 ←R3 − 7R2 b3 ←b3 − 7b2

= 0 27 17 7 21
   
2 − 0 2 2 =(11 − 7 × 3)
= 0 − 0 27 − 72 72 − 21
 
2 = − 10
 
= 0 0 −2

Dr. Sukanta Deb Solutions of Linear System of Equations

 Matrices and Linear System of Equations

Gauss Elimination Method

After first & second row operations, we get
   
2 1 1 10
A = 0 12 3
2 , b =
 3
0 0 −2 −10

Second Step: The reduced system of linear equations after row

operations becomes
    
2 1 1 x1 10
= 0 12 3  
2 x2 =  3 
0 0 −2 x3 −10

This system is solved by applying backward substitutions to get

b3 −10
x3 = = =5
a33 −2

Dr. Sukanta Deb Solutions of Linear System of Equations

 Matrices and Linear System of Equations

Gauss Elimination Method

b2 − a23 ∗ x3 (3 − 32 ∗ 5)
x2 = = 1
= −9
a22 2
b1 − (a12 ∗ x2 + a13 ∗ x3 ) 10 − (1 ∗ (−9) + 1 ∗ 5)
x1 = = =7
a11 1

Therefore, the required solution is

x1 = 7, x2 = −9, x3 = 5

Dr. Sukanta Deb Solutions of Linear System of Equations

 Matrices and Linear System of Equations

Gauss Elimination (with pivoting) Method

Solve the following system of linear equations with the

Gaussian elimination method with partial pivoting:

2x1 + 2x2 + x3 = 6
4x1 + 2x2 + 3x3 = 4
x1 − x2 + x3 = 0
 
2 2 1 6
Let A|b = 4 2 3 4. From the first column, we find that
1 −1 1 0
the pivot element (the largest absolute value element) is 4.
Interchanging the row 
corresponding  to this element with the first
4 2 3 4
row, we obtain A|b = 2 2 1 6.
1 −1 1 0
Dr. Sukanta Deb Solutions of Linear System of Equations
 Matrices and Linear System of Equations

Gauss Elimination (with pivoting) Method

Eliminating the first variable from the second row onwards using
the elementaryrow operations of basic Gauss elimination method,
4 2 3 4
we get A|b = 0 1 − 21 10 
3 . From the second row
3 1
0 −2 4 −1
onwards, the pivot element is searched in the second column. It
can be seen that it is − 32 . Interchanging the second row with the
 
4 2 3 4
pivot element row, A|b = 0 − 32 1
4 −1 . Eliminating the
1 10
0 1 −2 3
second variable from the second row  onwards using thebasic Gauss
4 2 3 4
3 1
elimination method, we get A|b = 0 − 2 
4 −1 .
1 10
0 0 −3 3

Dr. Sukanta Deb Solutions of Linear System of Equations

 Matrices and Linear System of Equations

Gauss Elimination (with pivoting) Method

Therefore, after elementary row operations with partial pivoting,
we obtain
   
4 2 3 4
A = 0 − 23 1
4 , b =  − 31 
1 10
0 0 −3 3

The reduced system of linear equations after row operations

becomes
    
4 2 3 x1 4
0 − 3 1 
x2 =  −1 
2 4
1 10
0 0 −3 x3 3

This system is solved by applying backward substitutions to get

10
b3
x3 = = 31 = −10
a33 −3

Dr. Sukanta Deb Solutions of Linear System of Equations

 Matrices and Linear System of Equations

Gauss Elimination (with pivoting) Method

b2 − a23 ∗ x3
x2 = = −1
a22
b1 − (a12 ∗ x2 + a13 ∗ x3 )
x1 = =9
a11
Therefore, the required solution is

x1 = 9, x2 = −1, x3 = −10

Dr. Sukanta Deb Solutions of Linear System of Equations

 Matrices and Linear System of Equations

Gauss-Jordan Method
The Gauss-Jordan method is a variation of Gauss elimination
method. The major difference is that

When an unknown is eliminated in

the Gauss-Jordan method, it is
eliminated from all other equations
rather than just the subsequent
ones.
In addition, all rows are normalized
by dividing them by their pivot
elements. Thus, the elimination
step results in an identity matrix
rather than a triangular matrix.
Consequently, it is not necessary to
employ back substitution to obtain Figure: Graphical depiction of
the solution. the Gauss-Jordan method.
Dr. Sukanta Deb Solutions of Linear System of Equations
 Matrices and Linear System of Equations

Problems on Gauss-Jordan Method

Solve the following using Gauss-Jordan method:

3x1 − 0.1x2 − 0.2x3 =7.85

0.1x1 + 7x2 − 0.3x3 = − 19.3
0.3x1 − 0.2x2 + 10x3 =71.4

The coefficient matrix is: Column vector b is:

   
3 −0.1 −0.2 7.85
A = 0.1 7 −0.3 , b = −19.3
0.3 −0.2 10 71.4
The augmented matrix is
 
3 −0.1 −0.2 7.85
A|b =A|b = 0.1 7 −0.3 −19.3
0.3 −0.2 10 71.4

Dr. Sukanta Deb Solutions of Linear System of Equations

 Matrices and Linear System of Equations

Problems on Gauss-Jordan Method

 
1 −0.033333 −0.066667 2.61667
A|b = 0.1 7 −0.3 −19.3  R1 ←R1 /3
0.3 −0.2 10 71.4
 
1 −0.033333 −0.066667 2.61667 
=  R2 ←R2 −0.1R1
0 7.00333 −0.293333 −19.5617 R3 ←R3 −0.3R1
0 −0.190000 10.0200 70.6150
 
1 −0.033333 −0.066667 2.61667 
=  R2 ←R2 /7.00333
0 1 −0.0418848 −2.79320
0 −0.190000 10.0200 70.6150
 
1 0 −0.0680629 2.52356 
=  R1 ←R1 +0.033333R2
0 1 −0.0418848 −2.79320 R3 ←R3 +0.190000R2
0 0 10.01200 70.0843
Dr. Sukanta Deb Solutions of Linear System of Equations
 Matrices and Linear System of Equations

Problems on Gauss-Jordan Method

 
1 0 −0.0680629 2.52356
A|b = 0 1 −0.0418848 −2.79320 R3 ←R3 /10.01200
0 0 1 7.0000
 
1 0 0 3.0000 
=  R1 ←R1 +0.0680629R3
0 1 0 −2.5000 R2 ←R2 +0.0418848R3
0 0 1 7.0000

Thus the coefficient matrix has beem transformed to the identity

matrix. The solution is obtained in the right-hand-side vector.
Therefore no back substitution was required to obtain the solution.
The solution is x1 = 3.0, x2 = −2.5, x3 = 7.0

Dr. Sukanta Deb Solutions of Linear System of Equations

 Matrices and Linear System of Equations

Gauss-Seidel Method

The two most commonly used iterative methods for solving a set
of linear equations are:
jacobi’s method
Gauss-Seidel method
In the case of Jacobi’s method, the values of the unknowns are
updated when an iteration has completed. On the other hand, in
the case of Gauss-Seidel method, the values of the unknowns are
updated immediately without waiting for the iteration to complete.
These two methods are basically similar, having the same formula
for updating the values. Both methods require the coefficient
matrix of the system to be a special type of matrix called a
diagonally-dominant matrix. The difference between them lies in
the way updates are made: Gauss-Seidel uses the latest values of
the variables, whereas Jacobi does not use the latest values.

Dr. Sukanta Deb Solutions of Linear System of Equations

 Matrices and Linear System of Equations

Gauss-Seidel Method
The difference between the Gauss-Seidel method and Jacobi
iteration is depicted in the following figure.

Figure: Graphical depiction of the difference between (a) the

Gauss-Seidel and (b) the Jacobi iterative methods for solving
simultaneous linear algebraic equations.

Dr. Sukanta Deb Solutions of Linear System of Equations

 Matrices and Linear System of Equations

Diagonally-dominant Matrix

A square matrix A is said to be diagonally-dominant if the absolute

value of its diagonal element in each row is greater than the sum
of the absolute value of all other elements in the same row. This
condition is expressed as follows:

n
X
|aii | > |aij | , for i = 1, 2, . . . , n
j=1
j6=i

In the above definition, a diagonally-dominant matrix must always

have the largest absolute values at its diagonal elements
Theorem
If the coefficient matrix A is diagonally-dominant, then the
Gauss-Seidel method converges for any starting vector.

Dr. Sukanta Deb Solutions of Linear System of Equations

 Matrices and Linear System of Equations

Diagonally-dominant Matrix
 
5 −1 3
The following matrix is diagonally-dominant: A = 2 −7 −1,
4 −2 8
since

Row1 :|5| > | − 1| + |3| (|a11 | > |a12 | + |a13 |),

Row2 :| − 7| > |2| + | − 1| (|a22 | > |a21 | + |a23 |),
Row3 :|8| > |4| + | − 2| (|a33 | > |a31 | + |a32 |)

The following matrix is not diagonally-dominant but becomes

diagonally-dominant by exchanging rows 2 and 3:
   
5 −1 3 5 −1 3
R2 ↔R3
2 3 −6 −−−−→ 3 −7 2
3 −7 2 2 3 −6

Dr. Sukanta Deb Solutions of Linear System of Equations

 Matrices and Linear System of Equations

Gauss-Seidel Method
Let us consider 3 × 3 set of equations:
a11 x1 + a12 x2 + a13 x3 = b1
a21 x1 + a22 x2 + a23 x3 = b2
a31 x1 + a32 x2 + a33 x3 = b3
If the diagonal elements aii 6= 0 (i = 1, 2, 3), the first equation can
be solved for x1 , the second for x2 and the third for x3 to yield
b1 − a12 x2 − a13 x3
x1 = (1)
a11
b2 − a21 x1 − a23 x3
x2 = (2)
a22
b3 − a31 x1 − a32 x2
x3 = (3)
a33
Now, we can start the solution process by choosing guesses for the
x’s.
Dr. Sukanta Deb Solutions of Linear System of Equations
 Matrices and Linear System of Equations

Gauss-Seidel Method
A simple way to obtain initial guesses is to assume that they are all
zero, i.e., x1 = x2 = x3 = 0. These zeros (x2 = x3 = 0) can be
substituted into Equation (1), which can be used to calculate a
new value for x1 = b1 /a11 . Then, we substitute this new value of
x1 along with the previous guess of zero for x3 into Equation (2) to
compute a new value for x2 . The process is repeated for
Equation (3) to calculate a new estimate for x3 . Then we return to
the first equation and repeat the entire procedure until our solution
converges closely enough to the true values.
Note: Before applying the Gauss-Seidel method, please ensure
that the coefficient matrix A is diagonally-dominant,i.e.,
n
X
|aii | > |aij | , for i = 1, 2, . . . , n
j=1
j6=i

Dr. Sukanta Deb Solutions of Linear System of Equations

 Matrices and Linear System of Equations

Problems on Gauss-Seidel Method

Solve the following system of equations using the Gauss-Seidel

method:

8x1 − 3x2 + 2x3 = 20

4x1 + 11x2 − x3 = 33 (Diagonally-dominant matrix)
x1 + x2 + 4x3 = 9

Using the iteration equations of the Gauss-Seidel method,

b1 − a12 x2 − a13 x3 20 + 3x2 − 2x3
x1 = = (4)
a11 8
b2 − a21 x1 − a23 x3 33 − 4x1 + x3
x2 = = (5)
a22 11
b3 − a31 x1 − a32 x2 9 − x1 − x2
x3 = = (6)
a33 4

Dr. Sukanta Deb Solutions of Linear System of Equations

 Matrices and Linear System of Equations

Problems on Gauss-Seidel Method

Let us assume the initial approximations as: x1 = x2 = x3 = 0
(Iteration = 0). Putting x2 = x3 = 0 in Equation (4), we get
x1 = 2.5000. Now using x1 = 2.5000 and x3 = 0 in Equation (5),
we get x2 = 2.0909. Using the value x1 = 2.5000 and x2 = 2.0909
in Equation (6), we get x3 = 1.1023. Therefore, after the first
iteration, we get the values as x1 = 2.5000, x2 = 2.0909,
x3 = 1.1023. For successive iterations, the following Table is
constructed using the above set of equations.
Iteration x1 x2 x3
0 0 0 0
1 2.5000 2.0909 1.1023
2 3.0085 2.0062 0.9963
3 3.0032 1.9985 0.9996
4 3.0 2.0 1.0

Therefore, the solution is x1 = 3, x2 = 2.0, x1 = 1.0 .

Dr. Sukanta Deb Solutions of Linear System of Equations
 Matrices and Linear System of Equations

Problems on Gauss-Seidel Method

Solve the following system of equations using the Gauss-Seidel

method:

28x1 + 4x2 − x3 = 32
x1 + 3x2 + 10x3 = 24 (Not a diagonally-dominant matrix)
2x1 + 17x2 + 4x3 = 35

We can see that the coefficient matrix is not a diagonally

dominant matrix. To make the coeficient matrix a diagonally
dominant matrix, we rearrange the equations as:

28x1 + 4x2 − x3 = 32
2x1 + 17x2 + 4x3 = 35 (Diagonally-dominant matrix)
x1 + 3x2 + 10x3 = 24

Dr. Sukanta Deb Solutions of Linear System of Equations

 Matrices and Linear System of Equations

Problems on Gauss-Seidel Method

Using the iteration equations of the Gauss-Seidel method,

b1 − a12 x2 − a13 x3 32 − 4x2 + x3
x1 = =
a11 28
b2 − a21 x1 − a23 x3 35 − 2x1 − 4x3
x2 = =
a22 2
b3 − a31 x1 − a32 x2 24 − x1 − 3x2
x3 = =
a33 10
Let us assume the initial approximations as: x1 = x2 = x3 = 0
(Iteration = 0). Putting x2 = x3 = 0 in the first Equation, we get
x1 = 1.1429. Now using x1 = 1.1429 and x3 = 0 in the second
Equation, we get x2 = 1.9244. Using the value x1 = 1.1429 and
x2 = 1.9244 in the third Equation, we get x3 = 1.7084.

Dr. Sukanta Deb Solutions of Linear System of Equations

 Matrices and Linear System of Equations

Problems on Gauss-Seidel Method

Therefore, after the first iteration, we get the values as
x1 = 1.1429, x2 = 1.9244, x3 = 1.7084. For successive iterations,
the following Table is constructed using the above set of equations.

Iteration x1 x2 x3
0 0 0 0
1 1.1429 1.9244 1.7084
2 0.9290 1.5476 1.8428
3 0.9876 1.5090 1.8485
4 0.9933 1.5070 1.8486
5 0.9936 1.5070 1.8485
6 0.9936 1.5070 1.8485

Therefore, the solution is

x1 = 0.9936, x2 = 1.5070, x3 = 1.8485

Dr. Sukanta Deb Solutions of Linear System of Equations

 Matrices and Linear System of Equations

Jacobi Method
A simple way to obtain initial guesses is to assume that they are all
zero, i.e., x1 = x2 = x3 = 0. These zeros (x2 = x3 = 0) can be
substituted into Equation (1), which can be used to calculate a
new value for x1 = b1 /a11 . Similarly the zeros (x1 = x3 = 0) can
be used to compute a new value for x2 using Equation (2). The
process is repeated for Equation (3) to calculate a new estimate for
x3 using x1 = x2 = 0. Then we return to the first equation and
repeat the entire procedure until our solution converges closely
enough to the true values.
Note: Before applying the Jacobi method, please ensure that the
coefficient matrix A is diagonally-dominant,i.e.,
n
X
|aii | > |aij | , for i = 1, 2, . . . , n
j=1
j6=i

Dr. Sukanta Deb Solutions of Linear System of Equations

 Matrices and Linear System of Equations

Problems on Jacobi’s Method

Solve the following system of equations using Jacobi’s
method:

28x1 + 4x2 − x3 = 32
2x1 + 17x2 + 4x3 = 35
x1 + x2 + 10x3 = 24

We can see that the coefficient matrix is a diagonally dominant

matrix. The iteration equations of the Jacobi method are:
b1 − a12 x2 − a13 x3 32 − 4x2 + x3
x1 = =
a11 28
b2 − a21 x1 − a23 x3 35 − 2x1 − 4x3
x2 = =
a22 17
b3 − a31 x1 − a32 x2 24 − x1 − x2
x3 = =
a33 10
Dr. Sukanta Deb Solutions of Linear System of Equations
 Matrices and Linear System of Equations

Problems on Jacobi’s Method

Let us start with the initial approximation: x1 = x2 = x3 = 0.

Then the first approximation is

32 − 4(0) + 0
x1 = = 1.1429
28
35 − 2(0) − 4(0)
x2 = = 2.0588
17
24 − 0 − 0
x3 = = 2.4000.
10
Repeating this procedure, we obtain the successive approximations
as shown below:

Dr. Sukanta Deb Solutions of Linear System of Equations

 Matrices and Linear System of Equations

Problems on Jacobi’s Method

Iteration x1 x2 x3
0 0 0 0
1 1.1429 2.0588 2.4000
2 0.9345 1.3597 2.0798
3 1.0229 1.4595 2.1706
4 1.0119 1.4278 2.1518
5 1.0157 1.4335 2.1560
6 1.0151 1.4320 2.1551
7 1.0152 1.4323 2.1553

Therefore, the solution of the given linear system of equations is

x1 = 1.015, x2 = 1.432, x3 = 2.155

, correct up to three decimal places.

Dr. Sukanta Deb Solutions of Linear System of Equations
 Matrices and Linear System of Equations

Internal Assessment-I

Solve the following system of equations using the Gauss

elimination method:

2x1 + 8x2 + 2x3 = 14

x1 + 6x2 − x3 = 13
2x1 − x2 + 2x3 = 5

Solve the following system of equations using the Gauss-Seidel

method:

12x1 + 3x2 − 5x3 = 1

x1 + 5x2 + 3x3 = 28
3x1 + 7x2 + 13x3 = 76

Dr. Sukanta Deb Solutions of Linear System of Equations

 Matrices and Linear System of Equations

Internal Assessment-I

Solve the following system of equations using the

Gauss-Jordan method:

x1 + x2 − x3 = −3
6x1 + 2x2 + 2x3 = 2
−3x1 + 4x2 + x3 = 1

Solve the following system of equations using the Jacobi

method:

6x1 + x2 + x3 = 20
x1 + 4x2 − x3 = 6
x1 − x2 + 5x3 = 7

Dr. Sukanta Deb Solutions of Linear System of Equations

 Matrices and Linear System of Equations

A Quote

Nature is beautiful!
“The scientist does not study nature because it is
useful; he studies it because he delights in it, and he
delights in it because it is beautiful. If nature were
not beautiful, it would not be worth knowing, and if
nature were not worth knowing, life would not be
worth living.”
—Henri Poincaré.

Dr. Sukanta Deb Solutions of Linear System of Equations

 Matrices and Linear System of Equations

Contact Me?
QPPPPPPRQPPPPPPRQPPPPPPRQPPPPPPRQPPPPPPRQPPPPPPRQPPPPPPRQPPPPPPRQPPPPPPR

THANK YOU!

For any queries, suggestions:

k sukantodeb@gmail.com

T 9810362087

Í https://sites.google.com/site/sukantodeb

QPPPPPPRQPPPPPPRQPPPPPPRQPPPPPPRQPPPPPPRQPPPPPPRQPPPPPPRQPPPPPPRQPPPPPPR
Dr. Sukanta Deb Solutions of Linear System of Equations