Chapter 3

Systems of Linear Equations

Consider a system of 𝑛 linear equations in n unknowns
𝑎11 𝑥1 + 𝑎12 𝑥2 + ⋯ + 𝑎1𝑛 𝑥𝑛 = 𝑏1
𝑎21 𝑥1 + 𝑎22 𝑥2 + ⋯ + 𝑎2𝑛 𝑥𝑛 = 𝑏2
… ….. …….
𝑎𝑛1 𝑥1 + 𝑎𝑛2 𝑥2 + ⋯ + 𝑎𝑛𝑛 𝑥𝑛 = 𝑏𝑛
Where 𝑎𝑖𝑗 ,𝑖, 𝑗 = 1,2, … 𝑛 are the know coefficients , 𝑏𝑖 , 𝑖 = 1,2 … 𝑛 are the known right
hand side values and 𝑥𝑖 , 𝑖 = 1,2, … , 𝑛 are the unknowns to be determined.
In matrix notation we write the system as
𝐴𝑋 = 𝐵 … … … … … … … … … … … … … . . (1)
 a11 a12 ... a1n   x1   b1 
     
a a 22 ... a 2 n   x2   b2 
Where 𝐴 =  21 , 𝑋 = , and B 
... ... ... ...   ...   ... 
     
a ... a nn  x  b 
 n1 an2  n  n
The matrix [𝐴: 𝐵] is obtained by appending the column B to the matrix A is called the
augmented matrix i.e.
Methods to Solve Systems of Linear Equation
The methods of solution of the linear algebraic system of equ.1 may be classified as
direct and iterative methods (indirect method).
Direct Method
3.1. Gaussian Elimination Method
The method is based on the idea of reducing the given system of equations 𝐴𝑋 = 𝐵 to an
upper triangular system of equations 𝑈𝑋 = 𝑍 using elementary row operations. We know
that these two systems are equivalent i.e. the solutions of both the systems are identical.
This reduced system 𝑈𝑋 = 𝑍 is then solved by the back substitution method to obtain the
solution vector 𝑋.
Let us illustrate the method using 𝑛 = 3 but the method is analogous for 𝑛 > 3.

1
Consider the system
𝑎11 𝑥1 + 𝑎12 𝑥2 + 𝑎13 𝑥3 = 𝑏1
𝑎21 𝑥1 + 𝑎22 𝑥2 + 𝑎23 𝑥3 = 𝑏2 }
𝑎31 𝑥1 + 𝑎32 𝑥2 + 𝑎33 𝑥3 = 𝑏3
…………………………………………………..(1)
The augmented matrix of this system is
a11
(1) (1)
a12 (1)
a13 : b1 
(1)

 (1) (1) 
A : B  a21 (1)
a 22 (1)
a 23 : b2  ……………………………………………..(2)
a (1) (1)
a32 (1)
a33 : b3 
(1)
 31
a 21 a
Performing R2  R2  R1 and R3  R3  31 R1 ,we get
a11 a11

a11
(1) (1)
a12 (1)
a13 : b1(1) 
A : B   0 ( 2)
a 22 ( 2)
a 23

: b2( 2)  ……………………………………………...(3)
 0 ( 2)
a32 ( 2)
a33 : b3( 2) 


 a 21(1)   a 21(1) 
Where a 22
( 2)
 a 22
(1)
 a12
(1)
 (1)  , a 23
( 2)
 a 23  a13
(1) (1)
 (1) 
a  a 
 11   11 

(1)  a  (1)  a 
(1) (1)

 a32  a12  31(1)   a33  a13  31(1) 

( 2) (1) ( 2) (1)
a32  , a33 
 a11   a11 
 a (1)  (1)  a (1)  (1)
b ( 2) 2  b2   21(1) b1  b3   31(1) b1
(1) ( 2) (1)
 , b3  here we assume
 a11   a11 
a11  0

 a   a 
We call   21  and   31  as multipliers for the first stage a11 is called first pivot.
 a11   a11 
a11
(1) (1)
a12 (1)
a13 : b1(1) 
 
Now applying A : B    0 ( 2)
a 22 ( 2)
a 23 : b2( 2)  ………………………..(4)
 0 0 ( 3)
a33 : b3(3) 

Where

2
 a  a 
 a33   32 a 23 and b3  b3( 2)   32 b2 , we have assume
( 3) ( 2) ( 3)
a33
 a 22   a 22 

 a 
a22  0 , here the multiplier is   32  and new pivot is a 22 .
 a 22 
The augmented matrix (4) corresponds to an upper triangular system which can be solved
by backward substitution.
Remark: when any one of the pivots is zero or it is a very small number as the elimination
progresses. If a pivot is zero then division by zero gives over flow error, since division by zero is
not defined and if a pivot is a very small number then division by it introduces large round off
errors and the solution may contain larger errors.

Example: Solve the following system of equations using Gaussian Elimination Method

𝑥1 + 3𝑥2 + 5𝑥3 = 14

2𝑥1 − 𝑥2 − 3𝑥3 = 3

4𝑥1 + 5𝑥2 − 𝑥3 = 7

Solution:

Step 1: write the equations in matrix form:

Then, for conciseness, we combine the matrix of coefficients with the column vector of right
hand sides to produce the augmented matrix:

If the general system of equations is written AX = B then the augmented matrix is written
[ A|B].
Step 1 has now been completed. We will next triangularise the matrix of coefficients by means of
row operations. There are three possible row operations:

3
  interchange two rows;
 multiply or divide a row by a non-zero constant factor;
 add to, or subtract from, one row a multiple of another row.

Note that interchanging two rows of the augmented matrix is equivalent to interchanging the
two corresponding equations. The shorthand notation we use is introduced by example. To
interchange row 1 and row 3 we write R1 ↔ R3. To divide row 2 by 5 we write R2 ÷ 5. To add
three times row 1 to row 2, we write R2 + 3R1. In the Task which follows you will see where
these annotations are placed.

Note that these operations neither create nor destroy solutions so that at every step the system
of equations has the same solution as the original system.
Step 2: Triangularization
The second stage proceeds by first eliminating x1 from the second and third equations using
row operations.

In the above we have subtracted twice row (equation) 1 from row (equation) 2.
In full these operations would be written, respectively, as

Now since all the elements in rows 2 and 3 are negative we multiply throughout by −1:

Finally, we eliminate x2 from the third equation by subtracting equation 2 from equation 3
i.e. R3 − R2:

4
The system is now in triangular form.
Step 3: Back Substitution
Here we solve the equations from bottom to top. At each step of the back substitution process
we encounter equations which only have a single unknown and so can be easily solved.
In full the equations are

From the last equation we see that x3 = 3.

Substituting this value into the second equation gives

Finally, using these values for x2 and x3 in equation 1 gives x1 − 6 + 15 = 14. Hence x1 = 5.

The solution is therefore [x1 , x2 , x3 ]T = [5, −2, 3]T

Example 2: Solve

Solution:
Augmented matrix

Now subtract suitable multiples of row 1 from row 2 and from row 3 to eliminate the x1
coefficient from rows 2 and 3:

5
Now divide row 2 by 5 and add a suitable multiple of the result to row 3:

The equations in full are

The last equation reduces to 𝑥3 = 32

Using this value in the second equation gives 𝑥2 − 3 = −2 𝑠𝑜 𝑡ℎ𝑎𝑡 𝑥2 = 1

Finally, 𝑥1 − 1 + 29 = 3 𝑠𝑜 𝑡ℎ𝑎𝑡 𝑥2 = − 12

The solution is therefore

Equations which have an infinite number of solutions

Consider the following system of equations

In augmented form we have:

Now performing the usual Gauss elimination operations we have

6
Now applying R2 ÷ −5 and R3 ÷ −2 gives

Then R2 − R3 gives

We see that all the elements in the last row are zero. This means that the variable x3 can take any

value whatsoever, so let x3 = t then using back substitution the second row now implies

and then the first row implies

In this example the system of equations has an infinite number of solutions:

Where t can be assigned any value. For every value of t these expressions for x1 , x2 and x3
will simultaneously satisfy each of the three given equations.
Systems of linear equations arise in the modelling of electrical circuits or networks. By breaking
down a complicated system into simple loops, Kirchhoff’s law can be applied. This leads to a set
of linear equations in the unknown quantities (usually currents) which can easily be solved by one
of the methods described in this material.
Example: In the circuit shown find the currents (i1 , i2 , i3 ) in the loops.

7
Example:

8
3.2. Gauss Jordan Method
This method is based on the idea of reducing the given system of equations 𝐴𝑥 = 𝑏 to a diagonal
system of equations 𝐼𝑥 = 𝑑 where I is the identity matrix using elementary row operations we
know that the solutions of both the systems are identical .this reduced systems gives the solution
vector 𝑥.
In this case after the eliminations are completed we obtain the augmented matrix for a 3x3
system as
1 0 0 : d1 
0 1 0 : d  ………………………………………………..(*)
 2

0 0 1 : d 3 

and the solution is 𝑥𝑖 = 𝑑𝑖 , 𝑖 = 1,2,3.

9
The Gauss-Jordan elimination method to solve a system of linear equations is
described in the following steps.
1. Write the augmented matrix of the system.
2. Use row operations to transform the augmented matrix in the form described below,
which is called the reduced row echelon form (RREF).
(a) The rows (if any) consisting entirely of zeros are grouped together at the bottom of
the matrix.

(b) In each row that does not consist entirely of zeros, the leftmost nonzero element is
a 1 (called a leading 1 or a pivot).

(c) Each column that contains a leading 1 has zeros in all other entries.

(d) The leading 1 in any row is to the left of any leading 1’s in the rows below it.
3. Stop process in step 2 if you obtain a row whose elements are all zeros except the last
one on the right. In that case, the system is inconsistent and has no solutions.
Otherwise, finish step 2 and read the solutions of the system from the final matrix.
Note: When doing step 2, row operations can be performed in any order. Try to choose row
operations so that as few fractions as possible are carried through the computation. This makes
calculation easier when working by hand.
Example: Solve the following system by using the Gauss-Jordan elimination method.

Solution: The augmented matrix of the system is the following.

We will now perform row operations until we obtain a matrix in reduced row echelon form.

10
From this final matrix, we can read the solution of the system. It is

Example: Solve the following system by using the Gauss-Jordan elimination method.

Solution: The augmented matrix of the system is the following.

Let’s now perform row operations on this augmented matrix.

11
We obtain a row whose elements are all zeros except the last one on the right. Therefore, we
conclude that the system of equations is inconsistent, i.e., it has no solutions.

Example: Solve the following system by using the Gauss-Jordan elimination method.

Solution: The augmented matrix of the system is the following.

We will now perform row operations until we obtain a matrix in reduced row echelon form.

12
This last matrix is in reduced row echelon form so we can stop. It corresponds to the augmented
matrix of the following system.

We can express the solutions of this system as

Since there is no specific value for z, it can be chosen arbitrarily. This means that there are
infinitely many solutions for this system. We can represent all the solutions by using a parameter
t as follows.

13
Any value of the parameter t gives us a solution of the system. For example,

Exercise: solve the following system of equations using gauss Jordan method.
x1  x 2  x3  1
1. 4 x1  3x 2  x3  6
3x1  5 x 2  3x3  4

x1  2 x 2  x3  4
2. 2 x1  3x 2  x3  3
3x1  x 2  2 x3  3

Iterative Method
This method is based on the idea of successive approximations, we start with an initial
approximation to the solution vector 𝑋 = 𝑋0 to solve the system of equations 𝐴𝑋 = 𝐵 and
obtain a sequence of approximate vectors 𝑥0 , 𝑥1 , … , 𝑥𝑘 , .. which in the limit as 𝑘 → ∞ converges
to the exact solution vector 𝑋.
3.3. Gauss Jacobi Iteration (Jacobi) Method
𝑎11 𝑥1 + 𝑎12 𝑥2 + 𝑎13 𝑥3 = 𝑏1
Consider 21 𝑥1 + 𝑎22 𝑥2 + 𝑎23 𝑥3 = 𝑏2 }………………………….(1)
𝑎
𝑎31 𝑥1 + 𝑎32 𝑥2 + 𝑎33 𝑥3 = 𝑏3
And assume that the pivots 𝑎𝑖𝑖 ≠ 0 for 𝑖 = 1,2,3.
Write equ.1 as
𝑎11 𝑥1 = 𝑏1 − 𝑎12 𝑥2 + 𝑎13 𝑥3
𝑎22 𝑥2 = 𝑏2 − 𝑎21 𝑥1 + 𝑎23 𝑥3
𝑎33 𝑥3 = 𝑏3 − 𝑎31 𝑥1 + 𝑎32 𝑥2
The Jacobi iteration method is defined as

14
 x1( k 1) 
1
a11

b1  (a12 x 2( k )  a13 x3( k ) ) 
x 2( k 1) 
1
a 22

b2  (a 21 x1( k )  a 23 x3( k ) )  , k = 0,1,2 … ………….(2)

x3( k 1) 
1

b3  (a31 x1( k )  a32 x 2( k ) ) 
a33 }
Remark: a sufficient condition for convergence of the Jacobi method is that the system
of equations is diagonally dominant i.e. the coefficient matrix is diagonally dominant.
∑𝒏𝒊=𝟎|𝒂𝒊𝒋 | ≤ |𝒂𝒊𝒊 |, this implies that convergence may be obtained even if the system is not
𝒊≠𝒋

diagonally dominant, we may exchange the equations and if the new system is diagonally
dominant then convergence is guaranteed
How do we find the initial approximations to start the iterations? If the system is
diagonally dominant then the iteration converges for any initial solution vector. If no
suitable approximation is available, we can choose 𝑋 = 0 i.e. 𝑥𝑖 = 0 𝑓𝑜𝑟 𝑎𝑙𝑙 𝑖 then the
𝑏
initial approximation becomes 𝑥𝑖 = 𝑎 𝑖 , 𝑓𝑜𝑟 𝑎𝑙𝑙 𝑖.
𝑖𝑖

Example 1: Use the Jacobi method to approximate the solution of the following system
of linear equations.

Continue the iterations until two successive approximations are identical when rounded
to three significant digits.
Solution: To begin, write the system in the form

Because you do not know the actual solution, choose

Initial approximation
as a convenient initial approximation. So, the first approximation is

15
Continuing this procedure, you obtain the sequence of approximations shown in Table
below

Because the last two columns in Table are identical, you can conclude that to three
significant digits the solution is

Exercise: solve the system of equations

4 x1  x 2  x3  2
x1  5 x 2  2 x3  6 ,
x1  2 x 2  3x3  4
Using the Jacobi iteration method with the initial approximations
a. 𝑥𝑖 = 0, 𝑖 = 1,2,3
b. 𝑥1 = 0.5 , 𝑥2 = −0.5 , 𝑥3 = −0.5
Perform five iteration in each case.

Note: at every stage in the iteration, values of the previous iterations are used even
though the updated values of the previous variables are available. If we use the updated
values of 𝑥1 , 𝑥2 … . 𝑥𝑖−1 in computing the value of the variable 𝑥𝑖 , then we obtain a new
method called gauss seidal method.

16
3.4. Gauss Seidal Method
In this method we use the updated values of 𝑥1 , 𝑥2 … . 𝑥𝑖−1 , in computing the value of
the variable 𝑥𝑖 . We assume that the pivots 𝑎𝑖𝑖 ≠ 0 , 𝑓𝑜𝑟 𝑎𝑙𝑙 𝑖, we write the equations as
𝑎11 𝑥1 = 𝑏1 − 𝑎12 𝑥2 + 𝑎13 𝑥3
𝑎22 𝑥2 = 𝑏2 − 𝑎21 𝑥1 + 𝑎23 𝑥3
𝑎33 𝑥3 = 𝑏3 − 𝑎31 𝑥1 + 𝑎32 𝑥2
The gauss seidal iteration method is defined as

x1( k 1) 
1
a11

b1  (a12 x 2( k )  a13 x3( k ) ) 
x 2( k 1) 
1
a 22

b2  (a 21 x1( k 1)  a 23 x3( k ) )  , k = 0,1,2 …

x3( k 1) 
1

b3  (a31 x1( k 1)  a32 x 2( k 1) ) 
a33 }
Example 2: Use the Gauss-Seidel iteration method to approximate the solution to the
system of equations given in Example 1.
Solution: The first computation is identical to that given in Example 1. That is, using
(𝑥1 , 𝑥2 , 𝑥3 ) = (0,0,0) as the initial approximation, you obtain the following new value
for 𝑥1 .

Now that you have a new value for 𝑥1 , however, use it to compute a new value for
𝑥2 .That is,

Continued iterations produce the sequence of approximations shown in Table below.

17
Note that after only five iterations of the Gauss-Seidel method, you achieved the same
accuracy as was obtained with seven iterations of the Jacobi method in Example 1.
Neither of the iterative methods presented in this section always converges. That is, it is
possible to apply the Jacobi method or the Gauss-Seidel method to a system of linear
equations and obtain a divergent sequence of approximations. In such cases, it is said that
the method diverges.
Example 3: Divergence. Apply the Jacobi method to the system

Using the initial approximation (𝑥1 , 𝑥2 ) = (0,0)and show that the method diverges.
Solution: As usual, begin by rewriting the given system in the form

Then the initial approximation (0, 0) produces

as the first approximation. Repeated iterations produce the sequence of approximations

shown in Table below.

For this particular system of linear equations you can determine that the actual solution is
𝑥1 = 1 𝑎𝑛𝑑 𝑥2 = 1. So you can see from the Table above that the approximations given
by the Jacobi method become progressively worse instead of better, and you can
conclude that the method diverges.
Exercise: Find the solution of the system of equations
45 x1  2 x 2  3x3  58
 3x1  22 x 2  2 x3  47
5 x1  x 2  20 x3  67
Correct to three decimal places using the gauss Seidal method.

18
 3.3 Systems of non-linear equations using Newton's method
Definition: A system of non-linear equations is a set of equations as the following:

Definition: A solution of a system of equations 𝑓1 , 𝑓2 , ⋯ , 𝑓𝑛 in n variables in a point

(𝑎1 , 𝑎2 , ⋯ , 𝑎𝑛 ) ∈ 𝑅 𝑛 such that 𝑓1 (𝑎1 , 𝑎2 , ⋯ , 𝑎𝑛 ) = ⋯ = 𝑓𝑛 (𝑎1 , 𝑎2 , ⋯ , 𝑎𝑛 ) = 0.
Newton's Method
Given the continuous functions

(*)

We would like to find the values 𝑥 = 𝑥̂ 𝑎𝑛𝑑 𝑦 = 𝑦̂ such that 𝑓(𝑥̂, 𝑦̂) = 0 𝑎𝑛𝑑 𝑔(𝑥̂, 𝑦̂) = 0.

Assume that an initial approximate solution to equation (*) is known as (𝑥𝑖 , 𝑦𝑖 ). Express
𝑓(𝑥, 𝑦) 𝑎𝑛𝑑 𝑔(𝑥, 𝑦) in two variables Taylor series about (𝑥𝑖 , 𝑦𝑖 ), we get

(**)

Here, 𝑓(𝑥𝑖+1 , 𝑦𝑖+1 ) = 0 and 𝑔(𝑥𝑖+1 , 𝑦𝑖+1 ) = 0, since (𝑥𝑖+1 , 𝑦𝑖+1 ) will be the roots. Then,
truncating equation (**) after the first derivative terms and rearranging yields

(***)
Where ∆𝑥𝑖 = 𝑥𝑖+1 − 𝑥𝑖 , ∆𝑦𝑖 = 𝑦𝑖+1 − 𝑦𝑖 , writing equation (***) in matrix notation yields the
small changes below:

19
Where J(x, y) is the Jacobian matrix. Thus the new approximate solution will be

or equivalently

Example: Solve the given system of non-linear equation

Using Newton's method with initial guess (0, 1) and (1, 1)

Solution: First let us observe their graphs in the Figure below:

Graphs of f (x, y) and g(x, y).

From the graph, the given system has two solution around (0, 1) and (1, 1).
Now, take(𝑥0 , 𝑦0 ) = (0, 1) as initial guess

20
21
Since |𝑥6 − 𝑥5 | = 0.000 < 0.0005, so one of the root is (0.5249, − 0.7240).
Similarly, one can find the 2nd root with the initial guess (1, 1) (do as exercise).

22