System of Linear Equations

Systems of linear equations lie at the heart of linear algebra, and this chapter uses them to
introduce some of the central concepts of linear algebra in a simple and concrete setting. Context
idea is decribed intuitively via concrete examples.

1 System of linear equation

A linear equation in the n variables x1 , x2 , ..., xn is an equation of form

a1 x 1 + a2 x 2 + · · · + an x n = b

where

• a1 , a2 , . . . , an : coefficients of x1 , x2 , . . . , xn

• b : constant term

Each variable in a linear equation occurs to the first power only.

A sequence (s1 , s2 , . . . , sn ) is a solution of the equation if

a1 s 1 + a2 s 2 + · · · + an s n = b

that is, if the equation is satisfied when the substitutions x1 = s1 , x2 = s2 , ..., xn = sn are made.

A finite collection of linear equations in the variables x1 , x2 , ..., xn is called a system of linear
equations in these variables.

A sequence of numbers is called a solution to a system of equations if it is a solution to

every equation in the system.

Example 1.1

x+y+ z =3
(

2x + y + 3z = 1

is system of linear equation in three variables x, y and z. The sequence (−2, 5, 0) is a solution of
this system because when we substitute x by -2, y by 5 and z by 0 into the first equation

−2 + 5 + 0 = 3

1
and into the second solution
2(−2) + 5 + 3(0) = 1
we see that both equations are satisfied. That is (−2, 5, 0) is a solution of both the first equation
and the second equation. Hence it is a solution of the system.

A system of linear equations can have no solution (inconsistent) or have at least one solution
(consistent) that is it has unique solution or many infinite solutions.

Example 1.2 The system

x+y =2
(

x+y =1
has no solution. So it is inconsistent.

Example 1.3 The system

x + y = 2y = 1
has unique solution (1, 1).

Example 1.4 For an arbitrate value of t, the sequence (1 − t, t) is a solution of the system

x+ y =1
(

2x + 2y = 1

For example, corresponding to t = 1, we have a solution (0, 1). Another possible solution cor-
responding to t = 3 is (−2, 3). So the set of all possible solution of this system equation is
{(1 − t, t) : t ∈ R} which is many infinite. The solution of form (1 − t, t) is said to be a general
solution in parametric form which depends on t called the parameter.

2 Gaussian Elimination Method

In order to solve a system of linear equation, we try to replace a system by a new system which is
easier to solve by a sequential steps as writing a series of systems, one after other such that each
equivalent to the previous system, that is they have the same set of solutions. Three elementary
operation used to simplify a linear system are

1. (Interchange) Interchange two equations.

2. (Scaling) Multiply one equation by a nonzero number.

3. (Replacement) Add a multiple of one equation to a different equation.

The following example illustrates how to solve a linear system by using eliminating method with
a sequence of elementary operations.

2
Example 2.1
x+ 2y + 3z = 6 x+ 2y + 3z = 6
 

 

eliminate x in the 2nd and 3th equation
 
2x − 3y + 2z = 14 −−−−−−−−−−−−−−−−−−−−−−→ − 7y − 4z = 2
 Eq2−2Eq1,Eq3−3Eq1 
3x + y − z = −2 − 5y − 10z = −20

 


x + 2y + 3z = 6 x + 2y + 3z = 6 x + 2y + 3z = 6
  

 
 

simplify Eq3 exchange Eq 2 and Eq3 eliminate y in Eq3
  
−−−−−−−→ − 7y − 4z = 2 −−−−−−−−−−−−−→ y + 2z = 4 −−−−−−−−−−→ y + 2z = 4
Eq3  Eq2↔Eq3  Eq3+7Eq2 
−5
y + 2z = 4 − 7y − 4z = 2 10z = 30

 
 


x + 2y + 3z = 6


simplify Eq3

−−−−−−−→ 
Eq3
y + 2z = 4
30
z=3



The last system equation is equivalent to the origin system but easier to solve by back substitution
as following: from the last equation, we have z = 3. Next substituiting z by 3 into the second
equation to get
y + 2(3) = 4 ⇒ y = −2
Finally, substituiting y = −2 and z = 3 into equation 1, we obtain
x + 2(−2) + 3(3) = 6 ⇒ x = 1
Hence the solution is (1, −2, 3).
The elimination procedure depends on the coefficient of variables and constant term in the system.
So we can store these value in an array or rectangle of numbers called a matrix. To illustrate the
matrix context with respect to a system of linear equation, we consider a concrete linear system
x+ 2y + 3z = 6





2x − 3y + 2z = 14
3x + y − z = −2



The matrix contains the coefficient called coefficient matrix

1 2 3
 

A = 2 −3 2 
 

3 1 −1
and the matrix contains the constant terms is said to be a constant matrix
6
 

b =  14 
 

−2
The matrix stores all coeffients and constant terms is called the augmented matrix
 

1 2 3 6
h i
A b = 2 −3 2 14
 
 
3 1 −1 2
Three elementary operations on linear system becomes three elementary row operations on
matrix

3
 1. (Interchange) Interchange two rows.
2. (Scaling) Multiply one row by a nonzero number.
3. (Replacement) Add a multiple of one row to a different row.
The elimination procedure on linear system is equivalent to the procedure called reduction pro-
cedure using the corresponding elementary row operations on the augmented matrices to end up
with an augmented matrix of stair form called row-echelon form. In the row-echelon form, the
leading 1s proceed “down and to the right” through the matrix. All entries below and to the left
of them are zero.

Example 2.2 Revisite Example 2.1, the corresponding augmented matrices in the reduction pro-
cedure are given by
   
1 2 3 6 1 2 3 6
r −2r ,r −3r
h i    
A b = 2 −3 2 14 −−−−−2−−−−−−−1−−−→ 0 −7
1 3
−4 2 
   
  eliminate x in Eq2 and Eq3  
3 1 −1 2 0 −5 −10 −20
   
r3
1 2 3 6 1 2 3 6
r ↔r
   
−−−−−−−−−−−−−→ 0 −7 −4 2 −−−−−−2−−− −−−→ 0 1 2 4
−5   3  
simplify Eq3 by scaling   exchange Eq2 and Eq3  
0 1 2 4 0 −7 −4 2
   

1 2 3 6 r3
1 2 3

6
r3 +7r2
−−−−−−−−−−→ 0 1 2 4  −−−−−−−10−−−−−−→ 0 1 2 4
   
eliminate y in Eq3   simplify Eq3 by scaling  
0 0 10 30 0 0 1 3

4
The last augmented matrix is in row-echelon form which corresponds to the system

x + 2y + 3z = 6



y + 2z = 4

z=3



that we obtain in the final step in Example 2.1.

Remark firstly that, from the previous reduction procedure, we can also obtain the corresponding
row - echelon form of the coefficient matrix which is
1 2 3
 

A → 0 1 2 .
 

0 0 1
Secondly the previous system is even easier to solve if we can eliminate y in the second equation
and eliminate both y and z in the first equation. To do so, we continuite to use elementary row
operations for eliminating procedure
     
1 2 3 6 1 2 0 −3 1 0 0 1
r −3r ,r −2r r −2r
     
0 1 2 4 −−−−−1−−−−−−−3−−−→ 0 1 0 −−→ 0 1 0
3 2 1 2
−2 −−−−− −−− −2
     
  eliminate y in Eq1 and Eq2   eliminate y in Eq1  
0 0 1 3 0 0 1 3 0 0 1 3
The system corresponds to the last augmented matrix is
=1

x


y = −2

z=3



And we can easy to read the solution (1, −2, 3) from the last column of the last augmented matrix.
This observation points out that we can find the solution (if exists) of a linear system the most
easily if in the corresponding augmented matrix, each leading 1 is the only non zero entry/element
in its column.

Such type of matrix is called reduced row echelon form.

In order to carry a matrix to (reduced) row echelon form, we use Gaussian algorithm as
following

5
Step 1 If the matrix consists entirely of zeros, stop

Step 2 Otherwise, find the first column from the left containing a nonzero entry (call it a), and
move the row containing that entry to the top position.

Step 3 Now multiply the new top row by 1/a to create a leading 1.

Step 4 By subtracting multiples of that row from rows below it, make each entry below the leading
1 zero.

Step 5 Repeat steps 1–4 on the matrix consisting of the remaining rows.

For a given matrix A, after reduction procedure with Gaussian algorithm, it can leads to
different row echelon forms but to the unique reduced row echelon form. Furthermore, the number
of 1’s leading in such row echelon forms is the same and said to be rank of matrix A, denoted
by r(A) or rank(A).

Example 2.3 As in Example 2.2, the reduction procedure leads a row echelon form of augmented
matrix and coefficient matrix
   
1 2 3 6 1 2 3 6
h i    
A b = 2 −3 2 14 → 0 1 2 4
   
   
3 1 −1 2 0 0 1 3
21 3 1 2 3
   

A= 2
−3 2  → 0 1 2
  

3 1 −1 0 0 1
h i h i
The row echelon form of the augmented matrix A b has three 1’s leading so r( A b ) = 3.
Similar we have r(A) = 3.

The following examples illustrate how rank of augmented matrix and coefficient matrix tell us
about the solution of a given linear system.

Example 2.4 In order to solve the system equation

y + 3z = 3





2x + 3y + 5z = 3
x + 2y + 4z = 5



we first reduce the augmented matrix to row echelon form with elementary row operations as fol-
lowing

6
    
0 1 3 3 1 2 4 5
 to create 1 leading in column 1 
h i  
A b = 2 3 5 3 −−−−−−−−−−−−−−−−−→ 2 3 5 3
 
  r ↔r 1 3  
1 2 4 5 0 1 3 3
 
1 2 4 5
turn all element below leading 1 in column 1 to 0

0 −1 −3
−−−−−−−−−−−−−−−−−−−−−−−−−−−→  −7

r2 −2r1  
0 1 3 3
 
1 2 4 5
create leading 1 in column 2
 
0 1 3
−−−−−−−−−−−−−−−−→  7

−r2  
0 1 3 3
 
1 2 4 5
turn all element below leading 1 in column 2 to 0
 
−−−−−−−−−−−−−−−−−−−−−−−−−−−→ 0 1 3 7
 
r3 −r2  
0 0 0 −4
 
1 2 4 5
create leading 1 in column 4
 
−−−−−−−−− −−−−−−−→ 0 1 3 7
 
r 3  
0 0 0 1
−4

The corresponding system is

x + 2y + 4z =5





y + 3z =7
0=1



which has no solution. The result comes from the last row
h i
0 0 0 1

which leads to a equation with no solution. Such kind of row also leads the different between rank
of coefficient matrix and augmented matrix.

In fact, from the last augmented matrix we can see obtain a row echelon form of coefficient
matrix
1 2 4
 

A → 0 1 3


0 0 0
h i
So r(A) = 2 but r( A b = 3) = r(A) + 1 due to the 1 leading in the last column. That is when
h i
r( A b = r(A) + 1, the system has no solution or it is inconsistent.

Example 2.5 Consider a linear system in three variables x, y, z such that after reduction proce-
dure, we obtain the row-echelon form of augmented matrix as following

7
  
1 2 4 15
h i reduction procedure  
A b −−−−−−−−−−−→ 0 1 3 7
 
 
0 0 1 2

The leading 1’s are the coefficient of variables x, y, z so these variables are called leading vari-
ables. In the corresponding system

x + 2y + 4z = 15



y + 3z = 7

z=2



h i
all variables are leading variables which implies that r( A b ) = r(A) = number of variables.
Now applying back substitution we find that the system has unique solution (5, 1, 2).

Example 2.6 Applying Gaussian algorithm to the augmented matrix of the system

x1 − 2x2 + 3x3 + x4 = −3
(

2x1 − 4x2 + 3x3 − x4 = 0

leads to row echelon form

 
h i reduction procedure 1 −2 3 1 −3
A b −−−−−−−−−−−→  
0 0 1 1 2

The corresponding system is

x1 − 2x2 + 3x3 + x4 = −3
(

x3 + x4 = 2
The 1’s
h leading
i are coefficients of x1 and x2 . So the leading variables are x1 , x3 . In this case
rank( A b ) = rank(A) = number of leading variables < number of variables in the system.
Using back substituition, from the last equation we can express the leading variable x3 in terms of
x4
x3 = 2 − x4
From the first equation, we can express x1 in terms of x2 , x3 , x4 and then in terms of x2 , x4

x1 = −3 + 2x2 − 3x3 − x4 = −3 + 2x2 − 3(2 − x4 ) − x4 = −9 + 2x2 + 2x4 .

For any arbitrary values of x2 , x4 , the correspond values of x1 , x3 from these previous formula give
a solution of the system. The variables x2 , x4 is called free variables.
So set the values of free variables x2 = s, x4 = t in terms of parameters s, t then x1 =
9 + 2s + 2t, x3 = 2 − t. So we obtain general solution in parametric form (9 + 2s + 2t, s, 2 − t, t).
The set of solution is {(9 + 2s + 2t, s, 2 − t, t) : s, t ∈ R} which is many infinite. The number of
parameters in general solution is equal to the number of free variable, that is the difference between
total number of variables and number of leading variables, in other word number of variables - r(A)

8
 In conclusion, by using Gaussian algorithm to carry augmented matrix h of a linear
i system in
row echelon form, we can determine the rank of augmented matrix rank( A b ) and rank of
coefficient matrix rank(A). Hence by comparing these ranks andcoefficient matrix and number of
variables in a linear system, we have information about solution of this system
h i
Theorem 2.1 Consider a system of linear equation with n variables and augmented matrix A b
then
h i
1. rank( A b ) is either rank(A) or rank(A) + 1
h i
2. The system is consistent if rank( A b ) = rank(A) = r

(a) if r = n, the system has unique solution

(b) if r < n, the system has infinite many solution. The set of solutions involves exactly
n − r parameters with respect to the free variables
h i
3. The system is inconsistent if rank( A b ) = rank(A) + 1

3 Homogeneous system
Definition 3.1 A system of equations in the variables x1 , x2 , ..., xn is called homogeneous if all
the constant terms are zero that is, if each equation of the system has the form

a1 x 1 + a2 x 2 + · · · + an x n = 0

Any homogenenous system admits a trivial solution x1 = 0, x2 = 0, . . . , , xn = 0

Any solution in which at least one variable has a nonzero value is called a nontrivial solution

A homogeneous system has either uniques (trivial) solution or infinitely may solution
In this section, we aim to represent the general solution of a homogeneous system in terms of
some special solutions called basis solutions. In order to do so, we first provide some ideas about
linear combination which will be reused in the vector space later.

Consider a set of matrix with 1 column and n rows which is called a column. Two columns are
said to be equal if they have the same number of entries and corresponding entries are the same.
Example 3.1
1
 
1 1
" # " #
A= , B = 2 , C =
 
2 2
3
We have A = C but A ̸= B.
We define the addition and scalar multiplication in this set of columns as following
x1 + y 1
       
x1 y1 kx1
Definition 3.2 If x = . . . and y = . . . then x + y =  . . .  and kx =  . . .  for any
       

xn yn xn + y n kxn
k ∈ R.

9
Definition 3.3 (Linear Combination) A sum of scalar multiples of several columns is called
a linear combination of these columns.

4 1 2
" # " # " #
Example 3.2 is a linear combination of and because
10 2 6

4 1 2
" # " # " #
=2 +
10 2 6

Now we turn to the problem of representing general solution of a homoegenous system via the
following example

Example 3.3 Using Gaussian algorithm, we can find that the general solution of the system
of homogeneous linear equations

x1 − 2x2 + 3x3 − 2x4 = 0





−3x1 + 6x2 + x3 =0

−2x1 + 4x2 + 4x3 − 2x4 = 0



is x1 = 2s + 51 t, x2 = s, x3 = 53 t, x4 = t and can be written in matrix form

2s 1
t
   
2s + 1
 
t 5
5 s 0
x= s  =   +  3  = sx1 + tx2
     
3  0   5 t
t t
5 0 t

2 1
   
5
1 0
where x1 =   and x2 =  3  which are called basis solutions. That is every solution of a
   
0 5
0 1
homogeneous system can be express as a linear combination of the basis solutions. Linear com-
bination is one of the best ways to describe the general solution of homogeneous system of linear
equations

Definition 3.4 (Basic solutions) The gaussian algorithm systematically produces solutions to
any homogeneous linear system, called basic solutions, one for every parameter.

A straight consequence of Theorem 2.1 for homogeneous system is the following result

Theorem 3.1 Let A be an m × n matrix of rank r, and consider the homogeneous system in n
variables with A as coefficient matrix. Then:

1. The system has exactly n − r basic solutions, one for each parameter.

2. Every solution is a linear combination of these basic solutions.

10