Lesson 04 Linear Algebra
Lesson 04 Linear Algebra
Linear Algebra
Learning Objectives
Direction of vector v
Direction of vector v
Length of vector v
->
v
O (0,0)
Depending on the context and mathematical notation, vectors can be represented in numerous
ways. Common representations of vectors include:
P (4,4)
-> 4
v ->
V =
4
O (0,0)
Geometric representation:
P (4,4)
->
v • The length of the arrow corresponds to the
magnitude of the vector
• The direction of the arrow represents the
direction of the vector.
O (0,0)
For example, in a two-dimensional space, a vector v can be represented as an arrow from the
origin (0,0) to the point (x, y).
Vector Representation
Algebraic representation:
Basis vector It forms a set of linearly independent vectors that span a vector space.
Properties of Vectors
i
Vector Operations
Vector Operations: Addition
Vector addition refers to the operation of combining two or more vectors to yield a
vector sum.
z=u+v
u
u
v
Vector Operations: Subtraction
Vector subtraction involves the process of deducting two or more vectors to derive
a vector difference.
z=u-v
-v
u
z v
Vector Operations: Multiplication
u*v=z u.v=z
u*v = (x2y3 + x3y2) î + (x1y3 + x3y1) ĵ + (x1y2 u.v = x1y1 + x2y2 + x3y3
z = u*v y1 y2 y3
u v θ
θ v
Vector Operations: Dot Product vs. Cross Product
The representations for dot and cross products in vector form are as follows:
a1 b1
a1 b1
→ → → →
a = b = a = b = b2
a2
a2 b2
a3 b3
a₂b₃ - a₃b₂
→ → → →
a b = a1b1 + a2b2 a X b = a₃b₁ - a₁b₃
a₁b₂ - a₂b₁
Vector Operations: Norm
In machine learning, the size of a vector is often referred to as its norm, which indicates the
distance from the origin to a specific point, represented by the vector's norm.
Here:
The squared L2 norm, denoted as xTx, is often used to measure vector size and is
preferred because its derivative depends on x.
Here,
2 • 𝑋 symbolizes a vector with 𝑥𝑖 representing
||𝑋||22 = σ𝑛𝑖 |𝑥𝑖 |
the vector's components.
• |𝑥𝑖| signifies the absolute value of 𝑥.
Vector Operations: Norm Features
𝑃: 𝑥 ∞ = 𝑚𝑎𝑥𝑖 𝑥𝑖
Where:
• In this context, 𝑥 represents a vector with 𝑥𝑖 signifying the vector's individual
components.
• |𝑥𝑖| indicates the absolute value of 𝑥𝑖
Vector: Magnitude and Norm
In vector terminology, magnitude and norm both denote the size or length of a vector,
representing the same concept, and are often used interchangeably.
A matrix is a rectangular array or table composed of rows and columns filled with numbers,
symbols, or expressions, representing a mathematical object or property.
2 1 3 2 1
A = B =
-1 2 4 3 4
Matrix Size
Where:
m is the number of rows
n is the number of columns
A matrix featuring two rows and three columns is known as a two-by-three matrix,
a 2 X 3 matrix, or a matrix of dimension 2 X 3.
Notation of Matrix
Columns
1 2 n
3
a31 a32 a3n
Rows = Am x n
1 2 3 4 0 0 1 0 0
𝐴= 4 5 6 𝐷= 0 5 0 𝐼= 0 1 0
7 8 9 0 0 6 0 0 1
Forms of Matrix
1 2 3 1 0 0 1 2 3
𝐴= 0 5 6 𝐴= 4 5 0 𝐼= 2 4 5
0 0 9 7 8 9 3 5 6
Matrix Operations
Matrix Operations: Addition
22 32 13 8
A= B=
11 16 13 16
22 + 13 32 + 8
A + B =
11 + 13 16 + 16
35 40
A + B =
24 32
Matrix Operations: Addition Rules
Matrices can only be added if they contain the same number of rows and
columns.
22 32 13 8
A= B=
11 16 13 16
22 - 13 32 - 8
A - B =
11 - 13 16 - 16
9 24
A - B =
-2 0
Matrix Operations: Subtraction Rules
Matrices can only be subtracted if they possess the same number of rows
and columns.
Matrix subtraction does not allow the commutative property, which can be
illustrated as A - B not being equal to B - A.
Matrix Operations: Subtraction Rules
Commutative property: A - B ≠ B - A
Associative property: (A - B) - C = A - (B + C)
Matrix Operations: Multiplication
22 32 13 8
A= B=
11 16 13 16
Associativity Distributivity
A transpose is a matrix formed by turning all the rows of a given matrix into columns and vice
versa. The transpose of matrix A is denoted as AT.
22 32 22 11
AT =
A=
11 16 32 16
Matrix Operations: Determinant
The determinant is a scalar value that can be computed from the elements of a square matrix and
encodes certain properties of the linear transformation described by the matrix
A= a A=
𝑎 𝑏 𝑎 𝑏 𝑐
𝑐 𝑑 A= 𝑑 𝑒 𝑓
det(A) = a 𝑔 ℎ 𝑖
det(A) = ad - bc
det(A) = aei + bfg + cdh − ceg − afh − bdi
Matrix Operations: Determinant
Example
9 8
A=
10 11
det(A) = 99 – 80
det(A) = 19
Matrix Operations: Inverse
If A is a non-singular square matrix, there exists an n x n matrix, identified as A's inverse matrix,
which satisfies the following property:
1 0
I =
0 1
AB = BA = In
Where:
In denotes the nXn identity matrix.
Special Matrix Types: Tensors
Source: wikipedia
Matrix: Applications in Data Science
A linear equation is an algebraic equation in which the highest power of the variable(s) is 1.
When plotted on a Cartesian plane, it represents a straight line.
a1x1+a2x2+…..+anxn =b
Here,
• x1+x2+…..+xn represent the unknown quantities to be found
• a1+a2+…..+an are the coefficients
• b is the constant term
Linear Equations
Linear equation with one Linear equation with two Linear equation with three
variable variables variables
(3/2)x +7 = 0 3a+2b = 5 a – 3b = c
98x = 49 6x+9y-12=0 3x + 12 y = ½ z
Identifying Linear and Non-linear Equations
y = 8x - 9 Linear
y + 3x - 1 = 0 Linear
The formula for one-variable single-line The formula for two-variable single-line
calculations is as follows: calculations is as follows:
Equation Equation
Ax + B = 0 Ax + B y = C
Where: Where:
• A and B are real integers • A, B, and C are real integers
• x is the variable • x and y are the variables
Linear Equation in Standard Form
Coefficient of x Coefficient of y
Ax + By = C
Constant
Variables
Linear Equation in Slope Intercept Form
Slope equation
y = mx + b
Where:
• m is the slope.
• b is the intercept.
• x and y represent the distances of a point
on the line from the x-axis and y-axis,
respectively.
Linear Equation in Slope Intercept Form
There are four types of slopes, each representing a different relationship between
the variables x and y. These include:
In point-slope form, a straight line is represented by its slope and a specific point on
the line.
Equation:
y – y1 = m(x – x1)
Where
(x1, y1) are the coordinates of the
point.
Linear Equations Forms: Example
x-3x = 2
x-5 = 3x-3
x-5 = 3(x-1) -2x = 2
x = 3x+2
x = -1
System of Linear Equations
Equation:
Solving linear equations holds critical importance as it enables the determination of variable
values that satisfy the provided equations.
• Find solutions
Here are several • Understand relationships
reasons underscoring
• Make predictions
the importance of
solving linear • Optimize solutions
equations: • Validate equations
• Analyze data
Solving a Linear Equation
1 Graphic method
2 Substitution method
3 Linear combination or
elimination method
4 Matrix method
Solving Systems of Linear Equations Using Graphing
-6 -5 -4 -3 -2 -1 1 2 3 4 5 6
-1
-2
-3
-4
-5
-6
The two lines intersect at the point (-2,-1). Therefore, the solution to this
system of equations is (-2,-1), where x equals -2 and y equals 1.
Solving Systems of Linear Equations Using Substitution: Steps
3x+2y = 19 1
x+y = 8 2
Solving Systems of Linear Equations Using Substitution
3x + 2y = 19 3x + 2(8 − x) = 19 3x + 16 − 2x = 19
y = 8- x y=8−x y=8−x
Solving Systems of Linear Equations Using Substitution
x + 16 = 19 x=3
x=3
y = 8- x y=8–3
y=8-x
y=5
Answer:
x = 3, y = 5
Solving Systems of Linear Equations Using Elimination: Steps
Example:
3x + 2y = 19
x+y=8
Solving Systems of Linear Equations Using Elimination
3x+2y=19 x=3 X =3
y=5
2x + 2y = 16 2x + 2y =16 X+Y=8
Y
The blue line denotes the set of points
10 where 3x + 2y = 19 holds true.
-5 0 5 X
Solving Systems of Linear Equations: Applications
Signal processing and Linear equations and systems of equations are extensively used in
image analysis signal processing and image analysis.
Key Takeaways
A. a must be invertible
D. None of these
Knowledge
Check
What is the necessary condition for a solution to exist for the system Ax = b?
1
A. a must be invertible
D. None of these
For the system of linear equations Ax = b to have a solution, it is necessary that the inverse of matrix
A exists (i.e., |A| ≠ 0).
Knowledge
Check Suppose the cost of 2 balls and 1 bat is 100 units. How can this problem be expressed in
2 linear algebra with the variables x and y?
A. 2x + y = 100
B. 2x + 2y = 100
C. 2x + 4y = 200
D. x + y = 100
Knowledge
Check Suppose the cost of 2 balls and 1 bat is 100 units. How can this problem be expressed in
2 linear algebra with the variables x and y?
A. 2x + y = 100
B. 2x + 2y = 100
C. 2x + 4y = 200
D. x + y = 100
Assume the price of a bat equates to x units and the price of a ball equates to y units. The values of x
and y can vary based on the given scenario, as they are variables. Consequently, the problem
translates into the linear algebraic equation: 2x + y = 100.
Knowledge
Check
What does a linear equation involving three variables represent?
3
A. A flat object
B. A line
C. A plane
D. Both A and C
Knowledge
Check
What does a linear equation involving three variables represent?
3
A. A flat object
B. A line
C. A plane
D. Both A and C
A linear equation involving three variables denotes a set of points. The coordinates of these points
satisfy the given equation. Essentially, a linear equation with three variables represents a plane.
Knowledge
Check
Which of the following is NOT a type of matrix?
4
A. Square matrix
B. Scalar matrix
C. Diagonal matrix
D. Term matrix
Knowledge
Check
Which of the following is NOT a type of matrix?
4
A. Square matrix
B. Scalar matrix
C. Diagonal matrix
D. Term matrix