NumPy - Linear Algebra

---

---

Linear Algebra in NumPy

Linear algebra is a branch of mathematics that deals with vectors, matrices, and linear transformations.

NumPy package contains numpy.linalg module that provides all the functionality required for linear algebra. Some of the important functions in this module are described in the following table.

Creating Matrices

In NumPy, we can create matrices using arrays. Matrices are simply two-dimensional arrays, and they can be created using the np.array() function. You can specify the elements of the matrix as nested lists.

Example

Following is the basic example where we create a matrix that consists two rows and three columns −

import numpy as np

# Creating a 2x3 matrix
matrix = np.array([[1, 2, 3], [4, 5, 6]])
print("Matrix:\n", matrix)

Following is the output obtained −

Matrix:
[[1 2 3]
 [4 5 6]]

Matrix Operations

Matrix operations are fundamental in linear algebra and involve performing arithmetic on matrices. In NumPy, you can easily perform addition, subtraction, multiplication, and transposition of matrices.

Matrix Addition and Subtraction

Matrix addition and subtraction are performed element-wise. This means corresponding elements from each matrix are added or subtracted. Both matrices must have the same shape for these operations.

Example

In the following example, we are performing element-wise matrix addition and subtraction using two 2x2 NumPy arrays, A and B −

import numpy as np
A = np.array([[1, 2], [3, 4]])
B = np.array([[5, 6], [7, 8]])

# Matrix Addition
C = A + B
print("Matrix Addition:\n", C)

# Matrix Subtraction
D = A - B
print("Matrix Subtraction:\n", D)

This will produce the following result −

Matrix Addition:
[[ 6  8]
 [10 12]]
Matrix Subtraction:
[[-4 -4]
 [-4 -4]]

Matrix Multiplication

Matrix multiplication can be done using the @ operator or the np.dot() function. Unlike element-wise multiplication, matrix multiplication involves summing the products of rows and columns.

Example

Here, we are performing matrix multiplication on two 2x2 NumPy arrays, A and B, using the @ operator and the np.dot() function −

import numpy as np
A = np.array([[1, 2], [3, 4]])
B = np.array([[5, 6], [7, 8]])

# Matrix Multiplication
C = A @ B
print("Matrix Multiplication with @:\n", C)

D = np.dot(A, B)
print("Matrix Multiplication with np.dot():\n", D)

Following is the output of the above code −

Matrix Multiplication with @:
[[19 22]
 [43 50]]
Matrix Multiplication with np.dot():
[[19 22]
 [43 50]]

Matrix Transposition

The transpose of a matrix is obtained by flipping it over its diagonal, effectively swapping rows with columns. This can be achieved using the .T attribute.

Example

Over here, we are transposing a 2x2 NumPy array A to obtain its transpose A_T −

import numpy as np
A = np.array([[1, 2], [3, 4]])

# Transposing the matrix
A_T = A.T
print("Transpose of A:\n", A_T)

The output obtained is as shown below −

Transpose of A:
[[1 3]
 [2 4]]

Determinants & Inverses

The determinant indicates whether a matrix is invertible (non-singular) or not. If the determinant of a matrix is non-zero, the matrix is invertible. Conversely, if the determinant is zero, the matrix is singular and not invertible. It is also used to −

• Solve the linear equations.
• Change variables in integrals.
• Calculate area and volume.
• Define characteristic polynomial of a square matrix.

The inverse of a matrix is a matrix that, when multiplied by the original, results in the identity matrix.

A X A-1 = A-1 = I

NumPy Provides various functions to calculate the determinant and inverse of a matrix.

Computing the Determinant

We can calculate the determinant of a matrix using the linalg.det() function. It internally uses LAPACK routine to calculate the determinant (via LU factorization).

Example

In the example below, we are calculating the determinant of a 2x2 NumPy array A using the np.linalg.det() function −

import numpy as np
A = np.array([[1, 2], [3, 4]])

# Determinant of the matrix
det = np.linalg.det(A)
print("Determinant of A:", det)

After executing the above code, we get the following output −

Determinant of A: -2.0000000000000004

Computing the Inverse

We can compute the inverse of a matrix using the linalg.inv() function.

Example

Here, we are calculating the inverse of a 2x2 NumPy array A using the np.linalg.inv() function −

import numpy as np
A = np.array([[1, 2], [3, 4]])

# Inverse of the matrix
A_inv = np.linalg.inv(A)
print("Inverse of A:\n", A_inv)

The result produced is as follows −

Inverse of A:
 [[-2.   1. ]
 [ 1.5 -0.5]]

Eigenvalues and Eigenvectors

Eigenvalues and eigenvectors are fundamental in understanding linear transformations. They can be computed using the np.linalg.eig() function.

Eigenvalues indicate the magnitude of the transformation, while eigenvectors indicate the direction.

Example

In the example shown below, we are computing the eigenvalues and eigenvectors of a 2x2 NumPy array A using the np.linalg.eig() function −

import numpy as np
A = np.array([[1, 2], [3, 4]])

# Computing eigenvalues and eigenvectors
eigenvalues, eigenvectors = np.linalg.eig(A)
print("Eigenvalues:", eigenvalues)
print("Eigenvectors:\n", eigenvectors)

We get the output as shown below −

Eigenvalues: [-0.37228132  5.37228132]
Eigenvectors:
[[-0.82456484 -0.41597356]
 [ 0.56576746 -0.90937671]]

Solving Linear Systems

In Numpy, linear systems of equations can be solved using the np.linalg.solve() function. This function finds the values of variables that satisfy the linear equations represented by the matrix equation −

Ax = b

Where, A represents matrix and b is a vector.

Example

In this example, we are solving a linear system of equations represented by the matrix equation Ax=b, where A is a 2x2 matrix and b is a vector. We use the np.linalg.solve() function to compute the values of x that satisfy the equation −

import numpy as np
A = np.array([[3, 1], [1, 2]])
b = np.array([9, 8])

# Solving the linear system Ax = b
x = np.linalg.solve(A, b)
print("Solution of the linear system:", x)

Following is the output obtained −

Solution of the linear system: [2. 3.]

Singular Value Decomposition (SVD)

SVD is a factorization of a matrix into three matrices: U (the left singular vectors), S (the singular values) and V (the right singular vectors). It is useful in various applications, including signal processing and statistics.

You can perform SVD using the np.linalg.svd() function in NumPy.

Example

In the following example, we are performing Singular Value Decomposition (SVD) on a 2x2 matrix A, which decomposes it into three components: U, S, and V −

import numpy as np
A = np.array([[1, 2], [3, 4]])

# Performing SVD
U, S, V = np.linalg.svd(A)
print("U matrix:\n", U)
print("Sigma values:", S)
print("V matrix:\n", V)

This will produce the following result −

U matrix:
[[-0.40455358 -0.9145143 ]
 [-0.9145143   0.40455358]]
Sigma values: [5.4649857  0.36596619]
V matrix:
[[-0.57604844 -0.81741556]
 [ 0.81741556 -0.57604844]]

Norms and Conditions

Norms measure the size or length of vectors and matrices, helping quantify their magnitude. Condition numbers indicate how sensitive a matrix's solution is to changes in its input, indicating how well it can be solved numerically.

Computing Norms

Norms measure the size or length of vectors and matrices. We can use the NumPy linalg.norm() function to compute different types of norms, such as the Frobenius norm and the Euclidean norm.

Example

In the following example, we are calculating the Frobenius norm of a 2x2 matrix A, which provides a measure of its overall magnitude, similar to the Euclidean norm for vectors.

We also compute the L2 norm (Euclidean norm) of a 3D vector, which quantifies its length in space −

import numpy as np
A = np.array([[1, 2], [3, 4]])

# Frobenius norm
norm = np.linalg.norm(A, 'fro')
print("Frobenius norm of A:", norm)

# L2 norm (Euclidean norm)
vector = np.array([1, 2, 3])
l2_norm = np.linalg.norm(vector)
print("L2 norm of vector:", l2_norm)

Following is the output of the above code −

Frobenius norm of A: 5.477225575051661
L2 norm of vector: 3.7416573867739413

Computing Conditions

The condition number of a matrix measures how sensitive the solution of a linear system is to errors in the data.

It can be computed using the NumPy linalg.cond() function. A high condition number indicates that the matrix is close to singular, making it more challenging to solve linear equations accurately.

Example

Here, we are calculating the condition number of a 2x2 matrix A −

import numpy as np
A = np.array([[1, 2], [3, 4]])

# Condition number
cond = np.linalg.cond(A)
print("Condition number of A:", cond)

The output obtained is as shown below −

Condition number of A: 14.933034373659268