Understanding of different types of numerical integration

Deep Vyawahare
IIST Department of Aerospace Engineering
(Dated: September 9, 2021)
Numerical analysis is the area of mathematics and computer science that creates, analyzes, and
implements algorithms for solving numerically the problems of continuous mathematics. Such prob-
lems originate generally from real-world applications of algebra, geometry, and calculus, and they
involve variables which vary continuously. For this report our focus will be on Numerical Integration
Methods that are used for solving the different types of ordinary differential equations. We will also
take a look at the different solution techniques used by different commercial packages.

I. INTRODUCTION also to achieve given accuracy, it takes much more

computational time to use an explicit method.
Numerical methods for ordinary differential equations
refers to methods used to find numerical approxima- Now the important question What are the conse-
tions to the solutions of ordinary differential equations quences of using an implicit versus an explicit solution
(ODEs). This is also call numerical integration or com- method for a time-dependent problem.
putational integration. They can also find the solution
of differential equation which donot ave analytical solu-
tions.The most commonly used numerical methods in-
clude numerical integration techniques like the Finite
Element Method, Finite Difference Method, the Runge
Kutta Methods, Euler Methods, et III.2. Single and Multi step methods

The another mode of classification is on single and

II. ORDINARY DIFFERENTIAL EQUATION
multi-step methods. Single-step methods (such as Euler’s
method) refer to only one previous point and discard all
An ordinary differential equation (also abbreviated as previous information before taking a second step. Multi-
ODE), in Mathematics, is an equation which consists of step methods tries to gain efficiency by keeping and using
one or more functions of one independent variable along the information from previous steps rather than discard-
with their derivatives. ing it. Consequently, multistep methods refer to several
previous points and derivative values. So all the previous
values will be used in multistep methods.

III. DIFFERENT TYPES OF NUMERICAL

INTEGRATION TECHNIQUES

Usually we classify the numerical integration methods

IV. STIFF EQUATION
for solving ordinary differential equations in two cate-
gories like explicit and implicit factors, but one can fur-
ther classify in another categories like single step methods An ordinary differential equation problem is stiff if
and multi step methods. the solution being sought is varying slowly, but there
are nearby solutions that vary rapidly, so the numeri-
cal method must take small steps to obtain satisfactory
III.1. Explicit and Implicit Methods results. Whenever we are solving using numerical in-
tegration techniques we need a reliable solution to the
Numerical solution schemes are often referred to as differential equation. So to make the solution reliable
being explicit or implicit. When a direct computation of sometime we may need the step size to be very small in
the dependent variables can be made in terms of known a region where the curve is smooth, this phenomenon is
quantities, the computation is said to be explicit. When referred as stiffness. Many times we may get two differ-
the dependent variables are defined by coupled sets of ent problems with the same solution, yet one is not stiff
equations, and either a matrix or iterative technique is and the other is. The phenomenon cannot therefore be a
needed to obtain the solution, the numerical method property of the exact solution, since this is the same for
is said to be implicit.Implicit methods are more time both problems, and must be a property of the differen-
consuming and complicated as compared to Explicit tial system itself. Such systems are thus known as stiff
methods, but generally provide more accurate results systems
 2

V. DIFFERENT NUMERICAL INTEGRATION

METHODS FOR ODE

V.1. Euler Method

There are many methods which are better that Euler

method but since Euler method is simple to understand
and easy to implement, which makes it a great starting
point.This method is a first-order method used for
solving Initial Value Problems. We use Taylor series
expansion and only 2 terms while higher order terms are
excluded. As we are excluding higher order terms from FIG. 1. Graphical output from running program 1 in MAT-
our equation the higher order terms will come as error. LAB

If we consider the following differential equation:

method is that it is much more accurate compare to the
Forward Euler method. The major disadvantage of this
ẏ = f (t, y), y(0) = y0 method is that the equation difficult to solve and is time
consuming.
Taylor series at a time ∆t into the future

dy t2 d2 y Program 1:MATLAB Program for Euler’s method for

y(4t) = y(0) + 4t +4 + .... the first order equation.
dt t=0 2 dt2 t=0
clear; %% clear exisiting workspace
y = 1; %% initial condition
dy dt = 0.5; %% set the time step interval
y(4t) = y(0) + 4t + E(4t2 ) time = 0; %% set the start time=0
dt t=0
t final = 2; %% end time of the simulation
Nsteps = round(t final/dt); %% number of time steps to
dy take, integer
y(4t) = y(0) + 4t plot(time,y,’*’); %% plot initial conditions
dt t=0
hold on; %% accumulate contents of the figure
Now we have not considered the higher terms in the for i = 1:Nsteps %% number of time steps to take
expression and E is the error. This method is typically y = y - dt*y; time = time + dt %% Increment time
used for approximating the solution over some range of plot(time,y,’*’); %% Plot the current point
t. In this method, the state of the system at a later time end
is calculated using the state of the system at the current t = linspace(0,t final,100); %% plot analytical solution
time and since the state of a system at a later time is y = exp(-t);
not required hence, this is an explicit method. plot(t,y,’r’)
xlabel(’time’); %% add plot labels
The major advantage of the system is that it is ylabel(’y’);
relatively easy to solve and we do-not require much cal-
culation hence it is also a faster method. But the major
disadvantage is that it does not give high accuracy.Also
we cannot use this method for solving stiff problems.

If instead we use the backward difference approxima-

tion, we will obtain:

y(t) − y(t − 4t)

ẏ(t) ≈
4t

⇒ yn+1 = yn + 4tf (tn+1 , yn+1 )

The solution obtained from the Backward Euler method

is an implicit method, even though it’s form is similar to
Euler method’s solution. FIG. 2. Graphical output from running program 2 in MAT-
The major advantage of this method over the previous LAB versus analytical solution
 3

Program 2: MATLAB Program to solver the mass

spring system using Euler’s method
1
yi+1 = yi + (k1 + 2k2 + 2k3 + k4)
clear; %% clear exisiting workspace 6
y(1) = 1; %% initial condition, position
y(2) = 0; %% initial condition, velocity
dt = 0.1; %% set the time step interval k1 = hf (xi , yi )
t final = 30; %% final time to integrate to
time = 0; %% set time= 0.
Nsteps = round(t final/dt) %% number of steps to take. h k1
plot(time,y(1),’*’); k2 = hf (xi + , yi + )
2 2
hold on; %% accumulate contents of the figure
for i = 1:NSteps %% number of time steps to take
dy(2) = -y(1) %% Equation for dv/dt h k2
dy(1) = y(2) %% Equation for dx/dt k3 = hf (xi + , yi + )
2 2
y = y + dt*dy %% integrate both equations with Euler
time = time + dt
plot(time,y(1),’*’); k4 = hf (xi + h, yi + k3 )
plot(time,cos(time),’r.’);
end Here h is the step size, know h is known we need
to find other unknowns such as k1 ,k2 ,k3 and k4 . So
from the above equations we calculate all the ki un-
knowns and after finding the ki we can find yi+1 using the
V.2. Runge-Kutta Methods
above equation and thus we will get some approximation.
Runge–Kutta method is an effective and widely used Similar to the 4th order Runge Kutta method we
method for solving the initial-value problems of differ- can have higher as well as lower order Runge Kutta
ential equations. Runge–Kutta method can be used method which we can compute using the accuracy we
to construct high order accurate numerical method by need. There are explicit and implicit Runge Kutta
functions’ self without needing the high order deriva- which have there own advantages and disadvantages.The
tives of functions.The Runge-Kutta-Fehlberg Method is main difference between the explicit and the implicit
one of the most extensively used adaptive time-stepping runge kutta method is the different values of aij for
method, which uses a fourth-order and a fifth-order different condition like i=j or i¿j or i¡j in the equation (1).
Runge-Kutta method which helps in reducing the num-
ber of evaluations required per unit time. It is commonly
The advantage of the Runge Kutta method is that it is
known as the RKF45 Method.
relatively more accurate than Euler method and is more
The problem with Euler method is that the methods
stable also it does have some higher order terms but still
have a significant error due not considering higher order
this method can widely use for solving ODE. The Runge
derivative but we can include higher order derivative in
Kutta method are one step method and have local and
Euler method to make it more stable, but the problem is
global error as same. To increase the accuracy we can
with including higher order derivative the equations are
decrease the step size and get solution to the ode. The
becoming very complicated hence we use Runge Kutta
major disadvantage is that compare to multi step meth-
methods which has higher order terms but makes calcu-
ods which gives us same accuracy the computational time
lation little simpler and thus gives us relatively accurate
require in Runge Kutta method is more.
and stable results without bringing complications due to
higher order terms.
For a generalise n stage Runge kutta method we define
following equation: V.3. Adams

n
X Adams method is most commonly used linear muli-
ki = f (t0 + ci 4t, yo + 4t aij kj ) (1) tistep method. It is use for solving ordinary differen-
j=1 tial equation . The Adams method approximate the
solution by referring to previously obtained solutions.
The Explicit Adams Methods is the The Adams Bash-
Similarly if we did for 4 Order Runge Kutta we would forth methods since these methods are explicit in nature,
get: whereas the Adams-Moulton methods are also called Im-
plicit Adams Methods, owing to the fact that they are
xi+1 = xi + h implicit in nature.
 4

V.3.1. Adams-Bashforth Methods numerical solutions when compared to the Adams Bash-
forth methods. But the main problem is that the extra
These are the most commonly used linear multiple-step term which we have in Adam-Moulton equation will in-
methods used for solving non-stiff equations. Adams- crease the computational time and will have give more
Bashforth method is an Explicit method. In Adams efforts for solving using Adams-Moulton equation. Also
Bashforth methood we assume a uniform discretization Adams-Moulton equation is an Implicit equation.
in the x-domain,

xi = a + ih V.4. Midpoint Rule

Where the expression of h is (N are some positive in- Le us assume that f : [a, b] → R is our function, which
teger N) : we want to integrate. In the case of midpoint rule, we
use value the value in eqn 1 as an approximation for the
b−a value in the sub interval.
h=
N
x1 + x2
Let the given differential equation is of the form: f( ) (5)
2
0
y = f (x, y) So we get,

Now we will integrate the differential equation from b−a

h= (6)
x=xi to x=xi+1 n
Which will give the following equation:
xi + xi+1
p1 = f ( ) (7)
Z xi 2
y(xi+1 ) − y(xi ) = f (x, y(x))dx (2)
xi+1 where h is the length of subintervals and pi is approxi-
mated value for whole subinterval. Then,
Now we get:
b n−1
f (x, y(x) = Pm−1 (x) + Rm−1 (x) (3)
Z X
f (x)dx = pi h (8)
a i=0
Now here Pm−1 is the Lagrangian polynomial of atmost
degree m-1 and Rm−1 is the remainder term. Midpoint rule is not very accurate method but it gives
us very good results when function doesn’t change a lot
Now generally Adams-Bashforth methods require in subintervals. Error of this rule is estimated by the
less computaional time than the single step explicit following formula
methods and Adams-Bashforth method are relatively
more accurate and more stable then the Runge Kutta (b − a)3 2
method. error = M (9)
24n2
Where M is maximum of second derivative of the function
f. According to this formula, we can see, that significant
V.3.2. Adams-Moulton Methods impact of the value of error has number n of subintervals.
The bigger n, the smaller error.
Now Adams-Bashforth method and Adams-Moulton Program 3: MATLAB Program to solver the
method are same the only difference is that we have a massspring system using Midpoint Method method:
extra term in Adam-Moulton equation which is xi+1 so Midpoint Solver
we will need to interpolate the values at xi+1 so which
will change the final form of equation which is: function [t,data] = midpointsolver(y,dt,t final,derivs
Handle)
time = 0;
f (x, y(x) = Pm−1 (x) + Rm−1 (x) (4) Nsteps = round(t final/dt) %% number of steps to take
t = zeros(Nsteps,1);
Now here Pm is the Lagrangian polynomial of atmost data = zeros(Nsteps,length(y));
degree m and Rm is the remainder term and no change t(1) = time; %% store intial condition
apart from this. data(1,:) = y’;
for i =1:Nsteps
The main advantage of Adam-moulton equation is that dy = feval(derivs Handle,time,y); %% evaluate the
such methods have smaller errors, , and have more stable initial derivatives
 5

yH = y + dy*dt/2; %% take Euler step to midpoint V.6. Simpsons

dy = feval(derivs Handle,time,y); %% re-evaluate the
derivs Simpson’s rule is based on polynomial interpolation
y = y + dy*dt; %% shoot across the interval and uses second degree polynomial. It is the most
time = time+dt; %% increment time acurate method compare to midpoint and trapezoidal
t(i+1) = time; %% store for output method. It is similar to trapezoid rule, because like
data(i+1,:) = y’; %% store for output there, here we use h, yi1 and yi as three sides of the
end figure, but the fourth side is parabola approximated
to graph of the function. Values at the beginning and
end of the subinterval are used as the points needed to
approximate this parabola For simpsons rule error has
following formula:

(b − a)5
error = − M (14)
90n5
Where M is maximum of the second derivative.

VI. DIFFERENT NUMERICAL INTEGRATION

METHODS FOR PDE

VI.1. Finite Element Method

FIG. 3. Comparison of the solution of the mass spring system
using the midpoint method and Euler’s method Program 2
AND 3. Finite element method is technique use for finding solu-
tion of the boundary value problem for differential equa-
tion. To minimise the error and to get a stable solution
it uses variational methods. The finite element method
V.5. Trapezoidal rule uses small sub domains to approximate the solution in
larger domain.
The trapezoidal rule is almost same as that of midpoint
rule, the fundamental difference between two rules are VI.2. Methods of Line
that while in mid point rule we consider the rectangle
whereas we will consider trapezoid in trapezoidal rule.
Method of Line is partial difeerential equation solv-
We will approximate by taking polygon chain on graph
ing technique in which we discretized the dimension. We
on different sub intervals .
use software developed for the numerical integration of
We calculate following values:
ordinary differential equations (ODEs) and differential
b−a algebraic equations. Method of line leads us to a system
h= (10) of ordinary differential equations to which a numerical
n
method for initial value ordinary equations can be ap-
plied
yi + yi−1
pi = h (11)
2
Where h is a length of sub interval and pi refers to VI.3. Finite Volume Method
means area of the trapezoid for interval [xi−1 , xi ].
The whole integral on the range of [a, b] equals to Finite volume method refers to method in which we
surround the node point with small volume on a mesh.
Z b n So the values are calculated on a different places on a
meshed geometry. ”Finite volume” refers to the small
X
f (x)dx = pi (12)
a i=1
volume surrounding each node point on a mesh. These
terms are then evaluated as fluxes at the surfaces of each
For trapezoid rule error has following formula: finite volume. Because the flux entering a given volume
is identical to that leaving the adjacent volume, these
(b − a)3 2 methods are conservative. Another advantage of the fi-
error = M (13) nite volume method is that it is easily formulated to allow
12n2 for unstructured meshes. The method is used in many
Where M is maximum of the second derivative. computational fluid dynamics packages.
 6

VI.4. Spectral Method frameworks. For hardened frameworks the Jacobian

grid might be treated in one or the other full or united
Spectral methods are techniques used in applied math- structure. LSODE can likewise be utilized when the
ematics and scientific computing to numerically solve Jacobian can be approximated by a band framework.
certain differential equations, spectral method use fast
Fourier transform. The idea is to write the solution of the
differential equation as a sum of certain ”basis functions”
and then to choose the coefficients in the sum that best VII.5. MATLAB
satisfy the differential equation. Spectral methods and
finite element methods are closely related and built on MATLAB is a widely used proprietary software for per-
the same ideas; the main difference between them is that forming numerical computations. It comes with its own
spectral methods use basis functions that are nonzero programming language, in which numerical algorithms
over the whole domain, while finite element methods use can be implemented. MATLAB has various solvers like
basis functions that are nonzero only on small subdo- for solving non stiff differential equation we can also use
mains the RK23 method in MATLAB. There are various meth-
ods which can be used and are writ tern down in the
table.
VII. NUMERICAL METHODS USED IN
Solver Method / Equations
COMMERCIAL PACKAGES
ode23 solver RK 23 Method
Stiff oridanary differential
VII.1. ABAQUS ode 23tb solver
equation
Trapezoidal Method For
ode 23t solver
It is a software application used for both the mod- Numerical Integration
eling and analysis of mechanical components and as- ode 23s solver Differential Algebric Equation
ode 45 solver RK 45 Method
semblies and visualizing the finite element analysis re-
ode113 solver Non stiff ordinary equation
sult.Standard, a general-purpose Finite-Element ana-
lyzer that employs implicit integration scheme. It also TABLE I. Different ODE Solver and there uses
have explicit solver as well as a Computational Fluid Dy-
namics software application.

VII.6. SUNIDALS
VII.2. ANSYS
SUNDIALS is a SUite of Nonlinear and DIfferen-
Ansys Mechanical finite element analysis software is tial/ALgebraic equation Solvers. It consists of the fol-
used to simulate computer models of structures, elec- lowing six solvers: CVODE, CVODES, ARKODE, IDA,
tronics, or machine components for analyzing strength, IDAS and KINSOL. SUNDIALS is implemented with the
toughness, elasticity, temperature distribution, electro- goal of providing robust time integrators and nonlinear
magnetism, fluid flow, and other attributes. Ansys has solvers that can easily be incorporated into existing sim-
trapezoidal method and RK 45 methods. They also have ulation codes
various other solver packages. They also have LSDYNA
solver.

VII.3. Python

Python is an open source software which we can use

to solve various differential equation. The different avail-
able methods on Python are namely, RK45 method, the
midpoint method(RK2), and a few other methods.

VII.4. LSODE

LSODE is a bundle of subroutines for the mathe-

matical arrangement of the underlying worth issue for
frameworks of standard differential conditions. The
bundle is reasonable for one or the other solid or nonstiff
 7

Package Purpose
solves initial value problems for ordinary
CVODE
differential equation (ODE) systems.
solves ODE systems and includes sensitivity
CVODES
analysis capabilities (forward and adjoint).
solves initial value ODE problems with additive
ARKODE Runge-Kutta methods, including
support for IMEX methods.
solves initial value problems for differential-
IDA
algebraic equation (DAE) systems.
solves DAE systems and includes sensitivity
IDAS
analysis capabilities (forward and adjoint).
KINSOL Solves nonlinear algebraic systems.

TABLE II. Different packages and Solvers in SUNIDAL

 8

[1] https://en.wikipedia.org/wiki/List_of_ [4] https://en.wikipedia.org/wiki/Numerical_methods_

numerical-analysis_software for_partial_differential_equations

[2] https://www.lstc.com/products/ls-dyna [5] https://en.wikipedia.org/wiki/Numerical_methods_

for_ordinary_differential_equations
[3] https://en.wikipedia.org/wiki/Abaqus [6] https://en.wikipedia.org/wiki/Numerical_
integration