Introduction

Sound Synthesis

The first line of ’Mary had a little lamb’

The following matlab code creates the first line of the popular song: Mary had a
little lamb.
fre=[329.6280 293.6650 261.6260 293.6650 0 329.6280 329.6280 .
329.6280 0 293.6650 293.6650 293.6650 0 329.6280 391.9950 3
f = @(f0,t) exp(-0.05*t) .* sin(2*pi*f0*t);
n = 10000; %number of sampling points for a half second note
t = [0:n]/(2*n); %play a note only a half second
song = [];
for k = 1:length(fre)
song= [song f(fre(k),t)];
end
sound(song,2*n)
Mary

Bishnu P. Lamichhane, bishnu.lamichhane@newcastle.edu.au Mathematics and Music

 Introduction
Sound Synthesis

Additive Synthesis: Fourier Series

By combining sine waves of di↵erent frequencies and amplitudes, we can create

new sounds. These are often called timbres.
The lowest frequency in a note is called the fundamental frequency.
A music instrument generates a tone which can be approximated by many sine
waves.
A tone having many frequencies has rich, complex tones. For example,

f (x) = a1 sin(⇡x) + a2 sin(2⇡x) + a3 sin(3⇡x) + a4 sin(4⇡x).

Bishnu P. Lamichhane, bishnu.lamichhane@newcastle.edu.au Mathematics and Music

 Introduction
Sound Synthesis

Fourier Series

A sufficiently smooth function f (x) is expanded in terms of sin and cos as

a0 X ⇣ ⇣ n⇡ ⌘ ⇣ n⇡ ⌘⌘
1
f (x) = + an cos x + bn sin x .
2 n=1
L L

The coefficients an and bn are given by the Euler’s formulas:

Z ⇣ n⇡ ⌘
1 L
(A) an = f (x) cos x dx for n 0.
L L L
Z ⇣ n⇡ ⌘
1 L
(B) bn = f (x) sin x dx for n 1.
L L L
These coefficients an , bn are called Fourier series coefficients of the function f (x).

Convergence theory?

Bishnu P. Lamichhane, bishnu.lamichhane@newcastle.edu.au Mathematics and Music

 Introduction
Sound Synthesis

Fourier Series: Sawtooth-like Wave

Find the Fourier series of
f (x) = x for ⇡ < x < ⇡, f (x + 2⇡) = f (x), x ⇡2
/ 2⇡Z.
In this case, the Fourier coefficients are given by
Z
1 ⇡
an = x cos(nx) dx = 0, n 0.
⇡ ⇡
Z ⇡
1 2 ( 1)n+1
bn = x sin(nx) dx = cos(n⇡) = 2 , n 1.
⇡ ⇡ n n
It can be proved that the Fourier series converges to f (x) at every point x where f is
di↵erentiable, and therefore:
1
a0 X
f (x) = + [an cos (nx) + bn sin (nx)]
2 n=1
X1
( 1)n+1
= 2 sin(nx), for x ⇡2
/ 2⇡Z.
n=1
n

Bishnu P. Lamichhane, bishnu.lamichhane@newcastle.edu.au Mathematics and Music

Synthesisers
Synthesisers create different sounds by combining some simple sounds
(waveforms) to create more complex sounds. Some simple sound
waves: sine, square, triangle and sawtooth waves.
 Introduction
Sound Synthesis

Fourier Series: Example for Partial Sums

Consider the partial sums of Fourier series:
a0 a0 ⇣⇡ ⌘ ⇣⇡ ⌘
S0 (x) = , S1 (x) = + a1 cos x + b1 sin x , ··· ,
2 2 L L
a0 X ⇣ ⇣ n⇡ ⌘ ⇣ n⇡ ⌘⌘
N
SN (x) = + an cos x + bn sin x .
2 n=1
L L

N =1 N =2

N =5 N = 25
Bishnu P. Lamichhane, bishnu.lamichhane@newcastle.edu.au Mathematics and Music
 Introduction
Sound Synthesis

Fourier series: Square Wave

Consider a square wave f (x) of period 2L. The following square wave is an odd function. So
a0 = 0, and an = 0 for all n 2 N, and

Z L
2 n⇡x
bn = f (x) sin( ) dx
L 0 L
2
= (cos(n⇡) 1).
n⇡

Hence
2 4
[1 ( 1)n ] =
bn = if n is odd,
(n⇡) (n⇡)
and bn = 0 if n is even. The Fourier series for L = 1 is, therefore,
1
X 1
4 1 4 X 1
f (x) = sin (n⇡x) = sin ((2n 1)⇡x)
⇡ n=1,3,5,... n ⇡ n=1 2n 1

N =1 N =2 N =5 N = 25
Bishnu P. Lamichhane, bishnu.lamichhane@newcastle.edu.au Mathematics and Music
 Introduction
Sound Synthesis

Additive Synthesis: Fourier Series

By combining sine waves of di↵erent frequencies and amplitudes, we can create new
sounds. These are often called timbres. The lowest frequency in a note is called the
fundamental frequency. A music instrument generates a tone which can be
approximated by many sine waves. A tone having many frequencies have rich,
complex tones.

Mary Additive
Mary Simple
Bishnu P. Lamichhane, bishnu.lamichhane@newcastle.edu.au Mathematics and Music
 Introduction
Sound Synthesis

Subtractive Synthesis: Fast Fourier Transform (FFT)

Subtractive synthesis is based on modifying the spectrum of the original signal.

Bishnu P. Lamichhane, bishnu.lamichhane@newcastle.edu.au Mathematics and Music

 Introduction
Sound Synthesis

Subtractive Synthesis: Sound Comparison

Many techniques in sound synthesis are based on modifying the spectrum of a

synthetic signal to match with a spectrum of an organic sound.

Original music using sawtooth wave

After using subtractive synthesis

Bishnu P. Lamichhane, bishnu.lamichhane@newcastle.edu.au Mathematics and Music

 Introduction
Sound Synthesis

Sound Synthesis with Optimisation

We want to break sound into smaller pieces. Since sound waves are periodic signals,
we can express using sine and cosine waves resulting in a Fourier series. Let
y : [0, T ] ! R be a continuous signal. For example, we can approximate a sound
signal y by using sin and cos like
Z n
!2
T X
ya = min y(t) (ai cos(wi t) + bi sin(wi t)) dt
0 i=1

1 This
Pn can be digitised by sampling the signal y and also the sum
i=1 (ai cos(wi t) + bi sin(wi t)) at a finite number of time points.
2 The optimisation problem is nonlinear if we minimise with respect to wi .
3 We can use some information of the sound signal like physical properties to
simplify wi .

Bishnu P. Lamichhane, bishnu.lamichhane@newcastle.edu.au Mathematics and Music

 Introduction
Sound Synthesis

Optimisation Bottleneck
If we approximate the global signal (a 1/2 second A4 note looks like below), we
do not capture the local property of the signal accurately.

If we approximate the the signal locally (using around 5-6 milliseconds note), we
lose the accuracy globally.

Bishnu P. Lamichhane, bishnu.lamichhane@newcastle.edu.au Mathematics and Music

 Introduction
Sound Synthesis

Frequency Modulation Synthesis

The main idea behind Frequency Modulation is that we modulate the frequency over
time resulting in change in frequency over time. A simple frequency modulation is
obtained by setting
F (t) = sin(2⇡fc t + I sin(2⇡fm t)),
where fc is called the carrier frequency, fm is called the modulation frequency and I
is called the index of modulation, which is also the strength of modulation or the
intensity of the modulation.
We can include more terms to model a wide variety of sounds:

F (t) = sin(2⇡fc t + I1 sin(2⇡fm1 t) + I2 sin(2⇡fm2 t)).

Yamaha’s First Synthesiser DX-7 is based on this technique.

Bishnu P. Lamichhane, bishnu.lamichhane@newcastle.edu.au Mathematics and Music

 Introduction
Sound Synthesis

Frequency Modulation
Using fc = 1, fm = 2 and I = 12 in the modulated signal

F (t) = sin(2⇡fc t + I sin(2⇡fm t)),

we have the following result. Here the carrier signal is

Fc (t) = sin(2⇡fc t).

Synthetic bass simulation using the frequency modulation:

Mary Frequency Modulation
Bishnu P. Lamichhane, bishnu.lamichhane@newcastle.edu.au Mathematics and Music
 Introduction
Sound Synthesis

Frequency Modulation

Fourier series for the frequency modulation:

sin(fc t + I sin(fm t)) =

P1 ⇥ ⇤
J0 (I) sin(fc t) + k=1 Jk (I) sin((fc + kfm )t) + ( 1)k sin((fc kfm )t) ,

where Jk is a Bessel function (first kind) of order k. The function Jk is a solution to

the following Bessel di↵erential equation

d2 y dy
x2 2
+x + x2 k2 y = 0
dx dx

Bishnu P. Lamichhane, bishnu.lamichhane@newcastle.edu.au Mathematics and Music

 Introduction
Sound Synthesis

Bessel Function

A summation formula for Jk (x) is

1
X ( 1)m ⇣ x ⌘2m+k
Jk (x) = .
m=0
m! (m + k + 1) 2

A more useful form in the frequency modulation for integer values of n is

Z Z ⇡
1 ⇡ 1
Jn (x) = cos(n⌧ x sin ⌧ ) d⌧ = < ei(x sin ⌧ n⌧ ) d⌧,
⇡ 0 2⇡ ⇡
p
where i = 1 is the imaginary number, and < stands for the real part of the
expression.

Bishnu P. Lamichhane, bishnu.lamichhane@newcastle.edu.au Mathematics and Music

 Introduction
Sound Synthesis

Wavetable Synthesis

Wavetable synthesis produces a sound signal by cycling through a table of values that
represent a waveform.The wavetable method generates di↵erent frequencies of a
signal by changing the speed with which it cycles through table entries.
1 Wavetable synthesis is based on creating periodic waveforms by employing
arbitrary periodic waveforms.
2 The output waveform changes in time as one wavetable is mixed with another,
creating a changing waveform.
3 Used in many commercial synthesisers: Waldorf, Ableton, PPG Wave, etc.

Bishnu P. Lamichhane, bishnu.lamichhane@newcastle.edu.au Mathematics and Music

 Introduction
Sound Synthesis

Physical Modelling Synthesis

This sound synthesis method is based on generating sound by using a mathematical

model. Modelling a physical source of sound from a musical instrument like piano or
flute leads to di↵erential equations, which are then approximated by some numerical
techniques.
1 This technique can produce a wide variety of sounds: the sound may be often
too refined or too good.
2 It requires not much memory compared to digital instruments based on sound
recording or other techniques.
3 Commercial packages: IK Multimedia’s Modo Bass and Modo Drum, Modartt
Pianoteq and Organteq, Yamaha used some ideas in Yamaha VL1 in 1994.

Bishnu P. Lamichhane, bishnu.lamichhane@newcastle.edu.au Mathematics and Music

 Introduction
Sound Synthesis

Some Interesting Facts

Human hearing range: 20 Hz to 20 kHz.

Children can hear a slightly wider range of frequencies around 15 Hz to 25 kHz.
For middle aged person, the upper limit is often 14 kHz.
Children voice frequencies range from 250 to 400 Hz
Adult female voice around 165 to 255 Hz
Adult male voice around 85 to 155 Hz

Bishnu P. Lamichhane, bishnu.lamichhane@newcastle.edu.au Mathematics and Music

 Introduction
Sound Synthesis

Final Slide

Play Three Instruments

Thank you. Questions.

Bishnu P. Lamichhane, bishnu.lamichhane@newcastle.edu.au Mathematics and Music