Slide 1 of 20

Time Series and Spectral Analysis

stae 2021

Part V. Spectral analysis

Sónia Gouveia
( sonia.gouveia@ua.pt )

Departamento de Electrónica, Telecomunicações e Informática (DETI)

Instituto de Engenharia Electrónica e Informática de Aveiro (IEETA)
Universidade de Aveiro (UA), Portugal
 Slide 2 of 20

stae 2021

Part V – Spectral analsysis

1. Motivation
1.1. Some ideas
1.2. What is spectral analysis?
1.3. Spectrum: definition and estimation
2. Parametric spectral densities
1.1. White noise
1.2. AR, MA and ARMA models
3. References
 Slide 3 of 20

1. Motivation Some ideas

What is spectral analysis?
Spectrum: definition and estimation
stae 2021

Fundamental concepts in time series

Figures from Brockwell and Davies (2016) and Shumway and Stoffer (2010).
 Slide 4 of 20

1. Motivation Some ideas

What is spectral analysis?
Spectrum: definition and estimation
stae 2021

Some ideas
Cyclic dynamics are the rule rather than the exception, as many data show such
temporal pattern…

What types of questions might arise from these data?

• How to estimate the periodicity of the time series using spectral analysis?
• How to characterize the frequency content of the time series?
 Slide 5 of 20

1. Motivation Some ideas

What is spectral analysis?
Spectrum: definition and estimation
stae 2021

Time Series and Spectral Analysis

What is a time series? A time series is a set of observations obtained by measuring a
single variable regularly over a period of time. Time series analysis
(temporal domain)

𝑥𝑡 Time series analysis

(temporal domain) 𝑥𝑡
… Prediction of future values
of the time series
(Model forecasting) Time scale 𝑡 (e.g. sec)

𝑡 = 1,2, … , 𝑛 Spectral analysis

(frequency domain)
Parametric Non-parametric
Knowledge improvement of the underlying (model-driven) (data-driven)
mechanism that generates the time series
(Data decomposition or Model fitting)
𝑋(𝜔)
𝑐𝑦𝑐𝑙𝑒𝑠
Frequency scale 𝜔 (e.g. Hz = )
𝑠𝑒𝑐
 Slide 6 of 20

1. Motivation Some ideas

What is spectral analysis?
Spectrum: definition and estimation
stae 2021

What is spectral analysis?

In short, spectral analysis corresponds to the decomposition of a time series into
underlying sine and cosine functions of different frequencies, which allows to
determine those frequencies that appear particularly strong or important.

𝑓 = 2 cycles per time unit

𝑆 = 1/2 time units per cycle

The frequency (𝜔 or 𝑓) of a sine or cosine function is such that 𝐴 cos 𝜔𝑡 + 𝜙 =

𝐴 cos 2 𝜋𝑓𝑡 + 𝜙 and 𝑓 is expressed in terms of the number of cycles per unit time.
The period (𝑆) is then defined as the amount of time required for one full cycle. Thus, it
is the reciprocal of the frequency, 𝑆 = 1/𝑓.
 Slide 7 of 20

1. Motivation Some ideas

What is spectral analysis?
Spectrum: definition and estimation
stae 2021

Fitting sine waves

One way of viewing spectral analysis is as a linear multiple regression problem, where
the dependent variable is the observed time series 𝑥𝑡 for 𝑡 = 1, … , 𝑛 (𝑛 even), and the
predictors are the 𝑛 − 1 sine functions of all possible (discrete) frequencies
𝑛 𝑛
2𝜋t 2𝜋t 2𝜋 − 1 𝑡 2𝜋 −1 𝑡
cos , sin , … . , cos 2 , sin 2 , cos(𝜋t)
𝑛 𝑛 𝑛 𝑛
If the estimated regression coefficients are denoted by 𝑎1 , 𝑏1 , … , 𝑎𝑛/2−1 , 𝑏𝑛/2−1 , 𝑎𝑛/2 ,
respectively then
𝑛/2−1
2𝜋𝑘𝑡 2𝜋𝑘𝑡
𝑥𝑡 = 𝑎0 + ෍ 𝑎𝑘 cos + 𝑏𝑘 sin + 𝑎𝑛/2 cos 𝜋𝑡
𝑛 𝑛
𝑘=1

The cosine parameters, 𝑎𝑘 , and sine parameters, 𝑏𝑘 , convey the degree to which the
respective functions are correlated with the data. This regression model is a finite
Fourier series for a discrete time series.
 Slide 8 of 20

1. Motivation Some ideas

What is spectral analysis?
Spectrum: definition and estimation
stae 2021

Fitting sine waves

𝑛/2−1
2𝜋𝑘𝑡 2𝜋𝑘𝑡
𝑥𝑡 = 𝑎0 + ෍ 𝑎𝑘 cos + 𝑏𝑘 sin + 𝑎𝑛/2 cos 2𝜋0.5𝑡
𝑛 𝑛
𝑘=1

Note that
• there are no degrees of freedom for error, because the number of coefficients equals
the length of the time series (𝑛);
• the intercept term 𝑎0 is just the mean of the time series;
• the lowest possible frequency is one cycle or 2𝜋 radians per record length, which is
2𝜋/𝑛 per sampling interval (𝑘 = 1);
• a general frequency, in this representation, is 𝑘 cycles per record length or 2𝜋𝑘/𝑛
radians per sampling interval.
• the highest frequency is 0.5 cycles per sampling interval (𝑘 = 𝑛/2) corresponding
to 𝜋 radians per sampling interval.
 Slide 9 of 20

1. Motivation Some ideas

What is spectral analysis?
Spectrum: definition and estimation
stae 2021

Nyquist frequency
𝑛/2−1
2𝜋𝑘𝑡 2𝜋𝑘𝑡
𝑥𝑡 = 𝑎0 + ෍ 𝑎𝑘 cos + 𝑏𝑘 sin + 𝑎𝑛/2 cos 2𝜋0.5𝑡
𝑛 𝑛
𝑘=1

Many time series are of a variable that is continuous in time but is sampled to give a
time series at discrete time steps.
The frequency resolution and frequency range are determined by the sampling interval
(Δ𝑡) and record length 𝑛 :
• the sampling interval (or sampling rate) constrains the highest frequency possible to
detect (also known as the Nyquist frequency),
• the length of the time series determines the lowest frequency possible to distinguish.
In general, the sampling frequency is 𝑓𝑠 = 1/Δ𝑡 and the Nyquist frequency becomes
𝑓𝑠 /2. For example, if 𝑥𝑡 is sampled every week (Δ𝑡 = 1 ⇒ 𝑓𝑠 /2 = 0.5) then it is not
possible to detect cycles less than 1/0.5 = 2 weeks in length.
 Slide 10 of 20

1. Motivation Some ideas

What is spectral analysis?
Spectrum: definition and estimation
stae 2021

Periodogram
The periodogram quantifies the contributions of the individual frequencies to the time
series regression and is defined as
𝑃𝑘 = 𝑎𝑘2 + 𝑏𝑘2
where 𝑃𝑘 is the periodogram value at frequency 𝑘 (for 𝑘 = 1, … , 𝑛/2 ). The
periodogram is calculated through the fast Fourier transform algorithm (FFT).
• The periodogram values can be interpreted in terms of variance of the data at the
respective frequency or period.
• If a large correlation (sine or cosine coefficient) is identified then there is a strong
periodicity of the respective frequency (or period) in the data.
delta_t=1
t = seq(0,100,by=delta_t)
x = cos(2*pi*t/16) + 0.75*sin(2*pi*t/5)
par(mfrow=c(1,2))
plot(t,x,'l')
stats::spectrum(x, log="no")
 Slide 11 of 20

1. Motivation Some ideas

What is spectral analysis?
Spectrum: definition and estimation
stae 2021

Spectral densities
Suppose that 𝑋𝑡 is a zero-mean stationary process with ACVF 𝛾𝑋 satisfying
σ∞
ℎ=−∞ 𝛾𝑋 ℎ < ∞. The spectral density of 𝑋𝑡 is the function
∞
1
𝑋 𝜔 = ෍ 𝛾𝑋 ℎ 𝑒 −𝑖ℎ𝜔 , −∞ ≤ 𝜔 ≤ ∞
2𝜋
ℎ=−∞

where 𝑒 𝑖𝜔 = cos 𝜔 + 𝑖 sin(𝜔) and 𝑖 = −1. The summability of 𝛾𝑋 ℎ implies

𝑖ℎ𝜔 2
that the series in 𝑋 𝜔 converges absolutely (since 𝑒 = 𝑐𝑜𝑠 2 ℎ𝜔 + sin2 (ℎ𝜔) =
1). As sinusoidal functions have period 2𝜋, so also does 𝑋 𝜔 and it suffices to confine
attention to the values of 𝑋 𝜔 for −𝜋 < 𝜔 ≤ 𝜋.
Basic properties of 𝑋 𝜔
• 𝑋 𝜔 is even i.e. 𝑋 𝜔 = 𝑋 −𝜔 ,
• 𝑋 𝜔 ≥ 0 for all 𝜔 ∈ (−𝜋, 𝜋],
𝜋 𝜋
• 𝛾𝑋 ℎ = ‫׬‬−𝜋 𝑋 𝜔 𝑒 𝑖ℎ𝜔 𝑑𝜔 = ‫׬‬−𝜋 𝑋 𝜔 cos(ℎ𝜔) 𝑑𝜔, for all integers ℎ.
 Slide 12 of 20

1. Motivation Some ideas

What is spectral analysis?
Spectrum: definition and estimation
stae 2021

Spectrum estimation
The spectrum estimation methods can be classified as:
• Parametric methods
Based on parametric models of a time series e.g. autoregressive moving average
(ARMA) models. The model parameters are first estimated from the data and then the
spectrum is obtained considering the estimated model. It is a model-driven approach.
• Nonparametric methods
It is a data-driven approach based on the discrete Fourier transform. Includes many
algorithms including the periodogram method and the Welch method (averaged
periodogram).

In general, parametric estimation methods lead

to a smoother spectrum estimate.
 Slide 13 of 20

1. Parametric spectra White noise

AR, MA and ARMA models

stae 2021

Spectral density of white noise

If 𝑋𝑡 is a white noise process, i.e. 𝑋𝑡 ~ WN 0, 𝜎𝑋2 where 𝜎𝑋2 is the constant variance,
then 𝛾𝑋 0 = 𝜎𝑋2 and 𝛾𝑋 ℎ = 0 for all ℎ. This process has a flat spectral density
𝜎𝑋2
𝑋 𝜔 = , 𝜔 ∈ (−𝜋, 𝜋]
2𝜋
A process with a constant spectral density is called white noise, since each frequency in
the spectrum contributes equally to the variance of the process.

# Simulate a white noise realization

w = rnorm(200, mean = 0, sd = 1)
par(mfrow=c(1,2))
astsa::arma.spec(ar = 0, ma = 0, col=1, lwd=1, main="", ylim=c(0,2))
s=stats::spectrum(w, log="no", ylim=c(0,2))
mean(s$spec)
 Slide 14 of 20

1. Parametric spectra White noise

AR, MA and ARMA models

stae 2021

Spectral density of an AR(1) process

If 𝑋𝑡 is a stationary AR(1) process satisfying the equation 𝑋𝑡 = 𝜙𝑋𝑡−1 + 𝑒𝑡 where
𝑒𝑡 ~ WN 0, 𝜎𝑒2 then 𝑋𝑡 has spectral density
∞
𝜎𝑒2
𝑋 𝜔 = 1 + ෍ 𝜙 ℎ 𝑒 −𝑖ℎ𝜔 + 𝑒 𝑖ℎ𝜔 =
2𝜋(1 − 𝜙 2 )
ℎ=1
𝜎𝑒2 𝜙𝑒 𝑖𝜔 𝜎𝑒2 𝜙𝑒 −𝑖𝜔
= 2
1+ 𝑖𝜔
+ −𝑖𝜔
= 1 − 2𝜙 𝑐𝑜𝑠 𝜔 + 𝜙 2 −1
2𝜋(1 − 𝜙 ) 1 − 𝜙𝑒 1 − 𝜙𝑒 2𝜋
# AR(1) model: theoretical spectrum
phi=0.7
par(mfrow=c(1,2))
astsa::arma.spec(ar = phi, ma = 0, col=1, lwd=1, main='')
astsa::arma.spec(ar = -phi, ma = 0, col=2, lwd=1, main='')
# Simulate a realization of the AR(1) process
par(mfrow=c(1,1))
AR1_sim1 = stats::arima.sim(list(order=c(1,0,0), ar=phi), n=100)
AR1_sim2 = stats::arima.sim(list(order=c(1,0,0), ar=-phi), n=100)
ts.plot(AR1_sim1, AR1_sim2, gpars = list(col = c("black", "red")))
 Slide 15 of 20

1. Parametric spectra White noise

AR, MA and ARMA models

stae 2021

Spectral density of an AR(1) process

𝜙 = 0.7 𝜙 = −0.7

Different patterns emerge by evaluating the spectra for e.g. 𝜙 = ±0.7:

• for 𝜙 = 0.7, the density is large for low frequencies and small for high frequencies.
This leads to a smooth realization of the process with relatively few high-frequency
components.
• for 𝜙 = −0.7, the realization of the process fluctuates rapidly about its mean value.
The series has a large contribution from high-frequency components as reflected by
the size of the spectral density near frequency 𝜋.
 Slide 16 of 20

1. Parametric spectra White noise

AR, MA and ARMA models

stae 2021

Spectral density of an MA(1) process

If 𝑋𝑡 = 𝜃𝑒𝑡−1 + 𝑒𝑡 where 𝑒𝑡 ~ WN 0, 𝜎𝑒2 then 𝑋𝑡 has spectral density
𝜎𝑒2 𝜎𝑒2
𝑋 𝜔 = 1+ 𝜃2 +𝜃 𝑒 −𝑖𝜔 + 𝑒 𝑖𝜔 = 1 + 2𝜃 𝑐𝑜𝑠 𝜔 + 𝜃 2 , 𝜔 ∈ (−𝜋, 𝜋]
2𝜋 2𝜋
This function is shown for 𝜃 = ±0.7 and an analogous interpretation to the AR(1)
spectrum can be made with respect to the frequency content.

𝜃 = 0.7

𝜃 = −0.7

# MA(1) model: theoretical spectrum

theta=0.7
par(mfrow=c(1,2))
astsa::arma.spec(ar = 0, ma = theta, col=1, lwd=1, main='')
astsa::arma.spec(ar = 0, ma = -theta, col=2, lwd=1, main='')
 Slide 17 of 20

1. Parametric spectra White noise

AR, MA and ARMA models

stae 2021

Spectral density of an 𝐀𝐑𝐌𝐀(𝒑, 𝒒) process

If 𝑋𝑡 is a stationary ARMA(𝑝, 𝑞) satisfying
𝝓𝑝 𝐵 𝑋𝑡 = 𝜽𝑞 𝐵 𝑒𝑡
where 𝑒𝑡 ~𝑊𝑁(0, 𝜎𝑒2 ), 𝝓𝑝 𝑧 = 1 − 𝜙1 𝑧 − ⋯ − 𝜙𝑝 𝑧 𝑝 is the AR polynomial and
𝜽𝑞 𝑧 = 1 + 𝜃1 𝑧 + ⋯ + 𝜃𝑞 𝑧 𝑞 is the MA polynomial, then
2
𝜎𝑒2 𝜽𝑞 𝑒 −𝑖𝜔
𝑋 𝜔 = 2,𝜔 ∈ (−𝜋, 𝜋]
2𝜋 𝝓 𝑒 −𝑖𝜔
𝑝

Because the spectral density of an ARMA process is a ratio of trigonometric

polynomials, it is often called a rational spectral density.
Example: For an ARMA(1,1) process, the spectral density in 𝜔 ∈ (−𝜋, 𝜋] becomes
2
𝜎𝑒2 𝜽1 𝑒 −𝑖𝜔 𝜎𝑒2 1 + 𝜃𝑒 −𝑖𝜔 1 + 𝜃𝑒 𝑖𝜔 𝜎𝑒2 1 + 𝜃 2 + 2𝜃 cos 𝜔
𝑋 𝜔 = = =
2𝜋 𝝓1 𝑒 −𝑖𝜔 2 2𝜋 1 − 𝜙𝑒 −𝑖𝜔 1 − 𝜙𝑒 𝑖𝜔 2𝜋 1 + 𝜙 2 − 2𝜙 cos 𝜔
 Slide 18 of 20

1. Parametric spectra White noise

AR, MA and ARMA models

stae 2021

Spectral density of an 𝐀𝐑(𝟐) process

For an AR(2) process, the spectral density in 𝜔 ∈ (−𝜋, 𝜋] becomes
2
𝜎𝑒2
𝜽0 𝑒 −𝑖𝜔𝜎𝑒2 1
𝑋 𝜔 = −𝑖𝜔 2
= −𝑖𝜔 −2𝑖𝜔 𝑖𝜔 2𝑖𝜔
=
2𝜋 𝝓2 𝑒 2𝜋 1 − 𝜙1 𝑒 − 𝜙2 𝑒 1 − 𝜙1 𝑒 − 𝜙2 𝑒
𝜎𝑒2 1
=
2𝜋 1 + 𝜙12 + 2𝜙2 + 𝜙22 + 2 𝜙1 𝜙2 − 𝜙1 cos 𝜔 − 4𝜙2 cos 2 (𝜔)
The spectra will exhibit a well-defined peak at 𝑋 𝜔m where 𝜔m can be obtained
explicitly by differentiating the 𝑋 𝜔 denominator with respect to cos 𝜔 and setting
the derivative equal to zero. This leads to
𝜙1 𝜙2 − 𝜙1 −1
𝜙1 𝜙2 − 𝜙1
cos 𝜔m = ֞ 𝜔m = cos
4𝜙2 4𝜙2
measured in radians/(time unit) implying that 𝜔m /(2𝜋) is in cycles/(time unit) and the
period of the dominant periodic componente corresponds to 2𝜋/𝜔m time units.
 Slide 19 of 20

1. Parametric spectra White noise

AR, MA and ARMA models

stae 2021

Spectral density of an 𝐀𝐑(𝟐) process

# AR(2) model: theoretical spectrum and frequency with highest spectrum amplitude
phi=c(1.3,-0.6); # Xt=1.3X(t-1)-0.6X(t-2)+e(t)
par(mfrow=c(1,2))
astsa::arma.spec(ar = phi, ma = 0, col=1, lwd=1, main='')
wmax1=acos((phi[1]*phi[2]-phi[1])/(4*phi[2]))/(2*pi) # cycles per time unit

# Simulate a realization of the AR(2) process

x = stats::arima.sim(list(order=c(2,0,0), ar=phi), n=512)

# Compute the periodogram

per = stats::spectrum(x, log="no")

# Find the dominant frequency (via spectrum)

S=forecast::findfrequency(x)
wmax2=1/S

# Repeat example for another AR(2)

phi=c(1,-0.9); # Xt=1X(t-1)-0.9X(t-2)+e(t)
 Slide 20 of 20

4. References
stae 2021

References
• Brockwell P.J. and Davies R.A. (2016), Introduction to Time Series and Forecasting (3rd ed), Springer
Texts in Statistics. (https://link.springer.com/content/pdf/10.1007%2F978-3-319-29854-2.pdf)