Random Processes, Correlation, and Power Spectral Density
Random Processes, Correlation, and Power Spectral Density
1 Random Processes
A random process X(t) is a set (or “ensemble”) of random variables expressed as a function
of time (and/or some other independent variables).
X1 (t)
X2 (t)
X(t) = ..
.
Xm (t)
For example, if X(t) represents be the wind at the top of the Cape Hatteras light house from
12:00 pm to 1:00 pm, then Xj (t) could be the wind speed at the top of the Cape Hatteras
light house from 12:00 pm to 1:00 pm on the j th day of the year.
1.1 Ensemble Average
The ensemble average is the average across the variables in the ensemble at a fixed point in
time, t.
m
1 X
E[g(X(t))] = E[g(X1 (t)), g(X2 (t)), · · · , g(Xm (t))] ≈ g(Xj (t))
m j=1
In general the ensemble average can change with time. A random process X is stationary if
ensemble statistics are equal for every point in time:
E[g(X(t1 ))] = E[g(X(t2 ))] ∀ t1 , t2
A process is weak-sense stationary if the first two moments of the probability density fX (X(t))
are time-independent. A process is strong-sense stationary if all moments of the probability
density fX (X(t)) are time-independent.
1.2 Time Average
The time average is the average along time for a variable Xj (t) in the ensemble.
1 Z T /2
hg(Xj )i = lim g(Xj (t)) dt
T →∞ T −T /2
60
sample record number
55
50
45
40
35
0 2 4 6 8 10
time, sec
60
sample record number
55
50
45
40
35
0 2 4 6 8 10
time, sec
CC BY-NC-ND January 9, 2018, H.P. Gavin
Power Spectral Density 3
The statistics of an ergodic process X(t) can be found from any single record Xj (t) from the
ensemble.
• mean value
n
1 Z T /2 1X
hX(t)i = E[X(t)] = lim Xj (t) dt = n→∞
lim xj (ti )
T →∞ T −T /2 n i=1
• mean-square value
n
D E 1 Z T /2 2 1X
X 2 (t) = E[X 2 (t)] = lim Xj (t) dt = n→∞
lim x2 (ti )
T →∞ T −T /2 n i=1 j
• variance
2
σX = E[X 2 ] − (E[X])2
A similar relation can be found by applying the Fourier transforms in the opposite order,
and gives the Fourier transform of x(t) = (1) cos(2πf 0 t).
Z ∞
0 0
δ(f ± f ) = (1)e−i2π(f ±f )t dt .
−∞
3 Parseval’s Theorem
For (generally) complex-valued functions x(t),
Z ∞ Z ∞
x∗1 (t)x2 (t) dt = X̄1∗ (f )X̄2 (f ) df
−∞ −∞
The proof of Parseval’s theorem involves the Fourier transform of the Dirac delta function:
Z ∞ Z ∞ Z ∞ Z ∞
+i2πf 0 t
x∗1 (t)x2 (t) dt = X̄1∗ (f )e−i2πf t df X̄2 (f )e df 0
dt
−∞ −∞ −∞ −∞
Z ∞ Z ∞ Z ∞
0
= X̄1∗ (f )X̄2 (f 0 ) e+i2πf t e−i2πf t df df 0 dt
−∞ −∞ −∞
Z ∞ Z ∞ Z ∞
i2π(f 0 −f )t
= X̄1∗ (f )X̄2 (f 0 ) e dt df 0 df
−∞ −∞ −∞
Z ∞ Z ∞
= X̄1∗ (f )X̄2 (f 0 ) δ(f 0 − f ) df 0 df
−∞ −∞
Z ∞ Z ∞
= X̄1∗ (f ) 0 0
X̄2 (f ) δ(f − f ) df 0
df
−∞ −∞
Z ∞
= X̄1∗ (f )X̄2 (f ) df
−∞
If x1 (t) and x2 (t) are both real, then x∗1 (t) = x1 (t) but X̄1∗ (f ) 6= X̄1 (f ), so,
Z ∞ Z ∞
x1 (t)x2 (t) dt = X̄1∗ (f )X̄2 (f ) df
−∞ −∞
4 Auto-correlation
For a zero-mean process X (E[Xj (t)] = 0 ∀ j), the covariance of the process at time t, X(t),
with the process at time t + τ , X(t + τ ), is
1 Z T /2
RXX (τ ) = hX(t)X(t + τ )i = E[X(t) · X(t + τ )] = lim x(t) · x(t + τ ) dt
T →∞ T −T /2
n
1 X
= lim x(ti ) · x(ti + τ )
n→∞ n + 1
i=1
The auto power spectral density SXX (f ) of a zero-mean random process x(t) is defined in
terms of finite-duration Fourier transforms,
" !∗ !#
1 Z T /2
−i2πf t
Z T /2
−i2πf t
SXX (f ) = lim E x(t)e dt x(t)e dt
T →∞ T −T /2 −T /2
• Z ∞ D E
SXX (f ) df = X 2 (t)
−∞
proof:
Z ∞ Z ∞ " !∗ !#
1 Z T /2 −i2πf t
Z T /2
SXX (f ) df = lim E x(t)e dt x(t)e−i2πf t dt df
−∞ −∞ T →∞ T −T /2 −T /2
Z ∞ " #
1 Z T /2 Z T /2
∗ +i2πf p −i2πf q
= lim E x (p)x(q) e e dp dq df
−∞ T →∞ T −T /2 −T /2
"Z #
1 T /2 Z T /2 Z ∞
∗ +i2πf (p−q)
= lim E x (p)x(q) e df dp dq
T →∞ T −T /2 −T /2 −∞
"Z Z ∞ #
1 T /2 Z T /2
∗ +i2πf (p−q)
= lim E x (p)x(q) e df dp dq
T →∞ T −T /2 −T /2 −∞
"Z #
1 T /2 Z T /2
∗
= lim E x (p)x(q) δ(p − q) dp dq
T →∞ T −T /2 −T /2
"Z #
1 T /2
∗
= lim E x (t) x(t) dt
T →∞ T −T /2
1 Z T /2 ∗ D E
= lim x (t) x(t) dt = X 2 (t)
T →∞ T −T /2
• Z ∞
SXX (f ) exp(+i2πf τ ) df = RXX (τ )
−∞
proof:
Z ∞ Z ∞ " !∗ !#
1 Z T /2 Z T /2
+i2πf τ
SXX (f )e df = lim E x(t)e−i2πf t dt x(t)e−i2πf t dt e+i2πf τ df
−∞ −∞ T →∞ T −T /2 −T /2
Z ∞ " #
1 Z T /2 Z T /2 ∗
= lim E x (p)x(q) e+i2πf p e−i2πf q dp dq e+i2πf τ df
−∞ T →∞ T −T /2 −T /2
"Z #
1 T /2 Z T /2 Z ∞
∗ +i2πf (p−q+τ )
= lim E x (p)x(q) e df dp dq
T →∞ T −T /2 −T /2 −∞
"Z Z ∞ #
1 T /2 Z T /2
∗ +i2πf (p−q+τ )
= lim E x (p)x(q) e df dp dq
T →∞ T −T /2 −T /2 −∞
"Z #
1 T /2 Z T /2
∗
= lim E x (p)x(q) δ(p − q + τ ) dp dq
T →∞ T −T /2 −T /2
"Z #
1 T /2
∗
= lim E x (t) x(t + τ ) dt
T →∞ T −T /2
1 Z T /2 ∗
= lim x (t) x(t + τ ) dt = RXX (τ )
T →∞ T −T /2
• Z ∞
RXX (τ ) exp(−i2πf τ ) dτ = SXX (f )
−∞
proof:
Z ∞ Z ∞ " #
−i2πf τ 1 Z T /2
RXX (τ ) e dτ = lim E x∗ (t) x(t + τ ) dt e−i2πf τ dτ
−∞ −∞ T →∞ T −T /2
"Z #
1 T /2 Z T /2
∗ −i2πf τ
= lim E x (t) x(t + τ ) e dτ dt
T →∞ T −T /2 −T /2
"Z #
1 T /2 Z T /2
∗ +i2πf (p−q)
= lim E x (p) x(q) e dp dq
T →∞ T −T /2 −T /2
" !∗ !#
1 Z T /2
−i2πf p
Z T /2
−i2πf q
= lim E x(p)e dp x(q) e dq
T →∞ T −T /2 −T /2
= SXX (f )
Thus, the auto-correlation function and the auto power spectrum are related by Fourier
transform pairs. These are the Wiener-Khintchine relations.
Note that the power spectral density is a density function. If the process X(t) has units of
‘m’ (meters), and the circular frequency, f , is the independent variable (with units of ‘Hz’ or
‘cycles/sec’), then the units of of the power spectral density is SXX (f ) is ‘m2 /Hz’.
Alternatively, if the angular frequency ω is the independent variable (dω = 2π df ), then the
units of the power spectral density SXX (ω) is ‘m2 /(rad/s)’.
So, to convert between frequencies f and ω, the value of the power spectral density needs to
be scaled as well.
SXX (ω) dω = 2π SXX (f ) df
The one-sided power spectral density function GXX (ω) is defined here so that
Z ∞ Z ∞
D E df 1 Z∞
X 2 (t) = GXX (f ) df = GXX (ω/(2π)) dω = GXX (ω) dω
0 0 dω 2π 0
Since SXX (f ) is symmetric about f = 0,
D E Z ∞
X 2 (t) = 2 SXX (f ) df ,
0
so, the one-sided psd GXX (f ) is twice the symmetric, two-sided psd SXX (f )
5 Cross-correlation
For zero-mean processes X and Y (E[Xi (t)] = 0 and E[Yj (t)] = 0 ∀ i, j), the covariance of
process X at time t, X(t), with a different process Y at time t + τ , Y (t + τ ), is
1 Z T /2
RXY (τ ) = hX(t)Y (t + τ )i = E[X(t) · Y (t + τ )] = lim x(t) · y(t + τ ) dt
T →∞ T −T /2
n
1 X
= lim x(ti ) · y(ti + τ )
n→∞ n + 1
i=1
If X and Y are ergodic random processes (E[g(X(t1 )] = E[g(X(t2 )] ∀ t1 , t2 and E[g(Y (t1 )] =
E[g(Y (t2 )] ∀ t1 , t2 ), then RXY (τ ) is independent of time.
Since X and Y are zero-mean random processes by assumption, then RXY (0) = E[X(t)Y (t)]
which is the covariance of X and Y , V[X, Y ].
The cross-correlation function is not symmetric: RXY (−τ ) 6= RXY (τ ),
however, RXY (τ ) = RY X (−τ ).
Z ∞
RXY (τ ) = SXY (f ) exp(+i2πf τ ) df
−∞
Z ∞
SXY (f ) = RXY (τ ) exp(−i2πf τ ) dτ · · · Wiener − Khintchine
−∞
7 Examples
• Dirac delta function x(t) = δ(t)
Z T /2
X̄(f ) = δ(t) exp(−i2πf t) dt =1
−T /2
h i
SXX (f ) = limT →∞ E 1
T
X̄ ∗ (f )X̄(f ) =0
Z ∞
RXX (τ ) = SXX (f ) exp(i2πf τ ) df = 0
−∞
Z ∞
RXX (0) = δ(t)2 dt =∞
−∞
where fk = (k∆f ) and θ is a random variable uniformly distributed between 0 and 2π.
• Case 1: The presence of two sinusoidal components is impossible to discern from the
time series. Two components at 10 Hz and 6.18 Hz are clearly apparent in the power
spectrum.
The spike in RXX (τ ) at τ = 0 is from the added white noise, w.
Two sinusoidal components are visibly present in the plot of RXX (τ ).
• Case 2: The time series contains a broad range of frequency components, with more
spectral power at the low frequencies.
The time series loses correlation at time lags τ greater than 0.1 s.
• The three mean square calculations give the same values in both cases.
6
4
2
0
x(t)
-2
-4
-6
-8
-4 -2 0 2 4
1.5
Sxx(f)
0.5
0
-20 -15 -10 -5 0 5 10 15 20
4
3
2
Rxx(τ)
1
0
-1
-2 -1.5 -1 -0.5 0 0.5 1 1.5 2
15
10
5
x(t)
0
-5
-10
-15
-4 -2 0 2 4
1
0.8
0.6
Sxx(f)
0.4
0.2
0
-20 -15 -10 -5 0 5 10 15 20
20
15
10
Rxx(τ)
5
0
-5
-2 -1.5 -1 -0.5 0 0.5 1 1.5 2
% All three mean_sq calculations are identical! % All six mean_sq calculations are identical! % All six mean_sq calculations are identical!
if epsPlots, print(’PSD2.eps’,’-color’,’-FHelvetica:10’); end % The Sxx from FFT of x equals generating Sxx
% The Rxx from IFFT of S equals analytic Rxx
% The Sxx from FFT of x equals generating Sxx
% The Rxx from IFFT of S equals analytic Rxx % ------------------------- example 3
% ------------------------- example 2
Power Spectral Density 13
1.5
x(t)
1
0.5
-Tp +Tp
0
-2 -1.5 -1 -0.5 0 0.5 1 1.5 2
0.1
0.08
0.06
Sxx(f)
0.04
0.02
-1/Tp +1/Tp
0
-2 -1.5 -1 -0.5 0 0.5 1 1.5 2
0.08
0.07
0.06
0.05
Rxx(τ)
0.04
0.03
0.02
0.01
0 -Tp +Tp
-2 -1.5 -1 -0.5 0 0.5 1 1.5 2
10
sum cosine
ifft
5
x(t)
-5
-10
-20 -15 -10 -5 0 5 10 15 20
2
model
1.5 data
So
Sxx(f)
0.5
-fc +fc
0
-3 -2 -1 0 1 2 3
8
model
6 data
4
Rxx(τ)
2
0
-2
-4/fc -3/fc -2/fc -1/fc 0 +1/fc +2/fc +3/fc +4/fc
10
sum cosine
ifft
5
x(t)
-5
-10
-20 -15 -10 -5 0 5 10 15 20
2
model
1.5 data
So
Sxx(f)
6
model
4 data
2
Rxx(τ)
0
-2
-4
-2 -1.5 -1 -0.5 0 0.5 1 1.5 2
0.35
U10= 5 m/s, σU= 1.2m/s
0.2
0.15
0.1
0.05
0
10-3 10-2 10-1 100 101 102
turbulence frequency, f, Hz
Figure 1. Davenport wind turbulence spectra with a Davenport length scale of L = 500 m and
terrain roughness k = 0.01.
0.25
U= 5 m/s, σU= 1.9m/s
U=10 m/s, σU= 2.6m/s
Kaimal wind turbulence PSD , f G uu(f) / σ2u
0.15
0.1
0.05
0
10-3 10-2 10-1 100 101 102
turbulence frequency, f, Hz
Figure 2. Kaimal wind turbulence spectra with a Kaimal length scale of L = 50 and a turbulence
intensity Ti = 0.2.
CC BY-NC-ND January 9, 2018, H.P. Gavin
16 CEE 541. Structural Dynamics – Duke University – Fall 2018 – H.P. Gavin
and define GZZ (0) = 0. This spectrum is scaled so that the mean-square wave height is
Z ∞ 4
D
2
E α U19.5
Z (t) = σZ2 = 2π GZZ (f ) df ≈
0 g 2 2.96
So the standard deviation of the wave height is proportional to the mean wind speed squared,
which is a strong-dependence on mean wind speed. The speed of the largest waves is
g
cp = ≈ (1.14)U19.5
2πfp
The speed of the largest waves is 14% faster than the mean wind speed.
2 0.22
• α = 0.066 (U10 /(gF ))
• γ is the magnification of the largest waves . . . 1 < γ < 7 . . . on average γ ≈ 3.3.
• fp = 2.84(g 2 /(U10 F ))0.33 is the frequency of the largest waves
• σ(f ) = 0.07 for |f | ≤ fp , and σ(f ) = 0.09 for |f | > fp
• psd scaling is: σZ2 = 2π 0∞ GZZ (f ) df
R
4
U= 9 m/s, σZ=0.43m
U=10 m/s, σZ=0.53m
U=12 m/s, σZ=0.77m
3.5 U=15 m/s, σZ=1.20m
2.5
1.5
0.5
0
0 5 10 15 20
wave period, s
5
U= 9 m/s, σZ=0.64m
4 U=10 m/s, σZ=0.70m
3 U=12 m/s, σZ=0.82m
U=15 m/s, σZ=1.00m
2
wave height power spectrum, Gzz(f), m2/Hz
1
0
0 5 10 15 20
12
U= 9 m/s, σZ=0.93m
10 U=10 m/s, σZ=1.02m
8 U=12 m/s, σZ=1.20m
6 U=15 m/s, σZ=1.46m
4
2
0
0 5 10 15 20
50
U= 9 m/s, σZ=1.54m
40 U=10 m/s, σZ=1.69m
30 U=12 m/s, σZ=1.99m
U=15 m/s, σZ=2.41m
20
10
0
0 5 10 15 20
wave period, s
Figure 4. JONSWAP spectra with γ = 3.3 and F = 100 km, 200 km, and 500 km.
(2ζg f /fg )2
GAA (f ) = ā2
(1 − (f /fg )2 )2 + (2ζg f /fg )2
is parameterized by:
However, unlike the models for wind turbulence and ocean waves described in the previous
sections, earthquake ground motions have time-varying amplitudes. An envelope function
that mimics the growth and decay of earthquake ground motions is
1 1 1
0 0 0
10-1 10 0 1
10 10-1 10
0
10
1
10-1 100 101
natural period, Tn, s natural period, Tn, s natural period, Tn, s
Table 1. Ground motion parameters fit to the ATC-63 ground motion suites.
P GV fg zg a b ā
FF 0.33 1.5 0.9 4.0 2.0 0.23
NF-NP 0.52 1.3 1.1 3.0 2.0 0.33
NF-P 0.80 0.5 1.8 1.0 2.0 0.43
m/s Hz - - s m/s2
Either of the two amplitude constants P GV (peak ground velocity) or ā may be used to
specify the amplitude of the ground motion. If P GV is used the ground motion is scaled to
the specified peak value.
ground acceleration power spectrum, Gaa(f)
0.12
far-field
0.1 near-fault without pulse
near-fault with pulse
0.08
0.06
0.04
0.02
0
0 2 4 6 8 10
ground motion frequency, Hz
0.8
envelope
0.6
0.4
0.2
0
0 5 10 15 20 25
time, s
Figure 5. Spectra and envelopes for three classes of earthquake ground motions.
Synthesized ground motion accelerations may be detrended so that the ground velocity is
zero and the ground displacement is nearly zero at the end of the synthesized ground motion,
as illustrated by the example realizations on the next page.
0.2
0.1
accel. g
0
-0.1
-0.2
0 5 10 15 20 25 30
20
veloc. cm/s
10
0
-10
-20
-30
0 5 10 15 20 25 30
25
20
displ. cm
15
10
5
0
0 5 10 15 20 25 30
0.4
0.3
0.2
accel. g
0.1
0
-0.1
-0.2
-0.3
-0.4
0 5 10 15 20 25 30
50
40
veloc. cm/s
30
20
10
0
-10
-20
-30
0 5 10 15 20 25 30
10
displ. cm
5
0
-5
-10
0 5 10 15 20 25 30
0.3
0.2
accel. g
0.1
0
-0.1
-0.2
-0.3
0 5 10 15 20 25 30
60
veloc. cm/s
40
20
0
-20
-40
-60
-80
0 5 10 15 20 25 30
60
40
displ. cm
20
0
-20
-40
0 5 10 15 20 25 30
with zero mean and a variance of 1/(∆t), where (∆t) is the sampling interval.
15 15
σ2
u = 1/dt = 1/0.05
10 10
5 5
time series, u
0 0
-5 -5
-10 -10
-15 -15
0 20 40 60 80 100 0 0.010.020.030.040.050.060.070.080.09
time, s p.d.f.
Figure 6. Gaussian unit white noise with (∆t) = 0.05 s,pn = 2048; computed with
u=randn(1,n)/sqrt(dt); T = (n)(∆t) = 102.4 s and σu = 1/(∆t). points: histogram of
sample, line: Normal distribution
The complex Fourier coefficients Ūq for this series can be computed with a discrete Fourier
transform,
1 n−1 2πqp
X h i
Ūq = up exp −i , q= − n2 + 1, − n2 + 2, · · · , −1, 0, 1, · · · . n
2
n p=0 n
If the input to a dynamical system is discrete unit Gaussian white noise, then the power
spectral density of the output of the system is equal to the magnitude-squared of the system
frequency response function. This is a common application of Gaussian white noise in the
context of dynamic systems analysis. It is therefore helpful to understand the statistical
properties of unit Gaussian white noise.
Finite time discrete unit Gaussian white noise has the following provable properties:
• The real parts of the Fourier coefficients Ūq are symmetric about q = 0 (even); the odd
parts of Ūq are anti-symmetric about q = 0 (odd). Both the real and imaginary parts
of Ūq have zero mean, and a variance of 1/(2T ) (figure 7).
• The auto-power spectral density of sampled unit white noise has an expected value of 1
over the Nyquist frequency range. The auto-power spectral density is a χ2 -distributed
random variable with a degree-of-freedom (d.o.f.) of 2, and a variance of 1. The
coefficient of variation of the power spectral density of unit Gaussian white noise is
therefore also 1 (figure 10).
• The auto-correlation R(τ ) of sampled unit white noise resembles δ(τ ) with R(0) = σu2 =
1/(∆t) (figure 11).
• The fact that the power spectral density estimate from a sample of Gaussian white noise
has a coefficient of variation of 1 means that the uncertainty in the PSD estimate is as
large as the mean PSD value. This presents a challenge in the estimation of PSD’s from
noisy data. This coefficient of variation can be reduced by averaging PSD’s together.
The average of K PSD’s of independent samples of unit Gaussian white noise is a χ2 -
distributed random variableq with 2K degrees of freedom, a covariance of 1/K and a
coefficient of variation of 1/K (figure 12).
Note that the generation of a random time series from a given power spectral density function
(as in example 2 of section 8) results in a time series with exactly the prescribed power spectral
density. If the generated time series is meant to represent a random phenomenon, then it
would be appropriate to randomize the Fourier amplitudes with a Rayleigh distribution.
The reduction of the variance of PSD estimates with averaging motivates the Welch method
for estimating power spectral densities from measurements of noisy time series.
0.4 0.4
real(U) ... even
imag(U) ... odd
2
0.3 0.3 σU = 1/(2T)
0.2 0.2
complex Fourier coefficients, U
0.1 0.1
0 0
-0.1 -0.1
-0.2 -0.2
-0.3 -0.3
-0.4 -0.4
-10 -5 0 5 10 0 1 2 3 4 5 6 7
frequency, f, Hz p.d.f.
Figure 7. Fourier transform coefficients of the sample of Gaussian unit white noise shown in
figure 6. computed with U=fft(u)/n; points: histogram of sample, line: Normal distribution
correlation between real and imaginary = -0.000
0.2
0.1
imag(U) (odd)
-0.1
-0.2
0.35 0.35
1 1
Fourier phase, <U/π (odd)
0.5 0.5
0 0
-0.5 -0.5
-1 -1
-10 -5 0 5 10 0 0.05 0.1 0.15 0.2 0.25 0.3
frequency, f, Hz p.d.f
Figure 9. Fourier transform magnitudes and phases of the sample of Gaussian unit white noise
shown in figure 6, The magnitudes are symmetric about f = 0 (even); the phases are anti-
symmetric about f = 0 (odd). points: histogram of sample, line: Rayleigh distribution for |U |
and uniform distribution for ∠U
K=1, coefficient of variation of S = 0.996
8
7
7
6
6 χ2 distribution
power spectral density, S
5
5 d.o.f. = 2
4 4
3 3
2 2
1 1
0 0
-10 -5 0 5 10 0 0.2 0.4 0.6 0.8 1
frequency, f, Hz p.d.f.
Figure 10. Power spectral density of the sample of unit Gaussian noise shown in figure 6.
computed with S=conj(U)*U/(df) Dashed lines show the 50% confidence interval for the
power spectral density. left: points = histogram of the PSD data, line = χ2 distribution
25
20
15
autocorrelation, R(τ)
10
-5
-60 -40 -20 0 20 40 60
time lag, τ, s
Figure 11. Auto-correlation function of the sample of unit Gaussian noise shown in figure 6.
computed with R=ifft(S)/df
K=10, coefficient of variation of S = 0.307
2.5 3
2.5
2
χ2 distribution
power spectral density, S
2 d.o.f. = 2K
1.5
1.5
1
1
0.5
0.5
0 0
-10 -5 0 5 10 0 0.050.10.150.20.250.30.350.40.450.5
frequency, f, Hz p.d.f.
Figure 12. Average of power spectral densities from K = 10 independent samples of unit
Gaussian white noise. Dashed lines show the 50% confidence interval for the power spectral
density. left: points = histogram of the PSD data, line = χ2 distribution with 2K degrees of
freedom.
11 Computing power spectral density from noisy data via FFT: psd.m
The previous section, and Figures 6 and 12 show that averaging the magnitude-square of the
Fourier amplitudes computed from several independent realizations of the same (stationary)
random process reduces the variance of the power spectral density estimate.
Fourier analysis of any finite-duration time series fundamentally presumes that the series
repeats periodically. If the series is representative of a discrete-time random process then a
sub-sequence of n points from the process does not repeat itself periodically; xk 6= xk+n ∀k, 1 ≤
k ≤ n. Furthermore, the periodic extension of a non-periodic n-point segment contains a
sharp discontinuity between x1 and xn . The Fourier components of sequences with sharp
discontinuities do not approach zero as the frequency approaches the Nyquist frequency. In
fact, these high-frequency spectral magnitudes can be quite a bit larger than those of the
true band limited process.
Welch’s method estimates the power spectral density of a discrete-time sequence by averaging
the FFT amplitudes of windowed sub-sequences. The steps of the algorithm are as follows:
1. Extract K sub-sequences of n points from a long time series (of N points). If the n-point
sub-sequences overlap by no points, K = b(N − no )/(n − no )c. For greatest numerical
efficiency, n should be a power of 2. To minimize the variance of the PSD estimate
from a record of pre-measured time-series, set no = n/2, which gives K = 2N/n − 1,
and a reduction in variance by a factor of approximately 9K/11 [2] . See Figure 13.
2. Detrend each sub-sequence, either by subtracting the mean of the sub-sequence, or by
subtracting the least-squares best-fit straight line through the sub-sequence.
3. Multiply each sub-sequence by a real-valued window sequence wk , (k = 1, ..., n), that
tapers the amplitude to zero at both ends ([6] section 13.4).
Common window choices are:
square: wk = 1
Welch: wk = 1 − (2k/n − 1)2
sine: wk = sin(π(k − 1/2)/n)
Lanczos: wk = sinc(2(k − 1)/(n − 1) − 1)
Hamming: wk = 0.53836 − 0.46146 cos(2πk/n)
Gauss: wk = exp(−(1/2)((k − n/2)/(n/5))2 )
Hanning: wk = (1 + cos(2π(k − n/2)/n))/2
Bartlett: wk = 1 − |2k/n − 1|
These functions, and their spectral shapes, are plotted in Figure 14.
The choice of the window function affects the width of spectral peaks, and the degree
to which spectral content that is not exactly matched to a Fourier frequency “leaks”
into adjacent frequency bins. If the signal is known to contain frequencies only at
the Fourier frequencies, then a square window will give the narrowest peaks, and the
spectral leakage from other (Fourier) frequencies will be precisely zero. This is almost
never the case. (The signal is almost surely to contain spectral content at frequencies
between the Fourier frequencies.) Windowing the data prior to computing the FFT
narrows the peaks associated with a non-Fourier frequency component, and suppresses
the leakage of signal energy throughout the spectrum. So, the choice of window function
controls the trade-off between narrow peaks and leakage into adjacent frequency bins.
Of the window functions illustrated here and excluding the square window, the Gauss
window has the widest peak but the least amount of leakage into adjacent frequencies,
whereas the Welch window has the most narrow peak, but with more leakage into
adjacent frequencies.
5. Compute the power spectral density by multiplying the averaged FFT magnitudes by
the time step, ∆t, and dividing by square of the norm of the window function, ||w||2 .
In this way the two-sided PSD agrees with Parseval’s theorem.
6. Confidence intervals for the PSD estimation may optionally be computed via the χ2
inverse CDF with (2K)(9K/11) degrees of freedom,
0.8
0.6
0.4
0.2
0
0 500 1000 1500 2000 2500
0.8
0.6
0.4
0.2
0
0 500 1000 1500 2000 2500
Figure 13. non-overlapping Welch windows and 50% overlapping Welch windows
1 square
Welch
Sine
0.8 Lancoz
Hamming
Gauss
Hanning
0.6
window, wk
Bartlett
0.4
0.2
0
0 0.2 0.4 0.6 0.8 1
k / n
square
10 0 Welch
Sine
Lancoz
Hamming
Gauss
10-1 Hanning
scaled PSD
Bartlett
10-2
10-3
-4 -2 0 2 4
frequency bin
Figure 14. Window functions for PSD computations and their associated spectral shapes
As an example of the Welch method to compute the PSD, we compute the PSD from a record
corresponding to the model power spectral density function:
10−6 1
SXX (f ) = −3
+
(((f /10) − 1) + 10 (f /10) )
2 2 2 2 (1 + (f /10)2 )2
In these calculations, ∆t = 0.01s; N = 4096, and n = 512 or 1024. The time-series sample is
computed from the power spectral density via the IFFT and uniformly-distributed phases.
1 theta = 2* pi *rand(1 , N /2) - pi ; % u n i f o r m l y −d i s t r i b u t e d p h a s e s
2 theta = [ 0 , theta , - theta ( N /2 -1: -1:1) ]; % p h a s e a n g l e s a n t i −symmetric a b o u t f =0
3 theta ( N /2+1) = 0; % f o r r e a l −v a l u e d s i g n a l s
4 x = r e a l ( i f f t ( sqrt ( Sxx * df ) .* exp( i * theta )))* N ; % imag p a r t ˜ 1e−16
Power spectral densities are then computed from the time-series using 7 windows of n = 1024
points and 15 windows of n = 512 points. The results are shown in Figure 15.
10
5
x(t)
0
-5
-10
n=1024; K= 7
1.4
1.2 n= 512; K=15
1
model
0.8
Sxx (f)
0.6
0.4
0.2
0
-20 -15 -10 -5 0 5 10 15 20
Figure 15. A random sequence corresponding to a particular power spectral density, and the
power-spectral density estimated via the Welch method. solid lines: PSD computed, dots: 50%
confidence interval
12 Computing frequency response functions from noisy data via FFT: tfe.m
In the frequency-domain, the relationship between the true inputs u and true outputs y is the
frequency-response function Ho (f ). The frequency response function may be easily estimated
from the Fourier transforms of the true inputs Ȳ (f ) and outputs Ū (f ).
Ȳ (f )
Ȳ (f ) = Ho (f ) Ū (f ) ⇔ Ho (f ) =
Ū (f )
In the absence of measurement noise, the frequency response function may be computed from
either of the ratios of Fourier transforms below.
Ȳ ∗ (f )Ȳ (f ) Ū ∗ (f )Ȳ (f )
Ho (f ) = =
Ȳ ∗ (f )Ū (f ) Ū ∗ (f )Ū (f )
However, measurements of the input and output û and ŷ typically contain random measure-
ment noise, n and m. (See Figure 16.) As described in section 10, noise in a finite-time
discrete random process propagates directly to randomness in the Fourier coefficients. So
an estimate of a frequency response function obtained by a ratio of noise-corrupted Fourier
transforms would also contain significant noise. Fortunately, noise in power spectral density
estimates of noisy data can be reduced, through a process of windowing and averaging, as
described in section 11. From section 5, the cross-power spectral density
Z ∞
1 ∗
SY U = lim E Ȳ (f ) Ū (f ) = RY U (τ )e−i2πf τ dτ
T →∞ T −∞
Assuming the noise processes n and m are uncorrelated with each other and with the true
input and the true output, the associated cross-power spectra are all zero:
SN M (f ) = 0; SM U (f ) = 0; SM Y (f ) = 0; SN U (f ) = 0; SN Y (f ) = 0.
u Ho y
input output
linear behavior
n + u m + y
noise msmnt noise msmnt
Figure 16. A block diagram of a linear system with noisy measurements.
With these assumptions on the noise processes, we can use power spectral density estimates
to compute lower and upper bounds H1 (f ) and H2 (f ) on the true frequency response function
Ho (f ).
S (f ) SY U (f )
H1 (f ) = Ŷ Û = < Ho (f )
SŶ Ŷ (f ) SY Y (f ) + SM M (f )
SÛ Û (f ) SU U (f ) + SN N (f )
H2 (f ) = = > Ho (f )
SÛ Ŷ (f ) SU Y (f )
An estimate of the frequency response function that simultaneously minimizes the effects of
the input measurement noise and the output measurement noise is the geometric mean of
the bounds [7]. q
Hv (f ) = H1 (f ) H2 (f )
Given a system model (A, B, C, D) and the power spectral density of a stationary input
process SU U (f ), the power spectral density of the output process SY Y (f ) is
SY Y (f ) = |H(f )|2 SU U (f )
from which, Z ∞
D E
2
y (t) = SY Y (f ) df
−∞
... etc.
References
[1] Bendat, J.S. and Piersol, A.G., Random Data: Analayis and Measurement Procedures,
4th ed. Wiley 2010.
[2] Childers, D.G., (ed.) Modern Spectrum Analysis, IEEE Press, 1978.
[3] Clough, R., and Penzien, J., Dynamics of Structures, 2nd ed., CSI Press, 2003. (Ch. 20-
21)
[4] Gavin, H.P., Fourier Series, Fourier Transforms, and Periodic Response to Periodic
Forcing, CEE 541 course notes, 2018
[6] Press, W.H., Flanner, W.H., Teukolsky, S.A., and Vetterling, W.T., Numerical Recipes
in C: The Art of Scientific Computing, 2nd ed., 1992. (Ch. 13)