Time Series Analysis: 3. Convolution

Time Series Analysis:
3. Convolution
P. F. Góra
http://th-www.if.uj.edu.pl/zfs/gora/
2009
Periodogram
The most popular estimate of the power spectrum is called the periodogram.
Because the power spectrum does not contain any information on the phase, we
no longer distinguish the positive and negative frequencies for the purpose of
calculating the periodogram.
P (0) = |G(0)|2 ,
P (fn) =
h
|G(fn
)|2
+ |G(f−n
P (fN/2) = |G(fN/2)|2 .
)|2
i
(1a)
, n = 1, 2,
N
−1
2
(1b)
(1c)
Caveat emptor! When calculating the periodogram, it is particularly important to take the proper
care on what components are stored where, what the normalisation is etc. The point of the
formula (1) is to bin together terms with the same “absolute” frequencies, but different pieces of
software may store them differently. And they do.
c 2009 P. F. Góra
Copyright °
3–2
Leakage
We expect the periodogram to project all the power from the interval
(fn − 1/(2N ∆), fn + 1/(2N ∆)). However, as f 6= fn have non-zero projections onto fm6=n, the power spectrum leaks to other channels. The leakage to
a channel offset by s is given by “the golden formula”:
"
P (k → s) =
c 2009 P. F. Góra
Copyright °
1
sin(πs)
N 2 sin(πs/N )
#2
.
(2)
3–3
1.0
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.0
-4
-3
-2
-1
0
s
1
2
3
4
Power leakage, formula (2)
c 2009 P. F. Góra
Copyright °
3–4
Windowing function
To prevent the leakage, sometimes the series is multiplied by a function that kills
its ends but is close to unity in the middle; this is supposed to smooth the series.
Then the DFT of the modified series is calculated:
−1
1 NX
Dn = √
gk wk e2πink/N .
N k=0
(3)
{wk } is the windowing function.
c 2009 P. F. Góra
Copyright °
3–5
The modified periodogram reads
1
|D0|2 ,
W
i
1 h
2
2
P (fn) =
|Dn| + |D−n| ,
W
³
´
1
P fNyq =
|DN/2|2 ,
W
P (0) =
W =
NX
−1
k=0
c 2009 P. F. Góra
Copyright °
wk2 .
(4a)
(4b)
(4c)
(4d)
3–6
Most popular windowing functions
Barlett window:
Hann window:
¯
¯
¯ k − N/2 ¯
¯
¯
wk = 1 − ¯
¯.
¯ N/2 ¯
·
µ
1
2πk
wk =
1 − cos
2
N
Welch window:
Ã
k − N/2
wk = 1 −
N/2
c 2009 P. F. Góra
Copyright °
(5)
¶¸
.
(6)
!2
.
(7)
3–7
Windowing function
1.0
Hann
Barlett
Welch
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.0
0
c 2009 P. F. Góra
Copyright °
N/2
N
3–8
The averaged spectrum, square window
10 3
10 2
P(f)
10 1
10 0
10-1
10-2
1.00
1.25
1.50
f
c 2009 P. F. Góra
Copyright °
3–9
The averaged spectrum, Barlett window
10 3
10 2
P(f)
10 1
10 0
10-1
10-2
1.00
1.25
1.50
f
c 2009 P. F. Góra
Copyright °
3–10
The averaged spectrum, Hann window
10 3
10 2
P(f)
10 1
10 0
10-1
10-2
1.00
1.25
1.50
f
c 2009 P. F. Góra
Copyright °
3–11
The averaged spectrum, Welch window
10 3
10 2
P(f)
10 1
10 0
10-1
10-2
1.00
1.25
1.50
f
c 2009 P. F. Góra
Copyright °
3–12
10 3
10 3
okno prostokatne
10 1
10 1
P(f)
10 2
P(f)
10 2
okno Barletta
10 0
10 0
10-1
10-1
10-2
1.00
10 3
1.25
10-2
1.00
1.50
f
10 3
okno Hanna
10 1
10 1
1.50
okno Welcha
P(f)
10 2
P(f)
10 2
1.25
f
10 0
10 0
10-1
10-1
10-2
1.00
1.25
f
c 2009 P. F. Góra
Copyright °
1.50
10-2
1.00
1.25
1.50
f
3–13
Power spectrum of nonstationary signals
To calculate the time-dependent power spectrum of nonstationary signals, divide
the signal into overlapping segments; ideally, each segment should be approximately stationary. Then calculate the powers spectrum for each segment.
c 2009 P. F. Góra
Copyright °
3–14
1.0
g(t)
P(f)
0.5
32
24
0.0
16
0
-0.5
8
1
2
f
0
4
8
12
16
t
20
24
28
t
3
4
0
32
Left — a nonstationary signal. Right — its time dependent spectrum.
c 2009 P. F. Góra
Copyright °
3–15
Another example of nonstationary data
80
Daily births in California in 1959
70
urodziny
60
50
40
30
20
10
0
64
128
192
256
320
dzień
c 2009 P. F. Góra
Copyright °
3–16
Time dependent spectrum
P(t,f)
2-1
32
64
2-2
96
128
160
192
224
t
256
288
c 2009 P. F. Góra
Copyright °
2-3
2-4
2-5
f
3–17
Convolution
s(t), r(t) — continuous signals. Their convolution:
Z∞
z(t) = (s ? r)(t) =
s(t0)r(t − t0) dt0
(8)
−∞
Fourier transforms fulfil Z(f ) = S(f )R(f ).
N −1
N −1
1
2
{sn}n=0
, {rn}n=0
— periodic discrete signals. The discrete convolution:
zj = (s ? r)j =
M/2
X
sj−k rk .
(9)
k=−M/2+1
where M = max(N1, N2) and the shorter signal is padded with zeros. (In
many situations, one of the “signals” is much shorter than the other.) Fourier
transforms fulfil Zk = Sk Rk .
c 2009 P. F. Góra
Copyright °
3–18
The response function of a filter
c 2009 P. F. Góra
Copyright °
3–19
Input
Output
011111 0000 00000
0
0.125
001111 1000 00000
000111 1100 00000
0.625
000011 1110 00000
1.625
000001 1111 00000
1.75
000000 1111 10000
1.75
000000 0111 11000
1.625
000000 0011 11100
1.125
000000 0001 11110
0.125
000000 0000 11111
0
0.125A + 0.5B + C + 0.125D
c 2009 P. F. Góra
Copyright °
3–20
A rectangular signal convoluted with the response function
in
t
out
c 2009 P. F. Góra
Copyright °
3–21
A convolution of a signal and a response function
original
convolution
response
0
0
The ends are spoiled!
c 2009 P. F. Góra
Copyright °
3–22
A convolution of a signal and a response function
original
convolution
response
0
0
The ends are repaired by padding with zeros
c 2009 P. F. Góra
Copyright °
3–23
The procedure
1. Pad the input signal with enough zeros to cover the longer tail of the response function (remember that the padded signal should have a “magic”
length).
2. Calculate the FFT of both the signal and the response in one go.
3. Multiply the transforms termwise.
4. Calculate the inverse FFT.
The total cost ∼ O(2N log N )
c 2009 P. F. Góra
Copyright °
3–24
Back to windowing functions
Remember Eq. (3):
−1
1 NX
gk wk e2πink/N .
Dn = √
N k=0
The signal, gk , and the windowing function, wk , are (inverse) Fourier transforms
of Gn, Wn. Multiplying by a windowing function is equivalent to convolving in the
Fourier domain.
c 2009 P. F. Góra
Copyright °
3–25
An unexpected (?) application of the convolution
Given two polynomials A(z), B(z) of degrees not larger than n
A(z) = anz n + an−1z n−1 + · · · + a1z + a0 ,
B(z) = bnz n + bn−1z n−1 + · · · + b1z + b0 .
(10a)
(10b)
Find the coefficients of their product C(z) = A(z) B(z).
c2n =
c2n−1 =
c2n−2 =
...
c0 =
anbn
anbn−1 + an−1bn
anbn−2 + an−1bn−1 + an−2bn
(11a)
(11b)
(11c)
a0b0
(11d)
The sum of indices in each term on rhs equals to the index on lhs.
c 2009 P. F. Góra
Copyright °
3–26
We have
cl =
n
X
ãk b̃l−k ,
l = 0, 1, . . . , 2n ,
(12a)
k=0
where

a , b
s s
ãs, b̃s =
0
s = 0, 1, . . . , n
s = n + 1, n + 2, . . . , 2n
(12b)
and periodicity (sic!) of the “signals” is used to calculate the coefficients with negative indices: b̃−j = b̃2n−j+1 = 0 for j = 1, 2, . . . , n. The coefficients of the
product are given by the convolution of the coefficients of the input polynomials.
The numerical cost of calculating the coefficients of the product is O(2n log 2n),
not O(n2), as it would appear.
c 2009 P. F. Góra
Copyright °
3–27
Calculating the products of large integers
x =
y =
n
X
j=0
n
X
aj 2j ,
(13a)
bj 2j ,
(13b)
j=0
where aj , bj = {0, 1}. The rhs are polynomials of the form (10), evaluated in
z = 2. The product xy is also a polynomial, and its coefficients are given by the
appropriate convolution. For a large n, calculating the product via FFT becomes
efficient! In addition, calculating the FFT of a “signal” consisting of {0, 1} is
particularly simple.
c 2009 P. F. Góra
Copyright °
3–28