Basics Rank-nullity theorem: dim(R(A)) + dim(N (A∗ )) = m (since R(A) = Dual basis; Φ̃ = Φ(ΦΦ∗ )−1 = ΦG−1 , G the Gram

A)) + dim(N (A∗ )) = m (since R(A) = Dual basis; Φ̃ = Φ(ΦΦ∗ )−1 = ΦG−1 , G the Gram matrix.
Sylvester’s inequality: If A is an M × N matrix and B is an N × K matrix, N (A∗ )⊥ ), dim(R(A∗ )) + dim(N (A)) = n (since R(A∗ ) = N (A)⊥ ). ChangeD of basis:
E x, y ∈ H, y = Ax, x = Φα, y = Ψβ, β = Γβ.
then rank(A) + rank(B) − N ≤ rank(AB). Invertibility (equivalences): For A square matrix, N (A) = {0} (trivial Γi,j = Aφj , ψ̃i .
Cholesky decomposition: Any A hermitian PD matrix can be expressed nullspace), full-rank matrix, no 0 eigenvalue, determinant equal to 0.
as A = LL∗ , where where L is an invertible lower triangular matrix with Gram-Schmidt procedure: Want to convert original set {s(k) } onto or-
real and positive diagonal entries. Hilbert spaces and projection operators (k)
thonormal set {u }. At each step k, p (k)
=s(k) Pk−1 D (n) (k) E (n)
− n=0 u , s u .
Vandermonde matrix: A Vandermonde matrix V of size M × N has en- Vector space: Set of vectors (RN , functions,...), field of scalars (real, com-
tries of the form Vi,j = tji for i = 0, . . . , M − 1, j Q u(k) = p(k) / p(k) .

= 0, . . . , N − 1, and it is plex), vector addition, scalar multiplication. Satisfy x + y = y + x (com-
of full rank if tl 6= tm when l 6= m since det(V ) = 0≤l<m≤N −1 (tl − tm ). mutativity), (x + y) + z = x + (y + z) and (αβ)x = α(βx) (associativity),
Circulant matrix: Implement circular convolution. Its rows are circular α(x + y) = αx + αy and (α + β)x = αx + βx (distributivity), ∃0 s.t. x + 0 = x Discrete Systems
rotations of a sequence. They are diagonalized bu the DFT matrix. (additive identity), ∃ − x s.t. x + (−x) = 0 (additive inverse), 1x = x (mul- Delta function: −∞
R∞
δ(t)dt = 1, δ(t) = 0 ∀t 6= 0, −∞
R∞
x(t)δ(t)dt = x(0),
Toeplitz matrix; Constant coefficients along diagonals. Implements lin- tiplicative identity). R∞
x(τ )δ(t − τ )dτ = x(t). δ(at) = δ(t)/|a|.
ear convolution. In a matrix representation of a linear and shift-invariant Subspace: S ⊆ V , V vector space and also S itself. S 6= ∅, closed −∞
P
system, the matrix will be Toeplitz. under vector addition (x + y ∈ S ∀x, y ∈ S) and scalar multiplication Impulse
P response: Let yk (t) = A(xk(t)). If A linear (A( k αk xk (t)) =
Unitary matrix: U −1 = U ∗ and its eigenvalues satisfy |λj | = 1. Unitary (αx ∈ S ∀x ∈ S, α ∈ F, so {0} ∈ S). k α k yk (t)) shift-invariant (A(x k (t − τ )) = y k (t − τ )) system, can express
span(S): { N
P A as convolution with impulse response h(t) = A(δ(t)); A(x(t)) = (x ∗ h)(t).
matrices are isometries, i.e. kU xk = kxk. k=0 αk φk : αk ∈ F, φk ∈ S, N ∈ N \ {∞}}. Smallest vector
Normal matrices: AA∗ = A∗ A (so need to be square). space containing the set of vectors S. For S infinite, span(S) (o.w. some Filter as projection: A filter is said to be a projection if hn = (h ∗ h)n ,
Eigenvalue decomposition: Ax = λx (characteristic equation), kxk = 1, vectors cannot be represented with finite linear combiation of φk ). Always or equivalently H(ejω ) = H 2 (ejω ). In case of being a projection, orthogonal
λ ∈ F. λ verifies det(λI − A) = 0. Square matrix diagonalizable if ∃S n × n a subspace. projection if hn = h∗ −n , or equivalently H(e
jω
) = H ∗ (ejω ).
Inner product: hx + y, zi = hx, zi + hy, zi (distributivity), hαx, yi =
and Λ diagonal matrix s.t. A = SΛS −1 . For all square matrices, A = U RU ∗
α hx, yi (linearity in the first argument), hx, yi∗ = hy, xi (hermitian symme- Transforms
with R upper triangular matrix, U unitary (Schur decomposition). For nor-
try), hx, xi ≥ 0 equality iff x = 0 (PD).
mal matrices, R = Λ, with Λ diagonal. R∞ Time domain Frequency domain
Singular value decomposition: A = U SV ∗ , U, V unitary, S (e.g. for tall On CR , hx, yi = −∞ x(t)y ∗ (t)dt.
Fourier Transform Continuous aperiodic Continuous aperiodic
matrix) hermitian diagonal block and 0 block, U = [U1 U0 ]. This leads Norm: kxk ≥ 0 equality iff x = 0 (PD), kαxk = |α| kxk (positive scal-
to thin SVD A = U1 S1 V ∗ , U1 , U0 form orthonormal basis for R(A), N (A∗ ) ability), kx + yk ≤ kxk + kyk equality iff y = αx (triangle inequality). Fourier Series Continuous periodic Discrete aperiodic
resp. U, V are resp- eigenvectors of normal matrices AA∗ and A∗ A. A† = hx, yi = kxk kyk cos α.
DTFT Discrete aperiodic Continuous aperiodic
Pythagorean theorem: For {xk } orthogonal, k xk 2 = 2
P
V S † U ∗ (S † is S T with inverted singular values).
P
k kxk .
Determinant of a matrix: Oriented volume of the hyper-parallelepiped Cauchy-Schwarz: | hx, yi | ≤ kxk kyk. Equality when x = αy (collinear). DFT Discrete periodic Discrete periodic
defined by column vectors of A. |det(U )| = 1 for unitary matrices (e.g. Parallelogram law: kx + yk2 + kx − yk2 = 2(kxk2 + kyk2 ).
rotation and reflection). Linear operator: A(x + y) = Ax + Ay (additivity), A(αx) = α(Ax) (scal- Fourier Transform
Pn k r n0 −r n+1 ability). Operator norm kAk = supkxk=1 kAxk.
Geometric series: For r 6= 1,
R∞ R∞
k=n r = . x(t)e−jωt dt, x(t) = 1 jωt
0 1−r
Adjoint: For A : H0 → H1 , hAx, yiH = hx, A∗ yiH . Exists and is Definition: X(ω) = −∞ 2π −∞ X(ω)e .
† 1 0 R∞
Pseudo-inverse: The pseudo-inverse of A denoted A satisfies all the four Continuous-time convolution: (x ∗ y)(t) = x(τ )y(t − τ )dτ . Eigen-
statements: AA† A = A, A† AA† = A, (AA† )∗ = AA† and (A† A)∗ = A† A. unique. kA∗ k = kAk. Operator is unitary iff A−1 = A . If a matrix is
∗
−∞

If A has linearly independent columns (non-singular), A∗ A invertible and self-adjoint (hermitian), its eigenvectors define an orthogonal basis. If A functions ejωt .
invertible, (A−1 )∗ = (A∗ )−1 . Parseval: hx(t), y(t)i = 1
2π hX(ω), Y (ω)i.
A† = (A∗ A)−1 A∗ is a left-inverse. If A has linearly independent rows (A∗
Projection: Bounded linear operator (kP k < ∞; linear operators with Properties: x(αt) ↔ α 1
X ω


non-singular), AA∗ invertible and A† = A∗ (AA∗ )−1 is a right-inverse. α , (x ∗ y)(t) ↔ X(ω)Y (ω), x(t)y(t) ↔
finite-dimensional domains are always bounded) that is idempotent P 2 = P −jωτ
Building projections: AA† orthogonal projection into R(A). A† A orthog-
1
(X ∗ Y )(ω), x(t − τ ) ↔ e X(ω), ejω0 t x(t) ↔ X(ω − ω0 ), x(−t) ↔
(in such case kP k ≥ 1). If self-adjoint P ∗ = P , orthogonal projection and 2π
dn x(t) dn X(ω)
onal projection into R(A∗ ) (so orthogonal projection into N (A) is I − A† A). all eigenvalues are real valued (o.w. oblique).Bounded linear operator P sat- X(−ω), x∗ (t) ↔ X ∗ (−ω), dtn ↔ (jω)n X(ω), (−jt)n x(t) ↔ dω n ,
For U orthonormal set of vectors, U † = U ∗ . isfies hx − P x, P yi = 0∀x, y ∈ H iff P orthogonal projection. If S, T closed X(ω)
Rt
−∞
x(τ )dτ ↔ jω , X(0) = 0.
kAxk
Spectral norm: kAk := supx6=0 kxk = supkxk=1 kAxk = σmax (A) = subspaces s.t. H = S ⊕ T ∃projection P on H s.t. S = R(P ), T = N (P ). In P 1
P 2πm j 2πmt
p an orthogonal projection, all eigenvalues are either 0 or 1, and we have that Poisson sum formula: n∈Z x(t − nT ) = T m∈Z X( T )e
T .
∗
λmax (A A). Spectral norm is submultiplicative: kABk ≤ kAk kBk.
  kP xk ≤ kxk (orthogonal projection is a contraction). Moreover, if range Proof: LHS is T −periodic so compute Fourier Series and anti-transform re-
 ∂x ∂x  space is not trivial, kP k = 1. sult.
RR RR  ∂r ∂φ 
↔ ωπ 1{ ω ∈ [−ω0 , ω0 ]}.
Change of coordinates: f (x, y)dxdy = f (r, φ)  ∂y ∂y  drdφ. Projection theorem: For S closed subspace of Hilbert space H and x ∈ H, sin ω0 t
C P  ∂r ∂φ
Common transforms: sinc(ω0 t) = ω0 t 0

kx − x̂k ≤ kx − sk ∀s ∈ S iff x − x̂ ⊥ S, x̂ = P x, P orthogonal projection.
t 1{ t ∈ [−t0 /2, t0 /2]} ↔
2π 2kπ 1
P P
In case δ(t − nT ) ↔ δ(ω − ).
p of switching from cartesian
y

to polar, x = r cos(φ), y = r sin(φ), Basis:P Φ = {φk }k∈K ⊂ V , V = span(Φ) so any x ∈ V can be expressed as
n∈Z T k∈Z T 0
r = x2 + y 2 , φ = arctan x and determinant becomes r. x= t0 ω
1 − |t|, |t| < 1 ↔ sinc2 ( ω
k∈K αk φk and expansion
P
coefficients αk are unique. sinc( 2 ). 2 ). 1 ↔ 2πδ(ω). δ(t) ↔ 1.

Trigonometric identities Linear independence: k∈K αk φk = 0 iff αk = 0 ∀k.

Riesz basis: Φ basisP of Hilbert space H, ∃0 < λmin ≤ λmax < ∞ s.t. Fourier Series (signal of period T )
Even/odd: sin(−x) = − sin(x), cos(−x) = cos(x), tan(−x) = − tan(x). ∀x ∈ H, λmin kxk2 ≤ 2 2 ∗
k∈K |αk | ≤ λmax kxk . Let G = Φ Φ the Gram T 2πkt 2πkt
Cofunction: sin( π π
2 − x) = cos(x), cos( 2 − x) = sin(x).
1
x(t)e−j Xk ej
R 2
P
matrix. λmax (G) = 1/λmin , λmin (G) = 1/λmax . Definition: Xk = T dt, x(t) = k∈Z
T .
T −T
Sum and difference of angles: sin(x + y) = sin x cos y + cos x sin y, Orthogonal projection: αk = hx, φk i, i.e. α = Φ∗ x. This gives orthogonal 2
sin(x − y) = sin x cos y − cos x sin y, cos(x + y) = cos x cos y − sin x sin y, projection to span({φk }) (also for orthonormal set, not necessarily basis). Circular continuous-time convolution: For x, y T −periodic. (x~y)(t) =
cos(x − y) = cos x cos y + sin x sin y. Gram matrix: G = Φ∗ Φ, Gi,k = hφk , φi i. R T2 j 2πkt
T x(τ )y(t − τ )dτ . Eigenfunctions e
T .
Double angles: sin(2x) = 2 sin x cos x, cos(2x) = cos2 x − sin2 x = Biorthogonal pairs of bases: Φ, Φ̃ Dboth bases for H and biorthogonal: −
2
2 cos2 x − 1 = 1 − 2 sin2 x. D E E
Parseval: hx(t), y(t)i = T hXk , Yk i.
Product to sum: sin x sin y = 21 [cos(x − y) − cos(x + y)], cos x cos y = φi , φ̃k = δi−k . In this case αk = x, φ̃k , i.e. α = Φ̃∗ x. Residual of
2πkt0 2πk0 t
1 1
2 [cos(x − y) + cos(x + y)], sin x cos y = 2 [sin(x + y) + sin(x − y)]. projection on biorthogonal pairs of sets, satisfies x − P x ⊥ span({φ̃}k∈I ) Properties: x(t − t0 ) ↔ e−j T Xk , ej T x(t) ↔ X(k − k0 ),
Derivatives: dx d
sin(x) = cos(x), dx d
cos(x) = − sin(x). Φ̃ = ΦG−1 if Φ Riesz basis. (h ~ x)(t) ↔ T Hk Xk , h(t)x(t) ↔ (H ∗ X)k .
P
Parseval equality: For Φ orthonormal basis for H, hx, yi = hΦ∗ x, Φ∗ yi = Common transforms: n∈Z δ(t − nT ) ↔ 1/T .
Ranges, nullspaces and invertibility hα, βi since Φ∗ Φ = I on `2 (K), ΦΦ∗ = I on H (unitary operator). For
For matrix A of size m × n. biorthogonal pairs of bases, hx, yi = hα̃, βi since Φ̃∗ Φ = I on `2 (K),
DFTF
∗
Range: R(A) = {y : y = Ax} (linear subspace of Fm ). xn e−jωn , xn = 2π
Rπ
ΦP˜hi = I on H. Definition: X(ejω ) = 1
X(ejω )ejωn dω.
P
n∈Z −π
Nullspace: N (A) = {x : Ax = 0} (linear subspace of Fn ). Bessel’s inequality: For orthonormal sets, Φ∗ Φ = I but ΦΦ∗ 6= I in gen- P jωn
Discrete convolution: (x ∗ y)n = k∈Z xk yn−k . Eigenfunctions e .
Rank: Cardinality of largest set of linearly independent columns (or rows) eral. kxk2 ≥ kΦ∗ 2
I xk with equality when Φ is a basis.
of A; rank(A) = dim(R(A)) = dim(R(A∗ )) ≤ min(m, n) (if equality, full Inverse problem: Find x s.t. y = Ax. Solution x̂ is consistent with mea- Properties: Transform is 2π−periodic. (h ∗ x)n ↔ H(ejω )X(ejω ), hn xn ↔
rank). rank(A) = rank(A∗ ). surements if {x̂ : x̂ = x + x̃, x̃ ∈ N (A)}, i.e. Ax̃ = y. 1
2π (H ~ X)(e
jω
).
DFT an image f (x, y) taken at angle θ gives a slice of its two-dimensional Fourier Stationary process: Joint distribution of (xn0 , . . . , xnL ) and
Definition: Xk =
PN −1 kn
xn WN , xn =
PN −1 −kn
Xk W N kn
, WN = transform, F (u, v), that subtends an angle θ with the u-axis. P (θ, ω) = ((xn0 +k , . . . , xnL +k )) are identical ∀{n0 , . . . , nL } ⊂ Z, ∀k ∈ Z and L
n=0 k=0
2πkn F (u, v)|u=ω cos θ,v=ω sin θ = F (ω cos θ, ω sin θ), where P (θ, ω) is the 1D-FT finite.
e−j N . of p(θ, t) w.r.t t. WSS: µx,n = µx , ax,n,k = ax,k n, k ∈ Z. x and y jointly WSS if each is
Discrete circular convolution: For x, y sequences of length N . (x ~ y)n = Fourier Reconstruction Method: Take the 1D Fourier transform of each WSS and cx,y,n,k = cx,y,k . With WSS, we have σx,n 2
= ax,0 − |µx |2 = σx
2
,
j 2πkn
ax,k = a∗ ∗
PN −1 projection (w.r.t. t). Insert the results in the appropriate slices in the (u, v)-
k=0 xk yn−k . Eigenfunctions e
N . Same as discrete convolution if x x,−k . With joint WSS, cy,x,k = cy,x,−k .
has support M , y has support L and N ≥ L + M − 1. plane. Resample on a rectangular grid in the (u, v)-plane. Take the 2D IFT 2 2 2
White noise:µx,n = 0, σx,n = σx and ax,k = σx δk (uncorrelated). Gaus-
kn of the formed 2D spectrum.
Properties: x(n−n0 ) mod N ↔ WN 0 Xk , Wn−k0 n ↔ X(k−k0 ) mod N , sian rv’s uncorrelated iff independent, so white Gaussian process is i.i.d.
(h ~ x)n ↔ Hk Xk , hn xn ↔ N1
(H ~ X)k , x∗ ∗ ∗
n = X−k mod N , x−n mod N ↔ Filtered Backprojection (FBP) Independent vs. uncorrelated: If X, Y independent, E[XY ] = E[X] E[Y ],
Xk∗ . which implies that they are uncorrelated, i.e. E[(X − E[X])(Y − E[Y ])] = 0.
Gives images very close to the original. FBP does not work easily with con- Converse is only true for zero-mean random variables and jointly Gaussian
straints. With limited data and/or non-uniform distribution of projection
Sampling and Interpolation angles, reconstruction with FBP contains artifacts. In theory, f (x, y) =
random variables.
Whitening or decorrelation: Processing that results in white noise pro-
Interpolation Φ̃, sampling Φ̃∗ . S̃ = N (Φ̃∗ )⊥ = R(Φ̃) (what can be mea- B[q](x, y) where q(θ, t) = 2π 1
R∞ |ω|
P (θ, ω) 2π ejωt dω. To proof this, do cess. Diagonalization of covariance matrix.
−∞
sured). S = R(Φ) (what can be reproduced). P = ΦΦ̃∗ projection (with Filtering WSS processes: y = x ∗ h with h the impulse response of BIBO-
inverse 2D-IFT of F (u, v). Change Cartesian to polar coordinates with
range S and x − x̂ ⊥ S̃, where x̂ = P x) iff Φ̃∗ Φ = I (or equivalently ω ∈ R+ , θ ∈ [0, 2π) (Recall that dudv = ωdθdω). Split integral in θ from 0 to stable LSIP system (BIBO means L1 norm of h finite). µy,n = µx H(ej0 ) = µy ,
hg(t − kT ), g̃ ∗ (nT − T )it = δk−n ). In this case, we say that sampling and ay,n,k = p∈Z ah,p axk−p = ay,k so if x WSS, y WSS and x and y jointly
π for θ and θ +π. Note that F̃ (ω, θ +π) = F̃ (−ω, θ) since cos(θ +π) = − cos θ
interpolation are consistent (and x̂ is the best least-squares approximation
 † and sin(θ + π) = − sin θ. Apply Fourier Slice Theorem. WSS. Cx,y (ejω ) = H ∗ (ejω )Ax (ejω ) and Cy,w (ejω ) = H(ejω )Ax (ejω ).
of x in S). When Φ = Φ̃∗ = Φ̃(Φ̃∗ Φ̃)−1 (for orthogonal vectors, pseudo- Convolution backprojection: Can also write f (x, y) = Power spectral density: x WSS. DTFT of its autocorrelation (FT
−jωk
in case of continuous time). Ax (ejω ) = . Ay (ejω ) =
R∞ R ∞ |ω| jωt P
k∈Z ax,k e
inverse is the adjoint), they form a biorthogonal pair of bases for S and hence 1
−∞
p(θ, t)h(x cos θ + y sin θ − t)dtdθ, where h(t) = 2π −∞ π
e dω.
S = S̃ and we say that operators are ideally matched (orthogonal projection). / L2 (R), its FT doesn’t exist, so in practice H(ω) =
Given that |ω| ∈
|ω| |H(ejω )|2 Ax (ejω ).
2π W (ω) Rπ
Ax (ejω ) = ax,0 .
 † 1
To prove S = S̃ when Φ = Φ̃∗ , note that Φ is a linear combination of |ω| 2 Power: Px = 2π −π
for W a suitable window. IFT of ramp filter is sinc − sinc (rectangle -
2π ∗
triangle). Orthogonal Stochastic Processes: cx,y,k,n = E[xk yk−n ] = 0 ∀k, n ∈ Z.
columns of Φ̃ with coefficients given by the corresponding column of (Φ̃∗ Φ̃)−1
Reconstruct image from sinogram: Take 1D FT of projections (w.r.t. t) For jointly WSS processes, equivalent to Cx,y (ejω ) = 0 ∀ω ∈ R.
(full rank matrix). Hence, columns of Φ, Φ̃ must span the same space.
apply filter H(ω) and take 1D IFT (if h(t) short, do convolution in spatial Wiener filtering: x̂ = h ∗ y, h = arg minh E[|en := xn − x̂n |2 ]. Assume
domain). Backproject filtered projections and sum backprojected images. x, w uncorrelated, WSS, zero-mean. x̂ ∈ S := span({yn−k }k∈Z ), so best
x(t)
Φ̃∗ yn
Φ x̃(t) x̂(t) Cx,y (ejω )
g̃(t) Sampling T g(t) Algebraic Reconstruction estimator e ⊥ S. This gives H(ejω ) = .
Deltas at nT Ay (ejω )
PN −1
R∞  Suppose f (x, y) = i=0 fi φi (x, y), where φk are the basis functions. Beam Beamforming
x(τ )g ∗ (τ − t)dτ  shapes are hi . Hence, bi = hf , hi i. With spline surface model, b (the mea-
P
yn = ; x̃(t) = n∈Z yn δ(t − nT )
 PM −1 ∗
−∞ t=nT
R∞ surement) results from b = Af , where A is the measurement matrix or for- In narrowband beamforming, output is yn = k=0 hk xk,n , M
P ∗ number of array elements. Array response vector: a(θ) =
x̂(t) = k∈Z yk g(t − kT ) = −∞ x̃(τ )g(t − τ )dτ = ΦΦ̃ x = P x. ward operator with Ai,k = hφk , hi i of size M × N and f the unknown pixel
In the orthogonal case, Φ̃∗ = Φ∗ and g̃(t) = g ∗ (−t) weight [H0 (ω)e−jωτ0 (θ) · · · HM −1 (ω)e−jωτM −1 (θ) ]T , Hk the transfer function of
√ vector. Usually, number of non-zero coefficients in each row of A
Shift-invariant subspace: A subspace S ⊂ L2 (R) is a shift-invariant sub- is O( N ), so A sparse (so avoid computing pseudo-inverse, usually dense). the k−th sensor, τk (θ) the time needed for the wave to travel from reference
Can view Af as forward projection, AT b as backprojection. In presence point to sensor k. If sensors ideal (Hk (ω) = 1 ∀k) and sensor 0 taken as
space with respect to shift T ∈ R+ if x(t) ∈ S =⇒ x(t − kT ) ∈ S ∀k ∈ Z.
s ∈ L2 (R) is called a generator of S when S = span({s(t − kT )}k∈Z ). of noise system can be inconsistent: Usually f̂ = arg minf kb − Af k22 . If reference, a(θ) = [1 e−jωτ1 (θ) · · · (ω)e−jωτM −1 (θ) ]T . xn = a(θ)sn + en
To checkP for latter, have to show that ∀x ∈ S, ∃{αk }k∈Z unique s.t. rank(A) = N , f̂ = (AT A)−1 AT b unique. If rank(A) = M < N , take so- For multiple sources x = [a(θ0 ) · · · a(θN −1 )][s0 · · · sN −1 ]T + e = As + e.
x(t) = k∈Z αk s(t − kT ).
Uniform linear array: Equispaced sensors in same line. Under plane-wave
lution with minimum norm f̂ = AT (AAT )−1 b (both interpreted as FBP).
Subspace of bandlimited functions: A function x(t) ∈ L2 (R) has band- assumption, τk = k d sin c
θ
, θ ∈ [− π π
2 , 2 ) (front-back ambiguity), c velocity of
With this approach, handle constraints of scanning topology.
ω
width ω0 ∈ [0, ∞) if its FT satisfies X(ω) = 0 ∀|ω| > 20 . This defines wave propagation, d distance between consecutive sensors.
the space of ω0 − bandlimited functions, BL[−ω0 /2, ω0 /2] which is a shift- Kaczmarz’s algorithm Beamformer’s response: r(θ) = h∗ a(θ). Beampattern is |r(θ)|2 . If
invariant subspace (proof by delay property of FT). a(θ1 ) = a(θ2 ) and θ1 6= θ2 , spatial aliasing, i.e. spacing between sensors
Let {rT T
0 , . . . , rM −1 } the rows of A. Hence hf , ri i = bi , which define affine is too large.
Sampling theorem:
If x ∈ BL[−π/T, π/T ], x(t) =
P π hyperplanes in RN . If solution to Af = b exists and is unique, it’s in- Signal model: For narrowband source with DOA θ and power σs2 , if there’s
n∈Z x(nT )sinc T (t − nT ) . This means that sinc is a generator for
2π √1 sinc πt

tersection of M affine hyperplanes. Kaczmarz’s satisfies constraints itera- no noise xn = a(θ)sn , so Rx = σs2 a(θ)a∗ (θ).
T −band limited functions. In this case, choose g(t) = T = D E
T bn − f (k−1) ,rn
g ∗ (−t) since it s a generator with shit T of BL[−π/T, π/T ] and {g(t − kT )}k tively starting from initial guess f (−1) . f (k) = f (k−1) + rn Data independent beamforming
krn k2
2
are orthonormal. If x ∈ BL[−ω0 /2, ω0 /2], we need T < 2π/ω0 (Nyquist (k−1) Phased array (delay-and-sum) beamformer: h = a(θ0 ), where signal
interval). ω0 /2π is called the Nyequist rate. for n = k mod D M . ThatE is, apply orthogonal projection of f onto
comes from single location θ0 . Can control beam width of main lobe and
Continuous-time convoluton via DSP: For x ∈ BL[−π/T, π/T ], y = h∗x Hn = {f (−1) : f (k−1) , rn = bn }. If there’s a unique solution, found with height of side lobes with tapering function h = T a(θ0 ), with T diagonal
can be computed by sampling, filtering the
resulting sequence and interpo- k → ∞. Ordering of rows influence convergence rate (can order to increase M × M with tapering weights.
π
lating the result of the convolution. h̃n = h(t), sinc( T (t − nT )) without krn k2 Response design: To find h giving response similar to rd (θ), pose as
√ angles between consecutive rows or select randomly with e.g. pn = 2 ).
pre-filter (first filter in sampling is multiplying by T ). kAk2
F overdetermined problem arg minh kA∗ h − rd k22 , A = [a(θ0 ) · · · a(θP −1 )],
Can incorporate constraints such as box constraint (0 ≤ fi ≤ 1) by project- rd = [rd (θ0 ) · · · rd (θP −1 )]∗ . If A full rank h = (AA∗ )−1 Ard .
Computational Tomography ing each step onto this convex set. White noise gain: Output power due to white noise of unit power, i.e.
Parametrize line with angle θ ∈ [0, π) of the line’s normal vector and signed h∗ h. If this is high, beamformer output could have poor SNR. Control it by
distance t ∈ R from the origin. Cannot use y = mx + n since vertical lines Cimmino’s method low-rank approximation of A or solving previous problem with regularization
cannot be described. +λ khk22 , which gives h = (AA∗ + λI)−1 Ard .
Instead of updating one row at a time, update once per sweep with av-
Explicit: x, y parametrized by s ∈ R, where Lθ,t = {(x(s), y(s))}. x(s) =
erage of all projections. f (k) = f (k−1) + AT W −1 (b − Af (k−1) ) where
t cos θ − s sin θ, y(s) = t sin θ + s cos θ
W = diag(M kr0 k2 , . . . , M krM −1 k2 ).
Data dependent beamforming
Implicit: Lθ,t = {(x, y) : x cos θ + y sin θ = t}. Output of beamformer approximates desired yd . Usually pose as
Radon transform: R ∞ R ∞Gives sinogram. R[f ](θ, t) = p(θ, t) = −1
arg minh E[|y − yd |2 ], which gives h = Rx rx,d .
R
f (x, y)ds = −∞ f (x, y)δ(x cos θ + y sin θ − t)dxdy. Stochastic Processes
L
θ,t −∞ LCMV: Constrain so that signals from desired directions have specified gain
Laminogram or Backprojection Summation: Adjoint of Radon trans- Covariance Σx = E[(x − µx )(x − µx )∗ ], which is PSD since u∗ Σx u = with C ∗ h = f . Solve minh h∗ Rx h (minimize power at beamformer’s output)
PN −1
form. Assign every point in the image along Lθ,t E[u∗ (x − µx )(x − µx )∗ u] = E[|u∗ (x − µx )|2 ]. hx, yi = ∗
n=0 E[xn yn ]. Auto- subject to constraints. This gives h = Rx −1
C(C ∗ Rx
−1
C)−1 f .
R the value p(θ, t). Gives
blurred reconstruction. B[p](x, y) = fb (x, y) = 0π p(θ, x cos θ + y sin θ)dθ. correlation ax,n,k = E[xn x∗ n−k ], crosscorrelation c ∗
x,y,n,k = E[xn yn−k ]. For GSC: Transform LCMV to unconstrained. Decompose h = h0 − g,
Fourier Slice Theorem: The Fourier transform of a parallel projection of i.i.d. process ax,n,k = |µx |2 + σx 2
δk . h0 ∈ R(C), g ∈ N (C ∗ ). h0 = C(C ∗ C)−1 f , the min norm solution
satisfying constraints (data independent), g = Cn hn , where Cn basis of B-Splines
N (C ∗ ) (has no contribution of satisfaction of constraint bt allows to min-
Elementary B-Spline of degree 0 is β (0) (t) = 1 for t ∈ [−0.5, 0.5) (0 o.w.)
imize objective). Solve minhn (h0 − Cn hn )∗ Rx (h0 − Cn hn ), which gives
∗ and of degree K, β (K) = β (K−1) ∗ β (0) (supported on [− K+1
2 ,
K+1
2 )). Shifts
hn = (Cn Rx Cn )−1 Cn
∗
Rx h0 . The data-dependent beamformer vector has
(K)
nulls in the directions of the constraints, which is ensured by the signal of this are called B-splines of degree K. FT is B (ω) = sincK+1 (ω/2).
blocking matrix Cn . For even K, β (K) smooth on (z − 0.5, z + 0.5) and if odd on (z, z + 1) z ∈ Z.
(0)
Causal B-Splines: Causal elementary B-Spline of degree 0 is β+ (t) = 1
Approximation Theory (K)
for t ∈ [0, 1) (0 o.w.) and of degree K, β+ = β (K) (t − 0.5(K + 1)). β+
(K)

Polynomials (finite interval) is a generator of shift-invariant subspace SK,Z , in fact the one with shortest
support. span({β (K) (t − k)}k∈Z ) = SK,Z for odd K and SK,Z+0.5 for even
Approximate x(t) in finite interval [a, b] by polynomial of order K pk (t) =
PK k K.
k=0 ak t . Approximation error eK (t) = x(t) − pK (t), t ∈ [a, b]. Smooth
0 (K−1)
d
(t − k), α0k = αk − αk−1 .
P
Differentiation: dt x(t) = k∈Z αk β+
approximation. Can approximate continuous functions arbitrarily well over Rτ P (1) (K+1) (1) Pk
finite intervals (Weierstrass theorem). Polynomials are infinitely differen- Integration: −∞ x(τ )dτ = k∈Z αk β+ (t − k), αk = m=−∞ αm .
tiable. Approximating continuous functions with high degree polynomials (1)
Canonical Dual Spline Basis: For dual basis of {β+ (t − k)}k∈Z , need
tends to be problematic (e.g. at ending points of interval). Cannot approxi- D
(1) (1)
E
(1)
mate discontinuous functions or over infinite intervals well. β̃+ (t − i), β+ (t − k) = δi−k . For canonical dual, need β̃+ ∈ S1,Z .
D E
P (1) (1)
Least square minimization x̂(t) = k∈Z x(t), β̃+ (t − k) β+ (t − k). Dual spline of degree 1 has
Rb infinite support but decays exponentially.
Minimize keK k22 = a
|x(t) − pK (t)|2 dt. Since PK ([a, b]) =
K 2 Polynomial reproduction (Strang-Fix theorem)
span({1, t, . . . , t }) ⊂ L ([a, b]), solution (by projection theorem) is pK (t) = R∞
(1 + |t|K )|φ(t)|dt < ∞ for k ∈ N and φ function with FT Φ, following
PK K If −∞
k=0 hx, φk i φk (t), {φk }k=0 orthonormal basis of PK ([a, b]). These inner
products with basis functions are not always easy to obtain (e.g. if we only are equivalent:
P
have samples). Gibbs phenomenon: No matter how large K is, absolute (i) pK (t) = k∈Z αk φ(t − k) for pK a polynomial of degree ≤ K.
error stays the same at the ripple near to the boundary. (ii) Φ and its first K derivatives satisfy Φ(0) 6= 0 and Φ(k) (2φl) = 0 for
Legendre polynomials: Orthonormal basis of PK ([−1, 1]). The Legendre k = 1, . . . , K, l ∈ Z \ {0}.
dn 2 n
P
polynomial of degree n, Ln (t) = 2n1n! dt n (t − 1) , has n real distinct zeros Partition of unity (case of Strang-Fix): φ1 (t) = n∈Z φ(t − n) = 1
in the interior of the interval [−1, 1] 1
(periodized version with period 1 of φ ∈ L (R)) iff Φ(2φk) = δk k ∈ Z.

Lagrange interpolation Series Truncation P

Given orthonormal expansion in infinite Hilbert space x = k∈Z ck φk ,
Observe function x(t) at points (nodes) t0 , . . . , tK . Constrain pK (ti ) =
x(ti )∀i. Leads to Vandermonde system, invertible iff {ti } distinct with so- ck = hx, φk i. Cannot store all coefficients. Linear approximation: Retain
PK t−t coefficients with a priori fixed set of indices. Don’t depend on x. Linear. Non
i = 0, i 6= kK t −ti .
Q
lution pK (t) = k=0 x(tk ) k i optimal in error. Nonlinear approximation: Retain M largest coefficients
Error: For x(t) ∈ CK + 1([a, b]) and {ti } distinct, |eK (t)| ≤ in absolute value. Depend on x. Non linear. Store index of coefficients.
QK
k=0 |t−tk | (K+1) Optimal in error. To proof latter
select indices in IM and apply Parseval:
maxη∈[a,b] |x (η)|. So error increases at the boundaries. P
(K+1)!
kx − x̂k2 = m∈Z\I cm φm = |cm |2 .
P
Last term means error higher for wigglier functions. M m∈Z\I M

Taylor series expansion

Uncertainty Principles
R∞ |x(t)|2 R∞ |x(t)|2 R∞ |X(ω)|2
µt := t dt, ∆2t := (t−µt )2 dt. µf := ω dω,
Assume x(t) ∈ C K ([a, b]) and find degree K polynomial with matching −∞ kxk2 −∞ kxk2 −∞ 2πkxk2
(t−t0 )k (k)
R∞ |X(ω)|2
derivatives at t0 ∈ [a, b]. Solution pK (t) =
PK ∆2f := (ω − µt )2 dω.
k=0 x(tk ) k! x (t0 ). −∞ 2πkxk2
|t−t0 |K+1
Error: For x(t) ∈ CK + 1([a, b]), |eK (t)| ≤ (K+1)!
maxη∈[a,b] |x(K+1) (η)|. Heisenberg principle
For x ∈ L2 (R) ∆2t ∆2f ≥ 14 with equality for Gaussian functions x(t) =
Minimax approximation 2
γe−αt , α > 0. To proof, suppose w.l.o.g. µt = µf = 0, use Cauchy-
Minimize keK k∞ = maxt∈[a,b] |eK (t)|. Non trivial since not Hilbert space. Schwarz, Parseval, integration by parts with (|x(t)|2 ) and limt→∞ tx2 (t) = 0
With polynomial of degree ≤ K, minimax approximation pK unique and (since x ∈ L2 (R), i.e. decays faster than 1/t). √
determined by at least K + 2 points a ≤ s0 < s1 < · · · < sK+1 ≤ b for Shifts and
√ scalings: Shifting changes µ in domain of shift. ax(at) ↔
which eK (sk ) = ±1(−1)k keK k∞ (Chebyshev equioscillation theorem). So X(ω/a)/ a gives µt /a, ∆t /a, aµf , a∆f .
expect to reach maximum error in K + 2 points with alternating sign. Nearly
solve by taking nodes minimizing maximum error for Lagrange interpolation. Heisenberg principle for infinite sequences
These optimal K + 1 nodes given by roots of K + 1−degree Chebyshev poly- Same definitions with discrete sum in time and DTFT (so integrals in
nomial (its scaled version 2−K TK+1 have
 the minimum `∞ norm among all [−π, π]). For x ∈ `2 (Z) and X(ejπ ) = 0 (necessary extra condition),
∆2n ∆2f > 14 and lower bound cannot be achieved.

2k+1
K + 1−degree polynomials): tk = cos 2(K+1)
π .
Shifts and scalings: Shifting in frequency gives µf + ω0 if sig-
Splines nal still X(ejπ ) = 0 (”signal not splitted”). Upsample and
postfilter gives N µn , N ∆n , µf /N, ∆f /N . Prefilter and downsample
Can approximate discontinuous functions well and over the entire real line. µn /N, ∆n /N, N µf , N ∆f if x ∈ BL[−π/N, π/N ].
Spline of degree K with knots τ (countable strictly-increasing sequence)
is a polynomial of degree ≤ K on [τn , τn+1 ) and its derivatives of order Uncertainty principle for finite-length sequences
0, . . . , K − 1 are continuous. SK,τ ⊂ L2 (R) is the spline space of degree xn ∈ CN and Xk its DFT with Nn and Nk nonzero coefficients respectively.
K with knots τ . When τ evenly spaced and doubly infinite, spline space Nn Nk ≥ N . So cannot be sparse in both domains.
called uniform. For spline of degree K with L + 1 knots, K − 1 degrees of
freedom. In practice for K = 3, specifies derivatives at end-points or make
sure 3rd derivative continuous at second and penultimate knots (not-a-knot
condition).