PCA, Eigenfaces, and Face Detection: CSC320: Introduction To Visual Computing Michael Guerzhoy

PCA, Eigenfaces, and Face Detection
Salvador Dalí, “Galatea of the Spheres”
Many slides from

CSC320: Introduction to Visual Computing
Noah Snavely, Derek Hoeim, Robert Collins
Michael Guerzhoy
What makes face detection hard?
*Face detection: given an image, find the coordinates of the faces
Variation in appearance: can’t match a single

face template and expect it to work.
(Note: it doesn’t work that great for eyes either)
slide by Derek Hoiem

Lighting

Occlusion
What makes face recognition hard?
Viewpoint

Face detection
• Do these images contain faces? Where?

Simple Idea for Face Detection
1. Treat each window in the image like a vector
x
2. Test whether x matches some in the database

2
SSD: 𝑦𝑗 − 𝑥
Cross-correlation: 𝑦𝑗 ⋅ 𝑥
NCC, zero-mean NCC…

The space of all face images
• When viewed as vectors of pixel values, face images are
extremely high-dimensional
– 100x100 image = 10,000 dimensions
– Slow and lots of storage
• But very few 10,000-dimensional vectors are valid face
images
• We want to effectively model the subspace of face images

The space of all face images
• Eigenface idea: construct a low-dimensional linear
subspace that contains most of the face images
possible (possibly with small errors)
• Here: a 1D subspace arguably suffices

The space of faces
= +
An image is a point in a high dimensional space

• An W x H intensity image is a point in RWH
• We can define vectors in this space as we did in the 2D case
Reconstruction
• For a subspace with the orthonormal basis of
size k Vk = 𝑣0 , 𝑣1 , 𝑣2 , … 𝑣𝑘 , the best
reconstruction of x in that subspace is:
𝑥𝑘 = 𝑥 ⋅ 𝑣0 𝑣0 + 𝑥 ⋅ 𝑣1 𝑣1 + ⋯ + 𝑥 ⋅ 𝑣𝑘 𝑣𝑘
– If x is in the span of 𝑉𝑘 , this is an exact
reconstruction
– If not, this is the projection of x on V
2
• Squared reconstruction error: 𝑥𝑘 − 𝑥
Reconstruction cont’d
• 𝑥𝑘 = 𝑥 ⋅ 𝑣0 𝑣0 + 𝑥 ⋅ 𝑣1 𝑣1 + ⋯ + 𝑥 ⋅ 𝑣𝑘 𝑣𝑘
• Note: in 𝑥 ⋅ 𝑣0 𝑣0 ,
– 𝑥 ⋅ 𝑣0 is a measure of how similar x is to 𝑣0
– The more similar x is to 𝑣0 , the larger the
contribution from 𝑣0 is to the sum
Representation and reconstruction
• Face x in “face space” coordinates:
=
• Reconstruction:
= +
^
x = µ + w1u1+w2u2+w3u3+w4u4+ …

Reconstruction
P=4
P = 200
P = 400
After computing eigenfaces using 400 face

images from ORL face database
Principal Component Analysis
• Suppose the columns of a matrix 𝑋𝑁×𝐾 are the datapoints (N
is the size of each image, K is the size of the dataset), and we
would like to obtain an orthonormal basis of size k that
produces the smallest sum of squared reconstruction errors
for all the columns of 𝑋 − 𝑋
– 𝑋 is the average column of X
• Answer: the basis we are looking for is the k
eigenvectors of 𝑋 − 𝑋 𝑋 − 𝑋 𝑇 that
correspond to the k largest eigenvalues
PCA – cont’d
• 𝑋 − 𝑋 𝑋 − 𝑋 𝑇 is called the covariance matrix
• If x is the datapoint (obtained after subtracting
the mean), and V an orthonormal basis, 𝑉 𝑇 𝑥 is a
column of the dot products of x and the elements
of x
• So the reconstruction for the centered x is
𝑥 = 𝑉(𝑉 𝑇 𝑥))
• PCA is the procedure of obtaining the k
eigenvectors 𝑉𝑘
NOTE: centering
• If the image x is not centred (i.e., 𝑋 was not
subtracted), the reconstruction is:
𝑥 = 𝑋 + 𝑉(𝑉 𝑇 (𝑥 − 𝑋))
Proof that PCA produces the best reconstruction
• (Fairly easy calculus – look it up, or we can talk in office hours, or possibly
we’ll do it next week)
Obtaining the Principal Components
• 𝑋𝑋 𝑇 can be huge
• There are tricks to still compute the EVs
PCA as dimensionality reduction
The set of faces is a “subspace” of the set of images

• Suppose it is K dimensional
• We can find the best subspace using PCA
• This is like fitting a “hyper-plane” to the set of faces
– spanned by vectors v1, v2, ..., vK
– any face
Eigenfaces example
Top eigenvectors: u1,…uk
Mean: μ

Another Eigenface set
Linear subspaces
convert x into v1, v2 coordinates
What does the v2 coordinate measure?

- distance to line
- use it for classification—near 0 for orange pts
What does the v1 coordinate measure?
- position along line
- use it to specify which orange point it is
Dimensionality reduction
How to find v1 and v2 ?
Dimensionality reduction
• We can represent the orange points with only their v1 coordinates
– since v2 coordinates are all essentially 0
• This makes it much cheaper to store and compare points
• A bigger deal for higher dimensional problems
Another Interpretation of PCA
The eigenvectors of the covariance matrix define a new
coordinate system
• eigenvector with largest eigenvalue captures the most
variation among training vectors x
• eigenvector with smallest eigenvalue has least variation
• The eigenvectors are known as principal components
Data Compression using PCA
• For each data point x, store 𝑉𝑘𝑇 𝑥 (a k-dimensional
vector). The reconstruction error would be the
smallest for a set of k numbers
Face Detection using PCA
• For each (centered) window x and for a set of
principal components V, compute the Euclidean
distance 𝑉𝑉 𝑇 𝑥 − 𝑥
• That is the distance between the reconstruction of x
and x. The reconstruction of x is similar to x if x lies in
the face subspace
• Note: the reconstruction is always in the face subspace
Issues: dimensionality
What if your space isn’t flat?
• PCA may not help
Nonlinear methods
LLE, MDS, etc.
Moving forward
• Faces are pretty well-behaved
– Mostly the same basic shape
– Lie close to a low-dimensional subspace
• Not all objects are as nice

Different appearance, similar parts

PCA, Eigenfaces, and Face Detection: CSC320: Introduction To Visual Computing Michael Guerzhoy

Uploaded by

Copyright:

Available Formats

PCA, Eigenfaces, and Face Detection: CSC320: Introduction To Visual Computing Michael Guerzhoy

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

PCA, Eigenfaces, and Face Detection: CSC320: Introduction To Visual Computing Michael Guerzhoy

Uploaded by

Copyright:

Available Formats

PCA, Eigenfaces, and Face Detection

Salvador Dalí, “Galatea of the Spheres”

Many slides from

Variation in appearance: can’t match a single

slide by Derek Hoiem

slide by Derek Hoiem

slide by Derek Hoiem

• Do these images contain faces? Where?

2. Test whether x matches some in the database

slide by Derek Hoiem

slide by Derek Hoiem

• Here: a 1D subspace arguably suffices

An image is a point in a high dimensional space

• Face x in “face space” coordinates:

slide by Derek Hoiem

After computing eigenfaces using 400 face

The set of faces is a “subspace” of the set of images

slide by Derek Hoiem

What does the v2 coordinate measure?

How to find v1 and v2 ?

• Not all objects are as nice

You might also like