Face Recognition Using PCA (Eigenfaces) and LDA (Fisherfaces)

Face Recognition
using
PCA (Eigenfaces) and LDA (Fisherfaces)
Slides adapted from Pradeep Buddharaju
Principal Component Analysis
A N x N pixel image of a face,

represented as a vector occupies a
single point in N2-dimensional image
space.
Images of faces being similar in overall

configuration, will not be randomly
distributed in this huge image space.
Therefore, they can be described by a

low dimensional subspace.
Main idea of PCA for faces:
To find vectors that best account for

variation of face images in entire
image space.
These vectors are called eigen
vectors.
Construct a face space and project the
images into this face space
(eigenfaces).
Image Representation
Training set of m images of size N*N are

represented by vectors of size N2
x1,x2,x3,,xM
Example
1 2 3
3 1 2
4 5 1
1
2

3
33
3
1
2
4
5
1
91
Average Image and Difference Images
The average training set is defined by

= (1/m) mi=1 xi
Each face differs from the average by vector

ri = xi
Covariance Matrix
The covariance matrix is constructed as

C = AAT where A=[r1,,rm]
Size of this matrix is N2 x N2
Finding eigenvectors of N2 x N2 matrix is intractable. Hence, use the

matrix ATA of size m x m and find eigenvectors of this small matrix.
Eigenvalues and Eigenvectors - Definition

If v is a nonzero vector and is a number such that
Av = v, then
v is said to be an eigenvector of A with eigenvalue .
Example
(eigenvalues)
2 1 1
1
1 2 1 3 1
v (eigenvectors)
Eigenvectors of Covariance Matrix
The eigenvectors vi of ATA are:
Consider the eigenvectors vi of ATA such that

ATAvi = ivi
Premultiplying both sides by A, we have

AAT(Avi) = i(Avi)
Face Space
The eigenvectors of covariance matrix are

ui = Avi
Face Space
ui resemble facial images which look ghostly, hence called Eigenfaces
Projection into Face Space

A face image can be projected into this face space by
pk = UT(xk ) where k=1,,m
Recognition
The test image x is projected into the face space to
obtain a vector p:
p = UT(x )
The distance of p to each face class is defined by
k2 = ||p-pk||2; k = 1,,m
A distance threshold c, is half the largest distance
between any two face images:
c = maxj,k {||pj-pk||}; j,k = 1,,m
Recognition
Find the distance between the original image x and its
reconstructed image from the eigenface space, xf,
2 = || x xf ||2 , where xf = U * x +
Recognition process:
IF c
then input image is not a face image;
IF <c AND kc for all k
then input image contains an unknown face;
IF <c AND k*=mink{ k} < c
then input image contains the face of individual k*
Limitations of Eigenfaces Approach
Variations in lighting conditions

Different lighting conditions for
enrolment and query.
Bright light causing image saturation.
Differences in pose Head orientation

- 2D feature distances appear to distort.
Expression
- Change in feature location and shape.
Linear Discriminant Analysis
PCA does not use class information
PCA projections are optimal for reconstruction from

a low dimensional basis, they may not be optimal
from a discrimination standpoint.
LDA is an enhancement to PCA
Constructs a discriminant subspace that minimizes

the scatter between images of same class and
maximizes the scatter between different class
images
Mean Images
Let X1, X2,, Xc be the face classes in the database and let
each face class Xi, i = 1,2,,c has k facial images xj, j=1,2,
,k.
We compute the mean image i of each class Xi as:
1 k
i x j
k j 1
Now, the mean image of all the classes in the database can
be calculated as:
1 c
i
c i 1
Scatter Matrices
We calculate within-class scatter matrix as:

c
SW
T
(
x
)
(
x
)
k i k i
i 1 xk X i
We calculate the between-class scatter matrix as:
S B N i ( i )( i ) T
i 1
Multiple Discriminant Analysis
We find the projection directions as the matrix W that maximizes
|W T SBW |
W argmax J(W )
|W T SW W |
^
This is a generalized Eigenvalue problem where the

columns of W are given by the vectors wi that solve
SB w i i SW w i
Fisherface Projection
We find the product of SW -1 and SB and then compute the Eigenvectors

of this product (SW -1 SB) - AFTER REDUCING THE DIMENSION OF
THE FEATURE SPACE.
Use same technique as Eigenfaces approach to reduce the

dimensionality of scatter matrix to compute eigenvectors.
Form a matrix W that represents all eigenvectors of SW -1 SB by placing

each eigenvector wi as a column in W.
Each face image xj Xi can be projected into this face space by the
operation
pi = WT(xj )
Testing
Same as Eigenfaces Approach
References
Turk, M., Pentland, A.: Eigenfaces for recognition. J. Cognitive

Neuroscience 3 (1991) 7186.
Belhumeur, P.,Hespanha, J., Kriegman, D.: Eigenfaces vs. Fisherfaces:

recognition using class specific linear projection. IEEE Transactions on
Pattern Analysis and Machine Intelligence 19 (1997) 711720.

Face Recognition Using PCA (Eigenfaces) and LDA (Fisherfaces)

Uploaded by

Copyright:

Available Formats

Face Recognition Using PCA (Eigenfaces) and LDA (Fisherfaces)

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Face Recognition Using PCA (Eigenfaces) and LDA (Fisherfaces)

Uploaded by

Copyright:

Available Formats

Face Recognition

Slides adapted from Pradeep Buddharaju

Principal Component Analysis

A N x N pixel image of a face,

Images of faces being similar in overall

Therefore, they can be described by a

Main idea of PCA for faces:

To find vectors that best account for

Training set of m images of size N*N are

Average Image and Difference Images

The average training set is defined by

Each face differs from the average by vector

The covariance matrix is constructed as

Finding eigenvectors of N2 x N2 matrix is intractable. Hence, use the

Eigenvalues and Eigenvectors - Definition

Eigenvectors of Covariance Matrix

The eigenvectors vi of ATA are:

Consider the eigenvectors vi of ATA such that

Premultiplying both sides by A, we have

The eigenvectors of covariance matrix are

ui resemble facial images which look ghostly, hence called Eigenfaces

Projection into Face Space

Limitations of Eigenfaces Approach

Variations in lighting conditions

Differences in pose Head orientation

Linear Discriminant Analysis

PCA does not use class information

PCA projections are optimal for reconstruction from

LDA is an enhancement to PCA

Constructs a discriminant subspace that minimizes

We compute the mean image i of each class Xi as:

We calculate within-class scatter matrix as:

We calculate the between-class scatter matrix as:

Multiple Discriminant Analysis

We find the projection directions as the matrix W that maximizes

This is a generalized Eigenvalue problem where the

We find the product of SW -1 and SB and then compute the Eigenvectors

Use same technique as Eigenfaces approach to reduce the

Form a matrix W that represents all eigenvectors of SW -1 SB by placing

Same as Eigenfaces Approach

Turk, M., Pentland, A.: Eigenfaces for recognition. J. Cognitive

Belhumeur, P.,Hespanha, J., Kriegman, D.: Eigenfaces vs. Fisherfaces:

You might also like