International Journal of Computer Sciences and Engineering Open

Access
Research Paper Vol.-6, Issue-10, Oct 2018 E-ISSN: 2347-2693

Modified Technique of Three Dimensional Face Recognition in the

Presences of Facial Expression

Kshipra Soni1*, Dr. R.K. Kapoor2

1
M.Tech Scholar, Department of Computer Engineering and Application, National Institute of Technical Teachers Training &
Research, RGPV, Bhopal, India
2
Associate Professor, Department of Computer Engineering and Application, , National Institute of Technical Teachers
Training & Research, RGPV, Bhopal, India
E-mail: kshiprasony@gmail.com, rkkapoor@nitttr.bpl.ac.in
*
Corresponding Author: kshiprasony@gmail.com, Tel.: 8305753566

Available online at: www.ijcseonline.org

Received: ………./2018, Revised: ………..2018, Accepted: ………2018, Published: 30/Oct/2018

Abstract— Face recognition has acquired abundant attention in market and research communities, but still remained very
accosting in real time applications. It is one of the various techniques used for identifying an individual. The major factors
affecting the face recognition system are pose, illumination, identity, occlusion and expression. The image variations due to the
change in face identity are less than the variations among the images of the same face under different illumination, expression,
occlusion and viewing angle. Among the several factors that influence face recognition, illumination and pose are the two
major challenges. Next to pose and illumination, the major factors that affect the performance of face recognition are occlusion
and expression. So in order to overcome these issues, we proposed an efficient 3d face recognition system based on partial
occlusion and expression. The similar blocks in the face image are identified. Then the occlusion can be recovered using the
block matching technique. Finally, the face can be recognized by using the PCA. From the implementation result, it is proved
that the proposed method recognizes the face images effectively.
Keywords—Facerecognition,OcclusionDetection,Expression,BlockmatchingAlgorithm,PrincipalComponentAnalysis(PCA)

© 2018, IJCSE All Rights Reserved 1

 I. INTRODUCTION Aleix M. MartõÂnez [11] has described a probabilistic
approach that was able to compensate for imprecisely
Among all the recent biometric techniques, face recognition
localized, partially occluded, and expression-variant faces
systems have received the most attention due to the non-
even at one single training sample per class was available to
intrusion nature of the person involved in recognition. It is
the system. To determine the occlusion problem, each face
more convenient to users when compared to other individual
was divided into k local regions which are analysed in
identification methods of biometric features. It involves
isolation. In contrast with other approaches where a simple
pattern recognition, image processing, intelligent learning and
voting space is used, they presented a probabilistic method
so on [1]. Face recognition, a kind of biometric identification,
that analyses how good a local match was.
researched in several fields such as computer vision, image
Kepenekci et al [12] have proposed an approach to feature
processing, and pattern recognition is a natural and direct
based frontal face recognition using Gabor wavelets. The
biometric method [2]. Automated methods that use facial
features were automatically extracted using the local
features as essential elements of distinction to determine
characteristics of each individual face in order to decrease the
identity are involved in the process of facial recognition. The
effect of occluded features. There were no training as in neural
recognition performance thoroughly degrades with pose and
network approaches, thus single frontal face for each
lighting variations [3]. Face recognition involves
individual was enough as reference.
categorization of extremely confusing multi-dimensional input
Dahua Lin and Xiaoou Tang [13] have proposed a method to
signals, and matching them with the known „signals‟ and it is
detect and recover the occluded facial region automatically. It
an extremely complicated type of pattern recognition.
frees the user from marking the occlusion area by
Information security, law enforcement and surveillance, smart
incorporating an automatic occlusion detector, that learns a
cards, access controls are some of the areas of applications for
face quality model as a criterion to guide the whole procedure
face recognition [4], [5], [6].
and it couples the detection and occlusion stages to achieve
A face is basically a 3D object amidst random background
accurate occlusion detection and high quality recovery
objects lighted by diverse lighting sources from several
simultaneously.
directions. Because of this, when a face is projected against a
Kazuhiro Hotta [14] has presented the use of Support Vector
2D image its appearance varies remarkably [7]. Major changes
Machine (SVM) with local Gaussian summation kernel for
in 2D appearance are also caused by diverse pose angles.
robust face recognition under partial occlusion. Because
Capability of figuring out identity in spite of such changes in
conventional methods apply one kernel to global features and
appearance that a face can have in a setting is needed for
global features were influenced easily by noise or occlusion,
robust face recognition. At the same time problems like noise,
the conventional methods are not robust to occlusion. The
camera distortion or image resolution should not affect the
recognition method based on local features, was robust to
working of the system. The variation caused by factors such as
occlusion because partial occlusion affects only specific local
occlusion, illumination, expression, pose, accessories and
features. In order to utilize their property of local features in
aging produces a performance drop in all the face recognition
SVM, local kernels were applied to local features. The
algorithms [8].
effectiveness and robustness of their method are shown by
comparison with global kernel based SVM.
II. RELATED WORK
Hyun Jun Oh et al. [15] have proposed a novel occlusion
Illumination, Occlusion, Expression and pose are the main invariant face recognition algorithm based on Selective Local
confronts to face recognition out of the diverse factors that Nonnegative Matrix Factorization (S-LNMF) technique. The
affect 3d face recognition. In that, occlusion and expression algorithm was composed of two phases; the occlusion
variation considerably reduces the performance of recognition. detection phase and the selective LNMF-based recognition
So to tackle this problem, the face with different occlusions phase. They used local approach to effectively detect partial
and expression is recognized under by our proposed system. occlusion in the input face image.
Few expressions based and occluded face recognition As a result of systematic literature review, numbers of open
approaches have been proposed earlier.
chances have been discussed. But identification of more
De Marsico et al. [9] proposed face recognition for occlusions specific parameters and area to be worked requires even more
and expression variations. It is based on the concept of
mysterious critical analysis which has been listed as gaps
partitioned iterated function systems (PIFS) which compute a found during the survey. In accordance with scope and
map of self-similarities within the whole input face image and
significance discussed in the earlier chapter, the actual and
search for the relation among small square regions. Algorithms specific problems have been stated in this section. This tends
of this kind suffer from local distortions such as occlusions. To
to be the base of this dissertation work. Following problems
overcome this, information extracted by PIFS is made local by have been identified:
working independently on each face component like eyes,
nose, and mouth. Distortions are further reduced by means of  There are two limitations in Two Dimensional Face
an ad hoc distance measure. Recognition are namely Translation and Rotation because
Tarrés et al [10] have presented a face recognition method that of this 3D Face Recognition is required.
could be able to manage with partial occlusion or variations in  LDA is unable to provide accurate results in terms of
face expression. Their method solves the face recognition Features classification that’s why k-NN is required.
problem from a near holistic perspective. The concept behind
this approach was to eliminate the few features that cause
reduction in recognition accuracy under variations in occlusion
or expression.
 III. METHODOLOGY One would like the correspondence to be such that geometric
features match across the curves as well as possible. In other
This paper presents a Riemannian framework for 3D facial
words, the lips should match the lips and the chin should
shape analysis. This framework is based on elastically
match the chin. The points A and B on β 1 will not match the
matching and comparing radial curves emanating from the tip
points A’ and B’ on β2 as they are not placed at the same
of the nose and it handles several of the problems described
distances along the curves. For curves, the problem of optimal
above. The main contributions of this paper are as follows:
registration is actually the same as that of optimal re-
 It extracts, analyses, and compares the shapes of radial
parameterization. This means that we need to find a re-
curves of facial surfaces. parameterization function γ(t) such that the point β 1(t) is
 It develops an elastic shape analysis of 3D faces by registered with the point β2(γ(t)), for all t.
extending the elastic shape analysis of curves to 3D facial As described in [16], this registration is accomplished by
surfaces. solving an optimizing problem using the dynamic
 To handle occlusions, it introduces an occlusion detection programming algorithm, but with an objective function that is
developed from a Riemannian metric. The chosen metric,
and removal step that is based on recursive-ICP.
termed an elastic metric, has a special property that the same
 To handle the missing data, it introduces a restoration step re-parameterization of two curves does not change the distance
that uses statistical estimation on shape manifolds of between them. This in turn enables us to fix the
curves. Specifically, it uses PCA on tangent spaces of the parameterization of one curve arbitrarily and to optimize over
shape manifold to model the normal curves and uses that the parameterization of the other. This optimization leads to a
model to complete the partially observed curves. proper distance (geodesic distance) and an optimal
deformation (geodesic) between the shapes of curves. In other
words, it results in their elastic comparisons.
A. Radial,Elastic Curves:Motivation
An important contribution of this paper is its novel use of C. Proposed Algorithm
radial facial curves studied using elastic shape analysis. Input: Facial surface s1 and s2.
The changes in facial expressions affect different regions of a Output: The distance ds.
facial surface differently. For example, during a smile, the top for i←1to2do
half of the face is relatively unchanged while the lip area for α←0to2∏do
changes a lot, and when a person is surprised the effect is often Extract the curve βαi;
the opposite. If chosen appropriately, curves have the potential if quality (βα1) =1 and quality (βα2) =1 then
to capture regional shapes and that is why their role becomes Compute the optimal rotational and re-parameterization
important. The locality of shapes represented by facial curves alignment Ơα* and γα*.
is an important reason for their selection. set qα2*(t) =√γα*(t)Ơα*qα2(γα*(t)))
The past usage of the level curves (of the surface distance compute
function) has the limitation that each curve goes through ds ([q1α],[q2α]) =cos-1(<q1α,q2α>)
different facial regions and that makes it difficult to isolate end
local variability. The previous work on shape analysis of facial end
curves for 3D face recognition was mostly based on level compute ds =1/n ΣαɛA ds (q1α, q2*α), Where n is the
curves [17], [18]. number of valid pairs of curves.
end.
B. Motivation for Elasticity
D. 3D Face Recognition on The Bosphorus Dataset:
Consider the two parameterized curves shown in Fig. 1; call
Recognition Under External Occlusion
them β1 and β2. Our task is to automatically match points on
In this section, we will use components I (occlusion
these radial curves associated with two different facial
detection and removal) and II (missing data restoration) in the
expressions. The expression on the left has the mouth open,
algorithm. The first problem we encounter in externally
whereas the expression on the right has the mouth closed. To
occluded faces is the detection of the external object parts. We
compare their shapes, we need to register points across those
accomplish this by comparing the given scan with a template
curves.
scan, where a template scan is developed using an average of
training scans that are complete, frontal, and have neutral
expressions. The basic matching procedure between a template
and a given scan is recursive ICP, which is implemented as
follows:
In each iteration, we match the current face scan with the
template using ICP and remove those points on the scan that
are more than a certain threshold away from the corresponding
points on the template. This threshold has been determined
using experimentation and is fixed for all faces. In each
iteration, additional points that are considered extraneous are
Fig.1. an example of matching radial curves extracted from two faces
belonging to the same person: a curve with an open mouth (on the left) and a incrementally removed and the alignment (with the template)
curve with a closed mouth (on the right). One needs a combination of based on the remaining points is further refined.
stretching and shrinking to match similar points (upper lips, lower lips, etc.).
 edges, linking them together into loops, and then
triangulating the resulting loops.
 A cropping filter cuts and returns part of the mesh inside a
Euclidean sphere of radius 75 mm cantered at the nose tip
to discard as much hair as possible.

Fig.2. Gradual removal of occluding parts in a face scans using recursive-

ICP.
Fig. 2 shows an example of this implementation. From left to
right, each face shows an increasing alignment of the test face
with the template, with the aligned parts shown in magenta,
and also an increasing set of points labelled as extraneous,
drawn in pink. The final result, the original scan minus the
extraneous parts, is shown in green at the end.
In the case of faces with external occlusion, we first restore
them and then apply the recognition procedure. That is, we
detect and remove the occluded part and recover the missing
part, resulting in a full face that can be compared with a Fig.4. the different steps of preprocessing: acquisition, filling holes, cropping,
gallery face using the metric dS. The recovery is performed and smoothing.
using the tangent PCA analysis and Gaussian models. To  A smoothing filter reduces high-frequency components
evaluate our approach, we perform this automatic procedure
(spikes) in the mesh, improves the shapes of cells, and
on the Bosphorus database. The Bosphorus database is suitable
for this evaluation as it contains scans of 60 men and 45 evenly distributes the vertices on a facial mesh.
women, 105 subjects in total, in various poses, expressions,
and in the presence of external occlusions (eyeglasses, hand, The proposed novel face recognition approach is implemented
and hair). For each subject there are four scans with occluded in NetBeans and face recognition was performed using the set
parts. These occlusions refer to- of Bosphorus Database, Yale Database and JAFFE database
1. Mouth occlusion by hand, under various expression and occlusion conditions. The results
2. Eyeglasses, show that proposed approach has an encouraging performance.
3. Occlusion of the face with hair, and B. Databases
4. Occlusion of the left eye and forehead regions by hands.
 The Japanese Female Facial Expression (JAFFE)
Database
The database contains 213 images of 7 facial expressions
posed by 10 Japanese female models. Each image has
been rated on 6 emotion adjectives by 60 Japanese
subjects [18].
Interpret the findings in view of the results obtained in
this and in past studies on this topic. State the conclusions
Fig.3. Examples of faces from the Bosphorus database. The un-occluded face
on the left and the different types of occlusions are illustrated. in a few sentences at the end of the paper. However, valid
colored photographs can also be published.
IV. RESULTS AND DISCUSSION
In the following section, a comparative performance analysis
of proposed method with other state-of-the-art solutions has
been provided using the Bosphorus dataset.
A. Data Preprocessing
Since the raw data contains a number of imperfections, such as
holes, spikes, and includes some undesired parts, such as
clothes, neck, ears, and hair, the data pre-processing step is Fig.5. Block division process in JAFFE Database images
very important and nontrivial. As illustrated in Fig. 5, this step
includes the following items:  Yale Database
 The hole-filling filter identifies and fills holes in input The database contains 165 GIF images of 15 subjects.
meshes. The holes are created either because of the There are 11 images per subject, one for each of the
following facial expressions or configurations: centre-
absorption of the laser in dark areas, such as eyebrows
light, w/glasses, happy, left-light, w/no glasses, normal,
and must matches, or self-occlusion or open mouths. They right-light, sad, sleepy, surprised, and wink [18].
are identified in the input mesh by locating boundary
 equation is used to calculate the accuracy measurement of the
overall approach,

Fig.6. Block division process in Yale Database images Genuine acceptance rate (GAR) is an overall accuracy
measurement of the approach. The following table gives the
In our proposed method, we used Bosphorus databases which percentage of the recognition rates and the accuracy rates.
contain the 3face images with various pose, illumination,
occlusion and expression.
TABLE.I. comparisons results of proposed hybridized
technique with other existing methods

Databases
V. CONCLUSION AND FUTURE SCOPE Yale Database Jaffe Database Bosphorus
Database
In this paper a framework for a statistical shape analysis of Method FR FA Acc FRR FA Acc FR FA Acc
facial surfaces have been presented. We have also presented s R R urac (%) R urac R R urac
results on 3D face recognition designed to handle variations of (% (% y (%) y (% (% y
) ) (%) (%) ) ) (%)
facial expression, pose variations, and occlusions between
gallery and probe scans. This method has several properties Face
that make it appropriate for 3D face recognition in non- recogni
tion 4.1 5.4 96. 4.75 5.9 95.8 1.0 1.1 99.4
cooperative scenarios. First, to handle pose variation and based 8 8 3 2 8 4 7
missing data, we have proposed a local representation by using on
a curve representation of a 3D face and a quality filter for Occlusi
selecting curves. Second, to handle variations in facial on
Face
expressions, we have proposed an elastic shape analysis of 3D recogni
faces. tion 5.3 6.7 95. 6.02 7.1 94.3 1.0 1.1 99.4
Finally, in the presence of occlusion, it is proposed to remove based 7 2 4 0 7 1
the occluded parts, then to recover only the missing data on the on
Express
3D scan using statistical shape models. That is, we have ion
constructed a low dimensional shape subspace for each element Propos
of the indexed collection of curves, and then represent a curve ed 3.6 4.8 98. 4.32 5.1 97.7 1.0 1.3 99.9
(with missing data) as a linear combination of its basis Techni 4 4 4 2 5 2 9
que
elements. The testing of proposed approach was carried out in a
simulated environment using NetBeans environment. NetBeans
provides an interactive environment with hundreds of reliable
and accurate built-in mathematical functions. Recognition
results demonstrate that it effectively minimize FAR and FRR.
Finally Elastic Distance Computation algorithm improves the
accuracy of the face recognition system. The research
proposals made out of this paper have opened several
challenging research directions, which can be further
investigated. The recognition rate using the proposed Elastic
Distance Computation algorithm approach applies to future
work. Also in the future it can be combined with other methods
and thus even improve the robustness and accuracy. Next, we
need to improve the performance of classifier and the face
potential area selection method. Future works can also include Fig.7. Comparison results of proposed hybridized technique with other
experiment this method on other 3D face databases. existing methods

A. False Acceptance Rate, False Rejection Rate

VI. COMPARATIVE ANALYSIS
In this table, we calculate the measures for Bosphorus
The recognition accuracy of the proposed approach is Database, Jaffe and Yale database. The result shows that our
compared with some existing approaches. Computing the false proposed method has a lower value of FAR and FRR error rate
acceptance rate (FAR) and false rejection rate (FRR) is the in both the databases. Moreover, the proposed system has a
common way to measure the biometric recognition accuracy. higher accuracy compared to the other two methods.
FAR is the percentage of incorrect acceptances i.e., percentage Bosphorus Database gives the better result when compared
of distance measures of different people’s images that fall with the Jaffe database and Yale database because it is a
below the threshold. FRR is the percentage of incorrect standard database when compared to the Jaffe database and
rejections - i.e., percentage of distance measures of same also the Bosphorus database contains more expression images
people’s images that exceed the threshold. The following than Jaffe database and Yale. Normally expression image is
more difficult to recognize than occluded image. So [18] C. Samir, A. Srivastava, M. Daoudi, and E. Klassen, “An Intrinsic
Framework for Analysis of Facial Surfaces,” Int’l J. Computer Vision,
Bosphorus database gives better accuracy in face recognition vol. 82, no. 1, pp. 80-95, 2009.
system. It is evident that the proposed face recognition system
efficiently recognizes the face under various expression and
occlusion.

ACKNOWLEDGMENT

We Thanks the anonymous references for their useful

suggestions. This Research work is useful in many security
purposes.

REFERENCES
[1] Hongzhou Zhang, Yongping Li, Lin Wang and Chengbo Wang, "Pose
insensitive Face Recognition Using Feature Transformation", IJCSNS
International Journal of Computer Science and Network Security, VOL.7
No.2, February 2007.
[2] Anil K. Jain, Arun Ross, and Salil Prabhakar, “An Introduction To
Biometric Recognition”, IEEE Transactions On Circuits And Systems
For Video Technology, Vol. 14, No. 1, Jan. 2004.
[3] Zhiwei Zhu and Qiang Ji, "Robust Pose Invariant Facial Feature Detection
and Tracking in Real-Time", In proc. of the 18th International
Conference on Pattern Recognition, pp.1092-1095, 2006.
[4] Shang-Hung Lin, "An Introduction to Face Recognition Technology",
Informing Science Special Issue on Multimedia Informing Technologies,
Vol.3, No. 1, 2000.
[5] Gregory Shakhnarovich and Baback Moghaddam, "Face Recognition in
Subspaces", Springer, Heidelberg, May 2004.
[6] Zhao, W. Chellappa, R., Phillips, P. J. and Rosenfeld, A., “Face
Recognition: A Literature Survey”, ACM Computing Survey, pp. 399-
458, Dec 2003.
[7] N.V.S.L. Swarupa and D.Supriya, "Face Recognition
System" ,International Journal of Computer Applications, Vol. 1,No.
29,pp.36-41,2010.
[8] Hazim Kemal Ekenel and Bulent Sankur, "Multiresolution face
recognition", Image and Vision Computing, Vol.23, pp.469-477, 2005.
[9] De Marsico, M. Nappi, M. Riccio, D., “Face Recognition Against
Occlusions and Expression Variations”Systems, Man and Cybernetics,
IEEE Transactions, Vol.40, Issue 1, pp. 121-132, 2010
[10] F. Tarrés, A. Rama, L. Torres, "A Novel Method for Face Recognition
under Partial Occlusion or Facial Expression Variations", In proc. of the
47th International Symposium (ELMAR-2005) on Multimedia Systems
and Applications, Zadar, Croatia, June 2005.
[11] Aleix M. MartõÂnez, "Recognizing Imprecisely Localized, Partially
Occluded, and Expression Variant Faces from a Single Sample per
Class", IEEE Transaction on Pattern Analysis and Machine Intelligence,
Vol.24, No.6, pp.748-763, 2002.
[12] B. Kepenekci, F. B. Tek, and G. B. Akar, "Occluded face recognition by
using gabor features", In proc. of the 3rd COST 276 Workshop on
Information and Knowledge Management for Integrated Media
Communication, Budapest, pp. 1-6,Oct. 2002.
[13] Dahua Lin and Xiaoou Tang, "Quality-Driven Face Occlusion Detection
and Recovery", In Proc. of the IEEE Conference on Computer Vision
and Pattern Recognition, pp.1-7, 2007.
[14] Kazuhiro Hotta, "Robust face recognition under partial occlusion based
on support vector machine with local Gaussian summation kernel",
Image and Vision Computing, Vol.26, No. 11, pp.1490–1498, 2008.
[15] Hyun Jun Oh, Kyoung Mu Lee and Sang Uk Lee, "Occlusion invariant
face recognition using selective local non-negative matrix factorization
basis images", Computer Vision, Vol.3851, pp.120-129, 2009.
[16] A. Srivastava, E. Klassen, S.H.S. Joshi, and I.H.I. Jermyn, “Shape
Analysis of Elastic Curves in Euclidean Spaces,” IEEE Trans. Pattern
Analysis and Machine Intelligence, vol. 33, no. 7, pp. 1415-1428, July
2011.
[17] C. Samir, A. Srivastava, and M. Daoudi, “Three-Dimensional Face
Recognition Using Shapes of Facial Curves,” IEEE Trans. Pattern
Analysis and Machine Intelligence, vol. 28, no. 11, pp. 1858-1863, Nov.
2006.
Authors Profile
Mss. Kshipra Soni in pursed Bachelor of Enggineering Sciences
from RGPV Bhopal, in year 2012. She is completed her master
degree from NITTTR, RGPV Bhopal, in 2017. She is currently
doing Guest Lecturer in Government enggineering college, sagar
(M.P.) She has published 2 research papers in reputed international
journals.

Dr. Ravi Kant Kapoor He is Associate Professor in National

Institute of Technical Teachers Training & Research, Bhopal. He is
completed his Ph.D from MANIT, Bhopal in year 2012. He has
published more than 10 research papers in reputed international
journals. He has 17 years of Teaching experience and Research too.

© 2015, IJCSE All Rights Reserved 7