DIGITAL IMAGE PROCESSING-

FACE RECOGNITION
 If you have found this useful please mail me your name,contact
number,your feed back
Urs

Page-2
Abstract: comparison between face images. In these
approaches, the overall face detection, facial
As one of the most successful feature localization, and face comparison is
applications of image analysis and carried out in a single step.
understanding, face recognition has recently The paper states the generic framework
received significant attention, and has an for the face recognition system, and the
important advantage over other biometric variants that are frequently encountered by
technologies as it is a non -intrusive and easy the face recognizer. Several face recognition
to use method. algorithms, will also be explained. It also
One of the fastest growing areas of throws some light on its important
advanced security involves biometric face applications areas .It concludes with the
recognition technologies. The art of picking a current state of the art and some backdrops.
face out of a crowd is a time honored skill.
Applying technology to such a pursuit has to
date proven both fruitful and frustrating.
Biometric face recognition technology offers
great promise in its ability to identify a single
face, from multiple lookout points,
from a sea of hundreds of thousands of other
faces. In addition to serving as a information
access control tool, biometric face
recognition technologies are being used to
safeguard international borders, financial
ATM transactions, prevent benefits and
identity fraud, and help combat terrorism.

The main idea and the driver of

further research in this area are security
applications and human-computer interaction.
Face recognition represents an intuitive and
method of recognizing people and this is why
it became one of three identification methods
used in e-passports and a biometric of choice
for many other security applications.
However, until the problems (illumination,
pose, aging , occlusions) are solved, it is
unrealistic to expect that the full deployment
potential of face recognition systems will be
realized. There are also many technological
issues to be solved, some of which have been
addressed in recent ANSI and ISO standards.

Face recognition is one of the few

biometric methods that possess the merits of
both high accuracy and low intrusiveness. It
has the accuracy of a physiological approach
without being intrusive.
In the general framework of face
recognition, a probe still-image is first
detected, analyzed and then matched against
a collection of images -- the database of the
people known by the system. As a result,
most of the work carried out in face
recognition by computer is limited to the
Contents :

 Introduction
 Organization of a Biometric Face
Recognizer
 Face detection
 Face detection algorithms
 Face Recognition
 View Based (Photometric) Face
Recognition Techniques
 Feature Based (Geometric) Face
Recognition Techniques
 Hybrid Approaches
 Application Areas
 Problems and Considerations
 Conclusions and Future
Developments
 References

Page-4
Introduction:

Definition: A face recognition system

recognizes an individual by matching the
input image against images of all users in a fig1: CCTV Cameras
database and finding the best match.
2. FACE DETECTION AND FEATURE
Face recognition has received significant EXTRACTION: The acquired face images
attention in the last 15 years, due to the are first scanned to detect the presence of
increasing number of commercial and law faces and find their exact location and size.
enforcement applications requiring reliable The output of face detection is an image
personal authentication (e.g. access control, window containing only the face area.
surveillance of people in public places, Irrelevant information, such as background,
security of transactions, mug shot matching, hair, neck and shoulders, ears, etc are
and human-computer interaction) and the discarded. The resulting face image is then
availability of low-cost recording devices. further processed to extract a set of salient or
discriminatory, local or global features,
Comparison with Other Biometrics: Despite which will be used by the face classifier to
the fact that there are more reliable biometric identify or verify the identity of an unknown
recognition techniques such as fingerprint face. Such features maybe the measurements
and iris recognition, these techniques are of local facial features (such as eyes, nose,
intrusive and their success depends highly on mouth, etc) characteristics or global features
user cooperation, since the user must position such as transformation coefficients of global
her eye in front of the iris scanner or put her image decomposition (PCA, LDA, wavelets,
finger in the fingerprint device. On the other etc). These features constitute the template or
hand, face recognition is non-intrusive since signature uniquely associated with the image.
it is based on images recorded by a distant
camera, and can be very effective even if the 3 CLASSIFICATION: In which the
user is not aware of the existence of the face template extracted during step 2 is
recognition system. The human face is compared against the stored templates
undoubtedly the most common characteristic in the database to generate matching
used by humans to recognize other people scores, which reveal how identical the
and this is why personal identification based faces in the probe and gallery images
on facial images is considered the friendliest are. Then, a decision-making module
among all biometrics. either confirms (verification) or
establishes (identification) the user’s
identity based on the matching score
Organization of a 4 SYSTEM DATABASE: It is used to
Biometric Face extract and store the templates of
enrolled users. This module is also
Recognizer: responsible for enrolling users in the
face recognition system database.
Generally consists of 4 parts: During the enrolment of an
individual, the sensor module records
1. SENSOR: Hardware that captures images of her face. These images are
face images of an individual. called gallery images and they are
Depending on the sensor modality, used for training the classifier that
the acquisition device maybe a black will perform face recognition. Most
and white or color camera, a 3D commonly, several frontal neutral
sensor capturing range (depth) data, views of an individual are recorded,
or an infrared camera capturing but often face images depicting
infrared images. different facial expressions (neutral,
smile, laugh, anger, etc) and presence

Page-5
 (or non-) of glasses are also acquired. faces in it (if any) and locate their exact
Sometimes gallery images are positions and size. Usually, face detection is a
recorded in more than one session. two-step procedure: first the whole image is
The time interval between different examined to find regions that are identified as
sessions may result in variations due “face”. After the rough position and size of a
to hairstyle, beard, make-up, etc being face are estimated, a localization procedure
present in gallery images. The follows which provides a more accurate
presence of such variations ensures a estimation of the exact position and scale of
more robust face recognition the face. So while face detection is most
performance. Given a user’s set of concerned with roughly finding all the faces
acquired images, a set of features is in large, complex images, which include
extracted similarly to step 3 above, many faces and much clutter, localization
and a template that provides a emphasizes spatial accuracy, usually
compact and expressive achieved by accurate detection of facial
representation of the user based on features.
her images is generated called
training. The training algorithm
depends on the face recognition
method employed by the face
recognition system. The aim of the
training is to encode the most
discriminative characteristics of a user
based on the classifier chosen, and to
determine the values of the different
thresholds.
: : fig2 :
FACE DETECTION

Face detection algorithms

These can be divided into four categories
according to:

1. KNOWLEDGE-BASED
METHODS are based on human
knowledge of the typical human face
geometry and facial features
arrangement. Taking advantage of
natural face symmetry and the natural
top-to-bottom and left-to-right order
in which features appear in the human
face, these methods find rules to
describe the shape, size, texture and
other characteristics of facial features
(such as eyes, nose, chin, eyebrows)
Face Detection: and relationships between them
(relative positions and distances).
Face detection is the first stage of
an automatic face recognition system, since a The main issue in such techniques
face has to be located in the input image is to find a successful way to translate
before it is recognized. A definition of face human knowledge about face geometry
detection could be: given an image, detect all into meaningful and well-defined rules.

Page-6
Another problem of such techniques is 4. APPEARANCE-BASED
that they do not work very well under METHODS. These methods use
varying pose or head orientations. large numbers of examples (images of
faces and \ or facial features)
2. FEATURE INVARIANT depicting different variations (face
APPROACHES aim to find shape, skin color, eye color,
structural features that exist even open\closed mouth, etc). Face
when the viewpoint or lighting detection can be viewed as a pattern
conditions vary and then use these to classification problem with two
locate faces. Different structural classes: “face” and “non-face”. The
features are being used: facial local “non-face” class contains images that
features, texture, and shape and skin may depict anything that is not a face,
color. Local features such as eyes, while the “face” class contains all
eyebrows, nose, and mouth are face images. Statistical analysis and
extracted using multi-resolution or machine learning techniques are
derivative filters, edge detectors, employed to discover the statistical
morphological operations or properties or probability distribution
thresholding. Statistical models are function of the pixel brightness
then built to describe their patterns of images belonging in the
relationships and verify the existence two classes. To detect a face in an
of a face .Skin color is another input image, the whole image is
powerful cue for detection, because scanned and image regions are
color scene segmentation is identified as “face” or “non face”
computationally fast. based on these probability functions.
Well-known appearance-based
Usually, they use features such methods used for face detection are
as texture, shape and skin color to find eigenfaces , LDA , neural networks,
face candidates and then use local facial support vector machines and hidden
features such as eyes, nose and mouth to Markov models.
verify the existence of a face. Feature
invariant approaches can be problematic
if image features are severely corrupted
Face Recognition
or deformed due to illumination, noise, Techniques
and occlusion.
Face recognition techniques can be
3. TEMPLATE-BASED METHODS. : roughly divided into two main categories:
To detect a face in a new image, first global approaches and feature based
the head outline, which is fairly techniques. In global approaches the whole
consistently roughly elliptical, is image serves as a feature vector, while in
detected using filters, edge detectors, local feature approaches a number of fiducial
or silhouettes. Then the contours of or control points are extracted and used for
local facial features are extracted in classification.
the same way, exploiting knowledge
of face and feature geometry. Finally,
the correlation between features
extracted from the input image and
predefined stored templates of face
and facial features is computed to
determine whether there is face
present in the image.

Page-7
 subset of these eigenfaces and the weights of
View Based this representation are used for recognition.
(Photometric) The identification of a test image is done by
locating the image in the database, whose
Approaches for Face weights are the closest to the weights of the
test image. The concept of eigenfaces can be
Recognition extended to eigenfeatures, such as eigeneyes,
eigenmouth, etc.
Also called as Global approaches or Holistic
approaches, model the variability of the face
by analyzing its statistical properties based on
a large set of training images. Representative Fig 3: Eigen
global techniques are eigenfaces, Linear faces
Discriminant Analysis (LDA), Support
Vector Machines (SVM) and neural
networks.

Current View Based (Photometric)

Approaches
Principal-component analysis (PCA)based

 EIGENFACES: Direct application of

PCA 2. PROBABILISTIC EIGENFACES:
 PROBABILISTIC EIGENFACES: Using a probabilistic measure of similarity
Two-class problem with prob. instead of the Euclidean distance between
measure weights, the eigenface approach was
 FISHER FACES/SUBSPACE LDA: extended to a Bayesian approach based on
FLD on eigenspace image differences Face recognition is viewed
 SVM: Two-class problem based on as a two-class classification problem. The
SVM(Support Vector Machines) first class contains intensity differences
 EVOLUTION PURSUIT: Enhanced between images of the same individual
GA learning (depicting variations in expression,
 FEATURE LINES: Point-to-line illumination, head orientation, use of
distance based cosmetics, etc) and represents the
 ICA: Independent Component intrapersonal facial variations. The second
Analysis based feature analysis class contains intensity differences between
images belonging to different people and
Other representations represents the extra personal facial variations
 LDA/FLD: LDA/FLD on raw image due to differences in identity. The distribution
 PDBNN: Probabilistic decision based probabilities of the two excluding classes are
Neural Network estimated using a large training set. The MAP
(Maximum a Posteriori) rule is used for face
1. EIGEN FACES : The first really recognition.
successful face recognition method (and a
reference point in face recognition literature) 3. LINEAR/FISHER DISCRIMINANT
is a holistic approach based on principal ANALYSIS (LDA): Face recognition
component analysis (PCA) applied on a set of techniques using Linear/Fisher Discriminant
images in order to extract a set of Eigen- Analysis (LDA) were also developed. LDA
images, known as eigenfaces. Every face is determines a subspace in which the between-
modeled as a linear combination of a small class scatter (extra personal variability) is as

Page-8
large as possible, while the within-class approximated using observations extracted by
scatter (intrapersonal variability) is kept scanning training images from left-to-right
constant. In this sense, the subspace obtained and top-to-bottom. To verify a face, first the
by LDA optimally discriminates the classes- observations are extracted from the input
faces. A combination of PCA and LDA was image and then their probability given the
also proposed. Other global techniques stored EHM model is calculated.
include Support Vector Machines (SVM) and
neural networks (NN).

Feature Based
(Geometrical) Face FICUDIAL IMAGE POINTS

Recognition Techniques
Main idea - discriminate among different
faces based on measurements of structural
attributes of the face. Most recent approaches
are Embedded Hidden Markov Models
(EHMMs), the Elastic Graph Matching and
Dynamic Link Architecture.

Current Feature-based methods

1. Pure Geometry
2. Dynamic link architecture: Graph
matching methods Fig 5: ELASTIC GRAPH MATCHING
3. Hidden Markov model (HMM)
methods 2. ELASTIC BUNCH GRAPH
4. Convolution Neural Network SOM MATCHING : It is based on the
learning based CNN methods Dynamic Link Architecture (DLA). The basic
idea of EGM is the representation of the face
1. EMBEDDED HIDDEN MARKOV using a set of local image features extracted
MODEL (HMM): For frontal views the from the intensity images over fiducial image
significant facial features appear in a natural points and the exploitation of their spatial
order from top to bottom (forehead, eyes, coherence using a connected graph. Each
nose, and mouth) and from left to right (e.g. node in the graph is assigned a set of Gabor
left eye, right eye). EHMMs model the face wavelet coefficients, over different scales and
as a sequence of states roughly corresponding orientations, extracted from the image
to facial features regions. The probability function. The graph is adapted to each face in
distribution functions of EHMM states are the face database by the minimization of a
cost function that locally.
Page-9
 images due to pose, illumination conditions,
facial expressions, use of cosmetics, different
hairstyle, presence of glasses, poses,
Hybrid Approaches beard,(occlusions), background Lighting etc.
Images of the same individual taken at
Approaches that use both global and local different times, may sometimes exhibit more
features have also been proposed. For variability due to the aforementioned factors
example, the modular eigenspace approach (intrapersonal variability), than images of
uses both eigenfaces and eigenfeatures, while different individuals due to gender, race, age
the Local Feature Analysis (LFA) extracts and individual variations (extra personal
topographic local features from the global variability)
PCA modes and uses them for recognition.
Another crucial parameter in face
Current Hybrid methods recognition is aging. A robust recognition
 Modular eigenfaces using Eigenfaces system should be able to recognize an
and eigenmodules Hybrid LFA Local individual even after some years, especially
feature method in mug-shot matching forensic applications.
 Shape-normalized Flexible This is a very challenging task, which has not
appearance models been successfully addressed yet.
 Component-based Face region and
components Recent public facial recognition
benchmarks have shown that in general, the
identification performance decreases linearly
Application Areas in the logarithm of number of people in the
gallery database.
1. ENTERTAINMENT: Video Games,
Virtual Reality, Training Programs, Human Also, in a demographic point of
Robot Interactions. view, it was found that the recognition rates
for males were higher than for females, and
2. SMART CARDS: e Passports, Drivers that the recognition rates for older people
Licenses, Entitlement Programs, national ID, were higher than for younger people.
Voter Registration, Welfare Funds
These tests also revealed that while
3. INFORMATION SECURITY: the best recognition techniques were
PERSONAL DEVICE Login, Desktop successful on large face databases recorded in
Login, Database Security, File Encryption, well-controlled environments, their
Intranet Security, Application Security, performance was seriously deteriorated in
Medical records, Secure Business Trading uncontrolled environments, mainly due to
Terminals variations in illumination and head rotations.
Such variations have proven to be one of the
4. LAW ENFORCEMENT AND biggest problems of face recognition systems.
SURVEILLANCE: Advanced Video
Surveillance, Portal control, Post-event
control, Suspect tracking and Investigation,
Conclusions and Future
CCTV control Developments
Problems and The problem of machine face
recognition has been an ongoing subject of
Considerations research for more than 20 years. Although a
large number of approaches have been
. The main problem of face proposed in the literature and have been
recognition is large variability of the recorded implemented successfully for real-world
applications, robust face recognition is still a
Page-10
challenging subject, mainly because of large
facial variability, pose variations and
uncontrolled environmental conditions. The
use of novel sensors, such as 3D, can help
overcome limitations due to viewpoint and
lighting variations. On the other hand, it has
been acknowledged that there is no perfect
biometric and thus the combination of
different modalities, e.g. face combined with
speaker, fingerprint and/or hand recognition,
is required to achieve the desired level of
performance.

The computer based face recognition

has made much useful advancement in the
past decade however the need for high
accurate systems remains. Through the
determination of industry, government
evaluations and organized standard bodies,
growth and progress will continue, raising the
bar for face recognition.

References:
1. Face Recognition: A
Literature Survey by (W.
ZHAO, R. CHELLAPPAA.
ROSENFELD)
2. Recent Advances in Face
Recognition by (KRESIMIR
DELAC , MISLAV GRGIC
And Marian Stewart Bartlett)
3. www.face-rec.org
4. Wikipedia

Page-11