Face Detection
FACE DETECTION IN IMAGES
A project report submitted in partial fulfilment of the requirements
for the award of the degree of
Bachelor of Technology
In
Department Of Computer Science And Engineering
By
Akash Roy Choudhury-)
Rithik Banerjee-)
DEPARTMENT OF INFORMATION TECHNOLOGY
JALPAIGURI GOVERNMENT ENGINEERING
COLLEGE
JALPAIGURI, WEST BENGAL-735102.
FEBRUARY 2017
BONAFIDE CERTIFICATE
This is to certify that the project titled FACE DETECTION IN IMAGES
is a bonafide record of the work done by
Akash Roy Choudhury-)
Rithik Banerjee-)
in partial fulfilment of the requirements for the award of the degree of
Bachelor of Technology in Department of Computer Science and
Engineering of the JALPAIGURI GOVERNMENT ENGINEERING
COLLEGE during the year-.
NAME
Guide
DR. DIPAK KUMAR KOLE
Head Of Department
Project Viva-voce held on _____________________________
Internal Examiner
External Examiner
i
ABSTRACT
The given problem falls under the domain of IMAGE PROCESSING.
Face recognition is an important application of Image processing owing to
it's use in many fields. The project presented here was developed after study
of various face recognition methods and their efficiencies. An effective and
real time face recognition system based on OpenCV and Python is
developed in the project. The system was tested on YALE Face database
B.
ii
ACKNOWLEDGEMENT
I deeply express my sincere thanks to my Head of Department Dr. Dipak
Kumar Kole , for encouraging and allowing us to present the project on the
topic “Face Detection in Images” using Python. I would like to take this
opportunity to thank all the lecturers who have directly or indirectly helped
this project. I pay my respects and love to I parents and all other family
members for their love and encouragement throughout. Last but not the
least I would like to express my gratitude to my friends for their
cooperation and support.
iii
TABLE OF CONTENTS
Title
Page No.
Abstract………………………..………………………………….ii
Acknowledgement……….……………………………………….iii
Table of Contents………………………………………………...iv
List of Figures……..……………………………………………...vi
CHAPTER 1
Introduction
1.1
Project Objective..…………………………………………1
1.2
General Introduction…………………….……….………..1
CHAPTER 2
Litrature Review
2.1
Face Detection Approaches ………….…………….……..2
2.2
Face Recognition Approaches ...………………………….3
CHAPTER 3
Methodology
3.1
Language Used…………………………………………….4
3.2
Project Design……………………………………...……...4
3.3
Detabase……….……………………………………...…...4
3.4
Implementation..…………………………………………..5
CHAPTER 4
Results and Discussion
4.1
Experimental Result……………………………….….…..8
4.2
System Information……………..……………….….…...11
4.3
Limitations………………………...………………….….11
iv
CHAPTER 5
Summary and Conclusion
5.1
Summary…………………………………….…………13
5.2
Conclusion……………………………………………..13
5.3
Scope and Future Work………………………..………14
Reference…….……………………………………...……….15
v
LIST OF FIGURES
Fig no.
Title
Page no.
1.
Set of images for individual number ………….5
2.
Result of code…………………..……………10
vi
CHAPETER 1
INTRODUCTION
1.1
PROJECT OBJECTIVE
The goal of this project was to create a program that can take an image
input and detect the faces in it.
1.2
GENERAL INTRODUCTION
Face recognition has been a sought after problem of biometrics and it has
a variety of applications in modern life. The problems of face recognition
attracts researchers working in biometrics, pattern-recognition field and
computer vision . An efficient face recognition system can be of great help
in forensic sciences, identification for law enforcement, surveillance,
authentication for banking and security system, and giving preferential
access to authorised users i.e. access control for secured areas etc. The
project on face recognition involved a detailed survey of a number of face
recognition algorithms along with their advantages and limitations as a
background study. Areal time face recognition system based on PCA and
Mahalanobis distance is presented in the project.
1
CHAPTER 2
LITERATURE REVIEW
The project on face recognition had helped the author to have a detailed
survey of a number of face recognition algorithms along with their
advantages and limitations. Some of the important methods studied will be
described in this section. Face recognition systems architecture broadly
consists of the three following tasks:
1. Acquisition(Detection, Tracking of face-like images)
2. Feature extraction (Segmentation, alignment & normalization of the face
image)
3. Recognition
2.1 Face Detection Approaches
Some of the main face detection methods are discussed here.
1) Knowledge based methods are developed on the rules derived from the
researchers knowledge of human faces. Problem in this approach is the
difficulty in translating human knowledge into well-defined rules.
2) Featured-based methods: Invariant features of faces are used for
detecting texture, skin colour. But features from such algorithm can be
severely corrupted due to illumination, noise and occlusion.
3) Template matching: Input image is compared with predefined face
template. But the performance here suffers due to variations in scale, pose
and shape.
4) Appearance-based method: In template matching methods, the templates
are predefined by experts. Whereas, the templates in appearance based
methods are learned from examples in images. Statistical analysis and
2
machine learning techniques can be used to find the relevant characteristics
of face and non-face images.
2.2 Face Recognition Approaches
LFA method of recognition analyses the face in terms of local features, e.g.,
eyes, nose, etc. referred to as LFA kernels. LFA technique offers better
robustness against local variations on the facial image in carrying out a
match, but does not account for global facial attributes. Neural Network are
based on learning of the faces in an example set by the machine in the
`training phase and
carrying out recognition in the `generalization phase. But in order to
succeed in a practical set-up, the examples should sufficiently large in
number to account for variations in real life situations. Model Matching
methods of face recognition (like Hidden Markov Model (HMM) ) train a
model for every person during model learning and choose the best
matching model, given a query image. Here also a big realistic
representative models is necessary for good results. A recognition system
based on sparse representation computed by l^1 -minimization works with
the basic idea of casting the recognition as a sparse representation problem.
The main concern in this approach is the presence of sufficiently large
number of features and correct computation of sparse representation. It is a
robust and scalable algorithms for face recognition based on linear or
convex programming.
3
CHAPTER 3
METHODOLOGY
3.1
Language Used: Python.
3.2
Project Design
I decided to use Python because of its simplicity and adaptability. As for
the images, I used the uncompressed, grayscale .pgm image format. This
allowed us to confirm the accuracy of the outputs because of the
uncompressed nature of the .pgm format, which means that the image
information doesn't need to be altered before being saved. Also, it is much
easier to code using the .pgm format since it can be read in as and saved as
a string without using any packages or software, also making it more
reliable.
The whole process can be divided in three major steps 1. The first step is to find a good database of faces with multiple images
for each individual.
2. The next step is to detect faces in the database images and use them
to train the face recognizer.
3. The last step is to test the face recognizer to recognize faces it was
trained for.
3.3 Database
For this tutorial, I will use the Yale Face Database that contains 165
grayscale images of 15 individuals in pgm format, There are 11 images for
each individual. In each image, the individual has a different facial
expression like happy, sad, normal, surprised, sleepy etc. Indeed, there are
166 images with 12 images for the first individual.
4
Figure 1: Set of images for individual number 1. Each image has a different facial
expression.
I will use this database by using 10 images of the total 11 images of each
individual in training my face recognizer and the remaining single image
of each individual to test my face recognition algorithm. The images
corresponding to each individual are named like subject. where number ranges from 01, 02, 03…, 14, 15 and facial
expression is the expression that the individual has in the image. I will use
these images to test the face recognizer.
3.4
IMPLEMENTATION
Now, I have an understanding of how my database looks like and it’s time
to start programming the face recognition algorithm.
3.4.1 Import the required modules
cv2 - This is the OpenCV module and contains the functions for face
detection and recognition.
os - This module will be used to manoeuvre with image and directory
names. First, I will use this module to extract the image names in the
database directory and then from these names I will extract the individual
number, which will be used as a label for the face in that image.
5
Image - Since, the dataset images are in gif format and as of now,
OpenCV does not support gif format, I will use Image module from PIL to
read the image in grayscale format.
numpy - My images will be stored in numpy arrays.
3.4.2 Load the face detection Cascade
The first step is to detect the face in each image. Once, I get the region of
interest containing the face in the image, I will use it for training the
recognizer. For the purpose of face detection, I will use the Haar Cascade
provided by OpenCV. The haar cascades that come with OpenCV are
located in the /data/haarcascades> directory of your OpenCV installation.
I will use haarcascade_frontalface_default.xml for detecting the face. So, I
load the cascade using the cv2.CascadeClassifier function which takes the
path to the cascade xml file.
3.4.3 Create the Face Recognizer Object
The next step is creating the face recognizer object. The face recognizer
object has functions like FaceRecognizer.train to train the recognizer
and FaceRecognizer.predict to recognize a face. OpenCV currently
provides 3 face recognizers 1. Eigenface Recognizer - createEigenFaceRecognizer()
2. Fisherface Recognizer - createFisherFaceRecognizer()
3. Local
Binary
Patterns
Histograms Face
Recognizer
- createLBPHFaceRecognizer()
I will use Local Binary Patterns Histograms Face Recognizer.
6
3.4.4 Create the function to prepare the training set
Now, I will define a function get_images_and_labels that takes the absolute
path to the image database as input argument and returns tuple of 2 list, one
containing the detected faces and the other containing the corresponding
label for that face. For example, if the I th index in the list of faces
represents the 5th individual in the database, then the corresponding ith
location in the list of labels has value equal to 5.
3.4.5 Preparing the training set
I pass the get_images_and_labels function with the path of the database
directory. This path has to be the absolute path. This functions returns the
features (images) and labels (labels) which will be used to train the face
recognizer in the next step.
3.4.6 Perform the training
I perform the training using the FaceRecognizer.train function. It requires
2 arguments, the features which in this case are the images of faces and the
corresponding labels assigned to these faces which in this case are the
individual number that I extracted from the image names.
7
CHAPTER 4
RESULTS AND DISCUSSION
4.1
Experimental Result
I completed my project as much as I had originally expected. I will test the
results of the recognizer by using the images with .sad extension which I
had not used earlier. As done in the get_images_and_labels function, I
append all the image names with the .sad extension in a image_paths list.
Then for each image in the list, I read it in grayscale format and detect faces
in it. Once, I have the ROI containing the faces, I pass the ROI to
the FaceRecognizer.predict function which will assign it a label and it will
also tell us how confident it is about the recognition. The label is an integer
that is one of the individual numbers I had assigned to the faces earler. This
label is stored in nbr_predicted . The more the value of confidence variable
is, the less the recognizer has confidence in the recognition. A confidence
value of 0.0 is a perfect recognition. I check if the recognition was correct
by comparing the predicted label nbr_predicted with the actual label
nbr_actual. The label nbr_actual is extracted using the os module and the
string operations from the name of the image. I also display the confidence
score for each recognition.
The Code for detecting face in images is below:
import cv2, os
import numpy as np
from PIL import Image
cascadePath = "haarcascade_frontalface_default.xml"
faceCascade = cv2.CascadeClassifier(cascadePath)
8
recognizer = cv2.createLBPHFaceRecognizer()
def get_images_and_labels(path):
image_paths = [os.path.join(path, f) for f in os.listdir(path) if not
f.endswith('.sad')]
images = []
labels = []
for image_path in image_paths:
image_pil = Image.open(image_path).convert('L')
image = np.array(image_pil, 'uint8')
nbr
=
int(os.path.split(image_path)[1].split(".")[0].replace("subject", ""))
faces = faceCascade.detectMultiScale(image)
for (x, y, w, h) in faces:
images.append(image[y: y + h, x: x + w])
labels.append(nbr)
cv2.imshow("Adding faces to traning set...", image[y: y + h, x:
x + w])
cv2.waitKey(50)
return images, labels
path = './yalefaces'
images, labels = get_images_and_labels(path)
cv2.destroyAllWindows()
recognizer.train(images, np.array(labels))
image_paths = [os.path.join(path, f) for f in os.listdir(path) if
f.endswith('.sad')]
for image_path in image_paths:
predict_image_pil = Image.open(image_path).convert('L')
predict_image = np.array(predict_image_pil, 'uint8')
faces = faceCascade.detectMultiScale(predict_image)
9
for (x, y, w, h) in faces:
nbr_predicted, conf = recognizer.predict(predict_image[y: y + h,
x: x + w])
nbr_actual
=
int(os.path.split(image_path)[1].split(".")[0].replace("subject", ""))
if nbr_actual == nbr_predicted:
print
"{}
is
Correctly
Recognized
with
confidence
{}".format(nbr_actual, conf)
else:
print "{} is Incorrect Recognized as {}".format(nbr_actual,
nbr_predicted)
cv2.imshow("Recognizing Face", predict_image[y: y + h, x: x +
w])
cv2.waitKey(1000)
RESULTS:
Below is the result of the code. As it can be seen the output shows
all
the
images
in
which
a
face
was
successfully
detected.
Figure 3: Each recognized face with the corresponding confidence.
10
4.2
System Information
OS Name:
Windows 10 Home Single Language
OS Manufacturer:
Microsoft Corporation
Processor:
Intel(R) Core™ i5-5200U CPU @ 2.20GHz
Installed Memory (RAM): 8.00 GB(7.89 GB usable)
System Type:
4.3
64-bit Operating System, x64-based processor
Limitations
Several
factors
limit
the
effectiveness
of
facial-recognition
technology:
1. Image quality
Image quality affects how well facial-recognition algorithms work.
The image quality of scanning video is quite low compared with that
of a digital camera.
2. Image size
When a face-detection algorithm finds a face in an image or in a still
from a video capture, the relative size of that face compared with the
enrolled image size affects how well the face will be recognized. An
already small image size, coupled with a target distant from the
camera, means that the detected face is only 100 to 200 pixels on a
side. Further, having to scan an image for varying face sizes is a
processor-intensive activity. Most algorithms allow specification of a
11
face-size range to help eliminate false positives on detection and speed
up image processing
3. Face angle
The relative angle of the target’s face influences the recognition score
profoundly. When a face is enrolled in the recognition software,
usually multiple angles are used (profile, frontal and 45-degree are
common). Anything less than a frontal view affects the algorithm’s
capability to generate a template for the face. The more direct the
image (both enrolled and probe image) and the higher its resolution,
the higher the score of any resulting matches.
12
Chapter 5
SUMMARY AND CONCLUSION
5.1
Summary
So, in this tutorial I performed the task of face recognition using OpenCV
in less than 40 lines of python codes. The code above assigns a label to
each image that is to recognized. But, what if the face to be recognized is
not even in the database. In that case, the confidence score comes to my
rescue. When, the face is not known by the face recognizer, the value of
confidence score will be very high and you can use a threshold to ascertain
that the face was not recognized.
5.2
Conclusion
This project on Face recognition had given us an opportunity to study about
many popular methods used in the field of face recognition. The elaborate
literary survey provided us with the pros and cons of many recognition
systems and the trade-off associated with them. I also came to know that
combining two or more techniques can improve the accuracy of system
greatly.
The facial expression recognition system presented in this research
work contributes a resilient face recognition model based on the mapping
of
behavioural
characteristics
with
the
physiological
biometric
characteristics. The physiological characteristics of the human face with
relevance to various expressions such as happiness, sadness, fear, anger,
surprise and disgust are associated with geometrical structures which
restored as base matching template for the recognition system.
13
5.3
Scope and Future Work
Face recognition systems used today work very well under constrained
conditions, although all systems work much better with frontal mug-shot
images and constant lighting. All current face recognition algorithms fail
under the vastly varying conditions under which humans need to and are
able to identify other people. Next generation person recognition systems
will need to recognize people in real-time and in much less constrained
situations.
I believe that identification systems that are robust in natural environments,
in the presence of noise and illumination changes, cannot rely on a single
modality, so that fusion with other modalities is essential . Technology used
in smart environments has to be unobtrusive and allow users to act freely.
Wearable systems in particular require their sensing technology to be small,
low powered and easily integrable with the user's clothing. Considering all
the requirements, identification systems that use face recognition and
speaker identification seem to us to have the most potential for wide-spread
application.
Cameras and microphones today are very small, light-weight and have been
successfully integrated with wearable systems. Audio and video based
recognition systems have the critical advantage that they use the modalities
humans use for recognition. Finally, researchers are beginning to
demonstrate that unobtrusive audio-and-video based person identification
systems can achieve high recognition rates without requiring the user to be
in highly controlled environments.
14
REFERENCES
[1]
http://home.iitk.ac.in/~draj/cs676/project/
[2]
http://hanzratech.in/2015/02/03/face-recognition-usingopencv.html
[3]
http://www.makematics.com/research/viola-jones/
[4]
http://www.pyimagesearch.com/2015/05/11/creating-a-facedetection-api-with-python-and-opencv-in-just-5-minutes/
[5]
https://www.youtube.com/watch?v=WfdYYNamHZ8
15