Facial Expression Recognition

© June 2019 | IJIRT | Volume 6 Issue 1 | ISSN: 2349-6002 Facial Expression Recognition Tanay Srivastava1, Yashika Gohil2, Akshay Pratap Singh3, Suryansh Parihar4, Amol K. Kadam5 1,2,3,4,5 Bharati Vidyapeeth (Deemed to be University) College of Engineering, Pune, India Abstract- The objective of this paper is used to detect the facial expression of human faces. The facial acknowledgment depends on the feeling of the human which is appeared in the face at the specific time. As we know, With the assistance of the OpenCV bundle and Machine Learning we are going to catch the articulation in the specific picture to discover either on the off chance that they are happy or sad. And if we want to use it for many other kinds of expression, we can find it out. Through this paper, our principle reason we are going to demonstrate we can probably catch the outward appearance and with the assistance AI we can most likely identify the expression of any individual. Index Terms- dlib, Emotion Detection, Facial Detection, Fisherface, Machine Learning, OpenCV, SVM I. INTRODUCTION The study of face and its features is an active research area from past few decades. Pose variation, illumination conditions, bad lighting etc., are still challenging factors faced by all algorithms. Face recognition and emotion detection system are the major applications of recognition system, in which many algorithms have tried to solve these problems. The face recognition is the basic part in modern authentication/identification applications; the accuracy of this system should be high for better results. Fisher face algorithm presents high accurate approach for face recognition; it performs two classes of analyses to achieve recognition i.e. principal component analysis (PCA) and linear discriminant analysis (LDA) respectively. While dealing with the machine learning problems, dimensionality is the biggest issue. Therefore, PCA is used to reduce the dimensionality of the images/frames. It converts the high dimensional space into low dimensional space. By reducing the dimensions, the number of features per image also be reduced. LDA is a discriminant method used in many recognition problems; it computes the group of IJIRT 148330 characteristic features that normalizes the different classes if image data for classification. Fisherface is the best algorithm among others by its accuracy of around 96%. Detection of face in an image or video is the fundamental step in any recognition system. It is difficult for computer to find the face in light of the fact that the number of features in an image is extremely high. Viola and Jones introduce different classifiers which groups the number of features into different classifiers. Using these classifiers, the different facial features are detected which can be used for further processing. But this method is limited to only frontal view of the face. Therefore, in order to detect the facial features of the person’s face under different poses, face landmark annotation is required. For annotation of facial landmarks, the popular method active shape model (ASM) or open source software dlib can be used. Emotion detection is based on different expressions of face and these expressions are generated by variations in facial features. II. NEED OF FACIAL AND EMOTION DETECTION II.1. Making cars safer and personalized Car Manufacturers around the world are increasingly focusing on making cars more personal and safer for us to drive. n their interest to construct progressively brilliant vehicle highlights, it bodes well for producers to utilize AI to enable them to comprehend the human feelings. Utilizing facial feeling location keen vehicles can caution the driver when he is feeling tired. The US Department of Transportation claims that driving-related errors cause around 95% of fatal road accidents. Facial Emotion Detection can discover unpretentious changes in facial miniaturized scale articulations that goes before sluggishness and send INTERNATIONAL JOURNAL OF INNOVATIVE RESEARCH IN TECHNOLOGY 469 © June 2019 | IJIRT | Volume 6 Issue 1 | ISSN: 2349-6002 customized cautions to the driver approaching him to stop for a recess, change music or temperature. II.2. Facial Emotion Detection in Interviews A candidate-interviewer interaction is susceptible to many categories of judgment and subjectivity. Such subjectivity makes it hard to determine whether candidate’s personality is a good fit for the job. Identifying what a candidate is trying to say is out of our hands because of the multiple layers of language interpretation, cognitive biases, and context that lie in between. That’s where AI comes in, which can measure candidate’s facial expressions to capture their moods and further assess their personality traits. Employee morale can also be deduced using this technology by holding and recording interactions. As an HR tool, it can help not only in devising recruiting strategies but also in designing HR policies that bring about best performance from employees. II.3. Testing for Video Games Computer games are structured remembering a particular target group of spectators. Every computer game intends to summon a specific conduct and set of feelings from the clients. During the testing phase, users are asked to play the game for a given period and their feedback is incorporated to make the final product. Utilizing facial feeling location one can help in understanding which feelings a client is experiencing continuously as he is playing without dissecting the total video physically. II.4. Market Research Traditionally, market research companies have employed verbal methods such as surveys to find the consumers wants and needs. However, such methods assume that consumers can formulate their preferences verbally and the stated preferences correspond to future actions which may not always be right. Another popular approach in market research industry is to employ behavioural methods that observe user’s reaction while interacting with a brand or product. Such methods are considered more objective than verbal methods. Behavioural methods use video feeds of users interacting with the product which are then analysed manually to observer their reactions and emotions. However, such techniques can quickly become very labour intensive as the sample size increase Facial Emotion Recognition can IJIRT 148330 act the hero by permitting statistical surveying organizations to quantify minute by-minute outward appearances of feelings (facial coding) consequently and total the outcomes. III. WORKING III.1. Facial Detection Face recognition is a process of identification of a person’s face in an image or video which includes several steps need to be processed. Figure.1 shows the block diagram of face recognition system, which includes face detection, face extraction and face matching. Block diagram of Face recognition system. Recognition is based on the stored image data of the different group of persons. Input images are of any type can be used for recognition, 1. Still images. 2. Video frames or video stills. 3. Video. Input image is subjected for face detection to detect the face. Detected face is then extracted from the image and these images are saved as a database. Saved images are used to compare with the input image. The matching of input image is performed to identify the user’s identity. The recognition result gives identification of the person (particularly his/her name). Figure 2 depicts the step by step architecture developed for face recognition. INTERNATIONAL JOURNAL OF INNOVATIVE RESEARCH IN TECHNOLOGY 470 © June 2019 | IJIRT | Volume 6 Issue 1 | ISSN: 2349-6002 applied for classification of different users. Fisherface algorithm generates the fisherfaces of each image that are used for recognition. In Fisherface algorithm, it performs “leave-one-out” cross validation to validate the user identification. III.2. EMOTION DETECTION Face emotion detection is used to predict the emotion state of the person based on their face expressions. The overview of the emotion detection system is shown in the figure 3 as follows. Block diagram of Emotion detection system. The algorithm is designed in such a way that, if the person is recognizing for the first time then the system considers him as a new user and performs each step of operation. But if the person/user data is already stored then it is considered as “Registered user” and it performs only matching operation to recognize the user identity. OpenCV contains cascade classifiers in which Viola & Jones face detection algorithm is implemented. By using these classifiers, the face region is detected from the image. It classifies the images into positive and negative images respectively. The images consist of face region is considered as positive and the images without face as negative images. These negative images are ignored for further processing. The stored images consist of face images of dimension 273x273, the greater number of the images higher the recognition rate. If the stored images consist false images or wrong extensions then recognition is not possible. Therefore, care should be taken while capturing input images. Fisherface algorithm is IJIRT 148330 Here input images are classified into two types, - Training images. - Testing images. Training images are used for training of classifier. Testing images are used to verify the algorithm by predicting the different emotions of the face. Block diagram of Emotion detection system. PCA is applied to training images to reduce the dimensionality. Because training images are more compared to testing and if the dimension is high then the time taken for processing will be more. Support vector machine classification is done for classifying different emotions flow diagram is shown in figure. INTERNATIONAL JOURNAL OF INNOVATIVE RESEARCH IN TECHNOLOGY 471 © June 2019 | IJIRT | Volume 6 Issue 1 | ISSN: 2349-6002 C. Hybrid approach: Hybrid approach is blend of feature based and holistic approaches. In this approach both local and whole face is used as the input to face detection system. V. APPLICATIONS 1. 2. 3. Facial features such as eyes, nose, lips and face contour are considered as the action units of face and are responsible for creation of expressions on face, are extracted using open source software called dlib. SVM classifier compares the features of training data and testing data to predict any emotion of the face. Here facial features are considered as the key points which are used for training and testing. Support vector machine is the supervised learning method of machine learning. Machine learning algorithms are advantageous over other algorithms, because of less error rate and faster results. Linear SVC which is also called as Multi SVM is used for classification. It uses “one-vs-all” strategy for training of n-class models.      IV. APPROACHES There are basically three approaches for face recognition: A. Feature base approach: In feature-based approach, the features like nose, eyes are segmented and it tends to be utilized as info information in face identification to simpler the errand of face acknowledgment. B. Holistic approach: In holistic approach the whole face taken to the face discovery framework to perform face acknowledgment. IJIRT 148330     Verification (one-to-one matching): When presented with a face image of an unknown individual along with a claim of identity, ascertaining whether the individual is who he/she claims to be. Identification (one-to-many matching): Given an image of an unknown individual, determining that person’s identity by comparing (possibly after encoding) that image with a database of (possibly encoded) images of known individuals. There are various application territories in which face acknowledgment can be abused for these two purposes, a couple of which are laid out beneath. Security (access control to buildings, airports/seaports, ATM machines and border checkpoints; computer/ network security; email authentication on multimedia workstations). Surveillance (a large number of CCTVs can be monitored to look for known criminals. General character checks (discretionary enlistment, banking, electronic business, recognizing new-borns, national IDs, identifications, drivers' licenses, worker IDs). Criminal equity frameworks (mug-shot/booking frameworks, post-occasion examination, criminology). Image database examinations (seeking picture databases of authorized drivers, advantage beneficiaries, missing kids, outsiders and police appointments). “Smart Card” applications. Multi-media environments with adaptive human computer interfaces. (part of ubiquitous or context aware systems, behaviour monitoring at childcare or old people’s centres, recognizing a customer and assessing his needs). Video indexing (labelling faces in video). Witness face reconstruction, other than these applications these procedures in the present face INTERNATIONAL JOURNAL OF INNOVATIVE RESEARCH IN TECHNOLOGY 472 © June 2019 | IJIRT | Volume 6 Issue 1 | ISSN: 2349-6002 acknowledgment innovation have additionally been altered and utilized for related applications, for example, sex characterization, demeanor acknowledgment and facial element acknowledgment and following; Face recognition is also being used in conjunction with other biometrics such as speech, iris, fingerprint, ear and gait recognition in order to enhance the recognition performance of these methods. VI. CONCLUSION Face recognition which is implemented in real-time helps to recognize the human faces can be used for person identification and authentication purposes. Face emotion detection is implemented using support vector machine classifiers which are capable of classifying different class of emotions accurately. The accuracy of both face recognition and emotion detection can be increased by increasing the number of images during training. The detection time is significantly less and hence the system yields less run-time along with high accuracy. The future work includes the implementation of the system in android improves the availability of the system to more users. [4] Samiksha Agrawal, Pallavi Khatri, “Facial Expression Detection Techniques: Based on Viola and Jones algorithm and Principal Component Analysis”, Fifth International Conference on Advanced Computing Communication Technologies, pp:108-112, February 2015. [5] Ibrahim A.Adeyanju, Elijah O. Omidiora, Omobolaji F. Oyedokun, “Performance Evaluation of Different Support Vector Machine Kernels for Face Emotion Recognition”, SAI Intelligent Systems Conference: 804 806, November 2015. [6] Ambika Ramchandra, Ravindra Kumar, “Overview of Face Recognition System Challenges”, International journal of scientific technology research, Vol.2, pp: 234-236, August 2013. [7] M. M. Nordstrom, M. Larsen. Sierakowski,M. B. Stegmann, “The imam face database-an annotated dataset of 240 face images”, Elsevier, DTU Informatics, Tech. Rep. , 2004. REFERENCES [1] Hyung-Ji Lee, Wan-Su Lee, Jae-Ho Chung, “Face recognition using Fisherface algorithm and elastic graph matching”, IEEE International Conference on Image Processing, Vol.1, pp: 998- 1001, October 2001. [2] Adrian Dinculescu, Cristian Vizitiu, Alexandro Nistorescu, Mihaela Marin, Andrea Vizitiu, “Novel Approach to Face Expression Analysis in Determining Emotional Valence and Intensity with Benefit for Human Space Flight Studies”, 5th IEEE International Conference on E-Health and Bioengineering - EHB, pp:1 - 4, November 2015. [3] Rajesh K M, Naveen Kumar M, “An AdaptiveProfile Modified Active Shape Model for Automatic Landmark Annotation Using Open CV”, International Journal of Engineering Research in Electronic and Communication Engineering (IJERECE), Vol.3, Issue.5, pp:1821, May 2016. IJIRT 148330 INTERNATIONAL JOURNAL OF INNOVATIVE RESEARCH IN TECHNOLOGY 473