Skip to content

Commit a72ec31

Browse files
justadudewhohacksjustadudewhohacks
committed
added sections for face expression recognition
1 parent ffe26b3 commit a72ec31

File tree

1 file changed

+65
-0
lines changed

1 file changed

+65
-0
lines changed

‎README.md

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@ Table of Contents:
1616
* **[Face Detection Models](#models-face-detection)**
1717
* **[68 Point Face Landmark Detection Models](#models-face-landmark-detection)**
1818
* **[Face Recognition Model](#models-face-recognition)**
19+
* **[Face Expression Recognition Model](#models-face-expression-recognition)**
1920
* **[Getting Started](#getting-started)**
2021
* **[face-api.js for the Browser](#getting-started-browser)**
2122
* **[face-api.js for Nodejs](#getting-started-nodejs)**
@@ -61,6 +62,10 @@ Check out my face-api.js tutorials:
6162

6263
![preview_face-similarity](https://user-images.githubusercontent.com/31125521/40316573-0a1190c0-5d1f-11e8-8797-f6deaa344523.gif)
6364

65+
## Face Expression Recognition
66+
67+
![preview_face-expression-recognition](https://user-images.githubusercontent.com/31125521/50575270-f501d080-0dfb-11e9-9676-8f419efdade4.png)
68+
6469
## Face Landmark Detection
6570

6671
![face_landmarks_boxes_2](https://user-images.githubusercontent.com/31125521/46063404-00928b00-c16d-11e8-8f29-e9c50afd2bc8.jpg)
@@ -161,6 +166,12 @@ The neural net is equivalent to the **FaceRecognizerNet** used in [face-recognit
161166

162167
The size of the quantized model is roughly 6.2 MB (**face_recognition_model**).
163168

169+
<a name="models-face-expression-recognition"></a>
170+
171+
## Face Expression Recognition Model
172+
173+
The face expression recognition model is lightweight, fast and provides reasonable accuracy. The model has a size of roughly 310kb and it employs depthwise separable convolutions and densely connected blocks. It has been trained on a variety of images from publicly available datasets as well as images scraped from the web. Note, that wearing glasses might decrease the accuracy of the prediction results.
174+
164175
<a name="getting-started"></a>
165176

166177
# Getting Started
@@ -228,6 +239,7 @@ await faceapi.loadSsdMobilenetv1Model('/models')
228239
// await faceapi.loadFaceLandmarkModel('/models')
229240
// await faceapi.loadFaceLandmarkTinyModel('/models')
230241
// await faceapi.loadFaceRecognitionModel('/models')
242+
// await faceapi.loadFaceExpressionModel('/models')
231243
```
232244

233245
All global neural network instances are exported via faceapi.nets:
@@ -354,6 +366,42 @@ Detect the face with the highest confidence score in an image + computes 68 Poin
354366
const result = await faceapi.detectSingleFace(input).withFaceLandmarks().withFaceDescriptor()
355367
```
356368

369+
### Recognizing Face Expressions
370+
371+
**Face expressions recognition can be performed for detected faces as follows:**
372+
373+
Detect all faces in an image + recognize face expressions. Returns **Array<[WithFaceExpressions<WithFaceDetection<{}>>](#usage-utility-classes)>**:
374+
375+
``` javascript
376+
const detectionsWithExpressions = await faceapi.detectAllFaces(input).withFaceExpressions()
377+
```
378+
379+
Detect the face with the highest confidence score in an image + recognize the face expression for that face. Returns **[WithFaceExpressions<WithFaceDetection<{}>>](#usage-utility-classes) | undefined**:
380+
381+
``` javascript
382+
const detectionWithExpressions = await faceapi.detectSingleFace(input).withFaceExpressions()
383+
```
384+
385+
### Composition of Tasks
386+
387+
**Tasks can be composed as follows:**
388+
389+
``` javascript
390+
// all faces
391+
await faceapi.detectAllFaces(input)
392+
await faceapi.detectAllFaces(input).withFaceExpressions()
393+
await faceapi.detectAllFaces(input).withFaceLandmarks()
394+
await faceapi.detectAllFaces(input).withFaceExpressions().withFaceLandmarks()
395+
await faceapi.detectAllFaces(input).withFaceExpressions().withFaceLandmarks().withFaceDescriptors()
396+
397+
// single face
398+
await faceapi.detectSingleFace(input)
399+
await faceapi.detectSingleFace(input).withFaceExpressions()
400+
await faceapi.detectSingleFace(input).withFaceLandmarks()
401+
await faceapi.detectSingleFace(input).withFaceExpressions().withFaceLandmarks()
402+
await faceapi.detectSingleFace(input).withFaceExpressions().withFaceLandmarks().withFaceDescriptor()
403+
```
404+
357405
### Face Recognition by Matching Descriptors
358406

359407
To perform face recognition, one can use faceapi.FaceMatcher to compare reference face descriptors to query face descriptors.
@@ -611,6 +659,23 @@ export type WithFaceDescriptor<TSource> TSource & {
611659
}
612660
```
613661

662+
<a name="with-face-expressions"></a>
663+
664+
### WithFaceExpressions
665+
666+
``` javascript
667+
export type FaceExpression = 'neutral' | 'happy' | 'sad' | 'angry' | 'fearful' | 'disgusted' | 'surprised'
668+
669+
export type FaceExpressionPrediction = {
670+
expression: FaceExpression,
671+
probability: number
672+
}
673+
674+
export type WithFaceExpressions<TSource> TSource & {
675+
expressions: FaceExpressionPrediction[]
676+
}
677+
```
678+
614679
<a name="other-useful-utility"></a>
615680

616681
## Other Useful Utility

0 commit comments

Comments
 (0)