A New Face Recognition Method based on SVD Perturbation

for Single Example Image per Person

Daoqiang Zhang1, Songcan Chen1, 2*and Zhi-Hua Zhou2
1
Department of Computer Science and Engineering

Nanjing University of Aeronautics and Astronautics, Nanjing 210016, China

2
National Laboratory for Novel Software Technology

Nanjing University, Nanjing 210093, China

Abstract

At present, there are many methods for frontal view face recognition. However, few of them can

work well when only one example image per class is available. In this paper, we present a new

method based on SVD perturbation to deal with the 'one example image' problem and two

generalized eigenface algorithms are proposed. In the first algorithm, the original image is linearly

combined with its derived image gotten by perturbing the image matrix's singular values, and then

principal component analysis (PCA) is performed on the joined images. In the second algorithm,

the derived images are regarded as independent images that could augment training image set, and

then PCA is performed on all the training images available, including the original ones and the

derived ones. The proposed algorithms are compared with both the standard eigenface algorithm

and the (PC)2A algorithm which is proposed for addressing the ‘one example image’ problem, on

the well-known FERET database with three different image resolutions. Experimental results

show that the generalized eigenface algorithms are more accurate and use far fewer eigenfaces

than both the standard eigenface algorithm and the (PC)2A algorithm.

Keywords: Face recognition; Principal component analysis; Eigenface; Extended PCA; Singular
value decomposition

*
Corresponding author. Tel.:+86-25-8489-2805.

E-mail address: s.chen@nuaa.edu.cn (S. Chen), daoqz@mail.com (D. Zhang), zhouzh@nju.edu.cn (Z.-H. Zhou)
1 Introduction

Face Recognition has been an active research area of computer vision and pattern recognition for

decades [1-3, 13-15]. Many face recognition methods have been proposed to date and according to

Brunelli and Poggio [1], these methods can be roughly classified into two categories, i.e.,

geometric feature-based algorithms and template-based ones. In the first category, the most often

used method is the elastic bunch graph matching [3], while in the second category the most widely

used algorithm is the eigenface [2]. Recently, neural networks [5-7], support vector machines [8],

kernel methods [9], and ensemble techniques [8] also find great applications in this area.

In some specific scenarios such as law enforcement, there may be only one image per person can

be used for training the face recognition system. It is unfortunate that most face recognition

algorithms may not work well in such a scenario where only one example image per class is

available. For example, most subspace methods such as Linear Discriminant Analysis (LDA) [11,

9], discriminant eigenfeatures [12] and fisherface [10] can hardly be used because in order to

obtain good recognition performance, they require there exist at least two example images per

class so that the intra-class variation could be considered against the inter-class variation. Recently,

a few researchers begin to address this issue [16, 19]. In [16], a method called (PC)2A was

proposed as an extension of the standard eigenface technique, which combines the original face

image with its first-order projection and then performs principal component analysis (PCA) on the

enriched version of the image. It was reported that (PC)2A outperformed the standard eigenface

technique when only one example image per class is available [16]. In [19], a probabilistic

approach was described, in which the model parameters were estimated by using a set of images

generated around a so-called representative sample image, each with small localized errors within

the eigenspace, or partially occluded and expression-variant faces corresponding to the sample

image.

In this paper, we generalize the standard eigenface technique along two ways. In the first way, we

combine the original image linearly with its derived image gotten by perturbing the image matrix's

singular values, and then apply PCA on the joined images. In the second one, instead of

combining the original image with its derivations, we regard the derived images as independent

images that could augment training information, and then apply PCA on all the images available,
including the original ones and the derived ones. The idea behind these two ways is to squeeze as

much information as possible from the single example images. These squeezed information can

drive some features that are important in face recognition with one example image per class

become more salient, therefore we get the first extended version. These information can also be

used to provide each class with several imitated example images so that the problem of face

recognition with one example image per class become a common face recognition problem,

therefore we get the second extended version. Experiments on a subset of the well-known FERET

database under three different image resolutions show that both the extensions of eigenface get

improved recognition accuracy while the number of eigenfaces used is far fewer than that used by

standard eigenface and (PC)2A.

The rest of this paper is organized as follows. In Section 2, we briefly introduce the eigenface

technique. In Section 3, we present two ways to generalize standard eigenface technique. In

Section 4, we report our experiments. Finally in Section 6, we draw the conclusion and point out

some directions for future research.

2 Eigenface

Eigenface technique is an application of classical PCA into face recognition field. It regards each

face image as a feature vector by concatenating the rows or columns of the image together, using

the intensity of each pixel as a single feature. Thus each image can be represented as an

n-dimensional vector xk , where n is the number of pixels in each image. Let {x1 , x2 ,..., xM } be a

set of M images, with each image belonging to one of c classes {X 1 , X 2 ,..., X c }. Consider a linear

transformation mapping the original n-dimension image space into an m-dimension feature space,

where m<<n. The new m-dimension feature vectors yk are defined by the linear

n×m
transformation y k = U xk , where U ∈ R
T
is a matrix with orthonormal columns.

Define the total scatter matrix ST as follows

 M
ST = ∑ ( xk − x )( xk − x )T (1)
k =1

T
where x is the mean image of all samples. Then, after applying the linear transformation U ,

the scatter of the transformed feature vectors {y1 , y2 ,..., yM } is U T ST U . In PCA, by

maximizing the determinant of the total scatter matrix of the projected samples, the optimal

projection U = [u1 , u 2 ,..., u m ] is obtained, where ui is the set of n-dimensional eigenvectors of

ST corresponding to the m largest eigenvalues. The uis are usually called eigenfaces in face

recognition. The extracted m-dimensional feature vectors, i.e. yks, instead of the original

n-dimensional ones are used in the subsequent recognition process. Usually, the number of

eigenvectors or eigenfaces, i.e. m, is controlled by setting a threshold as follows

∑λ i
i =1
n
≥θ (2)
∑λ
i =1
i

where λ1 , λ2 ,..., λn is the n biggest eigenvalues and θ is a pre-set threshold.

3 Generalized eigenface

According to Jain et al. [17], a typical statistical pattern recognition system can be divided into

three successive stages, i.e. preprocessing, feature extraction or selection, and learning. At present,

most of the extensions to eigenface technique focus on the latter two stages, i.e., feature extraction

or selection and learning. However, there are few works on the first stage, i.e., preprocessing,

except for a few standard image processing techniques such as histogram equalization. Recently, a

method called (PC)2A was proposed for addressing the issue of face recognition with one example

image per class, which designs a specific preprocessing technique to be used together with the

eigenface technique. In detail, (PC)2A combines the original face image with its first-order

projection and then performs PCA on the enriched version of the image [16]. In this section, we

follow the line of (PC)2A but use a novel singular value perturbing technique to obtain the derived

images. In fact, we generalize the standard eigenface along two ways that are presented in Section

3.1 and 3.2, respectively..

3.1 SPCA

In order to effectively recognize faces with only one example image per class, we derive an image

from the original image by perturbing the face matrix's singular values. Let I be an intensity

image of size N1 × N 2 , where I ( x, y ) ∈ [0,1] . The derived image P is defined as

P = U ⋅ Σn ⋅V T (3)

where U is an N1 × N1 orthogonal matrix, V is an N 2 × N 2 orthogonal matrix, and Σ is

an N1 × N 2 diagonal matrix consisted of singular values of I , i.e., its entry

sij = 0 if i ≠ j and sii = si ≥ 0 . n is a real value typically between 1 and 2. U , V and Σ

are determined by the following singular value decomposition scheme [20]

I = U ⋅ Σ ⋅V T (4)

Then we combine I linearly with P to generate a new image according to the following equation
I +α P
J= (5)
1+α
where α is a control parameter and its value is between 0.0 and 1.0. In the rest of the paper, α is set

to 0.25 if it is not explicitly stated. Then, PCA can be performed on J instead of I, therefore we get

a method called SPCA, i.e. Singular-value-perturbed version of PCA.

Fig. 1 shows an example of the original image, its derived images and the combined images,

where α is set to 0.25, and n is set to 5/4 for Fig. 1(b) and 3/2 for Fig. 1(c) respectively. Since the

pixels of the derived image and the combined image may fall out of [0 1], these images are

normalized into [0 1] for better display.

Note that when n equals to 1, the derived image P is equivalent to the original image I . If we

choose n>1, then the singular values satisfying si > 1 will be magnified. Thus the reconstructed

image P emphasizes the contribution of the large singular values, while restraining that of the

small ones. So by integrating P into I , we get a combined image J which keeps the main

information of the original image and is expected to work better against minor changes of

expression, illumination and occlusions.

3.2 SPCA+

In fact, the main difficulty of the ‘one example image’ problem of face recognition lies in that

since only one example image is available for each class, the intra-class variation can hardly be

considered against the inter-class variation. Since the derived image generated in last section, i.e.

P, can be regarded as a new image for a specific class, it is natural to try to use the derived image

as an additional example image for the class. Therefore, the problem of face recognition with one

example image becomes a common face recognition problem where each class has several

training images.

According to Eq. (3), a series of derived images can be generated by setting n to different values.

Without loss of generalization, assume that we have obtained a set of d derived images from each

original face image through singular value perturbing. Therefore, together with the original image,

each class now has (d+1) images for training, in this way, PCA can be performed on (d+1)M

training images, where M is the number of classes to be predicted. Such an extension to eigenface

is named as SPCA+.

4. Experiments

4.1 Data Set

In our experiments, the new methods presented in Section 3 are compared with both (PC)2A and

the standard eigenface technique. The experimental configuration is similar as that was described

in [16]. The experimental face database comprises 400 gray-level frontal view face images from

200 persons, with the size of 256x384. There are 71 females and 129 males, each person has two

images (fa and fb) with different facial expressions. The fa images are used as gallery for training

while the fb images as probes for testing. All the images are randomly selected from the FERET

face database [18]. No special criterion is set forth for the selection. So, the face images used in

the experiments are very diversified, e.g. there are faces with different race, different gender,

different age, different expression, different illumination, different occlusion, different scale, etc.,

which greatly increases the difficulty of the recognition task. See [16] for some concrete face

samples.
Before the recognition process, the raw images are normalized according to some constraints so

that the face area could be appropriately cropped. Those constraints include that the line between

the two eyes is parallel to the horizontal axis, the inter-ocular distance (distance between the two

eyes) is set to a fixed value, and the size of the image is fixed. Here in our experiments, the eyes

are manually located, and three different image resolutions are used. The first cropped image size

is 60x60 pixels with the inter-ocular distance as 28 pixels; and the second and the third images are

obtained by scaling the 60x60 size images to 30x30 and 15x15 size respectively.

4.2 Results

At first, we compare the recognition performance of the methods proposed in Section 3 with that

of (PC)2A and the standard eigenface technique when the size of the face database increases

gradually form 20 to 200 with 20 as the interval. For each size of the database, we repeat the

experiments for 100 times through randomly selecting faces from the database. When a probe, i.e.,

an unknown face image, is presented, its corresponding feature vector is constructed from the

eigenfaces. Then the distance between the probe's feature vector and that of the gallery images are

computed, and the k best-matched image (with the minimum distance) in the gallery is considered

as the top k match result. In this paper, we only consider the case k=1, i.e., the top 1 match.

Fig. 2 and 3 show two examples of image that is misrecognized by Eigenface and (PC)2A, while

recognized correctly by SPCA and SPCA+. Here the number of eigenfaces used is determined by

Eq. (2). The parameter θ is set to 0.95, α is set to 0.25, and n in Eq. (3) is set to 3/2 for SPCA

and 5/4 for SPCA+ respectively. These values will be used in the rest of this paper if no specific

value is explicitly stated. Note in Fig 2 and 3, we also give the reconstructed images from their

corresponding feature vectors by Eigenface, (PC)2A, SPCA and SPCA+ respectively. Fig 2 and 3

reveal that the reconstructed images of SPCA and SPCA+ more emphasize the main information

(e.g. the eyes and mouth) while restraining the trivial ones of the original image compared with

those of Eigenface and (PC)2A. Thus SPCA and SPCA+ are more suitable to recognize faces

under minor changes of expression, illumination and occlusions.

The whole top 1 match result on images with size of 60x60 is depicted in Fig. 4. From Fig. 4, we

know that for all cases, SPCA and SPCA+ achieve higher recognition accuracy than both (PC)2A
and the standard eigenface technique, and the difference is more and more distinct as the size of

database increases. The averaged recognition accuracy under different size of the face database is

shown in the first row of Table 1. It can be found that the performance of SPCA+ is comparable to

that of SPCA. These results supports our claim that through perturbing the singular values of the

original images to enlarge the training face database, the problem of face recognition with one

training image per person can be transformed to be a common face recognition problem to solve.

Although (PC)2A can achieve better recognition accuracy than the standard eigenface technique,

its biggest strength is that it can use significantly fewer (about 10-15% ) eigenfaces to achieve

similar performance of the standard eigenface technique. Therefore, it is very important to

compare the number of eigenfaces used by SPCA and SPCA+. Fig. 5 shows the comparison

results. It is impressive that the number of eigenfaces used by SPCA and SPCA+ is even far fewer

than that used by (PC)2A, and the difference is more and more distinct as the size of database

increases. The averaged number of eigenfaces used under different size of database for 60x60

image size is shown in the fourth row of Table 1. In average, SPCA and SPCA+ use nearly half

fewer eigenfaces than standard eigenface and (PC)2A. Recall that the number of eigenfaces used

determines the dimensionality of the feature vectors that are extracted for representing the face

images. So, it is obvious that using fewer eigenfaces means that less computational cost, less

storage cost, and less matching time are required, which is of great benefit for large-size face

databases in real-world tasks.

Table 1 also shows the performance of eigenface, (PC)2A, SPCA and SPCA+ under 30x30 and

15x15 image sizes. It can be seen that as the image size becomes smaller, the differences among

SPCA and SPCA+ and the other algorithms are not so distinct. We guess the reason is that when

the image size is small enough, the image will be very smooth, thus it will be less affected by the

singular value perturbations than large images.

In SPCA and SPCA+, there is a parameter n which is the order of the derived images from Eq. (3).

In order to know the influence of n, more experiments are performed. Fig. 6 shows the top 1

match recognition accuracy, and Fig. 7 shows the number of eigenfaces used by the methods,
where n = 1 + 1/(11 − m) and m varies between 1 and 10, thus n is between 1 and 2. Note that the

results are averaged under different size of face database. It can be seen that the best recognition

performances for SPCA and SPCA+ are gotten when m=9 （n=3/2） and m=7 （n=5/4）,

respectively. Note that for SPCA+, there is a notable degeneracy of recognition accuracy at m=8

（n=4/3）. We guess that when the order n takes too large values, the derived image will become

too even and be not appropriate for representing faces any more. Therefore we suggest use 3/2 and

5/4 as the default value of n for SPCA and SPCA+, respectively.

5 Conclusions

Most face recognition techniques require that there exist at least two example images per class.

Recently, a method called (PC)2A is proposed to address the 'one example image' problem. In this

paper, following up the way of (PC)2A, two generalized eigenface algorithms utilizing

singular-value-perturbation are proposed. Experiments show that both algorithms can achieve

good recognition accuracy with far fewer eigenfaces than both eigenface and (PC)2A.

Moreover, the second algorithm we proposed, i.e. SPCA+, is a general paradigm for dealing with

'small sample problem', in which we enlarge the original image database by appending the derived

images. This paper shows that this paradigm works well in the scenario of face recognition with

one example image per class. We believe that this method is also effective in scenarios where each

class has two (or more, but still 'small sample') example images, which is another interesting issue

for future work..

Although face recognition technology is very useful, it cannot be used to distinguish between

twins. Remember that the human beings do not rely on the face images solely to identify twins,

but also use other information such as gesture, behavior, even speech and voice. So combining

these information with the face recognition deserve further research in future.

Also we cannot utilize only face to determine the national or regional origin of an individual,

because face recognition is not omnipotent and we should combine it with other biometric

identification technology for practical use.

Acknowledgements

This work was supported by the National Natural Science Foundation of China under the Grant

No. 60271017, the National Outstanding Youth Foundation of China under the Grant No.

60325237, the Jiangsu Natural Science Foundation under the Grant No. BK2002092, and the

Returnee Foundation of Visitor Abroad. Portions of the research in this paper use the FERET

database of facial images collected under the FERET program.

References

[1] R. Brunelli and T. Poggio. Face recognition: Features versus templates. IEEE Trans. PAMI,

15(10)(1993) 1042--1062

[2] M. A. Turk and A. P. Pentland. Face Recognition Using Eigenfaces. Proc. of IEEE Conf. on

Computer Vision and Pattern Recognition, pp. 586-591, June 1991

[3] L. Wiskott, J.-M. Fellous, N. Kruger, and C. von der Malsburg. Face recognition by elastic

bunch graph matching. IEEE Tans. on Pattern Analysis and Machine Intelligence 19 (7) (1997)

775-779

[4] S. Lawrence, C. L. Giles, A. Tsoi, and A. Back. Face recognition: A convolutional

neural-network approach. IEEE Trans. on Neural Networks, 8(1997) 98--113

[5] B. Raytchev, H. Murase. Unsupervised face recognition by associative chaining. Pattern

Recognition 36 (2003) 245-257

[6] J. Zhang, Y. Yan, and M. Lades. Face recognition: Eigenfaces, elastic matching, and neural

nets. Proceedings of IEEE, 85(1997) 1422-1435

[7] D. Valentin, H. Abdi, A.J. O'toole, G.W. Cottrell. Connectionist models of face processing: a

survey. Pattern Recognition 27(9) (1994) 1209-1230

[8] S. Pang, D. Kim, S.Y. Bang. Membership authentication in the dynamic group by face

classification using SVM ensemble. Pattern Recognition Letters 24 (2003) 215-225

[9] J. Lu, K.N. Plataniotis, A.N. Venetsanopoulos. Face Recognition using kernel direct

discriminant analysis algorithms. IEEE Trans. on Neural Networks 14(1) (2003) 117-126

[10] P. Belhumeur, J. Hespanha, D. Kriegman. Eigenfaces vs. Fisherfaces: recognition using class

specific linear projection. IEEE Trans. on Pattern Analysis and Machine Intelligence 19(7)

(1997) 711-720
[11] K. Etemad, R. Chellappa. Discriminant analysis for recognition of human face images.

Journal of the Optical Society of America A: Optics Image Science and Vision 14(8) (1997)

1724-1733

[12] D.L. Swets, J. Weng. Using discriminant eigenfeatures for image retrieval. IEEE Trans. on

Pattern Analysis and Machine Intelligence 18(8) (1996) 831-836

[13] W. Zhao, R. Chellappa, A. Rosenfeld, P.J. Phillips. Face recognition: a literature survey.

http://citeseer.nj.nec.com/374297.html, 2000

[14] G. Sukthankar. Face recognition: a critical look at biologically-inspired approaches. Technical

Report: CMU-RI-TR-00-04, Carnegie Mellon University, Pittsburgh, PA, 2000

[15] R. Chellappa, C.L. Wilson, S. Sirohey. Human and machine recognition of faces: a survey.

Proceedings of the IEEE 83(5) (1995) 705-740

[16] J. Wu, Z.-H. Zhou. Face Recognition with one training image per person. Pattern Recognition

Letters 23(14) (2002) 1711-1719

[17] A.K. Jain, R.P.W, Duin, J. Mao. Statistical pattern recognition: a review. IEEE Trans. on

Pattern Analysis and Machine Intelligence 22(1) (2000) 4-37

[18] P.J. Phillips, H. Wechsler, J. Huang, P.J. Rauss. The FERET database and evaluation

procedure for face-recognition algorithms. Image and Vision Computing, 16(5), 1998:

295-306

[19] A.M. Martinez, Recognition imprecisely localized, partially occluded, and expression variant

faces from a single sample per class, IEEE Trans. on Pattern Analysis and Machine

Intelligence, 25(6) (2002) 748-763

[20] G.H. Golub, C.F. Van Loan. Matrix Computations. Johns Hopkins University Press, Baltimore,

Maryland, 1983
Figure captions

Fig. 1 Example of an original image, its derived images, and combined images

Fig. 2 Example of an image that is misrecognized by Eigenface and (PC)2A, while recognized

correctly by SPCA and SPCA+

Fig. 3 Another example of image that is misrecognized by Eigenface and (PC)2A, while

recognized correctly by SPCA and SPCA+

Fig. 4 Comparison of the averaged recognition performance of 100 times

Fig. 5 Comparison of averaged number of eigenfaces used of 100 times

Fig. 6 averaged recognition accuracy under different size of database with different values of m

Fig. 7 averaged number of eigenfaces used under different size of database with different m

List of tables

Table 1 Comparison of averaged recognition accuracy (ra %) and number of eigenfaces used (ne)

under different image sizes

 (a) (b) (c)

(d) (e)

Fig. 1 Example of an original image, its derived images, and combined images: a) original face

image, b) derived image, n =5/4, c) derived image, n =3/2, d) combined image of a) and b), e)

combined image of a) and c)

 (a) (b) (c)

(d) (e) (f)

Fig. 2 Example of an image that is misrecognized by Eigenface and (PC)2A, while recognized

correctly by SPCA and SPCA+: a) original training image, b) reconstructed image by Eigenface, c)

reconstructed image by (PC)2A, d) reconstructed image by SPCA, e)reconstructed image by

SPCA+, f)the test image

 (a) (b) (c)

(d) (e) (f)

Fig. 3 Another example of image that is misrecognized by Eigenface and (PC)2A, while

recognized correctly by SPCA and SPCA+: a) original training image, b) reconstructed image by

Eigenface, c) reconstructed image by (PC)2A, d) reconstructed image by SPCA, e)reconstructed

image by SPCA+, f) the test image

Fig. 4 Comparison of the averaged recognition performance of 100 times

Fig. 5 Comparison of averaged number of eigenfaces used of 100 times

Fig. 6 averaged recognition accuracy under different size of database with different values of m

Fig. 7 averaged number of eigenfaces used under different size of database with different m
Table 3 Comparison of averaged recognition accuracy (ra %) and number of eigenfaces used (ne)

under different image sizes

eigenface (PC)2A SPCA SPCA+

60x60 86.42 86.72 87.35 87.49
ra 30x30 87.26 87.45 87.64 87.88
15x15 87.81 87.88 87.88 87.23
60x60 48.1 41.7 28.7 22.2
ne 30x30 38.9 33.5 26.1 21.4
15x15 25.7 22.1 19.0 16.8