Skip to main content
Springer Nature Link
Log in

Data-dependent kernel sparsity preserving projection and its application for semi-supervised classification

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Dimensionality reduction methods (DR) have been commonly used as a principled way to understand the high-dimensional data. In this paper, a novel semi-supervised nonlinear method called semi-supervised data-dependent kernel sparsity preserving projection (SDKSPP) is proposed for dimensionality reduction. To achieve performance improvements, SDKSPP adopts a data-dependent kernel (DK) instead of a standard kernel. The coefficients in DK are optimized with labeled samples by using the Fisher criterion. Then the labeled and unlabeled samples are mapped into a high dimensional space by DK. The sparse reconstructive relationship among the whole samples is calculated by minimizing a l1 regularization-related objective function. Finally, a transform matrix that can preserve this relationship is obtained to project the mapped data into a low-dimensional space. The effectiveness of the proposed method is tested and compared with seven methods on four popular datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
€32.70 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (France)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Amari S, Wu S (1999) Improving support vector machine classifiers by modifying kernel functions. Neural Netw 12(6):783–789

    Article  Google Scholar 

  2. Belhumeur PN, Hespanha JP, Kriengman DJ (1997) Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720

    Article  Google Scholar 

  3. Boyd S, Vandenberghe L (2004) Convex optimization. Cambridge

  4. Cai D, He X, Han J (2007) Margin based semi-supervised elastic embedding for face image analysis. In: The IEEE international conference on computer vision, pp 1313–1320

  5. Cai D, He X, Han J (2007) Semi-supervised discriminant analysis. In: International conference on computer vision. https://doi.org/10.1109/ICCV.2007.4408856

  6. Chen B, Liu H, Bao Z (2008) Optimizing the data-dependent kernel under a unified kernel optimization framework. Pattern Recogn 41(6):2107–2119

    Article  MATH  Google Scholar 

  7. Cristianini N, Kandola J, Elisseeff A et al (2002) On kernel-target alignment. Adv Neural Inf Process Syst 179(5):367–373

    Google Scholar 

  8. Cristianini N, Ghaoui LE, Lanckriet GRG, Bartlett PL, Jordan MI (2004) Learning the kernel matrix with semi-definite programming. J Mach Learn Res 5(1):323–330

    MATH  Google Scholar 

  9. Fan M, Gu N, Qiao H, Zhang B (2011) Sparse regularization for semi-supervised classification. Pattern Recogn 44(8):1777–1784

    Article  MATH  Google Scholar 

  10. Friedman J, Hastie T, Tibshirani R (2010) Regularization paths for generalized linear models via coordinate descent. J Stat Softw 33(1):1

    Article  Google Scholar 

  11. Gao S, Tsang WH, Chia LT (2010) Kernel sparse representation for image classification and face recognition. In: European conference on computer vision, pp 1–14

  12. Gao Q, Wang Q, Huang Y, Gao X, Hong X, Zhang H (2015) Dimensionality reduction by integrating sparse reduction and fisher criterion and its application. IEEE Trans Image Process 24(12):5684–5694

    Article  MathSciNet  Google Scholar 

  13. Georghiades A (1997) Yale Face Database, Center for Computational Vision and Control at Yale University. http://cvc.yale.edu.proje/yalefaces/yales.html

  14. Gu N, Wang D, Fan M, Meng D (2014) A kernel-based sparsity preserving method for semi-supervised classification. Neurocomputing 139:345–356

    Article  Google Scholar 

  15. He Z, Li J (2015) Multiple data-dependent kernel for classification of hyperspectral images. Expert Syst Appl 42(3):1118–1135

    Article  MathSciNet  Google Scholar 

  16. He X, Cai D, Han J (2008) Learning a maximum margin subspace for image retrieval. IEEE Trans Knowl Data Eng 20(2):189–201

    Article  Google Scholar 

  17. Hull JJ (1994) A database for handwritten text recognition research. IEEE Trans Pattern Anal Mach Intell 16(5):550–554

    Article  Google Scholar 

  18. Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. In: Proceedings of the IEEE, vol 86, no 11, pp 2278–2324

  19. Lee MM, Keerthi SS, Ong CJ, Decoste D (2004) An efficient method for computing leave-one-out error in support vector machines with Gaussian kernels. IEEE Trans Neural Netw 15(3):750–757

    Article  Google Scholar 

  20. Lin C, Wang B, Zhao X, Pang M (2013) Optimizing kernel PCA using sparse representation-based classifier for MSTAR SAR image target recognition. Math Probl Eng 2013(6):707–724

    Google Scholar 

  21. Liu Y, Nie L, Han L, Zhang L, Rosenblum DS (2015) Action2Activity: recognizing complex activities from sensor data. In: Proceedings of the 24th international conference on artificial intelligence, pp 1617–1623

  22. Liu Y, Liang Y, Liu S, Rosenblum D, Zheng Y (2016) Predicting urban water quality with ubiquitous data. arXiv:161009462

  23. Liu Y, Zhang L, Nie L, Yan Y, Rosenblum DS (2016) Fortune teller: predicting your career path. In: Proceedings of the thirtieth AAAI conference on artificial intelligence, pp 201–207

  24. Liu Y, Zheng Y, Liang Y, Liu S, Rosenblum DS (2016) Urban water quality prediction based on multi-task multi-view learning. In: Proceedings of the twenty-fifth international joint conference on artificial intelligence, pp 2576–2582

  25. Lou S, Zhao X, Chuang Y, Zhang S (2016) Graph regularized sparsity discriminant analysis for face recognition. Neurocomputing 173(P2):290–297

    Article  Google Scholar 

  26. Luo L, Bao S, Mao J, Tang D (2016) Nonlinear process monitoring based on kernel global-local preserving projections. J Process Control 38:11–21

    Article  Google Scholar 

  27. Meng M, Wei J, Wang J, Ma Q, Wang X (2017) Adaptive semi-supervised dimensionality reduction based on pairwise constraints weighting and graph optimizing. Int J Mach Learn Cybern 8(3):793–805

    Article  Google Scholar 

  28. Motai Y, Yoshida H (2013) Principal composite kernel feature analysis: data-dependent kernel approach. IEEE Trans Knowl Data Eng 25(8):1863–1875

    Article  Google Scholar 

  29. Ong CS, Smola AJ, Williamson RC (2005) Learning the kernel with hyperkernels. J Mach Learn Res 6(1):1043–1071

    MathSciNet  MATH  Google Scholar 

  30. ORL face database. AT&T Laboratories, Cambridge. http://www.cam-orl.co.uk/facedatabase.html

  31. Qiao L, Chen S, Tan X (2010) Sparsity preserving projections with applications to face recognition. Pattern Recogn 43(1):331–341

    Article  MATH  Google Scholar 

  32. Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323

    Article  Google Scholar 

  33. Sugiyama M, Ide T, Nakajima S, Sese J (2006) Semi-supervised local Fisher discriminant analysis for dimensionality reduction. Mach Learn 78:35–61

    Article  MathSciNet  Google Scholar 

  34. Tenenbaum JB, De SV, Langford JC (2000) A global geometric framework for nonlinear dimensionality reduction. Science 290(5500):2319

    Article  Google Scholar 

  35. Turk MA, Pentland AP (1991) Face recognition using eigenfaces. In: International conference on computer research and development, pp 302–306

  36. Wright J, Yang A, Ganesh A, Sastry S, Ma Y (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell 31 (2):210–227

    Article  Google Scholar 

  37. Xiong H, Swamy MN, Ahmad MO (2005) Optimizing the kernel in the empirical feature space. IEEE Trans Neural Netw 16(2):460–474

    Article  Google Scholar 

  38. Xiong H, Zhang Y, Chen XW (2007) Data-dependent kernel machines for microarray data classification. IEEE/ACM Trans Comput Biol Bioinform 4(4):583–595

    Article  Google Scholar 

  39. Yang Y, Wang Y, Xue X (2016) Discriminant sparse locality preserving projection for face recognition. Multimed Tools Appl 76(2):1–16

    Google Scholar 

  40. Yin J, Liu Z, Jin Z, Yang W (2012) Kernel sparse representation based classification. Neurocomputing 77(1):120–128

    Article  Google Scholar 

  41. Zhang L, Zhou WD (2016) Fisher-regularized support vector machine. Inf Sci 343–344:79–93

    Article  MathSciNet  Google Scholar 

  42. Zhang D, Zhou ZH, Chen S (2007) Semi-supervised dimensionality reduction. In: SIAM international conference on data mining, pp 629–634

  43. Zhang P, You X, Ou W, Chen CLP, Cheung YM (2016) Sparse discriminative multi-manifold embedding for one-sample face identification. Pattern Recogn 52(C):249–259

    Article  Google Scholar 

Download references

Acknowledgements

This work is partially supported by the National Natural Science Foundation of China (No. 61573088, No. 61573087 and No. 61433004).

Author information

Authors and Affiliations

  1. College of Information Science and Engineering, Northeastern University, Shenyang, China

    Ao Zhang & Xianwen Gao

Authors
  1. Ao Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xianwen Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xianwen Gao.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, A., Gao, X. Data-dependent kernel sparsity preserving projection and its application for semi-supervised classification. Multimed Tools Appl 77, 24459–24475 (2018). https://doi.org/10.1007/s11042-018-5707-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-5707-0

Keywords

Search

Navigation