Abstract
Large-scale and high-quality test samples are extremely scarce in deep neural networks(DNN) testing. Existing test sample optimization methods exhibit the problem of low efficiency and low neuron coverage of optimized test samples, which consistently fail to expose erroneous behaviors of DNNs with corner-case inputs. In this paper, we propose DeepMC, an image classification DNN test sample optimization method jointly guided by misclassification and coverage. Specifically, we select the seed sample from the original test samples according to the misclassification probability. To maximize the misclassification probability and neuron coverage, we construct the joint optimization problem for the seed samples and use the gradient ascent to solve the joint optimization problem. We evaluate this method on two well-known datasets and prevalent image classification DNN models. Compare with DeepXplore, a DL white-box testing framework, DeepMC does not require multiple DNN models with similar functions for cross-referencing, saves 90% time consumption on MNIST, averagely covers 1.87% more neurons, and optimized test samples with more than 69% attack success rate. In addition, the test sample optimized by DeepMC can also be applied to optimize the robustness of the corresponding DNN with an average 3% improvement of the model’s accuracy.











Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Wei K T, Ismail M, Husin Z, Yasruddin M L (2022) Automated tomato grading system using computer vision (CV) and deep neural network (DNN) algorithm. In: Proceedings of the 2022 IEEE 12th symposium on computer applications & industrial electronics (ISCAIE), pp 22–27. https://doi.org/10.1109/ISCAIE54458.2022.9794557
Tang H, Liu H, Xiao W, Sebe N (2021) When dictionary learning meets deep learning: deep dictionary learning and coding network for image recognition with limited data. IEEE Trans Neural Netw Learn Syst 32(5):2129–2141. https://doi.org/10.1109/TNNLS.2020.2997289
Shelke N, Chaudhury S, Chakrabarti S, Bangare SL, Yogapriya G, Pandey P (2022) An efficient way of text-based emotion analysis from social media using LRA-DNN. Neurosci Inf 2(3):100048. https://doi.org/10.1016/j.neuri.2022.100048
Kahn G, Villaflor A, Ding B, Abbeel P, Levine S (2018) Self-supervised deep reinforcement learning with generalized computation graphs for robot navigation. In: Proceeding of the 2018 IEEE international conference on robotics and automation, pp 5129–5136. https://doi.org/10.1109/ICRA.2018.8460655
Dhamija T, Gupta A, Gupta S, Anjum KR (2022) Semantic segmentation in medical images through transfused convolution and transformer networks. Appl Intell. https://doi.org/10.1007/s10489-022-03642-w
Lee DH, Chen KL, Liou KH, Liu CH, Liu JL (2021) Deep learning and control algorithms of direct perception for autonomous driving. Appl Intell 51:237–247. https://doi.org/10.1007/s10489-020-01827-9
Kadhim TA, Zghal NS, Hariri D, Aissa DB (2022) Face recognition in multiple variations using deep learning and convolutional neural networks. In: Proceeding of the 2022 IEEE 9th international conference on sciences of electronics, Technologies of Information and Telecommunications (SETIT), pp 305–311. https://doi.org/10.1109/SETIT54465.2022.9875530
Golwalkar R, Mehendale N (2022) Masked-face recognition using deep metric learning and FaceMaskNet-21. Appl Intell. https://doi.org/10.1007/s10489-021-03150-3
Zhang X, Mahadevan S (2019) Ensemble machine learning models for aviation incident risk prediction. Decis Support Syst 116:48–63. https://doi.org/10.1016/j.dss.2018.10.009
Sun H, Chen J, Lei L, Ji K, Kuang G (2021) Adversarial robustness of deep convolutional neural network-based image recognition models: a review. J Radars 10(4):571–594. https://doi.org/10.12000/JR21048
Berghoff C, Neu M, Twickel AV (2020) Vulnerabilities of connectionist AI applications: evaluation and defense. Frontiers Big Data 3:23. https://doi.org/10.3389/fdata.2020.00023
Newaz AI, Haque NI, Sikder AK, Rahman MA, Uluagac AS (2020) Adversarial attacks to machine learning-based smart healthcare systems. In: Proceeding of the 2020–2020 IEEE Global Communications Conference, pp 1–6. https://doi.org/10.1109/GLOBECOM42002.2020.9322472
Kumar K, Nair S, Roy DG, Rajalingam B, Kumar RS (2021) Security and privacy-aware artificial intrusion detection system using federated machine learning. Comput Electr Eng 96:107440. https://doi.org/10.1016/j.compeleceng.2021.107440
Ramanagopal MS, Anderson C, Vasudevan R, Johnson-Roberson M (2018) Failing to learn: autonomously identifying perception failures for Selfdriving cars. IEEE Robot Autom Lett 3(4):3860–3867. https://doi.org/10.1109/LRA.2018.2857402
Hyun K, Yongchul K, Ki-Woong P, Hyunsoo Y, Choi D (2018) Friend-SafeEvasion Attack: An Adversarial Example That is Correctly Recognized by A Friendly Classifier. Comput Secur 78:380–397. https://doi.org/10.1016/j.cose.2018.07.015
Wang Z, Yan M, Liu S, Chen J, Zhang D, Wu Z et al (2020) Survey on Testing of Deep Neural Networks. J Softw 31(5):1255–1275. https://doi.org/10.13328/j.cnki.jos.005951
Wang S Y, Zhang Z H, Sun J Z (2020) Test case prioritization based on coverage compaction. Journal of Xi’an University of Posts and Telecommunications 03:82–87. https://doi.org/10.13682/j.issn.2095-6533.2020.03.014
Wang Z, Zheng Y, Hai Z, Chang Y, Chen T (2022) Transferable adversarial examples can efficiently fool topic models. Comput Secur 118:102749. https://doi.org/10.1016/j.cose.2022.102749
Ma L, Juefei-Xu F, Zhang F, Sun J, Xue M, Li B et al (2018) Deepgauge: multi-granularity testing criteria for deep learning systems. In: Proceeding of the 2018 33rd IEEE/ACM international conference on automated software engineering, pp 120–131. https://doi.org/10.1145/3238147.3238202
Serban A, Poll E, Visser J (2020) Adversarial examples on object recognition: a comprehensive survey. ACM Comput Surv 53(3):1–38. https://doi.org/10.1145/3398394
Yuan X, He P, Zhu Q, Li X (2019) Adversarial examples: attacks and defenses for deep learning. IEEE Trans Neural Netw Learn Syst 30(9):2805–2824. https://doi.org/10.1109/TNNLS.2018.2886017
Zhang S, Zuo X, Liu J (2019) The problem of the adversarial examples in deep learning. Chin J Comput 2(8):1886–1904. https://doi.org/10.11897/SP.J.1016.2019.01886
Zhang J, Qian W, Nie R, Cao J, Xu D (2022) Generate adversarial examples by adaptive moment iterative fast gradient sign method. Appl Intell. https://doi.org/10.1007/s10489-022-03437-z
Liu J, Tian Y, Zhang R, Sun Y, Wang C (2020) A two-stage generative adversarial networks with semantic content constraints for adversarial example generation. IEEE 8:205766–205777. https://doi.org/10.1109/ACCESS.2020.3037329
He Z, Lan X, Yuan J, Wen C (2022) Multi-layer noise reshaping and perceptual optimization for effective adversarial attack of images. Appl Intell 52:1289–1305. https://doi.org/10.1007/s10489-022-03838-0
Li C, Zhang X, Yin F, Liu C (2022) Decision-based adversarial attack with frequency Mixup. EEE Trans Inf Forensic Secur 17:1038–1052. https://doi.org/10.1109/TIFS.2022.3156809
Hayes J, Danezis G (2018) Learning universal adversarial perturbations with generative models. In: Proceeding of the 2018 IEEE security and privacy workshops, pp 43–49. https://doi.org/10.1109/SPW.2018.00015
Pei K, Cao Y, Yang J, Jana S (2019) Deepxplore: automated Whitebox testing of deep learning systems. Commun ACM 62(11):137–145. https://doi.org/10.1145/3361566
Ma L, Juefei-Xu F, Xue M, Li B, Li L, Liu Y, et al (2019) Deepct: tomographic combinatorial testing for deep learning systems. In: Proceeding of the 2019 IEEE 26th international conference on software analysis, pp 614−618. https://doi.org/10.1109/SANER.2019.8668044
Tian Y, Pei K, Jana S, Ray B (2018) DeepTest: automated testing of deep-neural-network-driven autonomous cars. In: Proceeding of the 2018 IEEE/ACM 40th international conference on software engineering, pp 303–314. https://doi.org/10.1145/3180155.3180220
SudKul (2018) Self-driving-car Datasets.[updated 17 May 2018; cited 10 February 2022]. Available from: https://github.com/udacity/self-driving-car/tree/master/datasets. Accessed 12 Nov 2021
Guo J, Jiang Y, Zhao Y, Chen Q, Sun J (2018) DLFuzz: differential fuzzing testing of deep learning systems. In: Proceeding of the 2018 26th ACM joint meeting on European software engineering Conf. And Symp. On the foundations of software engineering, pp 739−743. https://doi.org/10.1145/3236024.3264835
Le Cun Y, Cortes C, Burges C J (2018) Handwritten Digital Image Dataset MNIST. [updated 9 February 2018; cited 10 February 2022]. Available from: http://yann.lecun.com/exdb/mnist. Accessed 12 Nov 2021
An LF Image Database ImageNet. [updated 11 March 2021; cited 10 February 2022]. Available from: http://www.image-net.org. Accessed 12 Nov 2021
Feng Y, Shi Q, Gao X, Wan J, Fang C, Chen Z (2020) Deepgini: prioritizing massive tests to enhance the robustness of deep neural networks. In proceeding of the 29th ACM Sigsoft international symposium on software testing and analysis, pp.177-188. https://doi.org/10.1145/3395363.3397357
Tan S, Tan Z (2019) Improved LeNet-5 model based on handwritten numeral recognition. In: Proceeding of the 2019 Chinese control and decision conference, pp 6396–6399. https://doi.org/10.1109/CCDC.2019.8833112
Zhang X (2021) The AlexNet, LeNet-5 and VGG NET applied to CIFAR-10. In: Proceeding of the 2021 2nd international conference on big Data & Artificial Intelligence & software engineering, pp 414–419. https://doi.org/10.1109/ICBASE.53849.2021.00083
Mascarenhas S, Agarwal M (2021) A comparison between VGG16, VGG19 and ResNet50 architecture frameworks for image classification. In: Proceeding of the 2021 international conference on disruptive Technologies for Multi-Disciplinary Research and Applications (CENTCON), pp 96–99. https://doi.org/10.1109/CENTCON52345.2021.9687944
Guo J, Zhao Y, Jiang Y, Song H, Jiang Y (2021) Coverage guided differential adversarial testing of deep learning systems. IEEE Trans Netw Sci Eng 8(2):933–942. https://doi.org/10.1109/TNSE.2020.2997359
Huang S (2020) Influence of different convolutional neural network settings on the performance of MNIST handwritten digits recognition. In: Proceeding of the 2020 international conference on artificial intelligence and education (ICAIE), pp 1–6. https://doi.org/10.1109/ICAIE50891.2020.00008
Acknowledgements
The work is supported by the National Natural Science Foundation of China (Grant No. 61876138, No.62272387), the Key R & D Project of Shaanxi Province (2020GY-010), the Key Industrial Chain Core Technology Research Project of Xi’an (Grant No.2022JH-RGZN-0028), and the Special Fund for Key Discipline Construction of General Institutions of Higher Learning from Shaanxi Province.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Sun, J., Li, J. & Wen, S. DeepMC: DNN test sample optimization method jointly guided by misclassification and coverage. Appl Intell 53, 15787–15801 (2023). https://doi.org/10.1007/s10489-022-04323-4
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-022-04323-4