Skip to main content
Springer Nature Link
Log in

Multi-feature fusion for fine-grained sketch-based image retrieval

  • 1227: Content-based Image Retrieval
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Fine-grained sketch-based image retrieval has become an important research topic in the computer vision area. To take advantage of more fine-grained information, we proposed a multi-feature fusion network for fine-grained sketch-based image retrieval. Multi-feature consists of a coarse-grained feature and two fine-grained features which can make better use of fine-grained information. Moreover, a mixed attention module is introduced into the network to extract more discriminating features. Finally, we use the DR-triplet loss to achieve more optimal directions of pair displacements to improve the retrieval performance. Experiments on two extended FG-SBIR datasets, QMUL-Shoe and QMUL-Chair, prove the effectiveness of our method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
€32.70 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (France)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Anjan D, Zeynep A (2020) Semantically tied paired cycle consistency for Any-Shot Sketch-Based image retrieval. Int J Comput Vis 128(10):2684–2703

    MATH  Google Scholar 

  2. Deen Dayal M, Nishant S, Dennis F, Srirangaraj S, Venu G (2020) Moving in the right direction: a regularization for deep metric learning. In: CVPR, vol 2020, pp 14579–14587

  3. Fei W, Mengqing J, Chen Q, Shuo Y, Cheng L, Honggang Z, Xiaogang W, Xiaoou T (2017) Residual attention network for image classification. In: CVPR, pp 6450–6458

  4. Florian S, Dmitry K, James P (2015) FaceNet: A unified embedding for face recognition and clustering. In: CVPR, pp 815–823

  5. Hua Z, Peng S, Yong L, Gan J, Xiaochun C, Hassan F (2019) Learning structural representations via dynamic object landmarks discovery for sketch recognition and retrieval. IEEE Trans Image Process 28(9):4486–4499

    Article  MathSciNet  MATH  Google Scholar 

  6. Jifei S, Qian Y, Yi Zhe S, Tao X, Timothy MH (2017) Deep spatial semantic attention for fine-grained sketch-based image retrieval. In: IEEE Int Conf Comput Vis, pp 5551–5560

  7. Jingyi Z, Fumin S, Li L, Fan Z, Mengyang Y, Ling S, Heng Tao S, Luc Van G (2018) Generative domain-migration hashing for sketch-to-image retrieval. In: ECCV, pp 304–321

  8. Jun-Yan H, Xiao W, Yu Gang J, Bo Z, Qiang P (2017) Sketch recognition with deep visual-sequential fusion model. In: ACM Multimedia, pp 448–456

  9. Kaihao Z, Wenhan L, Lin M, Hongdong L (2019) Cousin network guided sketch recognition via latent attribute warehouse. In: AAAI, pp 9203–9210

  10. Lawrence CZ, Piotr D (2014) Edge boxes: Locating object proposals from edges. In: ECCV, pp 391–405

  11. Mathias E, James H, Marc A (2012) How do humans sketch objects? ACM Trans Graph 31(4):44,1–44,10

    Google Scholar 

  12. Ming Z, Chun C, Nian W, Jun T, Chen Z (2021) Mixed attention dense network for sketch classification. Appl Intell 1–8

  13. Ming Z, Chun C, Nian W, Jun T, Wenxia B (2019) Gradually focused fine-grained sketch-based image retrieval. PLoS One 14(5):1–12

    Google Scholar 

  14. Mustafa S, Emel B (2019) Sketch recognition using transfer learning. Multimed Tools Appl 78(12):17095–17112

    Article  Google Scholar 

  15. Ni K, Hong H, Zhe B (2020) Lightweight neural network for sketch recognition on mobile phones. In: 5th EAI International Summit, pp 428–439

  16. Qian Y, Feng L, Yi Zhe S, Tao X, Timothy MH, Chen Change L (2016) Sketch me that shoe. In: CVPR, pp 799–807

  17. Qian Y, Yongxin Y, Yi Zhe S, Tao X, Timothy MH (2015) Sketch-a-net that beats humans. In: BMVC

  18. Radenovic F, Tolias G, Chum O (2018) Deep shape matching. In: ECCV, pp 774–791

  19. Sarvadevabhatla R, Babu R (2022) Freehand sketch recognition using deep features. arXiv:1502.00254

  20. Sasi Kiran Y, Shiva Krishna MR, Ashish M, Anurag M (2018) A zero-shot framework for sketch based image retrieval. In: ECCV, pp 316–333

  21. Shi Y, Wang K (2019) Sketch recognition based on deformable convolutional network. Comput Sci Eng 6(2):2456–1843

    Google Scholar 

  22. Xingyuan Z, Yaping H, Qi Z, Yanting P, Runsheng Z, Song W (2020) A Hybrid convolutional neural network for sketch recognition. Pattern Recognit Lett 130:73–82

    Article  Google Scholar 

  23. Yang Y, Hospedales T (2022) Deep neural networks for sketch recognition. arXiv:1501.07873v1

  24. Yi L, Timothy MH, Yi Zhe S, Shaogang G (2014) Fine-grained sketch-based image retrieval by matching deformable part models. In: BMVC

  25. Yu X, Shuangbu W, Yanran L, Lihua Y, Xiaosong Y, Jian Jun Z (2019) Fine-grained color sketch-based image retrieval. In: CGI, pp 424–430

  26. Yuming S, Li L, Fumin S, Ling S (2018) Zero-shot sketch-image hashing. In: CVPR, pp 3598–3607

  27. Zhang XL, Shen ML, Li XM, Feng FX (2022) A deformable CNN-based triplet model for fine-grained sketch-based image retrieval. Pattern Recogn 125:108508

    Article  Google Scholar 

  28. Zhantao Y, Xiaoguang Z, Jiuchao Q, Peilin L (2021) Dark-aware network for fine-grained sketch-based image retrieval. IEEE Signal Process Lett 28:264–268

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by The Open Research Fund of National Engineering Research Center for Agro-Ecological Big Data Analysis and Application, Anhui University (No. AE201903), and the National Natural Science Foundation of China (No. 61772032, 61901006).

Author information

Authors and Affiliations

  1. Anhui University, Hefei, China

    Ming Zhu, Chen Zhao, Nian Wang & Jun Tang

  2. Anhui Jianzhu University, Hefei, China

    Pu Yan

Authors
  1. Ming Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chen Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nian Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jun Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Pu Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ming Zhu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhu, M., Zhao, C., Wang, N. et al. Multi-feature fusion for fine-grained sketch-based image retrieval. Multimed Tools Appl 82, 38067–38076 (2023). https://doi.org/10.1007/s11042-022-14115-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-14115-0

Keywords

Search

Navigation