Skip to main content

Advertisement

Springer Nature Link
Log in

Supervised deep semantics-preserving hashing for real-time pulmonary nodule image retrieval

  • Special Issue Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Hashing-based medical image retrieval has drawn extensive attention recently, which aims at providing effective aided diagnosis for medical personnel. In the paper, a novel deep hashing framework is proposed in the medical image retrieval, where the processes of deep feature extraction, binary code learning, and deep hash function learning are jointly carried out in supervised fashion. Particularly, the discrete constrained objective function in the hash code learning is optimized iteratively, where the binary code can be directly solved with no need for relaxation. In the meantime, the semantic similarity is maintained by fully exploring supervision information during the discrete optimization, where the neighborhood structure of training data is preserved by applying a graph regularization term. Additionally, to gain the fine-grained ranking of the returned medical images sharing the same Hamming distance, a novel image re-ranking scheme is proposed to refine the similarity measurement by jointly considering Euclidean distance between the real-valued feature descriptors and their category information between those images. Extensive experiments on the pulmonary nodule image dataset demonstrate that the proposed method can achieve better retrieval performance over the state of the arts.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
€32.70 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (France)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. It is renamed as ’latent layer’ in this paper for more intuitive understanding.

  2. In the training set, pulmonary nodules are classified into six categories according to their malignant degree, see Sect. 4.

  3. https://www.tensorflow.org.

References

  1. Torre, L.A., Siegel, R.L., Jemal, A.: Lung cancer statistics. Adv. Exp. Med. Biol. 893, 1–19 (2016)

    Article  Google Scholar 

  2. Bach, P.B., Mirkin, J.N., Oliver, T.K., Azzoli, C.G., Berry, D., Brawley, O.W., Byers, T., Colditz, G.A., Gould, M.K., Jett, J.R.: Benefits and harms of ct screening for lung cancer: a systematic review. JAMA J. Am. Med. Assoc. 307(22), 2418 (2012)

    Article  Google Scholar 

  3. Li, Z., Zhang, X., Müller, H., Zhang, S.: Large-scale retrieval for medical image analytics: a comprehensive review. Med. Image Anal. 43, 66–84 (2018)

    Article  Google Scholar 

  4. Sundararajan, K., Woodard, D.L.: Deep learning for biometrics: a survey. ACM Comput. Surv. 51(3), 65–16534 (2018)

    Article  Google Scholar 

  5. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  6. Gionis, A., Indyk, P., Motwani, R.: Similarity search in high dimensions via hashing. In: International Conference on Very Large Data Bases(VLDB), Edinburgh, Scotland, UK, September 7–10, pp. 518–529 (1999)

  7. Wu, G., Han, J., Lin, Z., Ding, G., Zhang, B., Ni, Q.: Joint image-text hashing for fast large-scale cross-media retrieval using self-supervised deep learning. IEEE Trans. Ind. Electron. 66(12), 9868–9877 (2019)

    Article  Google Scholar 

  8. Weiss, Y., Torralba, A., Fergus, R.: Spectral hashing. In: Twenty-Second Annual Conference on Neural Information Processing Systems(NIPS), Vancouver, British Columbia, Canada, December 8–11, pp. 1753–1760 (2008)

  9. Wang, J., Zhang, T., Song, J., Sebe, N., Shen, H.T.: A survey on learning to hash. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 769–790 (2018)

    Article  Google Scholar 

  10. Chi, L., Zhu, X.: Hashing techniques: a survey and taxonomy. ACM Comput. Surv. 50(1), 11–11136 (2017)

    Article  Google Scholar 

  11. Kulis, B., Darrell, T.: Learning to hash with binary reconstructive embeddings. In: Annual Conference on Neural Information Processing Systems(NIPS), Vancouver, British Columbia, Canada, December 7–10, pp. 1042–1050 (2009)

  12. Norouzi, M., Fleet, D.J.: Minimal loss hashing for compact binary codes. In: International Conference on Machine Learning(ICML), Bellevue, Washington, USA, June 28–July 2, pp. 353–360 (2011)

  13. Liu, W., Wang, J., Ji, R., Jiang, Y., Chang, S.: Supervised hashing with kernels. In: IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Providence, RI, USA, June 16–21, pp. 2074–2081 (2012)

  14. Gong, Y., Lazebnik, S., Gordo, A., Perronnin, F.: Iterative quantization: a procrustean approach to learning binary codes for large-scale image retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 35(12), 2916–2929 (2013)

    Article  Google Scholar 

  15. Liu, W., Wang, J., Kumar, S., Chang, S.: Hashing with graphs. In: International Conference on Machine Learning(ICML), Bellevue, Washington, USA, June 28–July 2, pp. 1–8 (2011)

  16. Shen, F., Shen, C., Liu, W., Tao Shen, H.: Supervised discrete hashing. In: IEEE Conference on Computer Vision and Pattern recognition(CVPR), Boston, MA, USA, June 7–12, pp. 37–45 (2015)

  17. Jiang, Q.-Y., Li, W.-J.: Discrete latent factor model for cross-modal hashing. IEEE Trans. Image Process. 28(7), 3490–3501 (2019)

    Article  MathSciNet  Google Scholar 

  18. Lin, K., Yang, H., Hsiao, J., Chen, C.: Deep learning of binary hash codes for fast image retrieval. In: IEEE Conference on Computer Vision and Pattern Recognition(CVPR) ,Boston, MA, USA, June 7–12, pp. 27–35 (2015)

  19. Yang, H., Lin, K., Chen, C.: Supervised learning of semantics-preserving hash via deep convolutional neural networks. IEEE Trans. Pattern Anal. Mach. Intell. 40(2), 437–451 (2018)

    Article  Google Scholar 

  20. Lai, H., Pan, Y., Liu, Y., Yan, S.: Simultaneous feature learning and hash coding with deep neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Boston, MA, USA, June 7–12, pp. 3270–3278 (2015)

  21. Cao, Z., Long, M., Wang, J., Yu, P.S.: Hashnet: Deep learning to hash by continuation. In: IEEE International Conference on Computer Vision(ICCV), Venice, Italy, October 22–29, pp. 5609–5618 (2017)

  22. Cai, Y., Li, Y., Qiu, C., Ma, J., Gao, X.: Medical image retrieval based on convolutional neural network and supervised hashing. IEEE Access 7, 51877–51885 (2019)

    Article  Google Scholar 

  23. Yu, C.J., Joachims, T.: Learning structural svms with latent variables. In: International Conference on Machine Learning(ICML), Montreal, Quebec, Canada, June 14–18, pp. 1169–1176 (2009)

  24. Zhang, P., Zhang, W., Li, W., Guo, M.: Supervised hashing with latent factor models. In: International ACM SIGIR Conference on Research and Development in Information Retrieval(SIGIR), Gold Coast, QLD, Australia, July 06–11, pp. 173–182 (2014)

  25. Lin, G., Shen, C., Suter, D., van den Hengel, A.: A general two-step approach to learning-based hashing. In: IEEE International Conference on Computer Vision(ICCV), Sydney, Australia, December 1–8, pp. 2552–2559 (2013)

  26. Xia, R., Pan, Y., Lai, H., Liu, C., Yan, S.: Supervised hashing for image retrieval via image representation learning. In: Twenty-Eighth AAAI Conference on Artificial Intelligence(AAAI), Québec City, Québec, Canada, July 27–31, pp. 2156–2162 (2014)

  27. Zhang, X., Liu, W., Dundar, M., Badve, S., Zhang, S.: Towards large-scale histopathological image analysis: Hashing-based image retrieval. IEEE Trans. Med. Imag. 34(2), 496–506 (2015)

    Article  Google Scholar 

  28. Zhao, J., Pan, L., Zhao, P., Tang, X.: Medical sign recognition of lung nodules based on image retrieval with semantic features and supervised hashing. J. Comput. Sci. Technol. 32(3), 457–469 (2017)

    Article  Google Scholar 

  29. Conjeti, S., Paschali, M., Katouzian, A., Navab, N.: Deep multiple instance hashing for scalable medical image retrieval. In: Medical Image Computing and Computer Assisted Intervention(MICCAI), Quebec City, QC, Canada, September 11–13, pp. 550–558 (2017)

  30. Liu, H., Wang, R., Shan, S., Chen, X.: Deep supervised hashing for fast image retrieval. In: IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Las Vegas, NV, USA, June 27–30, pp. 2064–2072 (2016)

  31. Fu, H., Kong, X., Wang, Z.: Binary code reranking method with weighted hamming distance. Multimed. Tools Appl. 75(3), 1391–1408 (2016)

    Article  Google Scholar 

  32. Ye, F., Dong, M., Luo, W., Chen, X., Min, W.: A new re-ranking method based on convolutional neural network and two image-to-class distances for remote sensing image retrieval. IEEE Access 7, 141498–141507 (2019)

    Article  Google Scholar 

  33. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations(ICLR), San Diego, CA, USA, May 7–9 (2015)

  34. Cai, D., He, X., Han, J., Huang, T.S.: Graph regularized nonnegative matrix factorization for data representation. IEEE Trans. Pattern Anal. Mach. Intell. 33(8), 1548–1560 (2011)

    Article  Google Scholar 

  35. Lin, Z., Ding, G., Hu, M., Wang, J.: Semantics-preserving hashing for cross-view retrieval. In: IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Boston, MA, USA, June 7–12, pp. 3864–3872 (2015)

  36. Petersen, K.B., Pedersen, M.S.: The Matrix Cookbook, p. 6. Technical University of Denmark, Cambridge (2012)

    Google Scholar 

  37. Shen, F., Shen, C., Liu, W., Shen, H.T.: Supervised discrete hashing. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, June 7–12, pp. 37–45 (2015)

  38. Armato, S.G.I., Mclennan, G., Bidaut, L., Mcnitt-Gray, M.F., Clarke, L.P.: The lung image database consortium (lidc) and image database resource initiative (idri): a completed reference database of lung nodules on ct scans. Med. Phys. 38(2), 915–931 (2011)

    Article  Google Scholar 

  39. Ren, S., He, K., Girshick, R.B., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Annual Conference on Neural Information Processing Systems(NIPS), Montreal, Quebec, Canada, December 7–12, pp. 91–99 (2015)

  40. Lin, T., Dollár, P., Girshick, R.B., He, K., Hariharan, B., Belongie, S.J.: Feature pyramid networks for object detection. In: IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Honolulu, HI, USA, July 21–26, pp. 936–944 (2017)

  41. Shen, Z., Liu, Z., Li, J., Jiang, Y., Chen, Y., Xue, X.: DSOD: learning deeply supervised object detectors from scratch. In: IEEE International Conference on Computer Vision(ICCV), Venice, Italy, October 22–29, pp. 1937–1945 (2017)

  42. Davis, J., Goadrich, M.: The relationship between precision-recall and ROC curves. In: International Conference on Machine Learning(ICML), Pittsburgh, Pennsylvania, USA, June 25–29, pp. 233–240 (2006)

Download references

Acknowledgements

This work is supported in part by the Natural Science Foundation of China under Grant 61702157 and 41804118, in part by Hebei Province Department of Education Fund under Grant QN2018085, in part by Innovation Capacity Improvement Project of Hebei Province 199676146H, and in part by the Key Program from NSF of North China Institute of Aerospace Engineering under Grant ZD-2013-05.

Author information

Authors and Affiliations

  1. State Key Laboratory of Reliability and Intelligence of Electrical Equipment, Hebei University of Technology, Tianjin, China

    Yongjun Qi & Junhua Gu

  2. Laboratory of Electromagnetic Field and Electrical Apparatus Reliability, Hebei University of Technology, Tianjin, China

    Yongjun Qi & Junhua Gu

  3. Information Technology Center, North China Institute of Aerospace Engineering, Langfang, China

    Yongjun Qi

  4. School of Artificial Intelligence, Hebei University of Technology, Tianjin, China

    Junhua Gu, Yajuan Zhang & Feng Wang

  5. School of Computing and Communications, Lancaster University, Lancaster, UK

    Gengshen Wu

  6. Hebei Province Key Laboratory of Big Data Calculation, Hebei University of Technology, Tianjin, China

    Junhua Gu & Yajuan Zhang

Authors
  1. Yongjun Qi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Junhua Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yajuan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gengshen Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Feng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Junhua Gu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Qi, Y., Gu, J., Zhang, Y. et al. Supervised deep semantics-preserving hashing for real-time pulmonary nodule image retrieval. J Real-Time Image Proc 17, 1857–1868 (2020). https://doi.org/10.1007/s11554-020-00963-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-020-00963-2

Keywords

Search

Navigation