Skip to main content

Advertisement

Springer Nature Link
Log in

Sub-RENet: a wavelet-based network for super resolution of diagnostic ultrasound

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

In past few years artificial intelligence in super resolution (SR) has taken a big leap. However, there is still scope of improvement for SR of diagnostic ultrasound (US) images. So, we have proposed a Sub-band Resolution Enhancement Network (Sub-RENet) for SR of the diagnostic US images. The efficiency of wavelets in multiresolution analysis, inspired us to use wavelet sub-bands for training. Sub-RENet has two modules B-net and A-net. B-net takes the resolution to next scale while A-net takes it to higher scales. Both networks use residual and skip connections and unique set of loss function. We proposed a weighted multi-network multi-scale perceptual loss to preserve image detail. The self-supervised A-net uses distribution loss to avoid artifacts. The B-net had average mean squared error and structural similarity of 628 and 0.79 for breast US image dataset and 459, and 0.82 for liver US image dataset, respectively. For output of ×8 input resolution, the values of blind image spatial quality evaluator and just noticeable blur were 43.3 and 0.875, for breast US image dataset and 43.7, and 0.873 for liver US image dataset, respectively, which is far better than recent SR networks. The consistency in values of evaluation metric for two different diagnostic US images shows the applicability of Sub-RENet in any diagnostic US images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
€32.70 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (France)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

The ImageNet and US datasets used for training and testing of proposed strategy, are publicly available. Following are the reference to access the US datasets [54,55,56]. And to access ImageNet dataset follow this link: https://www.image-net.org/.

References

  1. Xiao, J., Cheng, X., Xu, S., Tao, W., Xiao, Y.: Boosting image denoising effect via low-level noise injection. Signal Image Video Process. (2023). https://doi.org/10.1007/s11760-023-02785-8

    Article  Google Scholar 

  2. Zhou, R., Wang, R., Wang, Y., Zhou, F., Guo, N.: Research on low-light image enhancement based on MER-Retinex algorithm. Signal, Image Video Process. 18, 803–811 (2024). https://doi.org/10.1007/s11760-023-02801-x

    Article  Google Scholar 

  3. Qureshi, I., Yan, J., Abbas, Q., Shaheed, K., Bin Riaz, A., Wahid, A., Khan, M.W.J., Szczuko, P.: Medical image segmentation using deep semantic-based methods: a review of techniques, applications and emerging trends. Inf. Fusion. 90, 316–352 (2023). https://doi.org/10.1016/j.inffus.2022.09.031

    Article  Google Scholar 

  4. Ranjbarzadeh, R., Caputo, A., Tirkolaee, E.B., Jafarzadeh Ghoushchi, S., Bendechache, M.: Brain tumor segmentation of MRI images: a comprehensive review on the application of artificial intelligence tools. Comput. Biol. Med.. Biol. Med. 152, 106405 (2023). https://doi.org/10.1016/j.compbiomed.2022.106405

    Article  Google Scholar 

  5. Zhang, X., Cao, X., Wang, J., Wan, L.: G-UNeXt: a lightweight MLP-based network for reducing semantic gap in medical image segmentation. Multimed. Syst. 29, 3431–3446 (2023). https://doi.org/10.1007/s00530-023-01173-z

    Article  Google Scholar 

  6. Umirzakova, S., Ahmad, S., Khan, L.U., Whangbo, T.: Medical image super-resolution for smart healthcare applications: a comprehensive survey. Inf. Fusion. 103, 102075 (2023)

    Article  Google Scholar 

  7. Chen, Y., Xia, R., Yang, K., Zou, K.: MFFN: image super-resolution via multi-level features fusion network. Vis. Comput.Comput. 40(2), 1–16 (2023)

    Google Scholar 

  8. Lu, X., Xie, X., Ye, C., Xing, H., Liu, Z., Chen, Y.: Single-image super-resolution via a lightweight convolutional neural network with improved shuffle learning. Signal Image Video Process. 18, 233–241 (2024). https://doi.org/10.1007/s11760-023-02730-9

    Article  Google Scholar 

  9. Zhang, W., Han, Q., Li, Y., Li, H.: Separable feature complementary network with branch-wise and multi-scale spatial attention for lightweight image super-resolution, Signal. Image Video Process. (2023). https://doi.org/10.1007/s11760-023-02880-w

    Article  Google Scholar 

  10. Xiao, Z., Zhang, X., Liu, Y., Geng, L., Wu, J., Wang, W., Zhang, F.: RNN-combined graph convolutional network with multi-feature fusion for tuberculosis cavity segmentation. Signal Image Video Process. 17, 2297–2303 (2023). https://doi.org/10.1007/s11760-022-02446-2

    Article  Google Scholar 

  11. Sugashini, T., Balakrishnan, G.: YOLO glass: video-based smart object detection using squeeze and attention YOLO network, Signal. Image Video Process. (2023). https://doi.org/10.1007/s11760-023-02855-x

    Article  Google Scholar 

  12. Li, L., Li, X., Jiang, L., Su, X., Chen, F.: A review on deep learning techniques for cloud detection methodologies and challenges. Signal, Image Video Process. 15, 1527–1535 (2021). https://doi.org/10.1007/s11760-021-01885-7

    Article  Google Scholar 

  13. Zhou, Z.: M.M. Rahman Siddiquee, N. Tajbakhsh, J. Liang, UNet++: A Nested U-Net Architecture for Medical Image Segmentation. In: Stoyanov, D., Taylor, Z., Carneiro, G., Syeda-Mahmood, T., Martel, A., Maier-Hein, L., Tavares, J.M.R.S., Bradley, A., Papa, J.P., Belagiannis, V., Nascimento, J.C., Lu, Z., Conjeti, S., Moradi, M., Greenspan, H., Madabhushi, A. (eds.) Deep Learn, pp. 3–11. Springer International Publishing, Cham, Med. Image Anal. Multimodal Learn. Clin. Decis. Support (2018)

    Google Scholar 

  14. Chen, Z., Liu, J., Yang, J., Yang, W.: Super-resolution network-based fractional-pixel motion compensation. Signal Image Video Process. 15, 1547–1554 (2021). https://doi.org/10.1007/s11760-021-01887-5

    Article  Google Scholar 

  15. Huang, Y., De Bortoli, V., Zhou, F., Gilles, J.: Review of wavelet-based unsupervised texture segmentation, advantage of adaptive wavelets. IET Image Process. 12, 1626–1638 (2018). https://doi.org/10.1049/iet-ipr.2017.1005

    Article  Google Scholar 

  16. Chen, X., Yang, R., Guo, C.: A lightweight multi-scale residual network for single image super-resolution. Signal Image Video Process. 16, 1793–1801 (2022). https://doi.org/10.1007/s11760-022-02136-z

    Article  Google Scholar 

  17. Ren, C., He, X., Qing, L., Wu, Y., Pu, Y.: Remote sensing image recovery via enhanced residual learning and dual-luminance scheme. Knowledge-Based Syst. 222, 107013 (2021). https://doi.org/10.1016/j.knosys.2021.107013

    Article  Google Scholar 

  18. Guo, D., Xia, Y., Xu, L., Li, W., Luo, X.: Remote sensing image super-resolution using cascade generative adversarial nets. Neurocomputing 443, 117–130 (2021). https://doi.org/10.1016/j.neucom.2021.02.026

    Article  Google Scholar 

  19. Hou, M., He, X., Dou, F., Zhang, X., Guo, Z., Feng, Z.: Semi-supervised image super-resolution with attention CycleGAN. IET Image Process. 16, 1181–1193 (2022). https://doi.org/10.1049/ipr2.12401

    Article  Google Scholar 

  20. Tian, C., Yuan, Y., Zhang, S., Lin, C.-W., Zuo, W., Zhang, D.: Image super-resolution with an enhanced group convolutional neural network. Neural Netw.Netw. 153, 373–385 (2022). https://doi.org/10.1016/j.neunet.2022.06.009

    Article  Google Scholar 

  21. Ahmadian, H., Mageswaran, P., Walter, B.A., Blakaj, D.M., Bourekas, E.C., Mendel, E., Marras, W.S., Soghrati, S.: Toward an artificial intelligence-assisted framework for reconstructing the digital twin of vertebra and predicting its fracture response. Int. j. Numer. Method. Biomed. Eng. 38, e3601 (2022). https://doi.org/10.1002/cnm.3601

    Article  Google Scholar 

  22. Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38, 295–307 (2016). https://doi.org/10.1109/TPAMI.2015.2439281

    Article  Google Scholar 

  23. Y. Fan, H. Shi, J. Yu, D. Liu, W. Han, H. Yu, Z. Wang, X. Wang, T.S. Huang, Balanced Two-Stage Residual Networks for Image Super-Resolution, in: 2017 IEEE Conf. Comput. Vis. Pattern Recognit. Work., 2017: pp. 1157–1164. https://doi.org/10.1109/CVPRW.2017.154

  24. Shao, J., Zhuang, X., Wang, Z., Shen, W.: Pixel-level self-paced adversarial network with multiple attention in single image super-resolution. Signal, Image Video Process. 17, 1863–1872 (2023). https://doi.org/10.1007/s11760-022-02397-8

    Article  Google Scholar 

  25. Wu, Q., Zeng, H., Zhang, J., Xia, H.: Multi-image hybrid super-resolution reconstruction via interpolation and multi-scale residual networks. Meas. Sci. Technol. 34, 75403 (2023). https://doi.org/10.1088/1361-6501/accbdd

    Article  Google Scholar 

  26. Ma, X., Zhang, J., Li, T.: Geomagnetic reference map super-resolution using convolutional neural network. Meas. Sci. Technol. 35, 15014 (2023). https://doi.org/10.1088/1361-6501/acf7db

    Article  Google Scholar 

  27. Lu, X., Wu, S., Xiao, Z., Huang, X.: An enhanced multiscale generation and depth-perceptual loss-based super-resolution network for prostate ultrasound images. Meas. Sci. Technol. 34, 24002 (2022). https://doi.org/10.1088/1361-6501/ac99f1

    Article  Google Scholar 

  28. Zhang, D.: Wavelet Transform. In: Zhang, D. (ed.) Fundamentals of Image Data Mining: Analysis, Features, Classification and Retrieval, pp. 35–44. Springer International Publishing, Cham (2019). https://doi.org/10.1007/978-3-030-17989-2_3

    Chapter  Google Scholar 

  29. N. Ahn, B. Kang, K.-A. Sohn, Fast, Accurate, and Lightweight Super-Resolution with Cascading Residual Network BT - Computer Vision – ECCV 2018, in: V. Ferrari, M. Hebert, C. Sminchisescu, Y. Weiss (Eds.), Springer International Publishing, Cham, 2018: pp. 256–272.

  30. Liu, Z., Mao, X., Huang, J., Gan, M., Zhang, Y.: Stratified attention dense network for image super-resolution. Signal Image Video Process. 16, 715–722 (2022). https://doi.org/10.1007/s11760-021-02011-3

    Article  Google Scholar 

  31. Lu, Z., Chen, Y.: Single image super-resolution based on a modified U-net with mixed gradient loss. Signal Image Video Process. 16, 1143–1151 (2022). https://doi.org/10.1007/s11760-021-02063-5

    Article  Google Scholar 

  32. Jiang, Y., Sun, J., Ding, Y., Lu, W., Li, S.: Data generation model based on Gm-APD LiDAR data. Signal Image Video Process. 18, 669–677 (2024). https://doi.org/10.1007/s11760-023-02804-8

    Article  Google Scholar 

  33. Qiu, D., Cheng, Y., Wang, X.: Dual U-Net residual networks for cardiac magnetic resonance images super-resolution. Comput. Methods Programs Biomed.. Methods Programs Biomed. 218, 106707 (2022). https://doi.org/10.1016/j.cmpb.2022.106707

    Article  Google Scholar 

  34. Zang, H., Zhu, L., Ding, Z., Li, X., Zhan, S.: Cascaded Dense-UNet for Image Super-Resolution. J. Circuits Syst. Comput. 29, 2050121 (2020). https://doi.org/10.1142/S0218126620501212

    Article  Google Scholar 

  35. C. Ledig, L. Theis, F. Huszár, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, W. Shi, Photo-realistic single image super-resolution using a generative adversarial network, in: 2017 IEEE Conf. Comput. Vis. Pattern Recognit., 2017: pp. 105–114. https://doi.org/10.1109/CVPR.2017.19

  36. Song, J., Yi, H., Xu, W., Li, B., Li, X.: Gram-GAN: image super-resolution based on gram matrix and discriminator perceptual loss. Sensors. (2023). https://doi.org/10.3390/s23042098

    Article  Google Scholar 

  37. Lee, I.H., Chung, W.Y., Park, C.G.: Style transformation super-resolution GAN for extremely small infrared target image. Pattern Recognit. Lett. 174, 1–9 (2023). https://doi.org/10.1016/j.patrec.2023.08.013

    Article  Google Scholar 

  38. Meng, F., Wu, S., Li, Y., Zhang, Z., Feng, T., Liu, R., Du, Z.: Single remote sensing image super-resolution via a generative adversarial network with stratified dense sampling and chain training. IEEE Trans. Geosci. Remote Sens. 62, 1–22 (2024). https://doi.org/10.1109/TGRS.2023.3344112

    Article  Google Scholar 

  39. J. Park, S. Son, K.M. Lee, Content-aware local gan for photo-realistic super-resolution, in: Proc. IEEE/CVF Int. Conf. Comput. Vis., 2023: pp. 10585–10594.

  40. Zhou, D., Sun, S., Li, M., Liu, X., Mi, S.: GAN-based super-resolution for confocal superficial eyelid imaging: Real-time, domain generalization, and noise robustness. Biomed. Signal Process. Control 89, 105668 (2024). https://doi.org/10.1016/j.bspc.2023.105668

    Article  Google Scholar 

  41. Park, H.: Semantic super-resolution via self-distillation and adversarial learning. IEEE Access. 12, 2361–2370 (2024). https://doi.org/10.1109/ACCESS.2023.3349023

    Article  Google Scholar 

  42. Wong, C.Y., Jiang, G., Rahman, M.A., Liu, S., Lin, S.C.-F., Kwok, N., Shi, H., Yu, Y.-H., Wu, T.: Histogram equalization and optimal profile compression based approach for colour image enhancement. J. Vis. Commun. Image Represent.Commun. Image Represent. 38, 802–813 (2016). https://doi.org/10.1016/j.jvcir.2016.04.019

    Article  Google Scholar 

  43. Liu, H., Liu, J., Hou, S., Tao, T., Han, J.: Perception consistency ultrasound image super-resolution via self-supervised CycleGAN. Neural Comput. Appl. 35, 12331–12341 (2023). https://doi.org/10.1007/s00521-020-05687-9

    Article  Google Scholar 

  44. Tamang, L.D., Kim, B.-W.: Super-resolution ultrasound imaging scheme based on a symmetric series convolutional neural network. Sensors. 22, 3076 (2022). https://doi.org/10.3390/s22083076

    Article  Google Scholar 

  45. Shao, D., Qin, L., Xiang, Y., Ma, L., Xu, H.: Medical image blind super-resolution based on improved degradation process. IET Image Process. 17, 1615–1625 (2023). https://doi.org/10.1049/ipr2.12742

    Article  Google Scholar 

  46. Yuan, Z., Yang, D., Wang, W., Zhao, J., Liang, Y.: Self super-resolution of optical coherence tomography images based on deep learning. Opt. Express 31, 27566–27581 (2023). https://doi.org/10.1364/OE.495530

    Article  Google Scholar 

  47. Alzakari, S.A., Hassairi, S., Ali Alhussan, A., Ejbali, R.: A mobile Deep Sparse Wavelet autoencoder for Arabic acoustic unit modeling and recognition. Heliyon. 10, e26583 (2024). https://doi.org/10.1016/j.heliyon.2024.e26583

    Article  Google Scholar 

  48. Wang, Q., Chen, Z.: Parallel wavelet networks incorporating modality adaptation for hyperspectral image super-resolution. Expert Syst. Appl. 235, 121299 (2024). https://doi.org/10.1016/j.eswa.2023.121299

    Article  Google Scholar 

  49. Banu, A.S., Deivalakshmi, S.: AWUNet: leaf area segmentation based on attention gate and wavelet pooling mechanism. Signal, Image Video Process. 17, 1915–1924 (2023). https://doi.org/10.1007/s11760-022-02403-z

    Article  Google Scholar 

  50. M.X.B. Rodriguez, A. Gruson, L. Polania, S. Fujieda, F. Prieto, K. Takayama, T. Hachisuka, Deep adaptive wavelet network, in: Proc. IEEE/CVF Winter Conf. Appl. Comput. Vis., 2020: pp. 3111–3119.

  51. Babu, B.S., Venkatanarayana, M.: AQWT: adaptive quaternion wavelet transform and hybrid heuristic improvement for multi-modal medical image fusion model, Signal. Image Video Process. (2023). https://doi.org/10.1007/s11760-023-02760-3

    Article  Google Scholar 

  52. Quasdane, M., Ramchoun, H., Masrour, T.: Sparse smooth group L0∘L1/2 regularization method for convolutional neural networks. Knowledge-Based Syst. 284, 111327 (2024). https://doi.org/10.1016/j.knosys.2023.111327

    Article  Google Scholar 

  53. Yan, L., Ye, Y., Wang, C., Sun, Y.: LocMix: local saliency-based data augmentation for image classification. Signal Image Video Process. (2023). https://doi.org/10.1007/s11760-023-02852-0

    Article  Google Scholar 

  54. Yap, M.H., Edirisinghe, E.A., Bez, H.E.: A novel algorithm for initial lesion detection in ultrasound breast images. J. Appl. Clin. Med. Phys.Clin. Med. Phys. 9, 181–199 (2008). https://doi.org/10.1120/jacmp.v9i4.2741

    Article  Google Scholar 

  55. Al-Dhabyani, W., Gomaa, M., Khaled, H., Fahmy, A.: Dataset of breast ultrasound images. Data Br. 28, 104863 (2020). https://doi.org/10.1016/j.dib.2019.104863

    Article  Google Scholar 

  56. Vitale, S., Orlando, J.I., Iarussi, E., Larrabide, I.: Improving realism in patient-specific abdominal ultrasound simulation using CycleGANs. Int. J. Comput. Assist. Radiol. Surg. 15, 183–192 (2020). https://doi.org/10.1007/s11548-019-02046-5

    Article  Google Scholar 

  57. Stanke, L., Kubicek, J., Vilimek, D., Penhaker, M., Cerny, M., Augustynek, M., Slaninova, N., Akram, M.U.: Towards to optimal wavelet denoising scheme—a novel spatial and volumetric mapping of wavelet-based biomedical data smoothing. Sensors. (2020). https://doi.org/10.3390/s20185301

    Article  Google Scholar 

  58. Setiadi, D.R.I.M.: PSNR vs SSIM: imperceptibility quality assessment for image steganography. Multimed. Tools Appl. 80, 8423–8444 (2021). https://doi.org/10.1007/s11042-020-10035-z

    Article  Google Scholar 

  59. Gao, Q., Ma, L., Liu, W., Muhammad Khan, N., Cao, G., Fang, Y., Wang, H.: Research on the denoising method of infrared thermogram during rock fracture. Infrared Phys. Technol. 131, 104651 (2023). https://doi.org/10.1016/j.infrared.2023.104651

    Article  Google Scholar 

  60. Huang, J., Feng, H., Xu, Z., Li, Q., Chen, Y.: A robust deblurring algorithm for noisy images with just noticeable blur. Optik (Stuttg). 168, 577–589 (2018). https://doi.org/10.1016/j.ijleo.2018.04.052

    Article  Google Scholar 

  61. Gossler, F.E., Duarte, M.A.Q., Villarreal, F.: Design of nearly-orthogonal symmetric wavelet filter banks based on the wavelet orthogonalization process, circuits. Syst. Signal Process. 42, 234–254 (2023). https://doi.org/10.1007/s00034-022-02111-6

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electronics and Communication Engineering, Dr. B. R. Ambedkar National Institute of Technology, Jalandhar, Punjab, India

    Mayank Kumar Singh, Indu Saini & Neetu Sood

Authors
  1. Mayank Kumar Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Indu Saini
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Neetu Sood
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Mayank Kumar Singh [Corresponding Author] has conceived and designed the study, collected, analyzed, and interpreted the data during the research work, and drafted the manuscript. Dr Indu Saini has substantially contributed to the conceptualize, revision, drafting, and analysis of the research work. Dr Neetu Sood substantially contributed to the revision, drafting, and supervision of the research work.

Corresponding author

Correspondence to Mayank Kumar Singh.

Ethics declarations

Conflict of interest

There is no conflict of interest between the authors to declare.

Consent for publication

This manuscript presents our original work and this work is not under consideration for publication anywhere else.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Singh, M.K., Saini, I. & Sood, N. Sub-RENet: a wavelet-based network for super resolution of diagnostic ultrasound. SIViP 18, 5029–5041 (2024). https://doi.org/10.1007/s11760-024-03213-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-024-03213-1

Keywords

Search

Navigation