Skip to main content
Springer Nature Link
Log in

Simultaneous Detection and Classification of Partially and Weakly Supervised Cells

  • Conference paper
  • First Online:
Computer Vision – ECCV 2022 Workshops (ECCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13803))

Included in the following conference series:

  • 3594 Accesses

Abstract

Detection and classification of cells in immunohistochemistry (IHC) images play a vital role in modern computational pathology pipelines. Biopsy scoring and grading at the slide level is routinely performed by pathologists, but analysis at the cell level, often desired in personalized cancer treatment, is both impractical and non-comprehensive. With its remarkable success in natural images, deep learning is already the gold standard in computational pathology. Currently, some learning-based methods of biopsy analysis are performed at the tile level, thereby disregarding intra-tile cell variability; while others do focus on accurate cell segmentation, but do not address possible downstream tasks. Due to the shared low and high-level features in the tasks of cell detection and classification, these can be treated jointly using a single deep neural network, minimizing cumulative errors and improving the efficiency of both training and inference. We construct a novel dataset of Proteasome-stained Multiple Myeloma (MM) bone marrow slides, containing nine categories with unique morphological traits. With the relative difficulty of acquiring high-quality annotations in the medical-imaging domain, the proposed dataset is intentionally annotated with only \(5\%\) of the cells in each tile. To tackle both cell detection and classification within a single network, we model these as a multi-class segmentation task, and train the network with a combination of partial cross-entropy and energy-driven losses. However, as full segmentation masks are unavailable during both training and validation, we perform evaluation on the combined detection and classification performance. Our strategy, uniting both tasks within the same network, achieves a better combined Fscore, at faster training and inference times, as compared to similar disjoint approaches.

A. Golts, I. Livneh and A. Ciechanover—Equal contribution.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
€32.70 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
EUR 29.95
Price includes VAT (France)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
EUR 139.09
Price includes VAT (France)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
EUR 179.34
Price includes VAT (France)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://github.com/huiqu18/WeaklySegPartialPoints.

References

  1. Botta, C., et al.: Network meta-analysis of randomized trials in multiple myeloma: efficacy and safety in relapsed/refractory patients. Blood Adv. 1(7), 455–466 (2017)

    Article  Google Scholar 

  2. Chamanzar, A., Nie, Y.: Weakly supervised multi-task learning for cell detection and segmentation. In: 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), pp. 513–516. IEEE (2020)

    Google Scholar 

  3. Chandradevan, R., et al.: Machine-based detection and classification for bone marrow aspirate differential counts: initial development focusing on nonneoplastic cells. Lab. Invest. 100(1), 98–109 (2020)

    Article  Google Scholar 

  4. Cireşan, D.C., Giusti, A., Gambardella, L.M., Schmidhuber, J.: Mitosis detection in breast cancer histology images with deep neural networks. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013. LNCS, vol. 8150, pp. 411–418. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40763-5_51

    Chapter  Google Scholar 

  5. Cordell, J.L., et al.: Immunoenzymatic labeling of monoclonal antibodies using immune complexes of alkaline phosphatase and monoclonal anti-alkaline phosphatase (apaap complexes). J. Histochem. Cytochem. 32(2), 219–229 (1984)

    Article  Google Scholar 

  6. Coudray, N., et al.: Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nat. Med. 24(10), 1559–1567 (2018)

    Article  Google Scholar 

  7. Dagogo-Jack, I., Shaw, A.T.: Tumour heterogeneity and resistance to cancer therapies. Nat. Rev. Clin. Oncol. 15(2), 81–94 (2018)

    Article  Google Scholar 

  8. Erber, W., Mynheer, L., Mason, D.: APAAP labelling of blood and bone-marrow samples for phenotyping leukaemia. Lancet 327(8484), 761–765 (1986)

    Article  Google Scholar 

  9. Gao, Z., Puttapirat, P., Shi, J., Li, C.: Renal cell carcinoma detection and subtyping with minimal point-based annotation in whole-slide images. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12265, pp. 439–448. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59722-1_42

    Chapter  Google Scholar 

  10. Golts, A., Freedman, D., Elad, M.: Deep energy: task driven training of deep neural networks. IEEE J. Sel. Top. Sig. Process. 15(2), 324–338 (2021)

    Article  Google Scholar 

  11. Grady, L.: Random walks for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 28(11), 1768–1783 (2006)

    Article  Google Scholar 

  12. Han, Z., Wei, B., Zheng, Y., Yin, Y., Li, K., Li, S.: Breast cancer multi-classification from histopathological images with structured deep learning model. Sci. Rep. 7(1), 1–10 (2017)

    Google Scholar 

  13. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  14. Hou, L., Agarwal, A., Samaras, D., Kurc, T.M., Gupta, R.R., Saltz, J.H.: Robust histopathology image analysis: to label or to synthesize? In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8533–8542 (2019)

    Google Scholar 

  15. Javed, S., et al.: Cellular community detection for tissue phenotyping in colorectal cancer histology images. Med. Image Anal. 63, 101696 (2020)

    Article  Google Scholar 

  16. Javed, S., Mahmood, A., Werghi, N., Benes, K., Rajpoot, N.: Multiplex cellular communities in multi-gigapixel colorectal cancer histology images for tissue phenotyping. IEEE Trans. Image Process. 29, 9204–9219 (2020)

    Article  MATH  Google Scholar 

  17. Jia, Y., et al.: Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM International Conference on Multimedia, pp. 675–678. ACM (2014)

    Google Scholar 

  18. Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 694–711. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_43

    Chapter  Google Scholar 

  19. Kanavati, F., et al.: Weakly-supervised learning for lung carcinoma classification using deep learning. Sci. Rep. 10(1), 1–11 (2020)

    Article  Google Scholar 

  20. Kumar, N., Verma, R., Sharma, S., Bhargava, S., Vahadane, A., Sethi, A.: A dataset and a technique for generalized nuclear segmentation for computational pathology. IEEE Trans. Med. Imaging 36(7), 1550–1560 (2017)

    Article  Google Scholar 

  21. Li, C., Wang, X., Liu, W., Latecki, L.J., Wang, B., Huang, J.: Weakly supervised mitosis detection in breast histopathology images using concentric loss. Med. Image Anal. 53, 165–178 (2019)

    Article  Google Scholar 

  22. Li, J., et al.: Signet ring cell detection with a semi-supervised learning framework. In: Chung, A.C.S., Gee, J.C., Yushkevich, P.A., Bao, S. (eds.) IPMI 2019. LNCS, vol. 11492, pp. 842–854. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20351-1_66

    Chapter  Google Scholar 

  23. Litjens, G., et al.: Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis. Sci. Rep. 6(1), 1–11 (2016)

    Google Scholar 

  24. Liu, D., et al.: Unsupervised instance segmentation in microscopy images via panoptic domain adaptation and task re-weighting. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4243–4252 (2020)

    Google Scholar 

  25. Livneh, I., Cohen-Kaplan, V., Cohen-Rosenzweig, C., Avni, N., Ciechanover, A.: The life cycle of the 26s proteasome: from birth, through regulation and function, and onto its death. Cell Res. 26(8), 869–885 (2016)

    Article  Google Scholar 

  26. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)

    Google Scholar 

  27. Mahmood, F., et al.: Deep adversarial training for multi-organ nuclei segmentation in histopathology images. IEEE Trans. Med. Imaging 39(11), 3257–3267 (2019)

    Article  Google Scholar 

  28. Malon, C.D., Cosatto, E.: Classification of mitotic figures with convolutional neural networks and seeded blob features. J. Pathol. Inform. 4(1), 9 (2013)

    Article  Google Scholar 

  29. Naylor, P., Laé, M., Reyal, F., Walter, T.: Segmentation of nuclei in histopathology images by deep regression of the distance map. IEEE Trans. Med. Imaging 38(2), 448–459 (2018)

    Article  Google Scholar 

  30. Nishimura, K., Ker, D.F.E., Bise, R.: Weakly supervised cell instance segmentation by propagating from detection response. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11764, pp. 649–657. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32239-7_72

    Chapter  Google Scholar 

  31. Peikari, M., Salama, S., Nofech-Mozes, S., Martel, A.L.: A cluster-then-label semi-supervised learning approach for pathology image classification. Sci. Rep. 8(1), 1–13 (2018)

    Article  Google Scholar 

  32. Qu, H., Riedlinger, G., Wu, P., Huang, Q., Yi, J., De, S., Metaxas, D.: Joint segmentation and fine-grained classification of nuclei in histopathology images. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), pp. 900–904. IEEE (2019)

    Google Scholar 

  33. Qu, H., et al.: Weakly supervised deep nuclei segmentation using points annotation in histopathology images. In: International Conference on Medical Imaging with Deep Learning, pp. 390–400. PMLR (2019)

    Google Scholar 

  34. Qu, H., et al.: Weakly supervised deep nuclei segmentation using partial points annotation in histopathology images. IEEE Trans. Med. Imaging 39(11), 3655–3666 (2020)

    Article  MathSciNet  Google Scholar 

  35. Redmon, J., Farhadi, A.: Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767 (2018)

  36. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. Adv. Neural. Inf. Process. Syst. 28, 91–99 (2015)

    Google Scholar 

  37. Richardson, P.G., et al.: Bortezomib or high-dose dexamethasone for relapsed multiple myeloma. N. Engl. J. Med. 352(24), 2487–2498 (2005)

    Article  Google Scholar 

  38. Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    Chapter  Google Scholar 

  39. Russakovsky, O., et al.: Imagenet large scale visual recognition challenge. Int. J. Comput. Vision 115(3), 211–252 (2015)

    Article  MathSciNet  Google Scholar 

  40. Saha, M., Chakraborty, C.: Her2net: a deep framework for semantic segmentation and classification of cell membranes and nuclei in breast cancer evaluation. IEEE Trans. Image Process. 27(5), 2189–2200 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  41. Siegel, R.L., Miller, K.D., Fuchs, H.E., Jemal, A.: Cancer statistics, 2022. CA: Cancer J. Clinicians 72(1), 7–33 (2022). https://doi.org/10.3322/caac.21708

  42. Sirinukunwattana, K., Raza, S.E.A., Tsang, Y.W., Snead, D.R., Cree, I.A., Rajpoot, N.M.: Locality sensitive deep learning for detection and classification of nuclei in routine colon cancer histology images. IEEE Trans. Med. Imaging 35(5), 1196–1206 (2016)

    Article  Google Scholar 

  43. Slamon, D.J., et al.: Use of chemotherapy plus a monoclonal antibody against HER2 for metastatic breast cancer that overexpresses HER2. N. Engl. J. Med. 344(11), 783–792 (2001)

    Article  Google Scholar 

  44. Song, T.H., Sanchez, V., Daly, H.E., Rajpoot, N.M.: Simultaneous cell detection and classification in bone marrow histology images. IEEE J. Biomed. Health Inform. 23(4), 1469–1476 (2018)

    Article  Google Scholar 

  45. Tang, M., Perazzi, F., Djelouah, A., Ayed, I.B., Schroers, C., Boykov, Y.: On regularized losses for weakly-supervised CNN Segmentation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11220, pp. 524–540. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01270-0_31

    Chapter  Google Scholar 

  46. Wei, J.W., Tafe, L.J., Linnik, Y.A., Vaickus, L.J., Tomita, N., Hassanpour, S.: Pathologist-level classification of histologic patterns on resected lung adenocarcinoma slides with deep neural networks. Sci. Rep. 9(1), 1–8 (2019)

    Google Scholar 

  47. Wu, Y.Y., et al.: A hematologist-level deep learning algorithm (BMSNet) for assessing the morphologies of single nuclear balls in bone marrow smears: algorithm development. JMIR Med. Inform. 8(4), e15963 (2020)

    Article  Google Scholar 

  48. Xie, Y., Xing, F., Kong, X., Su, H., Yang, L.: Beyond classification: structured regression for robust cell detection using convolutional neural network. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 358–365. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_43

    Chapter  Google Scholar 

  49. Xie, Y., Xing, F., Shi, X., Kong, X., Su, H., Yang, L.: Efficient and robust cell detection: a structured regression approach. Med. Image Anal. 44, 245–254 (2018)

    Article  Google Scholar 

  50. Xing, F., Xie, Y., Yang, L.: An automatic learning-based framework for robust nucleus segmentation. IEEE Trans. Med. Imaging 35(2), 550–566 (2015)

    Article  Google Scholar 

  51. Xu, J., et al.: Stacked sparse autoencoder (SSAE) for nuclei detection on breast cancer histopathology images. IEEE Trans. Med. Imaging 35(1), 119–130 (2015)

    Article  MathSciNet  Google Scholar 

  52. Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a Gaussian denoiser: residual learning of deep CNN for image denoising. IEEE Trans. Image Process. 26(7), 3142–3155 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  53. Zhou, Y., Dou, Q., Chen, H., Qin, J., Heng, P.A.: SFCN-OPI: detection and fine-grained classification of nuclei using sibling FCN with objectness prior interaction. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)

    Google Scholar 

Download references

Acknowledgements

The study was supported by grants from the Dr. Miriam and Sheldon Adelson Medical Research Foundation (AMRF) and the Israel Precision Medicine Partnership (IPMP), administered by the Israel Science Foundation (ISF).

Author information

Authors and Affiliations

  1. Computer Science, Technion – Israel Institute of Technology, Haifa, Israel

    Alona Golts & Michael Elad

  2. Rapaport Faculty of Medicine, Technion – Israel Institute of Technology, Haifa, Israel

    Ido Livneh & Aaron Ciechanover

  3. Pathology Department, Rambam Medical Center, Haifa, Israel

    Yaniv Zohar

Authors
  1. Alona Golts
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ido Livneh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yaniv Zohar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Aaron Ciechanover
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Michael Elad
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alona Golts .

Editor information

Editors and Affiliations

  1. IBM Research - MIT-IBM Watson AI Lab, Massachusetts, USA

    Leonid Karlinsky

  2. Technion – Israel Institute of Technology, Haifa, Israel

    Tomer Michaeli

  3. Kyoto University, Kyoto, Japan

    Ko Nishino

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Golts, A., Livneh, I., Zohar, Y., Ciechanover, A., Elad, M. (2023). Simultaneous Detection and Classification of Partially and Weakly Supervised Cells. In: Karlinsky, L., Michaeli, T., Nishino, K. (eds) Computer Vision – ECCV 2022 Workshops. ECCV 2022. Lecture Notes in Computer Science, vol 13803. Springer, Cham. https://doi.org/10.1007/978-3-031-25066-8_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-25066-8_16

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-25065-1

  • Online ISBN: 978-3-031-25066-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation