Abstract
Medical image datasets are hard to collect, expensive to label, and often highly imbalanced. The last issue is underestimated, as typical average metrics hardly reveal that the often very important minority classes have a very low accuracy. In this paper, we address this problem by a feature embedding that balances the classes using contrastive learning as an alternative to the common cross-entropy loss. The approach is largely orthogonal to existing sampling methods and can be easily combined with those. We show on the challenging ISIC2018 and APTOS2019 datasets that the approach improves especially the accuracy of minority classes without negatively affecting the majority ones.
This study was supported by the Excellence Strategy of the German Federal and State Governments, (CIBSS - EXC 2189).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Aptos 2019 blindness detection (2019). https://www.kaggle.com/c/aptos2019-blindness-detection/data
Aleksey Nozdryn-Plotnicki, J.Y., Yolland, W.: Ensembling convolutional neural networks for skin cancer classification. ArXiv (2018)
Cao, K., Wei, C., Gaidon, A., Aréchiga, N., Ma, T.: Learning imbalanced datasets with label-distribution-aware margin loss. In: NeurIPS (2019)
Chan, H.P., Samala, R.K., Hadjiiski, L.M., Zhou, C.: Deep learning in medical image analysis (2020)
Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: SMOTE: synthetic minority over-sampling technique. JAIR 16, 321–357 (2002)
Chen, T., Kornblith, S., Norouzi, M., Hinton, G.E.: A simple framework for contrastive learning of visual representations. In: ICML (2020)
Codella, N.C.F., et al.: Skin lesion analysis toward melanoma detection 2018: a challenge hosted by the international skin imaging collaboration (ISIC). CoRR abs/1902.03368 (2019)
Cui, Y., Jia, M., Lin, T., Song, Y., Belongie, S.J.: Class-balanced loss based on effective number of samples. In: CVPR (2019)
Doi, K.: Computer-aided diagnosis in medical imaging: historical review, current status and future potential. Comput. Med. Imaging Graph. 31, 198–211 (2007)
Dong, Q., Gong, S., Zhu, X.: Imbalanced deep learning by minority class incremental rectification. IEEE TPAMI 41, 1367–1381 (2019)
Dosovitskiy, A., Fischer, P., Springenberg, J.T., Riedmiller, M.A., Brox, T.: Discriminative unsupervised feature learning with exemplar convolutional neural networks. IEEE TPAMI, 1734–1747 (2016)
Drummond, C., Holte, R.: C4.5, class imbalance, and cost sensitivity: why under-sampling beats oversampling. In: ICML Workshop (2003)
Fotedar, G., Tajbakhsh, N., Ananth, S., Ding, X.: Extreme consistency: overcoming annotation scarcity and domain shifts. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12261, pp. 699–709. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59710-8_68
Gong, L., Ma, K., Zheng, Y.: Distractor-aware neuron intrinsic learning for generic 2D medical image classifications. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12262, pp. 591–601. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59713-9_57
Guanjin Wang, K.W.W., Lu, J.: AUC-based extreme learning machines for supervised and semi-supervised imbalanced classification. IEEE Trans. Syst. Man Cybern.: Syst., 1–12 (2020)
Gutmann, M., Hyvärinen, A.: Noise-contrastive estimation: a new estimation principle for unnormalized statistical models. In: AISTATS (2010)
Gyawali, P.K., Ghimire, S., Bajracharya, P., Li, Z., Wang, L.: Semi-supervised medical image classification with global latent mixing. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12261, pp. 604–613. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59710-8_59
Han, H., Wang, W.-Y., Mao, B.-H.: Borderline-SMOTE: a new over-sampling method in imbalanced data sets learning. In: Huang, D.-S., Zhang, X.-P., Huang, G.-B. (eds.) ICIC 2005. LNCS, vol. 3644, pp. 878–887. Springer, Heidelberg (2005). https://doi.org/10.1007/11538059_91
Hayat, M., Khan, S., Zamir, S.W., Shen, J., Shao, L.: Gaussian affinity for max-margin class imbalanced learning. In: ICCV (2019)
He, H., Garcia, E.A.: Learning from imbalanced data. IEEE TKDE 21, 1263–1284 (2009)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR (2016)
Huang, C., Li, Y., Loy, C.C., Tang, X.: Learning deep representation for imbalanced classification. In: CVPR (2016)
Kang, B., et al.: Decoupling representation and classifier for long-tailed recognition. In: ICLR (2020)
Khan, S.H., Hayat, M., Zamir, S.W., Shen, J., Shao, L.: Striking the right balance with uncertainty. In: CVPR (2019)
Khosla, P., et al.: Supervised contrastive learning. In: NeurIPS (2020)
Kim, J., Jeong, J., Shin, J.: M2m: imbalanced classification via major-to-minor translation. In: CVPR (2020)
Li, X., Hu, X., Yu, L., Zhu, L., Fu, C.W., Heng, P.: CANet: cross-disease attention network for joint diabetic retinopathy and diabetic macular edema grading. IEEE Trans. Med. Imaging, 1483–1493 (2020)
Li, X., Yu, L., Jin, Y., Fu, C.-W., Xing, L., Heng, P.-A.: Difficulty-aware meta-learning for rare disease diagnosis. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12261, pp. 357–366. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59710-8_35
Li, Z., Zhong, C., Wang, R., Zheng, W.-S.: Continual learning of new diseases with dual distillation and ensemble strategy. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12261, pp. 169–178. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59710-8_17
Liao, H., Luo, J.: A deep multi-task learning approach to skin lesion classification. In: AAAI workshop (2017)
Lin, T., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: ICCV (2017)
Loshchilov, I., Hutter, F.: SGDR: stochastic gradient descent with warm restarts. In: ICLR (2017)
Mullick, S.S., Datta, S., Das, S.: Generative adversarial minority oversampling. In: ICCV (2019)
van den Oord, A., Li, Y., Vinyals, O.: Representation learning with contrastive predictive coding. CoRR abs/1807.03748 (2018)
Peilin Zhao, Steven C. H. Hoi, R.J., Yang, T.: Online AUC maximization. In: ICML (2011)
Peng, J., Bu, X., Sun, M., Zhang, Z., Tan, T., Yan, J.: Large-scale object detection in the wild from imbalanced multi-labels. In: CVPR (2020)
Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115, 211–252 (2015)
Shen, L., Lin, Z., Huang, Q.: Relay backpropagation for effective learning of deep convolutional neural networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9911, pp. 467–482. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46478-7_29
Shrivastava, A., Gupta, A., Girshick, R.B.: Training region-based object detectors with online hard example mining. In: CVPR (2016)
Shrivastava, A., Gupta, A., Girshick, R: Training region-based object detectors with online hard example mining. In: CVPR (2016)
Skorczyk-Werner, A., et al.: Fundus albipunctatus: review of the literature and report of a novel RDH5 gene mutation affecting the invariant tyrosine (p. Tyr175Phe). J. Appl. Genet. 56, 317–327 (2015)
Wei, D., Cao, S., Ma, K., Zheng, Y.: Learning and exploiting interclass visual correlations for medical image classification. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12261, pp. 106–115. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59710-8_11
Zhang, X., Fang, Z., Wen, Y., Li, Z., Qiao, Y.: Range loss for deep face recognition with long-tailed training data. In: ICCV (2017)
Zhuang, J.X., et al.: Skin lesion analysis towards melanoma detection using deep neural network ensemble (2018)
Zhuang, J., Cai, J., Wang, R., Zhang, J., Zheng, W.-S.: Deep kNN for medical image classification. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12261, pp. 127–136. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59710-8_13
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Marrakchi, Y., Makansi, O., Brox, T. (2021). Fighting Class Imbalance with Contrastive Learning. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12903. Springer, Cham. https://doi.org/10.1007/978-3-030-87199-4_44
Download citation
DOI: https://doi.org/10.1007/978-3-030-87199-4_44
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87198-7
Online ISBN: 978-3-030-87199-4
eBook Packages: Computer ScienceComputer Science (R0)
