Abstract
Most medical images feature a high similarity in the intensities of nearby pixels and a strong correlation of intensity profiles across different image modalities. One way of dealing with – and even exploiting – this correlation is the use of local image patches. In the same way, there is a high correlation between nearby labels in image annotation, a feature that has been used in the “local structure prediction” of local label patches. In the present study we test this local structure prediction approach for 3D segmentation tasks, systematically evaluating different parameters that are relevant for the dense annotation of anatomical structures. We choose convolutional neural network as learning algorithm, as it is known to be suited for dealing with correlation between features. We evaluate our approach on the public BRATS2014 data set with three multimodal segmentation tasks, being able to obtain state-of-the-art results for this brain tumor segmentation data set consisting of 254 multimodal volumes with computing time of only 13 s per volume.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Akselrod-Ballin, A., et al.: An integrated segmentation and classification approach applied to multiple sclerosis analysis. In: Computer Vision and Pattern Recognition (CVPR) (2006)
Chen, L.C., et al.: Learning a dictionary of shape epitomes with applications to image labeling. In: International Conference on Computer Vision (ICCV), pp. 337–344 (2013)
Dollar, P., Zittnick, C.L.: Structured forests for fast edge detection. In: International Conference on Computer Vision (ICCV), pp. 1841–1848 (2013)
Geremia, E., Menze, B.H., Ayache, N.: Spatially adaptive random forests. In: IEEE International Symposium on Biomedical Imaging (ISBI), pp. 1332–1335 (2013)
Iglesias, J.E., Konukoglu, E., Zikic, D., Glocker, B., Van Leemput, K., Fischl, B.: Is synthesizing MRI contrast useful for inter-modality analysis? In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013, Part I. LNCS, vol. 8149, pp. 631–638. Springer, Heidelberg (2013)
Kontschieder, P., Rota Bulo, S., Bischof, H., Pelillo, M.: Structured class-labels in random forests for semantic image labelling. In: International Conference on Computer Vision (ICCV), pp. 2190–2197 (2011)
Liao, S., Gao, Y., Oto, A., Shen, D.: Representation learning: a unified deep learning framework for automatic prostate MR segmentation. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013, Part II. LNCS, vol. 8150, pp. 254–261. Springer, Heidelberg (2013)
Menze, B.H., van Leemput, K., Lashkari, D., Weber, M.-A., Ayache, N., Golland, P.: A generative model for brain tumor segmentation in multi-modal images. In: Jiang, T., Navab, N., Pluim, J.P.W., Viergever, M.A. (eds.) MICCAI 2010, Part II. LNCS, vol. 6362, pp. 151–159. Springer, Heidelberg (2010)
Menze, B., et al.: The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans. Med. Imaging (TMI) 34(10), 1993–2024 (2015)
Pinheiro, P.H.O., Collobert, R.: Recurrent convolutional neural networks for scene labeling. In: International Conference on Machine Learning (ICML), pp. 82–90 (2014)
Pohl, K.M., et al.: A hierarchical algorithm for MR brain image parcellation. IEEE Trans. Med. Imaging (TMI) 26(9), 1201–1212 (2007)
Prasoon, A., Petersen, K., Igel, C., Lauze, F., Dam, E., Nielsen, M.: Deep feature learning for knee cartilage segmentation using a triplanar convolutional neural network. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013, Part II. LNCS, vol. 8150, pp. 246–253. Springer, Heidelberg (2013)
Prastawa, M., Bullitt, E., Ho, S., Gerig, G.: A brain tumor segmentation framework based on outlier detection. Med. Image Anal. 8, 275–283 (2004)
Roth, H.R., et al.: A new 2.5D representation for lymph node detection using random sets of deep convolutional neural network observations. In: Golland, P., Hata, N., Barillot, C., Hornegger, J., Howe, R. (eds.) MICCAI 2014, Part I. LNCS, vol. 8673, pp. 520–527. Springer, Heidelberg (2014)
Tong, T., et al.: Segmentation of MR images via discriminative dictionary learning and sparse coding: Application to hippocampus labeling. NeuroImage 76, 11–23 (2013)
Tustison, N., et al.: N4ITK: improved N3 bias correction with robust B-spline approximation. In: IEEE International Symposium on Biomedical Imaging (ISBI) (2010)
Urban, G., et al.: Multi-modal brain tumor segmentation using deep convolutional neural networks. In: MICCAI-BRATS, pp. 31–35 (2014)
Zhang, Y., Brady, M., Smith, S.: Segmentation of brain MR images through a hidden markov random field model and the expectation-maximization algorithm. IEEE Trans. Med. Imaging (TMI) 20(1), 45–57 (2001)
Zhu, L., et al.: Recursive segmentation and recognition templates for 2D parsing. In: Neural Information Processing Systems (NIPS), pp. 1985–1992 (2009)
Zikic, D., et al.: Decision forests for tissue-specific segmentation of high-grade gliomas in multi-channel MR. In: Ayache, N., Delingette, H., Golland, P., Mori, K. (eds.) MICCAI 2012, Part III. LNCS, vol. 7512, pp. 369–376. Springer, Heidelberg (2012)
Acknowledgments
PD acknowledges projects SIX CZ.1.05/2.1.00/03.0072, EU ECOP EE.2.3.20.0094, GACR 102/12/1104, and CZ.1.05/2.1.00/01.0017 (ED0017/01/01), Czech Republic. BM acknowledges support through the Technische Universität München-Institute for Advanced Study (funded by the German Excellence Initiative and the European Union Seventh Framework Programme under Grant agreement 291763), and the Marie Curie COFUND program of the European Union.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Dvořák, P., Menze, B. (2016). Local Structure Prediction with Convolutional Neural Networks for Multimodal Brain Tumor Segmentation. In: Menze, B., et al. Medical Computer Vision: Algorithms for Big Data. MCV 2015. Lecture Notes in Computer Science(), vol 9601. Springer, Cham. https://doi.org/10.1007/978-3-319-42016-5_6
Download citation
DOI: https://doi.org/10.1007/978-3-319-42016-5_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-42015-8
Online ISBN: 978-3-319-42016-5
eBook Packages: Computer ScienceComputer Science (R0)