SPECIAL SECTION ON SCALABLE DEEP LEARNING FOR BIG DATA

Received January 21, 2020, accepted February 17, 2020, date of publication March 5, 2020, date of current version March 30, 2020.
Digital Object Identifier 10.1109/ACCESS.2020.2978629

A Deep Learning Model Based on Concatenation

Approach for the Diagnosis of Brain Tumor
NEELUM NOREEN 1 , SELLAPPAN PALANIAPPAN1 , ABDUL QAYYUM 2, IFTIKHAR AHMAD 3,

MUHAMMAD IMRAN 4 , AND MUHAMMAD SHOAIB 4

1 Department of Information Technology, School of Science and Engineering, Malaysia University of Science and Technology, Petaling Jaya 47810, Malaysia
2 ImViA Laboratory, University of Bourgogne Franche-Comté, 21078 Dijon, France
3 Department of Information Technology, Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah 21589, Saudi Arabia
4 College of Computer and Information Sciences, King Saud University, Riyadh 11451, Saudi Arabia

Corresponding author: Neelum Noreen (neelum.noreen@pg.must.edu.my)

This project was funded by the Deanship of Scientific Research (DSR), King Abdulaziz University, Jeddah, under
grant No. D-433-611-1441. The authors, therefore, gratefully acknowledge DSR for technical and financial support.

ABSTRACT Brain tumor is a deadly disease and its classification is a challenging task for radiologists
because of the heterogeneous nature of the tumor cells. Recently, computer-aided diagnosis-based systems
have promised, as an assistive technology, to diagnose the brain tumor, through magnetic resonance imaging
(MRI). In recent applications of pre-trained models, normally features are extracted from bottom layers
which are different from natural images to medical images. To overcome this problem, this study proposes
a method of multi-level features extraction and concatenation for early diagnosis of brain tumor. Two pre-
trained deep learning models i.e. Inception-v3 and DensNet201 make this model valid. With the help of
these two models, two different scenarios of brain tumor detection and its classification were evaluated.
First, the features from different Inception modules were extracted from pre-trained Inception-v3 model
and concatenated these features for brain tumor classification. Then, these features were passed to softmax
classifier to classify the brain tumor. Second, pre-trained DensNet201 was used to extract features from
various DensNet blocks. Then, these features were concatenated and passed to softmax classifier to classify
the brain tumor. Both scenarios were evaluated with the help of three-class brain tumor dataset that is
available publicly. The proposed method produced 99.34 %, and 99.51% testing accuracies respectively
with Inception-v3 and DensNet201 on testing samples and achieved highest performance in the detection
of brain tumor. As results indicated, the proposed method based on features concatenation using pre-trained
models outperformed as compared to existing state-of-the-art deep learning and machine learning based
methods for brain tumor classification.

INDEX TERMS Deep learning, magnetic resonance imaging, brain tumor classification, pre-trained model,
dataset.

I. INTRODUCTION Meningioma represented 36.3% (29,320), Gliomas 26.5%

The advances in biomedical and human intelligence have (21,200), Pituitary tumors represented nearly 16.2% (13,210)
overcome diverse diseases in the last few years but people and rest of the cases belonged to other types of brain tumor
are still, suffering from cancer due to its unpredictable nature. such as Malignant, Medulloblastoma, and Lymphomas. The
This disease is still a significant problem for humanity. Brain principal causes of such disease are cancer-related ailment
tumor cancer is one of the serious and utmost emergent and morbidity. Effective handling of this disease is crucial
ailments. In the USA, almost 23,000 patients were identified which depends in its timely and accurate detection.
brain tumor cancer in 2015 [1]. The dedication of a therapy modality relies upon: the
In another report of cancer indicators [2], this disease is degree of the tumor at the time of the investigation, the patho-
uniform in both adults and children. Approximately 80,000 logical type, and the category of the tumor. The brain is the
fresh cases of primary brain tumors were reported in 2018 [3]: most complicated and key part of the human anatomy that
contains tissues and nerve cells to regulate the key actions
The associate editor coordinating the review of this manuscript and of the whole body like breathing, the operation of our senses
approving it for publication was Liangxiu Han . and muscles. A cell has its capabilities; with its functionality,

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
VOLUME 8, 2020 55135
 N. Noreen et al.: Deep Learning Model Based on Concatenation Approach for the Diagnosis of Brain Tumor

some cells are developed normally, some decrease their capa- as deep learning is nowadays used in many domains like
bilities, stop their growth, and then become abnormal. Such medical analyzation, object recognition system, and object
a bulk group of irregular cells produces the tissue which is detection [11]–[18]. To identify the different patterns in cell
described as a tumor. So, brain tumors are independent and images, the evaluation through deep learning has a significant
irregular propagation of brain cells [4], [5]. payback. The accuracy of prediction models and data analysis
The brain tumor analyzation, classification, and identifica- through DL techniques mainly rely on the data sample and its
tion are critical issues for a neurologist who is using CAD training as it needs more accurate data for better outcomes.
(computer-aided diagnosis) as a supportive tool for a medical To overcome shortcomings in training of data samples, trans-
operation. There are three noticeable kinds of brain tumors: fer learning can be applied to secure better performance.
meningioma, pituitary, and glioma. Precise and timely ana- Transfer learning is a DL technique in which trained features
lyzation of brain cancer is imperative for the satisfactory of large data can be deployed to small data sets where the
treatment of this ailment. The decision of a treatment depends large data is called a source or base dataset and small data
upon the pathological type, the phase at the moment of is named as target data set [18]. The fine-tune ConvNet and
examination, and the grade of the tumor. CAD systems have freeze the layers of ConvNet are considered as two main
been assisting neurologists in multiple ways. Besides, CAD scenarios of transfer learning where we can substitute and
applications in neurology are supporting in tumor grading, maintain the pre-trained ConvNet on a small dataset (target
classification, and detection [6]. dataset) to continue the back-propagation. After this, the tar-
A brain tumor is one of the most serious cancers among get dataset is classified by a fully connected layer.
children and adults. Initial recognition, classification, and The motivation to use pre-trained deep learnings is time
analyzation of brain tumors are particularly significant to saving, because it does not need a large data set to obtain
treat the tumor adequately. Recently, several systems of CAD results. These models also extracted random features from
have been introduced in the field of medical imaging to images for classification. The top layers extracted lower level
help clinicians and radiologists to diagnose various types of features such as texture, color, edges. The bottom layers of the
diseases and health related issues [7]. This work used brain models extracted high level features such as object and con-
tumor dataset Figshare which is publicly available. Different tours. In literature, normally features are extracted by using
researchers have already used this dataset to validate their pre-trained models are from bottom layers as the features on
models [8]. top layers of pre-trained models are almost similar in natural
Most of the methods of brain tumors’ classification depend and medical images. The bottom layers’ features would be
on segmentation. Unfortunately, less importance is given to different from natural images to medical images. The main
the problem of feature extraction and classification which idea is to extract features from various layers of pre-trained
is not only the most important step but can also improve models trained on our proposed dataset are combined or con-
the performance of computer-aided medical diagnosis. So, catenates to extract multiscale information from input images
the researchers are focusing on the classification tasks by to further enhance the features capability of the classifier
using deep learning techniques. Recently, some studies have model. The multi-level features information extracted from
employed deep learning to improve the performance of different bottom layers of the pre-trained models is the main
computer-aided medical diagnosis to investigate the brain contribution in this paper while other researchers are taking
tumor cancer. The deep learning strategies play important role single layer to extract features, according to literature review.
in medical field and proved, as the helpful tools, in many This paper is divided and organized into seven sec-
critical diseases such as lung cancer detection [9] and image tions. Section 2 provides literature review while section
analyzation of breast cancer [10]. 3 describes dataset. Section 4 describes the proposed model.
In the past, machine learning (ML) techniques were con- Section 5 illustrates the results of the proposed techniques
sidered as the foundation for the purpose to take over clas- and compares this model with some other existing methods.
sification and mining tasks. Recently the less accuracy in Section 6 provides discussion. Finally, last section concludes
prediction models and critical nature of the medical data the whole discussion and present the future work.
analyzation force researchers toward new methods of brain
tumor detection to improve classification accuracy. Conse- II. LITERATURE REVIEW
quently, deep learning (DL); a sub-field of machine learning, Machine learning approaches have been extensively
has become the center of attraction because it is capable employed in various domains including medical diagnos-
to provide efficient prediction models by using extensive tics and preventive medicine. A limited number of studies,
data such as images and text. Apart from it, predicting the however, have targeted diagnosis of brain tumor especially
model on a large dataset, deep learning is capable to provide employing magnetic resonance imaging (MRI). Mostly ML
the concluded results. In medical imaging, deep learning is methods train and test traditional ML algorithms on MRI
mostly used to identify damaged parts of any object such as data. Recently, some of the approaches have employed DL
effected part of the lungs, and it is also useful to classify for the diagnosis of brain tumor.
the objects images and prediction models. Application of Rehman et al. [5] proposed a framework that employed
deep learning approaches in pattern recognition is noticeable a setup called tri-architectural CNN (convolution neural

55136 VOLUME 8, 2020

N. Noreen et al.: Deep Learning Model Based on Concatenation Approach for the Diagnosis of Brain Tumor

network) for classification of tumors of different types data was passed to a fine-tuned CNN. Several experiments
(GoogLeNet, AlexNet and VGGNet). This classification were performed on two different sets – Radiopaedia and brain
involved pituitary gland tumors, glioma tumors, and menin- tumor. This study employed both original and augmented
gioma tumors types. The above mention algorithm sliced the datasets to determine systems accuracy i.e., 90.67%. Classi-
brain MRI to locate regions of interests. Fine tuning and fication of tumor into different grades from image data can
freezing were also applied to the sets of data for further be very useful in clinical practice as it can also accelerate
classification. The authors also considered data augmentation the treatment planning for a particular grade patient by over-
techniques to obtain results’ accuracy. This research attained coming the sampling error [23]. In this regard, Pereira et al.
an accuracy of 98.69% by employing VGG16 architecture for [23] proposed a 3D CNN that automatically grades glioma
enhancing classification and detection. from conventional multi-sequence MRI by defining a region
Deepak et al., [6] also adopted the concept of deep trans- of interest automatically. The anticipated grading system has
fer for classification of images and employed the same dual functions: extraction of regions of interests and predic-
data source discussed in [5]. The features of images were tion of glioma grade. The proposed system was assessed on
extracted, and these features were also used in aggregation of the publicly available dataset, called BRATS 2017 Training
testing and classification models. Employing a 5-fold model set in which subjects are classified as per images datasets. The
of classification at patients’ level, the authors achieved an accuracy of system was 89.5 %, in addition to the accuracy
accuracy of 98%. The study concluded that a fully auto- of predicting tumor that was achieved at 92.98% based on
mated classification can help in regions classification, and regions of interests.
performs better than manual regions classifications. Another Anaraki et al. [24] proposed a strategy based on CNN and
study, based on CapsNets as a Capsule network model, GA (genetic algorithm) to classify various types of Glioma
by Afshar et al., [19] classified brain tumors. The study images using MRI data. The proposed system used GA for an
improved the level of accuracy by bringing a variation in automatic selection of CNN structure. They obtained 90.9%
the maps of CapsNet at some convolution layer. The study accuracy predicting Glioma images of three types. Besides,
claimed a pronounced accuracy of 86.50% by using CapsNet the study brought an accuracy of 94.2 % in the classifica-
in convolution layer. The setup was achieved by using 64 fea- tion of Glioma, Meningioma, and Pituitary. Zhou et al. [25]
ture map to enhance the accuracy measures. purposed a method to use the 3D holistic image directly.
Another study by Abiwinanda et al. [20] adopted a deep First of all, 3D holistic image is converted into the 2D slices
learning model, that was based on CNN, applied to brain in the sequence, and then they applied DenseNet for the
tumor images classification. Although the study adopted extraction of the features from each 2D slice. After that,
5 classification models, it concluded model 2 as the best Recurrent Neural Network was applied on each 2D slice
approximation for enhancement of images classification. The which used the Long Short-Term Memory for the purpose
final architecture comprised of RELU layer and a maximum of the classification. They performed experiments on public
pool layer. This setup has 64 hidden neurons in the lay- and proprietary datasets. They also applied pure convolu-
ered architecture. The study claimed to achieve an accuracy tional neural network in the DenseNet as a convolutional auto
measure of 98.5% on training and 84.19% for validation. encoder for the sequence representation learning. So, they
The authors in [21] employed a two-dimensional discrete used the DenseNet long Short-Term Memory and DenseNet
transform based on wavelets and Gabor filters for extraction Convolutional Neural Network to perform tumor screening
of features of brain MRI. The study achieved an accuracy and tumor type classification. Their system achieved an accu-
measure of 91.9% by employing the aforementioned system racy of 92.13% with DenseNet-LSTM.
setup with backpropagation NN. Muneer et al. [26] used the real dataset from clinics of
Pashaei et al., [22] developed an architecture based on the United States. They used a customized classification
CNN for features extraction. They also designed a 5 layered algorithm based on Wndchrm. The tool has the concepts of
architecture having all layers as learnable layers with cus- neighborhood distance measure employing morphology and
tomized 3 × 3 layered setup. The study claimed to achieve an deep learning for CNN. The authors with this proposed setup
accuracy of 81% that was further enhanced by another fea- achieved an accuracy of 92.86% in addition to 98.25% that
tured classification model of CNN based on ELM (extreme was achieved through Wndchrm classification.
learning machine). The study noticed a limitation in the Banerjee et al. [27] discussed the convolutional neural
classifiers’ discrimination capability by vetting classification network to enhance MRI images classification by employing
differences in pituitary and meningioma images. a sequence of multiple MRI images. They proposed ConvNet
Sajjad et al. [7] proposed a system based on a neural models, that were customized to be developed from ab initio,
network classification that further aided a clear provision based on concepts of slicing and patching of MRI images.
of image segments (segmenting tumor region from dataset). The study aggregated the existing models i.e. ConvNets and
In addition, the study employed various noise suppression VGGNet that were developed for processing of different
techniques by using transformation and invariance concepts. MRI imaging. The study evaluated the performance of pro-
The CNN setup could tune the prediction accuracy for the posed models and claimed to achieve an accuracy measure
prediction of tumor grades. For the prediction accuracy, the of 97% tested on various datasets of MRI images. One of the

VOLUME 8, 2020 55137

 N. Noreen et al.: Deep Learning Model Based on Concatenation Approach for the Diagnosis of Brain Tumor

FIGURE 1. The brain tumor dataset sample for three classes: (a) glioma,
(b) meningioma, (c) pituitary.

FIGURE 3. Proposed model of concatenation of features and

classification of brain tumor by using Pre-Trained Inception-V3.

intensity value ranging between 0 to 1. The sample of three

types of brain tumor is shown in Figure.1.

IV. PROPOSED MODEL

FIGURE 2. The complete training, testing and validation process based on The proposed model with different number of layers and pre-
proposed deep learning models.
trained models has been explained in the following sections
in details. Figure 2 shows the process of pre-processing,
limitations in exisiting approaches is feature extraction from training, testing, and prediction of brain tumor. The pro-
bottom layers of pre-trained models which are different from posed model is based on deep learning which uses different
natural images to medical images. To overcome this problem, hyper parameters for training and optimizes these parameters
a method of multi-level features extraction is proposed which during training by using loss function and Adam optimizer.
enhances the capability of the model to classify the brain Machines learn through a loss function is a way of assessing
tumor. how quite a particular algorithm models the provided data.
Progressively, loss function learns to minimize the error in
III. DATASET prediction with the aid of some optimization function. Vari-
The brain dataset investigated in this study is comprised ous loss functions are available but our problem is multiclass,
of 3064 T1-weighted contrast MR images of 233 [5]. Three so, we used Cross-Entropy loss function which is also known
different types of tumors such as meningioma, glioma, as Softmax Loss. Adam Optimizer is an optimization algo-
and pituitary are existing in this dataset. The image res- rithm that can control sparse gradients on noisy problems.
olution 512 × 512 with voxel spacing size of 0.49 × Moreover, it blends the most suitable characteristics of the
0.49 mm2 consisted of axial (transverse plane), coronal AdaGrad and RMSProp algorithms to attain optimize param-
(frontal plane), or sagittal (lateral plane) planes has been used eters.
in this dataset. The axial plane distribution based on num-
ber of classes consisted of 708, 1426, 930 sample instances A. PRE-TRAINED INCEPTION-V3
of glioma, meningioma, and pituitary tumors respectively. The proposed model of pre-trained Inception-v3 is shown
A min-max normalization method was used within pixel in Figure 3. It consists of 11 number of inception modules

55138 VOLUME 8, 2020

N. Noreen et al.: Deep Learning Model Based on Concatenation Approach for the Diagnosis of Brain Tumor

and each module comprises of convolutional layer, activation

layer, pooling layer, and batch normalization layer. These
modules are concatenated to get multiscale maximal features
from input images. We have removed most of Inception
module at bottom layers of pre-trained Inception-v3 deep
learning model and concatenated features at the bottom layers
of proposed models for classification based on brain tumor
dataset. The global average pooling, and fully connected
layers are added with the last Inception module layer along
with different Inception modules features for brain tumor
classification. The dropout layer after global average pool-
ing layer has been used for regularization, and this layer
minimized the over fitting problem for training the proposed
model. The features are extracted from fully connected layer
and after the concatenation from different Inception blocks
are passed to the classifier. The features taken from Inception
block C, D and E modules are concatenated and passed to
the traditional classifier such as softmax. The features from
Inception block A and B did not produce good performance
and simply these features are not extracted for classification
is shown in Figure 3. The deep layers Inception module such
as C, D and E Inception blocks produced better performance
and these Inception module-based features have been used for
classification and assessment of multiclass brain tumor types.
The complete process is shown in Figure 3.

B. PRE-TRAINED DENSNET201
FIGURE 4. Proposed model of concatenation of features and
The pre-trained DensNet201 has been used for feature extrac- classification of brain tumor by using Pre-trained DensNet201.
tion by using brain tumor dataset. In this network, the features
are extracted from lower dense block and upper dense block.
In DensNet201, there are four denseblocks with different
evaluation, we only used three dense blocks from pre-trained
number of convolutional layers. The idea is to extract feature
DensNet201 model which have major contribution in effi-
from lower block2, middle block3 and end denseblock4 of
cient classification of brain tumor dataset
bottom layers of the DensNet201.
After feature extraction from every block, the average
V. RESULTS
pooling layer and fully connected layer have been used for
A. PERFORMANCE METRICS
feature concatenation and then passed these concatenated
features to softmax classifier. The softmax classifier has been To validate the performance of the proposed model, the fol-
applied on multilevel or fused features extracted features lowing performance metrics are used: the accuracy, precision,
from pre-trained DensNet201 model. After feature extraction recall, and F1 score to measure true and predicted classes
from different denseblocks, these features were concatenated which have already been represented in equations 1, 2, 3,
and passed to the softmax classifier for brain tumor detection and 4, respectively. The mathematical notation of each per-
and assessment. The complete process is shown in block formance metric is shown hereunder:
diagram of Figure 4.
The feature concatenation approach produced dense mul- TP + TN
tiscale information from input images for brain tumor clas- Accuracy = (1)
TP + FN + FP + TN
sification. The Denseblock having twelve (12) number of TP
convolutional layers blocks, forty-eight (48) number of con- Precision = (2)
TP + FP
volutional layers blocks and thirty-two convolutional layers TP
blocks had been used for feature extraction and features Recal = (3)
TP + FN
concatenation. These denseblocks extracted from bottom lay- Precision × Recall
ers of pre-trained DensNet201 model produced multiscale F1 = 2 × (4)
Precision + Recall
and dense information from input images of brain tumor
dataset for brain tumor multiclass classification. The dense- Here ‘‘TP’’ describes true positive, ‘‘TN’’ represents true
block comprised of six number of convolutional layers did negative, ‘‘FP’’ indicates to false positive, and ‘‘FN’’ denotes
not provide efficient performance. Based on experimental false negative

VOLUME 8, 2020 55139

 N. Noreen et al.: Deep Learning Model Based on Concatenation Approach for the Diagnosis of Brain Tumor

TABLE 1. The precision, recall, and F1 score based on different densnet TABLE 3. The precision, recall and F1 score based on concatenation of
block features. dens_block features and inception-based features.

TABLE 4. Average accuracy of densnet, inception, and the combined

approaches.

TABLE 2. The precision, recall and F1 score based on different inception

block features.

B. RESULT EVALUATION AND ANALYSIS

Various hyper-parameters were used to train our proposed
models. The number of epochs 100, learning rate 0.0001,
20 batch size, Adam optimizer, categorical cross entropy loss
function was used to trained proposed models. Softmax clas- features produced excellent performance for all brain tumor
sifier was used for pre-training and scratch based proposed classes as shown in Table.2.
deep learning models. The Keras with backend TensorFlow Table 3 highlights the results after concatenation of
library has been used to trained deep learning models. different features with densnet and inception. The per-
The proposed models are trained by using 80 percent data formance analysis for each brain tumor class based on
for training and 20 percent for testing. The evaluation metrics concatenation of features that use different bottom layers
have been computed for estimated labels and ground truth blocks. Table 3 highlights this performance. The concatena-
labels used in brain tumor dataset. The highest accuracy tion features of different blocks of proposed Incecption-v3
achieved by our proposed model based on concatenation and DensNet201 produced excellent performance as com-
of features. The precision, recall, and F1 score based on pared to individual block feature extraction approach which
different densnet block features are shown in Table 1. The Table 1 and 2 elaborate.
performance metrics for individual class of brain tumor are Table 4 shows accuracy of densnet, inception and com-
based on proposed model individual densnet blocks features bined approaches. The overall performance of combined
approach as shown in Table1. The precision values of class features, extracted from different layers of Inception block,
glioma and meningioma shows highest performance of den- is better as compared to individual Inception E, Inception D
snet block4 based features compared to other densnet blocks and Inception C blocks. The accuracy produced by proposed
features. Similarly, the other performance metrics such as techniques, based on feature concatenation, clearly indicated
recall and F1-score for all brain tumor types show better per- the highest performance. as compared to single block (Incep-
formance based on proposed deep learning features extraction tion E, Inception D and Inception C blocks) feature extraction
approach for classification of brain tumor. methods. Similarly, the second proposed DensNet201 based
The precision, recall and F1 score based on different incep- features concatenation (dense block2, dense block3, dense
tion block features have figured out in Table 2. block4) approach shows highest performance in brain tumor
The blocks, consisted of bottom layers of Incepion-v3, classification.
have been used to extract features for brain tumor classifi- The combined approaches such as combined inception
cation. The performance analysis of various bottom layers block and combined densnet block indicate their accuracies
blocks of Incecption-v3 has been compared to classify differ- 99.34% and 99.51 % respectively. Bold fonts in Table 4 indi-
ent types of brain tumor and the features based on Inception C cate the highest accuracy among the eight models.

55140 VOLUME 8, 2020

N. Noreen et al.: Deep Learning Model Based on Concatenation Approach for the Diagnosis of Brain Tumor

TABLE 5. Comparison with other existing techniques.

FIGURE 5. The ROC plot based on proposed combined_inception_block

and combined_densnet_block which is further sub-divided into:
(a) Combined_inception_block, (b) Combined_densnet_block.

The ROC (region of convergence) in Figure 5 indicates that

proposed model based on comined_features produced excel-
lent results for each tumor class (0, 1, 2). The 0-class denoted
as glioma, 1-class denoted as meningioma, and 2-class pitu-
itary. The ROC curve for class glioma and pituitary produced
true positive rate and class meningioma is less effective clas-
sified as compared toglioma and pituitary class.
Table 5 also shows comparison among other existing
approaches. The proposed model based Densnet block FIGURE 6. The feature maps for convolutional layer1 (FM-C1) for
using combined features produced excellent performance in proposed models (DensNet, Inception).
brain tumor classification which are shown in bold fonts
in Table 5. The Inception-v3 model based features concate-
nation method indicates 99.34% accuracy while DensNet201 model which shows the reconstruction of information same
model based features concatenation approach demonstrates like to original images.
99.51% accuracy. The feature extraction map for bottom convolutional lay-
The feature map activation for convolutional layers of the ers of our proposed models provided evidence to accurately
best proposed model based on denseblocks feature map and reconstruct the brain tumor information from input images.
Inception block feature map is shown in Figure 6. The acti- The classification of brain tumor with different types has
vation map for convolutional layer has been used in proposed been discussed. The shape of glioma is different and usually

VOLUME 8, 2020 55141

 N. Noreen et al.: Deep Learning Model Based on Concatenation Approach for the Diagnosis of Brain Tumor

encircled by edema. Meningioma commonly exists near significant feature extraction as well as classification. The
the skull, and cerebrospinal fluid. The well-known pituitary major issue in MR image classification and recognition is to
tumor is very close to sphenoidal sinus and optic chiasma. lessen the gap between the high-level data recognized by a
Therefore, the discriminative features and the most relevant human evaluator and a low-level visual data taken by a MRI
information related to any brain tumor is difficult to classify machine. The effectiveness of this scheme for classification
and the features have correlation with the position of tumor problem is to extract the best features that represent low-
area in any MR image along-with its shape, size, and bound- and high-level information representation by ignoring any
ary is varying based on tumor types. handcrafted features. The CNN deep learning models extract
The deep learning based proposed models extracted are important features automatically in a hierarchical learning
useful and discriminate features in order to classify the var- approach that proves deep learning models delivered the
ious type of tumor in a complex feature’s estimation of better results. The deep learning-based models extract simple
different tumor types. The glioma and pituitary tumor class structural feature information in earlier layers particularly
recovered accurately as compared to meningioma class based edges, shape, etc and final layers encode or construct abstract
on proposed tumor classification models. The results are interpretations for specific features. Deep learning-based
validated by using the ROC curves produced by proposed models provide an excellent self-learning feature extraction
models based on predicted samples. mechanism if they are compared to manual feature extraction
which lessen the need of domain knowledge for enhancing
VI. DISCUSSION the system performance. The concatenation of fusion of high
The traditional method based on human inspection has been level and low-level pre-trained-model layers features pro-
used for the detection and classification of MRI brain tumor duced an excellent performance, if it is compared to the last
and depends on the expertise of radiologists who examine and layer pre-trained deep learning-based feature extraction. The
investigate the components of images. The operator-assisted combination of a lower layer and higher layer deep feature
classification methods are non-reproducible and unrealistic would be a better choice for assessment and classification of
for a large amount of data because manually processing brain tumors.
large scale dataset is a time-consuming process. To overcome
such problems, computer-aided diagnosis tools are required VII. CONCLUSION
to process a large amount of data efficiently. According to This paper discussed the application of deep learning mod-
applications, the brain tumor classification is subdivided into els for the identification of brain tumor. In this paper, two
two types: 1) classification of MRI into normal and abnormal different scenarios were assessed. Firstly, pre-trained Den-
tumor, 2) classification within abnormal brain tumors into sNet201 deep learning model was used, and the features
different types of tumors. Automatic classification of brain were extracted from various DensNet blocks. Then, these
tumors into various pathological types relatively is a difficult features were concatenated and passed to softmax classifier to
problem as compared to binary classification (normal and classify the brain tumor. Secondly, the features from different
abnormal) of tumors. Inception modules were extracted from pre-trained Inception-
Mostly, the conventional feature extraction methods v3 model and concatenated and then, passed to the softmax
employed for machine learning to obtain handcraft features for the classification of brain tumors. Both scenarios were
depending on high-level and low-level features. This is a evaluated with the publicly available three-class brain tumor
prime problem for a tumor analyzation using machine learn- dataset. Consequently, the ensemble method based on con-
ing algorithms. In the CE-MRI dataset, a firm association catenation of dense block by using DensNet201 pre-trained
exists between the structure of tumor and nearby healthy model outperformed as compared to the current research
tissues along with edema. The shape of glioma is different methods for brain tumor classification problem. The pro-
and usually encircled by edema. Meningioma commonly posed method produced 99.51% testing accuracy on testing
exists near the skull, and cerebrospinal fluid. The well-known samples and achieved the highest performance in detection
pituitary tumor is very close to sphenoidal sinus and optic of brain tumor. In future, we will explore and apply fine-
chiasma. Therefore, the discriminative features and the most tune techniques on pre-trained models trained with a larger
relevant information related to any brain tumor have cor- number of layers and may also scratch-based models with
relation with the position of tumor area in any MR image data augmentation techniques to classify brain tumor. We will
along-with its shape, size, and boundary. Brain tumors have also explore ensemble method (fusion of classifiers output)
large deviations in shape, size, and intensity. So, handcrafted based on fine-tune and scratch-based features extracted from
features based on traditional machine learning methods may deep learning models.
not be a reasonable solution to evoke intensity of information.
Recently deep-learning-based automatic feature extraction
REFERENCES
and classification approach performed admirably to highlight
[1] R. L. Siegel, K. D. Miller, and A. Jemal, ‘‘Cancer statistics,’’ Cancer J.
computer-aided medical works. Deep Learning discovered Clin., vol. 65, no. 1, pp. 5–29, 2015.
notable features in a self-learning mode with the demand [2] R. Siegel, C. R. Miller, and A. Jamal, ‘‘Cancer statistics,’’ Cancer J. Clin.,
of a minimum data pre-processing in the process of most vol. 67, no. 1, pp. 7–30, 2017.

55142 VOLUME 8, 2020

N. Noreen et al.: Deep Learning Model Based on Concatenation Approach for the Diagnosis of Brain Tumor

[3] Brain Tumor Statistics, American Brain Tumor Association. Accessed: [23] S. Pereira, R. Meier, V. Alves, M. Reyes, and C. A. Silva, ‘‘Automatic brain
Oct. 26, 2019. [Online]. Available: http://abta.pub30.convio.net/ tumor grading from MRI data using convolutional neural networks and
[4] M. I. Razzak, M. Imran, and G. Xu, ‘‘Efficient brain tumor segmentation quality assessment,’’ in Understanding and Interpreting Machine Learning
with multiscale Two-Pathway-Group conventional neural networks,’’ IEEE in Medical Image Computing Applications. Cham, Switzerland: Springer,
J. Biomed. Health Informat., vol. 23, no. 5, pp. 1911–1919, Sep. 2019, 2018, pp. 106–114.
doi: 10.1109/JBHI.2018.2874033. [24] A. Kabir Anaraki, M. Ayati, and F. Kazemi, ‘‘Magnetic resonance imaging-
[5] A. Rehman, S. Naz, M. I. Razzak, F. Akram, and M. Imran, based brain tumor grades classification and grading via convolutional neu-
‘‘A deep learning-based framework for automatic brain tumors classifica- ral networks and genetic algorithms,’’ Biocybern. Biomed. Eng., vol. 39,
tion using transfer learning,’’ Circuits, Syst., Signal Process., vol. 39, no. 2, no. 1, pp. 63–74, Jan. 2019, doi: 10.1016/j.bbe.2018.10.004.
pp. 757–775, Sep. 2019, doi: 10.1007/s00034-019-01246-3. [25] Y. Zhou, Z. Li, H. Zhu, C. Chen, M. Gao, K. Xu, and J. Xu, ‘‘Holistic
[6] S. Deepak and P. M. Ameer, ‘‘Brain tumor classification using deep CNN brain tumor screening and classification based on DenseNet and recurrent
features via transfer learning,’’ Comput. Biol. Med., vol. 111, Aug. 2019, neural network,’’ in Proc. Int. MICCAI Brainlesion Workshop. Cham,
Art. no. 103345, doi: 10.1016/j.compbiomed.2019.103345. Switzerland: Springer, 2018, pp. 208–217.
[7] M. Sajjad, S. Khan, K. Muhammad, W. Wu, A. Ullah, and S. W. Baik, [26] K. V. A. Muneer, V. R. Rajendran, and J. K. Paul, ‘‘Glioma tumor grade
‘‘Multi-grade brain tumor classification using deep CNN with extensive identification using artificial intelligent techniques,’’ J. Med. Syst., vol. 43,
data augmentation,’’ J. Comput. Sci., vol. 30, pp. 174–182, Jan. 2019, no. 5, p. 113, Mar. 2019.
doi: 10.1016/j.jocs.2018.12.003. [27] S. Banerjee, S. Mitra, F. Masulli, and S. Rovetta, ‘‘Deep radiomics for
[8] J. Cheng, Brain Tumor Dataset. Figshare. Dataset. Accessed: brain tumor detection and classification from multi-sequence MRI,’’ 2019,
Sep. 19, 2019. [Online]. Available: https://doi.org/10.6084/m9.figshare. arXiv:1903.09240. [Online]. Available: http://arxiv.org/abs/1903.09240
1512427.v5
[9] Y. Gu, X. Lu, L. Yang, B. Zhang, D. Yu, Y. Zhao, and T. Zhou, ‘‘Automatic
lung nodule detection using a 3D deep convolutional neural network
combined with a multi-scale prediction strategy in chest CTs,’’ Comput,’’
Comput. Biol. Med., vol. 103, pp. 220–231, Dec. 2018. NEELUM NOREEN received the B.S. degree in computer science and the
[10] M. Yousefi, A. Krzyżak, and C. Y. Suen, ‘‘Mass detection in digi- Bachelor in Education degree from Hazara University, Mansehra, Pakistan,
tal breast tomosynthesis data using convolutional neural networks and in 2007 and 2009, respectively, and the M.S. degree in computer science
multiple instance learning,’’ Comput. Biol. Med., vol. 96, pp. 283–293, from the COMSATS Institute of Information Technology (COMSATS Uni-
May 2018. versity), Abbottabad, Pakistan, in 2011. She is currently a Ph.D. Scholar of
[11] A. Naseer, M. Rani, S. Naz, M. I. Razzak, M. Imran, and G. Xu, ‘‘Refin- informatics with the Information Technology Department, School of Science
ing Parkinson’s neurological disorder identification through deep trans- and Engineering, Malaysia University of Science and Technology, Malaysia.
fer learning,’’ Neural Comput. Appl., vol. 32, pp. 839–854, Feb. 2019, She has been a Faculty Member and a Research Assistant with various
doi: 10.1007/s00521-019-04069-0. institutes and universities. She has published articles in reputed journals
[12] H. Zuo, H. Fan, E. Blasch, and H. Ling, ‘‘Combining convolutional and and conferences. Her research interests include image processing, machine
recurrent neural networks for human skin detection,’’ IEEE Signal Process.
learning, and brain tumor classification based on MRI.
Lett., vol. 24, no. 3, pp. 289–293, Mar. 2017.
[13] O. Charron, A. Lallement, D. Jarnet, V. Noblet, J. B. Clavier, and P. Meyer,
‘‘Automatic detection and segmentation of brain metastases on multimodal
MR images with a deep convolutional neural network,’’ Comput. Biol.
Med., vol. 95, pp. 43–54, Apr. 2018.
SELLAPPAN PALANIAPPAN received the bach-
[14] L. Shao, F. Zhu, and X. Li, ‘‘Transfer learning for visual categoriza-
tion: A survey,’’ IEEE Trans. Neural Netw. Learn. Syst., vol. 26, no. 5, elor’s degree in statistics from the University of
pp. 1019–1034, May 2015. Malaya, the M.Sc. degree in computer science
[15] L. Zhou, Z. Zhang, Y. C. Chen, Z. Y. Zhao, X. D. Yin, and H. B. Jiang, from the University of London, U.K., and the
‘‘A deep learning-based radiomics model for differentiating benign and Ph.D. degree in interdisciplinary information sci-
malignant renal tumors,’’ Transl. Oncol., vol. 12, no. 2, pp. 292–300, ence from the University of Pittsburgh (USA).
2019. He is currently a Professor of IT, the Dean of
[16] E. Deniz, A. Şengür, Z. Kadiroğlu, Y. Guo, V. Bajaj, Ü. Budak, ‘‘Trans- the School of Science and Engineering, and the
fer learning based histopathologic image classification for breast cancer Provost of the Malaysia University of Science and
detection,’’ Health Inf. Sci. Syst., vol. 6, no. 1, p. 18, 2018. Technology. His research interests include data
[17] C. Tan, F. Sun, T. Kong, W. Zhang, C. Yang, and C. Liu, ‘‘A science, social media analytics, health informatics, blockchain, cloud com-
survey on deep transfer learning,’’ in Proc. Int. Conf. Artif. Neural puting, web services, cybersecurity, the IoT, and education 4.0.
Netw. Cham, Switzerland: Springer, 2018, pp. 270–279. [Online].
Available: https://link.springer.com/book/10.1007/978-3-030-01418-6,
doi: 10.1007/978-3-030-01424-7_27.
[18] M. Talo, U. B. Baloglu, and U. R. Acharya, ‘‘Application of deep transfer
learning for automated brain abnormality classification using MR images,’’ ABDUL QAYYUM received the bachelor’s degree
Cogn. Syst. Res., vol. 54, pp. 176–188, May 2019. in computer engineering from the Comsats Insti-
[19] P. Afshar, A. Mohammadi, and K. N. Plataniotis, ‘‘Brain tumor type clas- tute of Information Technology, Lahore, Pakistan,
sification via capsule networks,’’ in Proc. 25th IEEE Int. Conf. Image Pro- in 2007, the M.S. degree in electronics engineering
cess. (ICIP), Oct. 2018, pp. 3129–3133, doi: 10.1109/ICIP.2018.8451379. from International Islamic University, Pakistan,
[20] N. Abiwinanda, M. Hanif, S. T. Hesaputra, A. Handayani, and in 2012, and the Ph.D. degree in electrical and
T. R. Mengko, ‘‘Brain tumor classification using convolutional
electronics engineering with specialization in deep
neural network,’’ in Proc. World Congr. Med. Phys. Biomed.
learning and image processing from Universiti
Eng. Singapore: Springer, 2019, pp. 183–189. [Online]. Available:
https://www.springerprofessional.de/en/brain-tumor-classification-using- Teknologi Petronas Malaysia, in 2017. He is cur-
convolutional-neural-network/15802612 rently working as a Postdoctoral Researcher with
[21] M. R. Ismael and I. Abdel-Qader, ‘‘Brain tumor classification via statistical the ImViA Laboratory, University of Bourgogne Franche-Comté, Dijon,
features and back-propagation neural network,’’ in Proc. IEEE Int. Conf. France. He developed prototype for vital sign estimation, assessment of
Electro/Inf. Technol. (EIT), May 2018, pp. 0252–0257. stroke and arterial fibrillation using deep learning approach based on face
[22] A. Pashaei, H. Sajedi, and N. Jazayeri, ‘‘Brain tumor classification video analytical approach. His areas of interest are machine learning and
via convolutional neural network and extreme learning machines,’’ deep learning for signal processing, e.g., EEG, ECG, and biomedical images
in Proc. 8th Int. Conf. Comput. Knowl. Eng. (ICCKE), Oct. 2018, (Cardiac MRI, Liver CT, Prostate, and Kidney Tumor) segmentation and
pp. 314–319. classification.

VOLUME 8, 2020 55143

 N. Noreen et al.: Deep Learning Model Based on Concatenation Approach for the Diagnosis of Brain Tumor

IFTIKHAR AHMAD received the B.Sc. degree MUHAMMAD SHOAIB received the B.Eng. and
from Islamia University, Bahawalpur, Pakistan, M.Eng. degrees from the NED University of Engi-
in 1999, the M.Sc. degree in computer science neering and Technology, Karachi, and the Ph.D.
from the University of Agriculture, Faisalabad, degree in communication and information system
Pakistan, in 2001, the M.S./M.Phil. degrees in from the Beijing University of Posts and Telecom-
computer science from the COMSATS Institute munications, China, in 2010. He was a Senior
of Information Technology, Abbottabad, Pakistan, Manager (IP Operations, South) with Pakistan
in 2007, and the Ph.D. degree in information tech- Telecommunication Company Limited, Pakistan.
nology from Universiti Teknologi PETRONAS, He was a Maintenance Engineer with R. M. Inter-
Malaysia, in 2011. He has been a Faculty Member national. He is currently working as an Assistant
and a Research Supervisor with various universities, since 2001. He is Professor with the College of Computer and Information Sciences (Infor-
currently a Faculty Member with the Information Technology Department, mation Systems Department), King Saud University. His areas of research
Faculty of Computing and Information Technology, King Abdulaziz Univer- include video compression techniques, multilayer video coding, commer-
sity, Jeddah, Saudi Arabia. He has been involved in several funded projects cial data center facilities and IP packet based networks, infrastructure, and
as PI and Co-PI. He has published several articles in reputed journals and security.
conferences. He is also a member of several scientific and professional
bodies.

MUHAMMAD IMRAN received the Ph.D.

degree in information technology from Univer-
sity Teknologi PETRONAS, Malaysia, in 2011.
He is currently an Associate Professor with the
College of Applied Computer Science, King Saud
University, Saudi Arabia. His research interests
include the Internet of Things, mobile and wireless
networks, big data analytics, cloud computing, and
information security. His research is financially
supported by several grants. He has completed a
number of international collaborative research projects with reputable uni-
versities. He has published more than 150 research articles in top conferences
and journals. He has been involved in more than seventy conferences and
workshops in various capacities, such as the Chair, the Co-Chair, and a
Technical Program Committee Member. European Alliance for Innovation
(EAI) has appointed him as the Editor in Chief of EAI Transactions on
Pervasive Health and Technology. He serves as an Associate Editor for
reputable international journals, such as IEEE COMMUNICATIONS MAGAZINE,
Future Generation Computer Systems, IEEE ACCESS, Ad Hoc & Sensor
Wireless Networks (Journal) (SCIE), IET Wireless Sensor Systems, the Inter-
national Journal of Autonomous and Adaptive Communication Systems
(Inderscience). He served/serving as a Guest Editor for more than a dozen
special issues in journals, such as IEEE COMMUNICATIONS MAGAZINE, Com-
puter Networks (Elsevier), Future Generation Computer Systems (Elsevier),
Sensors (MDPI), the International Journal of Distributed Sensor Networks
(Hindawi), the Journal of Internet Technology, and the International Journal
of Autonomous and Adaptive Communications Systems.

55144 VOLUME 8, 2020