Since January 2020 Elsevier has created a COVID-19 resource centre with

free information in English and Mandarin on the novel coronavirus COVID-

19. The COVID-19 resource centre is hosted on Elsevier Connect, the
company's public news and information website.

Elsevier hereby grants permission to make all its COVID-19-related

research that is available on the COVID-19 resource centre - including this
research content - immediately available in PubMed Central and other
publicly funded repositories, such as the WHO COVID database with rights
for unrestricted research re-use and analyses in any form or by any means
with acknowledgement of the original source. These permissions are
granted for free by Elsevier for as long as the COVID-19 resource centre
remains active.
 Computers in Biology and Medicine 139 (2021) 105014

Contents lists available at ScienceDirect

Computers in Biology and Medicine

journal homepage: www.elsevier.com/locate/compbiomed

A deep learning approach using effective preprocessing techniques to detect

COVID-19 from chest CT-scan and X-ray images
Khabir Uddin Ahamed a, Manowarul Islam, PhD a, **, Ashraf Uddin a, Arnisha Akhter a,
Bikash Kumar Paul b, Mohammad Abu Yousuf c, Shahadat Uddin d, Julian M.W. Quinn e,
Mohammad Ali Moni, PhD e, f, *
a
Department of Computer Science and Engineering, Jagannath University, Dhaka, Bangladesh
b
Department of Information and Communication Technology, Mawlana Bhashani Science and Technology University, Bangladesh
c
Institute of Information Technology, Jahangirnagar University, Dhaka, Bangladesh
d
Complex Systems Research Group, Faculty of Engineering, The University of Sydney, Darlington, NSW, 2008, Australia
e
Healthy Ageing Theme, Garvan Institute of Medical Research, Darlinghurst, NSW, 2010, Australia
f
Artificial Intelligence & Digital Health Data Science, School of Health and Rehabilitation Sciences, Faculty of Health and Behavioural Sciences, The University of
Queensland, St Lucia, QLD, 4072, Australia

A R T I C L E I N F O A B S T R A C T

Keywords: Coronavirus disease-19 (COVID-19) is a severe respiratory viral disease first reported in late 2019 that has spread
Coronavirus worldwide. Although some wealthy countries have made significant progress in detecting and containing this
Respiratory diseases disease, most underdeveloped countries are still struggling to identify COVID-19 cases in large populations. With
Convolutional neural network
the rising number of COVID-19 cases, there are often insufficient COVID-19 diagnostic kits and related resources
Pneumonia
in such countries. However, other basic diagnostic resources often do exist, which motivated us to develop Deep
Deep learning
Rediology Learning models to assist clinicians and radiologists to provide prompt diagnostic support to the patients. In this
study, we have developed a deep learning-based COVID-19 case detection model trained with a dataset con­
sisting of chest CT scans and X-ray images. A modified ResNet50V2 architecture was employed as deep learning
architecture in the proposed model. The dataset utilized to train the model was collected from various publicly
available sources and included four class labels: confirmed COVID-19, normal controls and confirmed viral and
bacterial pneumonia cases. The aggregated dataset was preprocessed through a sharpening filter before feeding
the dataset into the proposed model. This model attained an accuracy of 96.452% for four-class cases (COVID-
19/Normal/Bacterial pneumonia/Viral pneumonia), 97.242% for three-class cases (COVID-19/Normal/Bacterial
pneumonia) and 98.954% for two-class cases (COVID-19/Viral pneumonia) using chest X-ray images. The model
acquired a comprehensive accuracy of 99.012% for three-class cases (COVID-19/Normal/Community-acquired
pneumonia) and 99.99% for two-class cases (Normal/COVID-19) using CT-scan images of the chest. This high
accuracy presents a new and potentially important resource to enable radiologists to identify and rapidly di­
agnose COVID-19 cases with only basic but widely available equipment.

1. Introduction world since it was reported in Wuhan, China [1,2]. COVID-19 has spread
rapidly through human-to-human transmission since transmission of the
In 2019, the COVID-19 pandemic appeared as a dangerous infectious virus can occur well before symptoms are evident. Over 130 million
disease caused by the SARS-CoV-2 virus that can result in severe respi­ people worldwide have been infected at the time of writing [3], repre­
ratory distress. The disease has caused millions of fatalities around the senting an enormous healthcare burden.

* Corresponding author. Artificial Intelligence & Digital Health Data Science, School of Health and Rehabilitation Sciences, Faculty of Health and Behavioural
Sciences, The University of Queensland, St Lucia, QLD, 4072, Australia.
** Corresponding author.
E-mail addresses: monzilahamed321@gmail.com (K.U. Ahamed), manowar@cse.jnu.ac.bd (M. Islam), ashraf@cse.jnu.ac.bd (A. Uddin), arnisha@cse.jnu.ac.bd
(A. Akhter), bikash@mbstu.ac.bd (B.K. Paul), yousuf@juniv.edu (M.A. Yousuf), shahadat.uddin@sydney.edu.au (S. Uddin), j.quinn@garvan.org.au
(J.M.W. Quinn), m.moni@uq.edu.au (M.A. Moni).

https://doi.org/10.1016/j.compbiomed.2021.105014
Received 13 October 2021; Received in revised form 1 November 2021; Accepted 1 November 2021
Available online 4 November 2021
0010-4825/© 2021 Elsevier Ltd. All rights reserved.
K.U. Ahamed et al. Computers in Biology and Medicine 139 (2021) 105014

accuracy in COVID-19 predictions, an approach that may be able to

complement RT-PCR tests and would be highly accessible. It may also be
possible to detect early mild symptomatic cases that give false negative
RT-PCR results due to low viral particle numbers in the upper respira­
tory tract. Two kinds of chest radiographic images have been used for
this approach: X-ray & computed tomography (CT) [15,16]. CT scans
provide very fine detail but require substantially more radiation expo­
sure than X-ray images and require high cost equipment [17]. X-ray
images are more accessible to a wider population through widely
available X-ray imaging facilities at low cost. CT scans can, however,
provide finer image features in 3 dimensions and can feasibly be used to
train a deep learning model to identify COVID-19 as they give very rich
datasets. Therefore, we investigated deep learning models trained with
datasets consisting of chest CT-scan and X-ray to determine whether this
would be a viable alternative to RT-PCR in detecting or confirming
COVID-19 cases.
In this study, we review the literature related to chest image classi­
fications. We have also investigated how effective deep learning ap­
proaches that employ chest X-ray and CT-scan images can classify
Fig. 1. Ground Glass Opacity was identified in the left middle to lower lung potential COVID-19 cases. The proposed architecture was able to
opacity (shown by a white arrow inside the red circle) [33]. (For interpretation distinguish cases of COVID-19 from pneumonia cases and normal con­
of the references to colour in this figure legend, the reader is referred to the trols with a high level of accuracy. We propose that such deep CNN-
Web version of this article.) based models trained on CT-scan and X-ray images could assist radiol­
ogists to make rapid, low cost and accessible diagnosis usefully early
Infections by SARS-CoV cause symptoms broadly similar to those detection of infected patients at an early stage in the disease progression.
caused by the related severe acute respiratory syndrome coronavirus The following is a summary of the main contributions:
(SARS-CoV) and Middle East respiratory syndrome coronavirus
(MERS-CoV), and range in severity from mild upper respiratory tract 1. We employed a preprocessing technique on the image dataset to
symptoms similar resembling a common cold to severe acute life- enable the dataset to be accurately and efficiently analysed by our
threatening illness [4]. Among the latter symptoms include fever, deep learning model.
headache, dry coughing and sore throat, severe pneumonia and acute 2. We developing an extended ResNet50V2-based deep learning model
respiratory distress syndrome(ARDS) that can often include severe where fine-tuning was performed to facilitate rapid detection and
hypoxia, and serious exacerbation of existing chronic pulmonary and diagnosis of COVID-19 cases with high accuracy.
respiratory conditions are often seen [5]. This virus has caused major 3. Classifying COVID-19 patient images from normal and typical
public health and economic problems and is particularly dangerous to pneumonia cases by considering two, three and four class categories
people with certain comorbidities such as diabetes, cardiovascular dis­ respectively.
ease and asthma [6–13]. 4. We conducted a comparative performance analysis of our proposed
Many early symptoms of COVID-19 are similar to those of the com­ methodology with other state-of-the-art approaches and showed that
mon cold and influenza making detection of early stage COVID-19 cases our model can identify COVID-19 cases with high accuracy using
problematic. Vaccines specific for SARS-CoV-2 have been developed and chest CT-scan and X-ray image datasets.
have been widely employed, reducing infection rates and greatly
improving patient survival, although many poorer countries have only The rest of this article is arranged as follows: Section 2 provides a
recently begun to vaccinate in significant numbers, so waves or surges of comprehensive overview of the studies in the field. The methodology of
infections are still being experienced. No other medication with high the proposed work is presented in Section 3. Experimental results with
efficacy against COVID-19 has been developed, although a number of discussion and dataset description are presented in Section 4. In Section
anti-inflammatory drugs and other re-purposed drugs have proved 5, we present our conclusions and proposals for future development
useful in reducing disease severity. Newly approved antivirals such as work.
Merck’s Molnupiravir are showing promise but currently are very
expensive. These factors mean that spread of SARS-CoV-2 is hard to 2. Related works
monitor, detect and overcome in less developed countries, particularly
with the emergence of newer more infectious strains. COVID-19 testing kits give a significant number of FN (false nega­
Reverse transcription-polymerase chain reaction(RT-PCR) is tive) so ideally need supplementing with an alternative rapid method to
currently the most commonly used for diagnosing COVID-19 patients, detect lower respiratory tract issues. This can be based on chest x-ray
and is the only fully reliable method for detecting early stage (pre- and CT scan imaging. In addition, in locations where RT-PCR testing is
symptom) SARS-CoV-2 infections. Cheaper antibody-based rapid tests not available due to cost and lack of expertise availability, chest x-ray
are not reliable for early stage infections or those in immunosuppressed imaging often is available [18]. This makes it appropriate to use in
people. SARS-CoV-2 RT-PCR-based tests respiratory swabs from naso­ COVID-19 diagnosis if the patient has begun to progress with the disease
pharyngeal or oropharyngeal. Although these may fail to identify and a differential diagnosis is needed to exclude other respiratory
COVID-19 cases in the early phases when viral load is low in the sampled ailments.
tissues [14]. However, a more significant issue is that RT-PCR is AI has proved effective for classifying a range of human activities
expensive, requires highly developed facilities and technical expertise [19] and AI use in the healthcare sector has risen dramatically in recent
that in many countries there is limited accessibility outside of large years, particularly in medical imaging technology. In imaging AI has
towns. particularly been used in the detection of cardiovascular diseases [20]
Several researchers have previously shown that deep learning and brain tumours [21]. In addition to these applications, there is now a
models trained on standard chest radiography images yield good move to use it in the diagnosis of COVID-19 cases, with CT scans and
chest X-rays a common focus.

2
K.U. Ahamed et al. Computers in Biology and Medicine 139 (2021) 105014

Fig. 2. Block Diagram of Proposed methodology.

There have been a number of suggestions that COVID-19 testing Resnet101 and MobilenetV2. These models are integrated to form a
should be performed using RT-PCR methods and a machine learning stack-like architecture. Furthermore, they used three separate classes for
analysis of chest imaging to accurately confirm COVID-19 cases, image classification and attained a higher accuracy. Jain et al. [41]
although the latter approach is unlikely to be suitable for rapid mass employed an unbalanced database of three groups of 1345 normal cases,
screening [22–27]. Studies have identified key changes that can be seen 3632 pneumonia cases, and 490 COVID-19 cases. They tested three ar­
in patient chest X-ray and CT-scan images after COVID-19 has developed chitectures, including Xception net, ResNeXt and Inception net V3, and
[28]. Yoon et al. [29] stated that one in every three patients had a single found that the Xception model had the highest accuracy of 97.97%. In
nodular opacity on the left side of the lower lung. Kong et al. [30] re­ Ref. [42], an AlexNet with a combination of SVM framework was pro­
ported that opacities of infrahilar airways were detected in the right side vided, with fine-tuning on the proposed design to distinguish COVID-19
in a COVID patient. However, most studies have found that ground glass instances from pneumonia and normal cases. Ouchicha et al. [43] pre­
opacity (GGO) is the most frequent finding in COVID-19 case lung im­ sented a deep CNN model called CVDNet to distinguish COVID-19 in­
ages. Widespread inflammation, thrombi and vascular ischaemic lesions fections from pneumonia and normal cases. To validate their system,
and other intrusive lung illnesses are associated with GGO [31]. GGO they utilized a 5-fold cross-validation technique. Ozturk et al. [33]
and blended GGO were discovered in the majority of patients, as well as presented an automated identification technique (DarkCovidNet) that
vascular dilatation with consolidation in the affected area, according to worked on 2-class classification categories (COVID-19 cases vs. Normal)
Zhao et al. [22]. Kanne et al. [32] observed that 50–75% patients had and a multi-class classification category (COVID-19 cases, Normal and
multifocal GGO as well as peripheral focal impacts on both lungs. Fig. 1 Pneumonia cases) with a binary class accuracy of 98.08%. However,
shows an example image of a GGO case. pre-processing actions on X-ray images were not considered in their
Researchers in this area have suggested a number of deep learning study.
architectures that can be trained with radiographic images to extract For the detection of COVID-19, a framework based on the CNN
COVID-19 status since radiography images are readily available. This is technique was applied in Ref. [44]. Ghavami et al. [45] suggested a
similar to recent work on skin cancer identification from images [34], comprehensible artificial intelligent approach for identifying COVID-19
pneumonia detection utilizing chest X-ray images [35], brain disorder patients by considering COVID-19 patients, healthy patients, and
classification [36], lung segmentation [37], detection of myocardial non-COVID-19 lung infections utilizing chest CT scan data. A combined
infarction [38]. These are just a few of the areas where Deep Learning deep transfer learning architecture was presented [46] where they used
methods have already been developed. 15 pre-trained convolutional neural networks (CNNs) models using
Chen et al. [39] designed a VGG-16 deep transfer learning approach chest CT-scan images. Li et al. [47] proposed a stacked auto-encoder
for identifying COVID-19 in chest X-ray images, which took into account based framework to classify covid-19 positive cases from the negative
two classes: COVID-19 and Non-COVID-19. Gupta et al. [40] proposed cases but they used a low amount of image data to train their model.
InstaCovNet-19 that is an integrated stacking model. They used a variety Heidarian et al. [48] proposed a framework called COVID-fact that was
of transfer learning frameworks, such as Nasnet, InceptionV3, Xception, trained with an unbalanced data set with three class categories,

3
K.U. Ahamed et al. Computers in Biology and Medicine 139 (2021) 105014

Fig. 3. Illustration of different filters a. Generated mask b. Variant of Laplacian filter c. Sharpening filter.

Fig. 4. Prepossessing operations on Chest CT-scan & X-ray images.

including covid-19, pneumonia cases and normal cases. VGG19 model. The model showed the highest accuracy (91%). Sedik
Using CT-scan images, Xu et al. [49] proposed an architecture called et al. [54] combined machine learning and deep learning algorithms to
”ResNet18 + Location attention” for identifying COVID-19 cases. identify and diagnose covid-19 cases from chest x-ray and ct-scan im­
Despite their efficient architecture, their overall accuracy level was only ages. The authors applied two methods of data-augmentation that
86.7%, which was insufficient. improve the learnability of the Convolutional Long Short-Term Memory
Some studies have used a combination of chest CT scans and X-ray and Convolutional Neural Networks based deep learning models.
images to identify and treat COVID-19. Using a balanced dataset of chest However, using machine learning (ML) methods has several limitations,
CT-scan and X-ray images, Mukherjee et al. [50] suggested a CNN-based including complexity, overfitting and poor performance while training
customized deep neural network (DNN) with extensive accuracy of with unbalanced datasets.
96.13% for X-ray and 95.83% for CT-scan. For identifying and diag­ The majority of the studies reviewed above used imbalanced datasets
nosing COVID-19, a standard tuned VGG-19 model was reported in as well as a small database of COVID-19 cases to train various machine
Ref. [51] utilizing chest CT-scan & X-ray images. Ahsan et al. [52] learning models. Datasets of small size are likely to result in a CNN-
proposed a pre-trained method to distinguish COVID-19 cases from based framework overfitting. As a result, the model would not report
non-COVID-19 cases obtaining an accuracy of 82.94% for CT-scan genuine and reliable classification performance outside the training
sample data and 93.94 for chest x-ray sample data. datasets. Furthermore, many existing methods are trained and tested
To distinguish COVID-19 from streptococcus and SARS virus in­ with raw images without any type of pre-processing, and augmentation.
fections, Dansana et al. [53] used an unbalanced dataset of x-ray and Thus, the network’s generalization error is increased, and the training
CT-scan images. They employed a tuned pre-trained VGG19 model, a advantages are minimized. Moreover, the majority of the studies have
pre-trained InceptionV3, and a decision tree classifier with the tuned used pre-trained approaches and trained their architectures using three

4
K.U. Ahamed et al. Computers in Biology and Medicine 139 (2021) 105014

Fig. 5. Samples of augmented images of Chest CT-scan & X-ray.

or two classes. To overcome these issues, we constructed a balanced

dataset and performed pre-processing and augmentation operations on
the collected images instead of using raw images. Furthermore, in our
study, a traditional deep learning model was modified and developed by
fine-tuning effectively and optimizing the hyper-parameters to improve
model robustness. In addition, multi-class comparisons among the image
classes, such as four-class, three-class, and two-class categories made the
suggested study more effective.

3. The proposed COVID-19 case detection model

This section describes data preprocessing, the CNN model and the
proposed architecture which was implemented before conducting ex­
periments to assess performance. The overview of the proposed meth­
odology is illustrated in Fig. 2. In this block diagram, chest CT-scan and
X-ray images are used as input for the suggested COVID-19 case detec­
tion method. Next, the collected images are preprocessed by considering
resizing, cropping and filtering techniques. After that, augmentation of
the image data is performed. Finally, the model was trained and tested
with the clinical datasets. The proposed model was built from the base
deep learning model named ResNet50V2. In this study, this model was
developed by adding extra layers to its base network. The added layers
were modified by applying regularization and effective fine-tuning
processes to make the experiment more robust and efficient. More­
over, this model can classify the images based on binary class, three class
and four class categories.

Fig. 6. The overview of Residual connection. 3.1. Data preprocessing

The original size of the collected images had different pixels for
different images. The inputs for the pre-trained models on ImageNet will

5
K.U. Ahamed et al. Computers in Biology and Medicine 139 (2021) 105014
Fig. 7. Proposed architecture.
6
K.U. Ahamed et al. Computers in Biology and Medicine 139 (2021) 105014

Fig. 8. Flattening operation..

Fig. 9. A sample of dropout operation.

A sharpening filter was then used to filter all of our collected images
Table 1 for enhancement. The concept of this filter comes from Laplacian filters
Summary of the proposed model based on four-class classification. picture or image highlights the regions of rapid intensity change and is
Layer (type) Output Shape Param # an illustration of a 2nd order or 2nd a derivative system of enhancement
[55]. This can be traditionally derived according to equation (1).
resnet50v2 (Functional) (None, 7, 7, 2048) 23564800
flatten (Flatten) (None, 100352) 0
∂2 f (x, y) ∂2 f (x, y)
(1)
2
dropout (Dropout) (None, 100352) 0 ▽f = +
dense (Dense) (None, 256) 25690368 ∂x2 ∂y2
dense_1 (Dense) (None, 4) 1028
Here.
Total parameters: 49,256,196
Trainable parameters: 49,210,756
∂2f (x,y)
Non-trainable parameters: 45,440
∂x2 = f(x + 1, y) + f(x − 1, y) − 2f(x, y)
∂2f (x,y)
∂y2 = f(x, y + 1) + f(x, y − 1) − 2f(x, y)
be smaller or equal to 224*224. In the case of transfer learning, the
inputs must be suited to the pre-trained model. Thus, for rigorous Now, from equation (1) we get the following output presented in
investigation purposes, all images were scaled down to 224*224 pixels equation (2).
to make the training model faster.
(2)
2
▽ f = [f (x + 1, y) + f (x − 1, y) + f (x, y + 1) + f (x, y − 1)] − 4f (x, y)
3.1.1. Cropping and filtering of images
From equation (2), a mask can be generated. Hence, the desired mask
In this study, cropping and sharpening filters of image processing
is represented in Fig. 3 (a). Besides that mask, other types of Laplacian
techniques were applied to the collected datasets to enhance the images
mask/filter also exist [56]. In this study, one of the variants of the
before feeding those images into the CNN model. However, cropping
Laplacian filter is used. The filter used in this study is shown in Fig. 3 (b).
methods were only applied on the CT scan images in order to extract the
From the given Laplacian filter, the intensity value is measured as the
main part of the lung image, i.e., removing the unwanted irrelevant
sum of center point of the mask along with the rest of the point that is
parts of the images. The CT scan images were cropped by considering the
computed as follows: W5 + (W1 + W2 + W3 + W4 + W6 + W7 + W8 +
proper height and width ratio. Fig. 4 shows the original and cropped
W9). Here, the intensity value “0” is found from the mask by adding the
image of the CT scan.

7
K.U. Ahamed et al. Computers in Biology and Medicine 139 (2021) 105014

Fig. 10. Sample images from the collected dataset a. Chest X-ray images of different types of cases of interest: COVID-19, normal (control), pneumonia with bacterial
infection and Pneumonia with viral infection. b. Chest CT-scan images: COVID-19, normal (control) and community acquired pneumonia cases.

Fig. 11. Schematic illustration of five -fold cross validation approach.

centre value and the other corresponding values. Again, when an orig­ shows the changes of the original image after applying the Laplacian
inal image is filtered through this mask, this produces a dark image mask. The original image can then be generated using the following
where only the edge of the image is found for the 0-intensity value. Fig. 4 rules given in equations (3) and (4).

8
K.U. Ahamed et al. Computers in Biology and Medicine 139 (2021) 105014

Table 2 2
g(x, y) = f (x, y) − ▽ f ; W5 < 0 (3)
Performance of the proposed model on each fold using four, three and two class
categories.
(4)
2
g(x, y) = f (x, y) + ▽ f ; W5 > 0
Class Folds Precision Recall F1- Accuracy
Score (%) Here, g (x, y) represents the output filter after performing the ex­
Four (covid-19/ Fold1 0.9625 0.9625 0.9625 96.09
pected operation. Therefore, if the center value of the Laplacian filter is
bacterial Fold2 0.9725 0.97 0.97 96.96 less than zero then it follows equation (3) and equation (4) otherwise.
pneumonia/ Fold3 0.97 0.9675 0.9675 96.85 Let us assume for the point f(x, y) → W5 and W1, W2………W9 for the
normal/viral Fold4 0.9675 0.965 0.9625 96.52 other corresponding points. By considering the corresponding Laplacian
pneumonia) Fold5 0.9625 0.9575 0.96 95.84
mask, from equation (4), equation (5) can be written.
Average 0.967 0.9645 0.9645 96.452

Three (covid-19/ Fold1 0.9767 0.9767 0.9767 97.54 g(x, y) = 9W5 − W1 − W2 − W3 − W4 − W6 − W7 − W8 − W9 (5)
bacterial Fold2 0.9833 0.9833 0.9833 98.12
pneumonia/ Fold3 0.9767 0.98 0.9767 97.68
Now from equation (5), the generated mask or filter is presented in
normal) Fold4 0.9467 0.9433 0.94 94.19 Fig. 3 (c). The generated filter is also called the sharpening filter. This
Fold5 0.9867 0.9867 0.9867 98.68 filter is used to sharpen as well as highlight the edges of the images.
Average 0.97402 0.974 0.97268 97.242 Besides this, it makes a transition between features, more recognizable
Two (covid-19/viral Fold1 0.98 0.98 0.98 98.26 and obvious compared to smooth noisy and blurry images.
pneumonia) Fold2 0.995 0.995 0.99 99.35
Fold3 0.985 0.985 0.99 98.70
Fold4 1.00 1.00 1.00 99.78 3.2. Data augmentation
Fold5 0.985 0.985 0.99 98.68
Average 0.989 0.989 0.99 98.954
Instead of gathering new data, practitioners can use data augmen­
tation to significantly boost the diversity of the data samples for the

Fig. 12. Classification performance results of 4-class using fold-3 chest x-ray dataset.

9
K.U. Ahamed et al. Computers in Biology and Medicine 139 (2021) 105014

Fig. 13. Classification performance results on 3-class using fold-3 chest x-ray datasets.

training models. Image augmentation approaches may help to reduce 3.3.1. Convolutional layer
network generalization errors, improve training amenities, and address The convolution layer is the main building block of a CNN (con­
data overfitting concerns. In this article, augmentation methods [57] on volutional neural network). Rather than basic matrix multiplication, it
image data were used to create the diversity of images based on performs a convolution operation, denoted by a*. Its parameters are
rescaling, zooming, horizontal flipping and shearing operations. These constructed using a collection of learnable filters, often known as ker­
procedures were carried out using the functionality of the Image­ nels. The purpose of this layer is to find features in the native regions of
DataGenerator from TensorFlow, Keras framework. In the image data input samples (here, the images) and produce a feature map that di­
augmentation settings, the values of the above mentioned criteria minishes the presence of the observed features in the input data. The
following rescaling = 1./255, zoom_range = 0.2, shear_range = 0.2 and basic convolution operation can be written according to equation (6).
horizontal_flip = True. Some samples of augmented images are pre­ ∑∑
sented in Fig. 5. F(i, j) = (I ∗ K)(i, j) = I(i + m, j + n)K(m, n) (6)
m n

Here, I refers to an input matrix (such as an image), m x n represents

3.3. CNN based transfer learning dimension, and K represents a 2D filter. The kernel is another name for
K. The outcome of the 2D characteristic map is F. F is generated by
The suggested model methodology is based on a deep transfer convolving input I with K. Therefore, I*K specifies the convolution ac­
learning architecture. Researchers have recently become interested in tion. Where * indicates a discrete convolution process. The matrix k
using transfer learning-based CNN models to handle a variety of com­ scans over the input matrix while taking the stride parameter into ac­
puter vision problems. Over the previous few decades, these models count. Furthermore, for the construction of non-linearity, the results of
have been widely employed in medical disease diagnostics [58], in­ each layer of the convolution are compiled utilizing a function called the
dustry and agriculture [59,60]. A CNN-based transfer learning archi­ activation function. Various types of activation functions have lately
tecture was developed and applied for chest CT-scan and X-ray image been used more commonly, with ReLU (rectified linear unit) being one
classifications in this research.

10
K.U. Ahamed et al. Computers in Biology and Medicine 139 (2021) 105014

Fig. 14. Classification performance results on 2-class using fold-2 chest x-ray dataset.

Table 3
Class-wise performance results on three and two class categories.
Class Task Precision Recall F1-Score Accuracy (%)

Three COVID-19 1.00 0.98 0.99

Normal 0.97 0.95 0.96 97.10
Viral Pneumonia 0.94 0.99 0.96

Two COVID-19 0.99 1.00 0.99 99.35

Normal 1.00 0.99 0.99

Two COVID-19 1.00 0.99 1.00 99.57

Bacterial Pneumonia 0.99 1.00 1.00

of the most well-known in the deep learning field. The activation func­
tion is usually calculated by normalizing the input to zero in ReLU. ReLU
also produces 0 output if the input is less than 0 and the raw output if the
input is more than 0. Equation (7) can be used to represent it
mathematically.
f (x) = max(0, x) (7)
So, if the input value of the x is less than zero, then the function f(x)
returns 0; if the input value of the x is larger than or equal to zero, the
Fig. 15. Average precision, recall & f1-score of 4-class, 3-class (a. covid vs
function f(x) returns 1.
pneu_bac vs normal, b. covid vs pneu_vir vs normal), 2-class (a. covid vs
pneu_vir, b. covid vs normal, c. covid vs pneu_bac).

11
K.U. Ahamed et al. Computers in Biology and Medicine 139 (2021) 105014

Table 4
Performance of the proposed model on each fold using three class categories.
Class Folds Precision Recall F1- Accuracy
Score (%)

Four(covid-19/ Fold1 0.9933 0.9933 0.99 99.00

Fig. 16. a. Cross folding accuracy on each fold considering 4-class, 3-class (covid vs pneu_bac vs normal), 2-class (covid vs pneu_vir) and b. Average accuracy on the considered class.
CAP/normal) Fold2 0.985 0.985 0.99 98.78
Fold3 0.995 0.995 0.99 99.25
Fold4 0.985 0.985 0.99 98.68
Fold5 0.995 0.995 0.99 99.35
Average 0.99066 0.99066 0.99 99.012

3.4. Subsampling (pooling) layer

Pooling layers are an important part of the convolution layer se­

quences in a CNN. These layers reduce spatial dimensions of the input
data by collecting the outputs of the neuron bunches at one layer and
turning them into a single neuron at the next layer. Pooling layers entail
sliding a 2-D(dimensional) filter over each channel of the feature map
and summarizing the features within the filter field of coverage. The
dimensions of a feature map can be written as nhxnwxnc, and the output
dimensions after a pooling layer operation can be derived using the
given formula.
(nh − f + 1)/sx(nw − f + 1)/sxnc (8)
Here, nh signifies the feature map height, nw indicates the feature
map width, and nc specifies the number of channels utilized in the
feature map, where f is filter size and s is stride length. Max pooling, L2-
norm pooling, global pooling layers and average pooling are some of the
pooling layers utilized in convolutional neural networks. Compared to
other pooling techniques, max pooling delivers the maximum value
while being employed in the input zone.

3.5. Fully connected layer

The fully connected layer is a fundamental part of CNN where the

entire neuron from the former layer is connected to the entire neuron in
the following layer and then conveys to the vaticination of how nearly
every value matches with every particular class. The final FC (fully
connected) layer output is then coupled with a function called ”activa­
tion function”, which provides output class scores. CNN employs a va­
riety of classifiers, including Sigmoid, SVM (support vector machine),
SoftMax etc. SoftMax, as indicated in equation (9), may calculate the
probability distribution of n number of output categories.
k
ex
Z k = ∑n xn
(9)
i=1 e

Here, the input vector is marked as x, n denotes the number of

classification classes while the output vector is labelled as Z, here, k = 1,
…,n. The sum of all outputs (Z) is equivalent to one. The Softmax clas­
sifier is used in this model to classify the input chest CT-scan and X-ray
images.

3.6. Model architecture and improvement

A CNN-based transfer learning architecture was developed and used

in this study. Transfer learning defines as a machine learning technique
that takes previously learned knowledge (a pre-trained model) and ap­
plies it to a new problem that is related [61]. Using a traditional CNN
model has some shortcomings, such as not working effectively with
insufficient data, as well as being time-consuming and costly in data
labelling and learning. Transfer learning methods can readily deal with
insufficient data in these circumstances and alleviate model completion
time. The proposed model of this study is created by extending and
tuning the ResNet50V2 CNN architecture. ResNet50V2 [62] is an
improved version of ResNet-50 that outperforms than ResNet50 and

12
K.U. Ahamed et al. Computers in Biology and Medicine 139 (2021) 105014

Fig. 17. Classification performance results on 3-class using fold-1 chest ct-scan dataset.

and hence improves the performance of very deep models. ResNet50V2

Table 5 has eliminated the very last non-linearity, resulting in an input-to-output
Class-wise performance results on two class categories. path that resembles the identity connection depicted in Fig. 6.
Class Task Precision Recall F1-Score Accuracy (%) The suggested basic model ResNet50V2 has fifty deeper layers and
Two COVID-19 1.00 1.00 1.00 99.99
25,613,800 parameters, according to Ref. [63]. The model receives
CAP 1.00 1.00 1.00 input of the pre-processed dataset of images as 224*224 pixels and
produces output like 7 × 7 × 2048 features map in its very last feature
Two COVID-19 1.00 1.00 1.00 99.99
Normal 1.00 1.00 1.00 extractor layer. As shown in Fig. 7, several operations are conducted on
the entire execution phase in the suggested core architecture of the
ResNet50V2. These operations include Convolution, Batch Normaliza­
ResNet101 on the ImageNet datasets. An adjustment or modification tion, Relu(rectified linear unit), Pooling and Padding. ReLU activation
was conducted to the extended formulation of the links within blocks in function and batch normalization are driven by ResNet50V2 in the input
ResNet50V2. Again, deeper models are better at extracting features in before the multiplication actions with weights matrix (operations of
general. However, due to the characteristics or diversity of feed-forward convolution). The Rectified Linear Unit (ReLU) is a form of activation
(passing inputs to the architecture to obtain a forecasting result from function that is often employed in deep learning frameworks or models.
complicated computations in the framework) and the back-propagation ReLU assists in the recording of non-linear influences and prevents
(weights of the parameter upgrade on the grounds of the prediction vanishing gradient problems.
outcome), which is also the particle of testing deep learning models,
heavily deep models are hard to train due to vanishing or exploding 3.6.1. Batch normalization
gradients. ResNets overcome this shortcoming by forming a residual Batch Normalization is usually referred to as bn_norm (“Batch
link, which reduces the influence of vanishing or exploding gradients Norm”). Recently this has been utilized in the deep learning field

13
 K.U. Ahamed et al.
14

Fig. 18. a. Cross folding accuracy on each fold considering 3-class (covid vs cap vs normal) b. Average precision, recall & f1-score of 3-class (covid vs cap vs normal), 2-class (a.covid vs cap, b.covid vs normal).

Computers in Biology and Medicine 139 (2021) 105014

K.U. Ahamed et al. Computers in Biology and Medicine 139 (2021) 105014

Table 6
Comparison of the proposed model with the other pretrained models.
Models Dataset Number of classes Preprocessing on image data Precision Recall F1-score Accuracy (%)

VGG19 chest x-ray 4 No 0.90 0.89 0.885 88.98

Yes 0.9425 0.93 0.945 94.56
ResNet50 chest x-ray 4 No 0.88 0.875 0.8725 87.50
Yes .0.93 0.9275 0.925 93.04
InceptionV3 chest x-ray 4 No 0.875 0.875 0.8725 87.39
Yes 0.94 0.9325 0.93 94.35
VGG19 chest x-ray 3 No 0.9325 0.94 0.93 94.35
Yes 0.96 0.955 0.9525 96.16
ResNet50 chest x-ray 3 No 0.95 0.94 0.935 94.88
Yes 0.9525 0.94 0.945 96.47
InceptionV3 chest x-ray 3 No 0.93 0.9275 0.9225 93.27
Yes 0.945 0.95 0.94 95.02
VGG19 chest ct-scan 3 No 0.82 0.815 0.8125 81.75
Yes 0.9625 0.96 0.955 96.18
ResNet50 chest ct-scan 3 No 0.78 0.775 0.7725 78.05
Yes 0.88 0.885 0.87 89.08
InceptionV3 chest ct-scan 3 No 0.83 0.82 0.83 83.55
Yes 0.955 0.96 0.9575 96.35
Propoosed model chest x-ray 4 without preprocessing 0.9015 0.8991 0.89706 89.902
chest x-ray 3 0.9525 0.95 0.9505 95.105
chest ct-scan 3 0.8375 0.8305 0.8335 83.605
Propoosed model chest x-ray 4 with preprocessing 0.967 0.9645 0.9645 96.452
chest x-ray 3 0.97402 0.974 0.97268 97.242
chest ct-scan 3 0.99066 0.99066 0.99 99.012

broadly. Batch normalization (BN/bn_norm) is a method that transforms role in controlling the overfitting issues of CNN based deep learning
the inter-layer outputs of the neural networks into conventional ordi­ architecture. Fig. 9 depicts the basic operation of the dropout layer.
nation, and it’s called normalizing. This approach effectively ’resets’ the The result was optimized and generated using two dense layers. Each
output distributions of the previous layer, allowing the current layer to neuron in the dense layer receives inputs from all the other neurons in
be processed more efficiently. Batch normalization increases Neural the previous layers, making them densely connected. The fully con­
Network performance during training, avoids overfitting, and provides nected layer is another name for the dense layer. Again, direct usage of
regularization. hyper-parameters in the model becomes critical because they control the
behaviour of the model directly. Due to this circumstance, fine-tuning of
3.6.2. Padding the hyper-parameters were performed to improve the performance of
The term “padding” is associated with convolutional neural networks the proposed model. The suggested model used the image size of
since it represents the number of pixel values that are added to a photo (224x224) and ImageNet weights with the Adam optimizer [64], batch
or image whenever it is created by the segment of a CNN. Padding works size with 32, learning rate with 1e-5, dropout ratio 0.5. In addition, the
by expanding the measurement range of convolutional neural networks. model utilized an activation function named SoftMax to classify the
“Kernel” is the general name for the neural networks filter. It scans over images into both two-class and multiclass categories. There were a total
each pixel in the target image and transforms the data samples or values of 49,256,196 parameters in the suggested model, including 49,210,756
into a larger or smaller format. To facilitate the kernel in managing the trainable and 45,440 non-trainable parameters. Table 1 shows the
photo or image, padding is assembled to the picture frame to give the output shape of the developed architecture along with a concise model
kernel enough room to cover the image. Incorporating padding to a summary.
CNN-processed image allows for more obvious image analysis.
4. Results analysis
3.6.3. Proposed transfer learning architecture
In order to make the proposed model more feasible, effective and 4.1. Dataset description
robust, we further developed the fundamental architecture of the pre-
trained ResNet50V2 by adding extra four layers. Firstly, we modify The dataset used for the study was collected from three separate
the top layer of the original ResNet50V2 to set the custom input of the publicly open and available sources. A total of 4593 chest X-rays and
images. Secondly, several layers are concatenated with the pre-trained 3000 chest CT-scan images are included in the dataset. Balanced data­
ResNet50V2 network. Finally, regularization and effective fine-tuning sets have been used in this study to avoid class imbalanced problems. To
operations were performed on the additional layers. As illustrated in begin, the dataset of chest X-ray images contains 1143 X-ray images of
Fig. 7, one flatten layer, then one dropout, and after that two fc (Fully COVID-19 cases, 1150 Normal, 1150 Viral Pneumonia images from
Connected) dense layers were assembled with the base architecture of Kaggle’s repository “COVID-19 Radiography Database” offered by
ResNet50V2. The flatten layer receives data from the previous layer and Tawsifur Rahman [65] and 1150 Bacterial Pneumonia images from Paul
converts the data into a 1-D (one-dimensional) array, which was fed to Mooney’s repository “Chest X-Ray Images (Pneumonia)” offered by Paul
the next layer as input. The output of the preceding convolutional layer Mooney [66]. Fig. 10 (a) depicts the sample chest X-ray images of four
was flattened to produce a single feature vector. The proposed model different cases. Again, in the dataset of CT-scan images, 1000 images of
was flattened to allow for rapid feedforward execution. Fig. 8 shows an COVID-19 cases, 1000 normal images and 1000 CAP
example of a flattening procedure. (community-acquired pneumonia) images are gathered from the Kag­
The dropout layer was the second adding layer used in the suggested gle repository “Large COVID-19 CT scan slice dataset” offered by Maede
framework. Dropout is one of the most commonly used regularization Maftouni [67]. Fig. 10 (b) shows the sample chest CT-scan images of
techniques used in deep learning. This layer acts as a mask, preventing three different cases. From these datasets of chest CT and X-ray images,
some neurons from contributing to the next layer while leaving the rest 80% of available dataset samples were used for training purposes, with
unconverted. The dropout layer was employed because this plays a vital the remaining 20% of samples used for testing.

15
K.U. Ahamed et al. Computers in Biology and Medicine 139 (2021) 105014

Table 7 pneumonia cases by considering four, three, and binary classes on the
Comparison of the proposed model with previously published state-of-art chest X-ray images dataset. The dataset containing CT-scan images was
models. used to classify COVID-19 cases, normal controls and CAP cases
Study Architecture Amount of chest CT or Class Accuracy considering three and two class categories respectively.
X-ray images (%)

Khan et al. CoroNet 284 covid-19, 327 viral 4 89.6 4.3. Evaluation of performance metrics
[68] pneumonia,
310 normal, 330 3 95
Four metrics were utilized in this study to evaluate the performance
bacterial pneumonia
X-ray images. 2 99
of the proposed architecture. The evaluation was performed with respect
Mahmud CovXNet 305 covid-19, 305 viral 4 90.3 to the accuracy, sensitivity(recall), precision and f1-score. The mathe­
et al. [69] pneumonia, matical formulae for these metrics are presented in the equations below
305 normal, 305 3 89.6 10,11,12 and 13 respectively.
bacterial pneumonia
X-ray images. 2 94.7 (TP + TN)
Ozturk et al. DarkCovidNet 125 covid-19, 500 3 98.08 Accuracy = (10)
(TP + FP + FN + TN)
[33] pneumonia, 500
no-findings X-ray 2 87.02
TP
images. Precision = (11)
Arsenovic ResNetCOVID19 434 covid-19, 1100 3 94.1 (TP + FP)
et al. [70] bacterial pneumonia,
1100 normal X-ray TP
images. Recall = (12)
(TP + FN)
Sethy et al. ResNet50 plus 25 covid-19 & 25 non- 2 95.38
[71] covid-19
Heidarian COVID-FACT 171 COVID-19, 60 3 90.82
(2 ∗ Precision ∗ Recall)
F1 − Score = (13)
et al. [48] Pneumonia, 76 normal (Precision + Recall)
CT-scan images.
Xu et al. [49] ResNet + Loc- 224 Influenza-A, 175 3 86.7 where TN, FN, TP, FP and represent true-negative, false-negative, true-
ation Attention Normal, 219 covid-19 positive and false-positive respectively.
CT-scan images.
Shalbaf et al. 15 Pre-Trained 349 covid-19 (+) & 397 2 85.0
Furthermore, the loss function was applied in this study to assess the
[46] (CNNs) models covid-19(− ) effectiveness of the predicted model. The model was trained using a
CT-scan images. categorical cross-entropy loss. The loss function was also utilized to
Mukherjee CNNs-tailored 168 covid-19 & 168 2 96.13 reduce the cost of the model parameters. The loss function will be
et al. [50] Deep NN non-covid-19 2 95.83
reduced by increasing the number of epochs. Equation (14) expresses
X-ray images.
+ the mathematical interpretation of the loss function.
168 covid-19 & 168 (∑ )
non-covid-19 ̂) = −
L(Y, Y ̂ ) + (1 − Y) ∗ log log(1 − Y
Y ∗ log( Y ̂) (14)
CT-scan images
Proposed Modified 1143 covid-19,1150 4 96.452 Here, Y = True label, Y
̂ = Predicted Labels & L(Y, Y)
̂ = Loss function.
model & Tuned normal, 1150 3 97.242
ResNet50V2 viral pneiumonia, 1150 2 99.35
bacterial 3 99.012 4.4. Evaluation of the model
pneumonia X-ray 2 99.99
images. A five-fold cross-validation technique was applied on four, three and
+
1000 covid-19,1000
binary class classifications. For each of the cases, 80% of the data is
normal, 1000 allotted for training and 20% for validation. As shown in Fig. 11, the
CAP(community operations are repeated five times.
acquired pneumonia)
CT-scan images.
4.4.1. Performance results on chest X-ray image dataset
The performance results on the chest x-ray image dataset were
4.2. Experimental setup and implementation evaluated for four, three and two class categories using a five-fold cross-
validation approach based on the given performance metrics in 4.3. As
The proposed framework was implemented in Keras with the Tensor demonstrated in Table 2, the overall performance was obtained by
flow GPU support. The entire experiment, training, as well as testing, averaging the values of each fold. The classification performance results
was carried out in the Google Colaboratory environment, which includes of fold-3 using 4-class and 3-class classification are presented in Figs. 12
a Tesla T4 graphics card, 12.75 GB RAM, & 68.50 GB of disk space. The and 13. Again, the performance results of fold-2 using 2-class classifi­
proposed architecture used Adam Optimizer which is an optimization cation are presented in Fig. 14.
technique for gradient descent. This method is quite efficient when The fivefold cross-validation approach was not applied on some sub-
dealing with problems involving a lot of data samples or parameters. class categories. The sub-classes are experimented with just once, where
Adam is not the same as the traditional stochastic gradient descent. For we classify 2-class COVID-19 vs pneumonia with bacterial infection
all weight updates, stochastic gradient descent used the same learning cases and COVID-19 vs normal control cases) and 3-class (COVID-19 vs
rate (called alpha), which remains constant during training. On the pneumonia with viral infection vs normal control cases). The class-wise
other hand, during training, Adam was utilizing the characteristics of performance results are presented in Table 3.
AdaGrad and the RMSProp algorithm to update weights iteratively in­ The performance analysis presented in Table 2 shows that the pro­
side a network. Adam configuration parameters include alpha, beta1, posed model achieved an average accuracy of 96.452% for 4-class,
beta2, epsilon and others. Here, alpha was regarded as the step size or 97.242% for 3-class and 98.954% for 2-class with a comprehensive
learning rate. Therefore, the proposed model used an adaptive learning number of average precision, recall and f1-score of 96.7%, 96.45% and
rate with 1e-5 from the Adam Optimizer. The proposed model was 96.45% for 4-class, 97.402%, 97.4% & 97.268% for 3-class and 98.9%,
implemented to classify COVID-19 cases from normal as well as regular 98.9% and 99.0% for 2-class respectively. Table 3 shows that the pro­
posed model achieved an accuracy of 97.10% for 3-class and with a high

16
K.U. Ahamed et al. Computers in Biology and Medicine 139 (2021) 105014

level of accuracy of 99.35% and 99.57% while categorizing COVID-19 architecture being far more effective and robust.
cases, normal cases and COVID-19 cases with bacterial pneumonia. The evaluation results of the proposed study were also compared
The average precision, recall and f1-score of all category classes are with examples of similar previous state-of-the-art work. The proposed
illustrated in Fig. 15 where the red bar represents precision, green and study obtained 96.452% accuracy for four class categories comprising
blue represent recall or sensitivity and f1-score respectively. The cross- COVID-19, pneumonia (bacterial and viral) cases and normal control
validation approach is applied for some classes including 4-class, 3- cases; these achieved 97.242% for three class categories considering
class (COVID19 vs pneumonia with bacterial infection vs normal control normal, COVID-19 cases and bacterial pneumonia cases, 99.35% for two
cases) and 2-class (COVID19 vs pneumonia with viral infection cases). class categories consisting of normal cases and COVID-19 cases using
The accuracy of each fold for the mentioned classes is presented in chest x-ray images as presented in Table 7. Besides this the proposed
Fig. 16 (a) where the blue line represents the 4-class, orange line and model achieved an accuracy of 97.10% for three class categories
green line represent the 3-class and 2-class respectively. The average considering viral pneumonia cases, normal cases and COVID-19 cases,
accuracy of these classes is plotted in Fig. 16 (b). 99.57% for two class categories comprising COVID-19 cases and pneu­
monia bacterial cases using x-ray images as presented in 4.4.1, Table 3.
4.4.2. Performance results on chest CT-scan image dataset However, Mahmud et al. [69] proposed a multi-dilation convolutional
Using a five-fold cross-validation approach based on the stated per­ neural network termed CovXNet for the identification and classification
formance metrics in 4.3, the performance results on the chest CT-scan of COVID-19 cases from other cases using chest x-ray images. This model
image dataset were evaluated on three class categories. As demon­ had a 90.3% accuracy for four different classes, including COVID-19
strated in Table 4, the overall performance was obtained by averaging cases, bacterial pneumonia cases, normal cases and viral Pneumonia
the values of each fold. The classification performance results of fold-1 cases, and 89.6% for three classes considering COVID-19 cases, viral
using 3-class classification are presented in Fig. 17. pneumonia cases and bacterial pneumonia cases and 94.7% while
The fivefold cross-validation approach was not applied on some sub- classifying just COVID-19 cases and bacterial pneumonia cases. Khan
class categories. The sub-classes are experimented with just once, where et al. [68] suggested a model CoroNet for COVID-19 identification using
we classify 2-class classifications including COVID-19 vs CAP and chest x-ray images, which obtained 89.6% accuracy in four-class clas­
COVID-19 vs normal control cases. The class-wise performance results sification, comprising COVID-19, normal controls, viral and bacterial
are presented in Table 5. pneumonia cases, 95% accuracy for three classes while comprising
The classification performance analysis reported in Table 4 shows COVID-19, cases with pneumonia and normal controls and 99.6% ac­
that the proposed model achieved an overall average accuracy of curacy achieved for the binary class considering normal controls and
99.012% on 3-class with high levels of average precision, recall and f1- COVID-19 cases. The aforementioned research did not employ further
score of 99.066%, 99.066% and 99.00% on 3-class respectively, while patient data. However, our proposed model employed larger amounts of
categorizing COVID-19, cases with CAP and normal images. Again from patient data and a balanced dataset of chest x-ray images for the
Table 5 shows the proposed model achieved comprehensively high ac­ experiment, and we obtained accuracy comparable to or greater than
curacy of 99.99% and 99.99% while categorizing COVID-19 & normal these studies, as shown in Table 7. Arsenovic et al. reported [70] a model
control cases and COVID-19 & CAP cases with almost a hundred percent ResNetCOVID19 to identify COVID-19 using X-ray images that achieved
precision, recall, and f1-score value for both classifications. Fig. 18 (b) 94.1% accuracy for three class incidents comprising pneumonia bacte­
depicts the average precision, recall, and f1-score of all category classes, rial cases, normal control cases & COVID-19 cases. They did not include
where the red bar represents precision, the green and blue bars represent data containing viral pneumonia but our proposed model did (Table 7)
recall or sensitivity, and the f1-score, respectively. As the cross- not only considering data that included viral pneumonia but also
validation approach is applied on the 3-class, so in this case, the accu­ comparing data from other different categories and achieved superior
racy of each fold on that class is presented in Fig. 18 (a) where the indigo accuracy when compared to their work on the three classes. Ozturk et al.
line represents the 3-class category. [33] proposed DarkCovidNet, an automated COVID-19 identification
system using x-ray images of the chest that attained 98.08% for binary
4.4.3. Discussion classes considering normal cases and COVID-19 cases and 87.02% ac­
In this study, the results of the proposed model were compared with curacy for three classes combining pneumonia cases with the other
other pre-trained models and some recent state-of-the-art studies from cases. For their raw data, they did not use any augmentation procedures.
this field. The proposed model was first compared with other pre-trained As data augmentation improves the robustness of the deep learning
models that included VGG19, ResNet50 and InceptionV3. In these cases, model, the proposed model in this study applied augmentation tech­
comparisons were performed by considering both the two datasets of niques on the raw images and obtained better results as demonstrated in
chest CT-scan and X-ray images with four class and three class classifi­ Table 7. Sethy et al. [71] established a model that coupled a transfer
cations. In four class classifications, the classifications were performed learning architecture ResNet50 with SVM to accurately distinguish
on images of COVID-19, viral pneumonia, bacterial pneumonia and select COVID-19 positive cases from COVID-19 negative cases using
uninfected (normal) control cases. Three-class classifications compared x-ray images of the chest with a 95.38% accuracy. They employed an old
COVID-19, pneumonia and normal cases, while another 3three-class transfer learning architecture and SVM (support vector machine) clas­
classification category compared COVID-19, community-acquired sifiers for case classification, whereas the proposed model utilized an
pneumonia and normal control cases. The comparisons between the upgraded version of ResNet50 and extended and tuned it for improved
proposed model and the other pre-trained models are demonstrated in performance in this study. Furthermore, instead of employing the SVM
Table 6. Thus, it is shown from Table 6 that, when the models are trained classifier, the proposed model employed SoftMax classifier and obtained
without using preprocessed data they produce low accuracy as well as better results from their model as shown in Table 7.
low precision, recall and f1-score value, compared to models trained Our proposed study obtained 99.012% accuracy for three classes
with preprocessed data. On the other hand, the proposed model ach­ comprising COVID-19, CAP cases, normal cases, 99.99% for two classes
ieved comprehensively high accuracy, specifically 96.452% for four considering normal cases and COVID-19 cases using chest CT-scan im­
class categories, 97.242% for three class categories using chest x-ray ages as presented in Table 7. Besides this, our model achieved an ac­
images and 99.012% for three class categories using chest CT-scan im­ curacy of 99.99% for two classes considering COVID-19 cases and CAP
ages. Moreover, the precision, recall and f1-score values achieved using cases using CT-scan images as presented in 4.4.2, Table 5. Heidarian
the proposed model were also very high, as shown in Table 6. Thus, it is et al. [48] proposed a model named COVID-FACT to identify COVID-19
evident that effective preprocessing on image data and development of using CT-scan images where they achieved an accuracy of 90.82% for
the pre-trained “ResNet50V2” model can result in the proposed three-class classification. A combined architecture ResNet + Location

17
K.U. Ahamed et al. Computers in Biology and Medicine 139 (2021) 105014

attention was built by Xu et al. [49] to identify COVID-19 patients using [4] A. Hafeez, S. Ahmad, S.A. Siddqui, M. Ahmad, S. Mishra, A review of covid-19
(coronavirus disease-2019) diagnosis, treatments and prevention, EJMO 4 (2)
chest CT-images. They worked on three-class classifications and ob­
(2020) 116–125.
tained a lower accuracy of 86.7% due to using the old architecture of [5] C.-C. Lai, T.-P. Shih, W.-C. Ko, H.-J. Tang, P.-R. Hsueh, Severe acute respiratory
ResNet. Shalbaf et al. [46] worked on 15 pre-trained models to identify syndrome coronavirus 2 (sars-cov-2) and coronavirus disease-2019 (covid-19): the
COVID-19 cases from non-COVID-19 cases and achieved a lower accu­ epidemic and the challenges, Int. J. Antimicrob. Agents 55 (3) (2020) 105924.
[6] Z Nain, SK Barman, MM Sheam, SB Syed, A Samad, JM Quinn, MM Karim,
racy of 85.0% using CT-scan images. Mukherjee et al. [50] proposed MK Himel, RK Roy, MA Moni, SK Biswas, Transcriptomic studies revealed
CNNs-tailored deep neural networks to identify COVID-19 cases from pathophysiological impact of COVID-19 to predominant health conditions,
non-COVID cases by using both the ct and x-ray images of the chest. The Briefings in Bioinformatics (2021 Jun 2), https://doi.org/10.1093/bib/bbab197.
[7] A. Nashiry, S. Sarmin Sumi, S. Islam, J.M. Quinn, M.A. Moni, Bioinformatics and
aforementioned studies did not deal with more patient data. Moreover, system biology approach to identify the influences of covid-19 on cardiovascular
using pre-trained models did not make their model more robust. and hypertensive comorbidities, Briefings Bioinf. 22 (2) (2021) 1387–1401.
Therefore, our proposed model employed more patient data with a [8] Nashiry MA, Sumi SS, Shohan MU, Alyami SA, Azad AK, Moni MA. Bioinformatics
and system biology approaches to identify the diseasome and comorbidities
balanced dataset of CT-scan images for the experiment and developed a complexities of SARS-CoV-2 infection with the digestive tract disorders. Briefings
pre-trained model to make the model more robust and obtain better in Bioinformatics. 2021 May 17. doi: 10.1093/bib/bbab126.
accuracy compared to these studies, as demonstrated in Table 7. [9] SH Mahmud, M Al-Mustanjid, F Akter, MS Rahman, K Ahmed, MH Rahman,
W Chen, MA Moni, Bioinformatics and system biology approach to identify the
In sum, the encouraging and promising results of our proposed model influences of SARS-CoV-2 infections to idiopathic pulmonary fibrosis and chronic
in the identification of COVID-19 cases from CT-scan and X-ray images obstructive pulmonary disease patients, Briefings in Bioinformatics (13 (Apr)
suggest that deep learning could play an important role in combating the 2021), https://doi.org/10.1093/bib/bbab115.
[10] M.A. Moni, P.-I. Lin, J.M. Quinn, V. Eapen, Covid-19 patient transcriptomic and
current pandemic in the near future.
genomic profiling reveals comorbidity interactions with psychiatric disorders,
Although we collected a large number of X-ray and CT-scan images Transl. Psychiatry 11 (1) (2021) 1–13.
to train our model, the model needs to be evaluated with a large number [11] M. Satu, K. Mizan, S.A. Jerin, M. Whaiduzzaman, A. Barros, K. Ahmed, M.A. Moni,
of patient images from different countries to ensure its robustness. To et al., Covid-hero: machine learning based covid-19 awareness enhancement
mobile game for children, in: International Conference on Applied Intelligence and
enhance the collected images, we sharped the images through a sharp­ Informatics, Springer, Cham, 2021, pp. 321–335.
ening filter. To improve the accuracy and robustness of the model, [12] M.S. Satu, M.I. Khan, M.R. Rahman, K.C. Howlader, S. Roy, S.S. Roy, J.M. Quinn,
advanced image processing techniques such as hybrid filtering (combi­ M.A. Moni, Diseasome and comorbidities complexities of sars-cov-2 infection with
common malignant diseases, Briefings Bioinf. 22 (2) (2021) 1415–1429.
nation of several filters) need to be incorporated. We used ResNet50V2 [13] M. Satu, K.C. Howlader, M. Mahmud, M.S. Kaiser, S.M. Shariful Islam, J.M. Quinn,
as our base model and also added some extra layers to it. Generally, S.A. Alyami, M.A. Moni, et al., Short-term prediction of covid-19 cases using
ResNet50V2 is a deeper model, and by adding some new layers to the machine learning models, Appl. Sci. 11 (9) (2021) 4266.
[14] T. Franquet, Imaging of pulmonary viral pneumonia, Radiology 260 (1) (2011)
existing layers, the proposed architecture becomes more deep and 18–39.
complex. Although deeper models perform well in feature extraction, [15] M.S. Satu, K. Ahammed, M.Z. Abedin, M.A. Rahman, S.M.S. Islam, A. Azad, S.
training the model with a large dataset is time-consuming. Hence, in A. Alyami, M.A. Moni, Satu MS, Ahammed K, Abedin MZ, Rahman MA, Islam SM,
Azad AK, Alyami SA, Moni MA. Convolutional Neural Network Model to Detect
future, we intend to build a deep learning model that might have low COVID-19 Patients Utilizing Chest X-ray Images. medRxiv (2021 Jan 1), 2020–06.
complexity and be more feasible and robust. [16] B. Pal, D. Gupta, M. Rashed-Al-Mahfuz, S.A. Alyami, M.A. Moni, Vulnerability in
deep transfer learning models to adversarial fast gradient sign attack for covid-19
prediction from chest radiography images, Appl. Sci. 11 (9) (2021) 4233.
5. Conclusion
[17] Li L, Qin L, Xu Z, Yin Y, Wang X, Kong B, Bai J, Lu Y, Fang Z, Song Q, Cao K.
Artificial intelligence distinguishes COVID-19 from community acquired
SARS-CoV-2 is a serious continuing threat to human health, but the pneumonia on chest CT. Radiology. 2020 Mar 19.
shortage of testing resources in many countries limits patient testing. [18] M. Chung, A. Bernheim, X. Mei, N. Zhang, M. Huang, X. Zeng, J. Cui, W. Xu,
Y. Yang, Z.A. Fayad, et al., Ct imaging features of 2019 novel coronavirus (2019-
Thus, alternative strategies may be needed to aid the rapid diagnosis of ncov), Radiology 295 (1) (2020) 202–207.
COVID-19 patients. Hence, we examined a deep learning framework [19] C. Dhiman, D.K. Vishwakarma, View-invariant deep architecture for human action
based on the ResNet50V2 architecture and effective preprocessing recognition using two-stream motion and shape temporal dynamics, IEEE Trans.
Image Process. 29 (2020) 3835–3844.
techniques identification and classification of COVID-19 cases using CT- [20] J. Wang, H. Ding, F.A. Bidgoli, B. Zhou, C. Iribarren, S. Molloi, P. Baldi, Detecting
scan and X-ray images. The architecture was examined using a balanced cardiovascular disease from mammograms with deep learning, IEEE Trans. Med.
and updated dataset collected from different open sources. The model Imag. 36 (5) (2017) 1172–1181.
[21] S. Deepak, P. Ameer, Brain tumor classification using deep cnn features via transfer
was capable of working with both the binary class and the multiclass learning, Comput. Biol. Med. 111 (2019) 103345.
classifications. Performance analysis shows that this model performed [22] W. Zhao, Z. Zhong, X. Xie, Q. Yu, J. Liu, Relation between chest ct findings and
well on the prepared datasets. This raises the possibility that this or a clinical conditions of coronavirus disease (covid-19) pneumonia: a multicenter
study, Am. J. Roentgenol. 214 (5) (2020) 1072–1077.
similar image analysis approach can assist radiologists and clinicians in [23] Y. Li, L. Xia, Coronavirus disease 2019 (covid-19): role of chest ct in diagnosis and
the diagnosis of COVID-19, and provide timely services to patients and management, Am. J. Roentgenol. 214 (6) (2020) 1280–1286.
thereby help to limit community transmission. Future research will [24] M.M. Ahamad, S. Aktar, M. Rashed-Al-Mahfuz, S. Uddin, P. Liò, H. Xu, M.
A. Summers, J.M. Quinn, M.A. Moni, A machine learning model to identify early
focus on federated learning and Blockchain technology to establish a
stage symptoms of sars-cov-2 infected patients, Expert Syst. Appl. 160 (2020)
distributed trust-less COVID-19 detection protocol. 113661.
[25] S Aktar, MM Ahamad, M Rashed-Al-Mahfuz, A Azad, S Uddin, AH Kamal,
Declaration of competing interest SA Alyami, PI Lin, SM Islam, JM Quinn, V Eapen, Predicting Patient COVID-19
Disease Severity by means of Statistical and Machine Learning Analysis of Clinical
Blood Testing Data, JMIR Medical Informatics 21 (2021 Mar).
We do not have any conflict of interest. [26] S. Aktar, A. Talukder, M. Ahamad, A. Kamal, J.R. Khan, M. Protikuzzaman,
N. Hossain, A. Azad, J.M. Quinn, M.A. Summers, et al., Machine learning
approaches to identify patient comorbidities and symptoms that increased risk of
References mortality in covid-19, Diagnostics 11 (8) (2021) 1383.
[27] S. Aktar, M.M. Ahamad, M. Rashed-Al-Mahfuz, A. Azad, S. Uddin, A. Kamal, S.
[1] F. Wu, S. Zhao, B. Yu, Y.-M. Chen, W. Wang, Z.-G. Song, Y. Hu, Z.-W. Tao, J.- A. Alyami, P.-I. Lin, S.M.S. Islam, J.M. Quinn, et al., Machine learning approach to
H. Tian, Y.-Y. Pei, et al., Author correction: a new coronavirus associated with predicting covid-19 disease severity based on clinical blood test data: statistical
human respiratory disease in China, Nature 580 (7803) (2020). E7–E7. analysis and model development, JMIR Medical Informatics 9 (4) (2021), e25884.
[2] M.S. Satu, M.I. Khan, M. Mahmud, S. Uddin, M.A. Summers, J.M. Quinn, M. [28] J.F.-W. Chan, S. Yuan, K.-H. Kok, K.K.-W. To, H. Chu, J. Yang, F. Xing, J. Liu, C.C.-
A. Moni, Tclustvid: a novel machine learning classification model to investigate Y. Yip, R.W.-S. Poon, et al., A familial cluster of pneumonia associated with the
topics and sentiment in covid-19 tweets, Knowl. Base Syst. 226 (2021) 107126. 2019 novel coronavirus indicating person-to-person transmission: a study of a
[3] Worldometer, Covid-19 Coronavirus Cases, 2021. https://www.worldometers. family cluster, Lancet 395 (10223) (2020) 514–523.
info/coronavirus/coronavirus-cases. accessed: 2021-08-05. [29] S.H. Yoon, K.H. Lee, J.Y. Kim, Y.K. Lee, H. Ko, K.H. Kim, C.M. Park, Y.-H. Kim,
Chest radiographic and ct findings of the 2019 novel coronavirus disease (covid-

18
K.U. Ahamed et al. Computers in Biology and Medicine 139 (2021) 105014

19): analysis of nine patients treated in korea, Korean J. Radiol. 21 (4) (2020) [50] H. Mukherjee, S. Ghosh, A. Dhar, S.M. Obaidullah, K. Santosh, K. Roy, Deep neural
494–500. network to detect covid-19: one architecture for both ct scans and chest x-rays,
[30] W. Kong, P.P. Agarwal, Chest imaging appearance of covid-19 infection, Appl. Intell. 51 (5) (2021) 2777–2789.
Radiology: Cardiothoracic Imaging 2 (1) (2020), e200028. [51] M. Y. Kamil, A deep learning framework to detect covid-19 disease via chest x-ray
[31] M. Infante, R. Lutman, S. Imparato, M. Di Rocco, G. Ceresoli, V. Torri, E. Morenghi, and ct scan images., Int. J. Electr. Comput. Eng. (2088-8708) 11 (1).
F. Minuti, S. Cavuto, E. Bottoni, et al., Differential diagnosis and management of [52] M. M. Ahsan, K. D. Gupta, M. M. Islam, S. Sen, M. Rahman, M. S. Hossain, et al.,
focal ground-glass opacities, Eur. Respir. J. 33 (4) (2009) 821–827. Study of Different Deep Learning Approach with Explainable Ai for Screening
[32] J.P. Kanne, B.P. Little, J.H. Chung, B.M. Elicker, L.H. Ketai, Essentials for Patients with Covid-19 Symptoms: Using Ct Scan and Chest X-Ray Image Dataset,
Radiologists on Covid-19: an Update—Radiology Scientific Expert Panel, 2020. arXiv preprint arXiv:2007.12525.
[33] T. Ozturk, M. Talo, E.A. Yildirim, U.B. Baloglu, O. Yildirim, U.R. Acharya, [53] D. Dansana, R. Kumar, A. Bhattacharjee, D.J. Hemanth, D. Gupta, A. Khanna,
Automated detection of covid-19 cases using deep neural networks with x-ray O. Castillo, Early diagnosis of covid-19-affected patients based on x-ray and
images, Comput. Biol. Med. 121 (2020) 103792. computed tomography images using deep learning algorithm, Soft Computing
[34] A. Esteva, B. Kuprel, R.A. Novoa, J. Ko, S.M. Swetter, H.M. Blau, S. Thrun, (2020) 1–9.
Dermatologist-level classification of skin cancer with deep neural networks, nature [54] A. Sedik, A.M. Iliyasu, A. El-Rahiem, M.E. Abdel Samea, A. Abdel-Raheem,
542 (7639) (2017) 115–118. M. Hammad, J. Peng, A. El-Samie, E. Fathi, A. El-Latif, et al., Deploying machine
[35] P. Rajpurkar, J. Irvin, K. Zhu, B. Yang, H. Mehta, T. Duan, D. Ding, A. Bagul, C. and deep learning models for efficient data-augmented detection of covid-19
Langlotz, K. Shpanskaya, et al., Chexnet: Radiologist-Level Pneumonia Detection infections, Viruses 12 (7) (2020) 769.
on Chest X-Rays with Deep Learning, arXiv preprint arXiv:1711.05225. [55] Chapter 4 - efficient medical image enhancement technique using transform hsv
[36] M. Talo, O. Yildirim, U.B. Baloglu, G. Aydin, U.R. Acharya, Convolutional neural space and adaptive histogram equalization, in: N. Dey, A.S. Ashour, F. Shi, V.
networks for multi-class brain disease detection using mri images, Comput. Med. E. Balas (Eds.), Soft Computing Based Medical Image Analysis, Academic Press,
Imag. Graph. 78 (2019) 101673. 2018, pp. 51–60, https://doi.org/10.1016/B978-0-12-813087-2.00003-8.
[37] A. Comelli, C. Coronnello, N. Dahiya, V. Benfante, S. Palmucci, A. Basile, [56] J.W. Woods, Chapter 7 - image enhancement and analysis, in: J.W. Woods (Ed.),
C. Vancheri, G. Russo, A. Yezzi, A. Stefano, Lung segmentation on high-resolution Multidimensional Signal, Image, and Video Processing and Coding, second ed.,
computerized tomography images using deep learning: a preliminary step for Academic Press, Boston, 2012, pp. 223–256, https://doi.org/10.1016/B978-0-12-
radiomics studies, Journal of Imaging 6 (11) (2020) 125. 381420-3.00007-2, second ed. Edition.
[38] M. Hammad, M.H. Alkinani, B. Gupta, A.A. Abd El-Latif, Myocardial infarction [57] C. Shorten, T.M. Khoshgoftaar, A survey on image data augmentation for deep
detection based on deep neural network on imbalanced data, Multimed. Syst. learning, J Big Data 6 (1) (2019) 1–48.
(2021) 1–13. [58] O.S. Lih, V. Jahmunah, T.R. San, E.J. Ciaccio, T. Yamakawa, M. Tanabe,
[39] A. Chen, J. Jaegerman, D. Matic, H. Inayatali, N. Charoenkitkarn, J. Chan, M. Kobayashi, O. Faust, U.R. Acharya, Comprehensive electrocardiographic
Detecting covid-19 in chest x-rays using transfer learning with vgg16, in: CSBio’20: diagnosis based on deep learning, Artif. Intell. Med. 103 (2020) 101789.
Proceedings of the Eleventh International Conference on Computational Systems- [59] J. Dekhtiar, A. Durupt, M. Bricogne, B. Eynard, H. Rowson, D. Kiritsis, Deep
Biology and Bioinformatics, 2020, pp. 93–96. learning for big data applications in cad and plm–research review, opportunities
[40] A. Gupta, S. Gupta, R. Katarya, et al., Instacovnet-19: a deep learning classification and case study, Comput. Ind. 100 (2018) 227–243.
model for the detection of covid-19 patients using chest x-ray, Appl. Soft Comput. [60] M. Rahimzadeh, A. Attar, Detecting and counting pistachios based on deep
99 (2021) 106859. learning, Iran J. Comput. Sci. (2021) 1–13.
[41] R. Jain, M. Gupta, S. Taneja, D.J. Hemanth, Deep learning based detection and [61] M. Hussain, J.J. Bird, D.R. Faria, A study on cnn transfer learning for image
analysis of covid-19 on chest x-ray images, Appl. Intell. 51 (3) (2021) 1690–1700. classification, in: UK Workshop on Computational Intelligence, Springer, 2018,
[42] M. Turkoglu, Covidetectionet: covid-19 diagnosis system based on x-ray images pp. 191–202.
using features selected from pre-learned deep features ensemble, Appl. Intell. 51 [62] K. He, X. Zhang, S. Ren, J. Sun, Identity mappings in deep residual networks, in:
(3) (2021) 1213–1226. European Conference on Computer Vision, Springer, 2016, pp. 630–645.
[43] C. Ouchicha, O. Ammor, M. Meknassi, Cvdnet: a novel deep learning architecture [63] Keras, Resnet50v2. https://keras.io/api/applications/, 2021, 2021-08-05.
for detection of coronavirus (covid-19) from chest x-ray images, Chaos, Solitons & [64] D. P. Kingma, J. Ba, Adam: A Method for Stochastic Optimization, arXiv preprint
Fractals 140 (2020) 110245. arXiv:1412.6980.
[44] J. Bullock, A. Luccioni, K.H. Pham, C.S.N. Lam, M. Luengo-Oroz, Mapping the [65] TawsifurRahman, Covid-19 Radiographydatabase, 2021. https://www.kaggle.
landscape of artificial intelligence applications against covid-19, J. Artif. Intell. com/tawsifurrahman/covid19-radiography-database. accessed: 2021-08-05.
Res. 69 (2020) 807–845. [66] PaulMooney, Chestx-rayimages(pneumonia). https://www.kaggle.com/paultimot
[45] R. Ghavami, M. Hamidi, S. Masoudian, A. Mohseni, H. Lotfalinezhad, M. A. hymooney/chest-xray-pneumonia, 2021 accessed: 2021-08-05.
Kazemi, B. Moradi, M. Ghafoori, O. Motamedi, O. Pournik, et al., Accurate and [67] MaedeMaftouni, Large Covid-19 Ct Scan Slice Dataset, 2021. https://www.kaggle.
Rapid Diagnosis of Covid-19 Pneumonia with Batch Effect Removal of Chest Ct- com/maedemaftouni/large-covid19-ct-slice-dataset. accessed: 2021-08-05.
Scans and Interpretable Artificial Intelligence, arXiv e-prints (2020) arXiv–2011. [68] A.I. Khan, J.L. Shah, M.M. Bhat, Coronet: a deep neural network for detection and
[46] A. Shalbaf, M. Vafaeezadeh, et al., Automated detection of covid-19 using diagnosis of covid-19 from chest x-ray images, Comput. Methods Progr. Biomed.
ensemble of transfer learning with deep convolutional neural network based on ct 196 (2020) 105581.
scans, Int. J. Comput. Assist.Radiol.Surg 16 (1) (2021) 115–123. [69] T. Mahmud, M.A. Rahman, S.A. Fattah, Covxnet: a multi-dilation convolutional
[47] D. Li, Z. Fu, J. Xu, Stacked-autoencoder-based model for covid-19 diagnosis on ct neural network for automatic covid-19 and other pneumonia detection from chest
images, Appl. Intell. 51 (5) (2021) 2805–2817. x-ray images with transferable multi-receptive feature optimization, Comput. Biol.
[48] S. Heidarian, P. Afshar, N. Enshaei, F. Naderkhani, M. J. Rafiee, F. B. Fard, K. Med. 122 (2020) 103869.
Samimi, S. F. Atashzar, A. Oikonomou, K. N. Plataniotis, et al., Covid-fact: a fully- [70] M. Arsenovic, S. Sladojevic, S. Orcic, A. Anderla, M. Sladojevic, Detection of Covid-
automated capsule network-based framework for identification of covid-19 cases 19 Cases by Utilizing Deep Learning Algorithms on X-Ray Images, 2020, pp. 1–8.
from chest ct scans, Front.Artif. Intell. 4. [71] P. K. Sethy, S. K. Behera, Detection of coronavirus disease (covid-19) based on deep
[49] X. Xu, X. Jiang, C. Ma, P. Du, X. Li, S. Lv, L. Yu, Q. Ni, Y. Chen, J. Su, et al., A deep features.
learning system to screen novel coronavirus disease 2019 pneumonia, Engineering
6 (10) (2020) 1122–1129.

19