Since January 2020 Elsevier has created a COVID-19 resource centre with

free information in English and Mandarin on the novel coronavirus COVID-

19. The COVID-19 resource centre is hosted on Elsevier Connect, the
company's public news and information website.

Elsevier hereby grants permission to make all its COVID-19-related

research that is available on the COVID-19 resource centre - including this
research content - immediately available in PubMed Central and other
publicly funded repositories, such as the WHO COVID database with rights
for unrestricted research re-use and analyses in any form or by any means
with acknowledgement of the original source. These permissions are
granted for free by Elsevier for as long as the COVID-19 resource centre
remains active.
 Chemometrics and Intelligent Laboratory Systems 224 (2022) 104539

Contents lists available at ScienceDirect

Chemometrics and Intelligent Laboratory Systems

journal homepage: www.elsevier.com/locate/chemometrics

Multi-classification deep CNN model for diagnosing COVID-19 using

iterative neighborhood component analysis and iterative ReliefF feature
selection techniques with X-ray images
Narin Aslan a, *, Gonca Ozmen Koca a, Mehmet Ali Kobat b, Sengul Dogan c
a
Department of Mechatronics Engineering, Faculty of Technology, Firat University, Elazig, Turkey
b
Department of Cardiology, Firat University Hospital, Elazig, Turkey
c
Department of Digital Forensics Engineering, Faculty of Technology, Firat University, Elazig, Turkey

A R T I C L E I N F O A B S T R A C T

Keywords: Background: The acute respiratory syndrome coronavirus 2 (SARS-CoV-2) disease seriously affected worldwide
COVID-19 health. It remains an important worldwide concern as the number of patients infected with this virus and the
Convolutional neural networks CNN death rate is increasing rapidly. Early diagnosis is very important to hinder the spread of the coronavirus.
Iterative neighborhood component analysis
Therefore, this article is intended to facilitate radiologists automatically determine COVID-19 early on X-ray
Iterative ReliefF
Feature selection
images. Iterative Neighborhood Component Analysis (INCA) and Iterative ReliefF (IRF) feature selection methods
are applied to increase the accuracy of the performance criteria of trained deep Convolutional Neural Networks
(CNN).
Materials and methods: The COVID-19 dataset consists of a total of 15153 X-ray images for 4961 patient cases. The
work includes thirteen different deep CNN model architectures. Normalized data of lung X-ray image for each
deep CNN mesh model are analyzed to classify disease status in the category of Normal, Viral Pneumonia and
COVID-19. The performance criteria are improved by applying the INCA and IRF feature selection methods to the
trained CNN in order to improve the analysis, forecasting results, make a faster and more accurate decision.
Results: Thirteen different deep CNN experiments and evaluations are successfully performed based on 80-20% of
lung X-ray images for training and testing, respectively. The highest predictive values are seen in the analysis
using INCA feature selection in the VGG16 network. The means of performance criteria obtained using the ac-
curacy, sensitivity, F-score, precision, MCC, dice, Jaccard, and specificity are 99.14%, 97.98%, 99.58%, 98.80%,
97.81%, 98.83%, 97.68%, and 99.56%, respectively. This proposed study is indicated the useful application of
deep CNN models to classify COVID-19 in X-ray images.

1. Introduction diagnosis of COVID-19 in patients with early onset of symptoms. Among

the nucleic acid-based tests, the Reverse Transcription Polymerase Chain
The coronavirus is a virus that causes respiratory system infection and Reaction (RT-PCR) test was used to confirm COVID-19 positive patients
can be transmitted from person to person. This virus was seen in the [2]. RT-PCR sometimes fails to diagnose COVID-19, and as a result, pa-
Wuhan region of China in early December 2019 and soon spread to other tients do not receive appropriate treatment in a timely manner. There-
countries. The name of the virus has been declared by the World Health fore, a negative RT-PCR test does not rule out COVID-19 infection [3].
Organization (WHO) as SARS-CoV-2. It also uses the term COVID-19 to Availability of testing kits poses important problem fruitful detection of
describe the illness caused by the virus. The coronavirus, which has been the illness. It is very important the application of deep learning and
mutating and spreading since August 2021, has infected approximately artificial intelligence based approaches for the efficient detection of
203,295,170 million people worldwide, according to the official state- disease from X-ray images. During the current COVID-19 pandemic,
ments of the WHO, and caused approximately 4,303,515 million deaths using such deep learning based approaches in real time, especially for
[1]. The most common symptoms are fever (39
and above), cough, rapid testing, successful implementation, and detection of disease, could
weakness, shortness of breath. Clinical features alone cannot define the potentially provide enormous benefits [4–8].

* Corresponding author.
E-mail addresses: narin.aslan@firat.edu.tr (N. Aslan), gozmen@firat.edu.tr (G. Ozmen Koca), mkobat@firat.edu.tr (M.A. Kobat), sdogan@firat.edu.tr (S. Dogan).

https://doi.org/10.1016/j.chemolab.2022.104539
Received 1 November 2021; Received in revised form 7 March 2022; Accepted 14 March 2022
Available online 30 March 2022
0169-7439/© 2022 Elsevier B.V. All rights reserved.
N. Aslan et al. Chemometrics and Intelligent Laboratory Systems 224 (2022) 104539

1.1. Related studies working hard since the beginning of the pandemic to find automated
COVID-19 detection systems that use DNN. A comparative of recent
Early detection of COVID-19 disease is very important. A certain studies is shown in Table 1. The data used in this study [22] were ac-
portion of the patient population affected by this disease, the infection quired from diverse sources and preprocessed. A deep CNN network was
can cause severe organ failure and death [9]. Although X-ray imaging is developed to enable detection of COVID-19 cases. When the dataset was
widely used in hospitals in almost most countries, unfortunately there is a small, data augmentation was used to artificially generate more samples
deficiency of specialists to analyze and interpret X-ray images in some from the same dataset instead of collecting more data. Similar to our
low-resource clinics and developing countries. A number of DL methods study, a three-class classification (Normal, Viral Pneumonia and
were tried for the analysis of these images. Data mining and data analysis COVID-19) were used. Two datasets were used in these studies. The
have identified the potential value of big data in the educational process initial dataset include only 180 images cases of COVID-19. 25 cases of
as part of popular technologies in the information fields. Some of these pneumocystis, SARS and Streptococcus were defined as pneumonia. The
studies will be discussed in this part of the study. In Ref. [10], the authors second dataset includes 8851 normal and 6012 pneumonia. As noted,
used image improving techniques such as histogram equalization. They 180 positive COVID-19 cases exist. In our study, more data was used than
also used gamma correction, balance contrast improvement techniques, the data of the study of [22]. ResNet18 network was designed to reduce
image complement, contrast limited adaptive histogram equalization to model complexity. As a result of the analysis, it was estimated with
research the effect of image improvement techniques on COVID-19 96.73% accuracy. The results of the cascade classifiers were proposed in
detection. In addition, the proposed U-net model was compared with the study [23], which included the best detections of bacterial pneu-
the standard U-Net model for lung image. Six pre-trained CNN; monia images, viral (non-COVID-19) pneumonia and COVID-19 were
ResNet101, ResNet18, InceptionV3, ResNet50, ChexNet and Dense- seen for the ResNet50V2, VGG16 and DenseNet169 networks with 99.9%
Net201 were analyzed on flat and segmented lung CXR images. Ucar and accuracy. However, despite the classification of four different diseases,
Korkmaz [11] developed COVID-19 method named COVIDdiagnosis-Net the number of dataset is less than our study. In the study of [23], 306
based on Deep Bayes-SqueezeNet, deep learning network, for diagnosis X-ray images with four classes as 79 images for viral (Non-COVID-19)
of COVID-19 with the help of Bayes optimization. In Ref. [12], X-ray pneumonia, 79 normal, 69 positive COVID-19 images, and 79 bacterial
images of various resolutions were trained with different ConvXNets pneumonia cases were used. Saha et al. [24] proposed an automated
forms designed. A stacking algorithm was used to optimize the estimates. detection called EMCNet to determine COVID-19 patients. The extracted
Also, they integrated a gradient-based discriminant localization to features, machine learning classifiers (SVM, Random Forest, DT, and
separate abnormal area of the X-ray images. In the study of Nour et al. AdaBoost) were developed for COVID-19 detection. Proposed EMCNet
[13], the CNN model was used as a deep feature extractor. Also, extracted dataset contains 4600 images. The data set was divided into three
deep distinguishing features were used machine learning algorithms with different sets: the training set with 3220 images at the rate of 70%, the
a Support Vector Machine (SVM), Decision Tree (DT) and k-Nearest validation set with 920 images at the rate of 20%, and the test set with
Neighbor (k-NN). Bayesian Optimization Algorithm was used for opti- 60% images at the rate of 10%. Compared to other DL based systems,
mize of machine learning models. Fuzzy tree transform was applied to EMCNet outperformed with 98.91% accuracy, 98.89% F1 score and
each image, and then sample splitting was applied to these images by 97.82% recall. An automated method of DL-assisted using X-ray images
Tuncer et al. [14]. They used the multicore native binary pattern for for early detection of COVID-19 infection was proposed [25]. The 286
feature generation and the features were selected by the iterative images in the training set contain 143 normal case and 143 COVID-19
neighboring component feature selector method. Then, images were images. In the test dataset, there are 60 normal case and 60 COVID-19
analyzed using several algorithms such as k-NN, SVM, DT. Ozturk et al. images. Classification of eight pre-trained CNN models from COVID-19
[15] suggested a model for early find of COVID-19 cases using X-ray and normal cases, such as VGG16, AlexNet, GoogleNet, SqueezeNet,
images. The DarkNet model was formed as the classifier YOLO object MobileNet-V2, ResNet34, ResNet50 and InceptionV3 networks, was
detection system model. They worked on automated detection of studied. The best performance was achieved by ResNet34 network with
COVID-19 based on combined hybrid feature selection and sample pyr- 98.33% accuracy. Demir [26] used a deep LSTM architecture called
amid feature extraction [16]. In addition, pyramid feature generation Deep-Coronet to automatically determine COVID-19 cases from X-ray
based on samples with fused dynamic dimensions, feature selection images. In addition, marker-controlled watershed segmentation and
based on ReliefF and iterative neighbor component analysis was per- Sobel gradient processes were implemented to the images to increase the
formed. Artificial neural networks (ANN) and Deep neural networks performance of the proposed model in the preprocessing phase. The
(DNN) were used to classify the selected most informative features. estimation accuracy was calculated as 100% for the reason of using small
Marqueset et al. [17] analyzed of the X-ray images obtained in two dataset as indicated in the paper. In our study, 15153 data are used: 80%
stages. Firstly, binary classification analysis was performed using X-ray of this data is used for training and 20% for testing. In the study of [26],
images taken from normal and COVID-19 images. Second, multiclass there was a total of 1061 data: 20% for testing, 80% of this data was used
results using images from normal, pneumonia and COVID-19 patients for training. 100% accuracy was obtained with the data set which is
were compared. The proposed EfficientNet was used for dual and mul- smaller than ours.
ticlass. A convolutional neural network-based Decomposition, Trans- There are some studies in which CT images, X-ray images, or two
mission, and Rendering (DeTraC) model were developed by Abbas et al.
[18]. Class boundaries were analyzed using the class decomposition
mechanism, which helps the model fit with any anomaly in the image Table 1
dataset. Karaknis et al. [19] applied synthetic images to rise the restricted Some studies diagnosing COVID-19 Chest X-ray with CNNs.
number of lung X-ray images. They used dual classification for normal Model Accuracy Classes Type
cases, COVID-19 and multiclass classification for normal cases, pneu- (%)
monia and COVID-19. Ismael et al. [20] extracted features from CXR Modified ResNet-18 [22] 96.73 COVID-19, Normal, CXR
images from a dataset with normal patient and COVID-19 images using a Pneumonia
pre-trained ResNet50 model. In the classification models achieved the VGG16,ResNet50V2, 99.9 Bacterial pneumonia, CXR
DenseNet169 [23] COVID-19, Non-COVID-19
best accuracy using the linear-core SVM. Also, in Ref. [21], Zebin and Viral
Rezvy applied pre-trained EfficientNetB0, ResNet50, VGG16 networks to EMCNet [24] 98.91 COVID-19, Normal CXR
determine COVID-19 and concerned infection based on lung X-ray im- ResNet-34 [25] 98.33 COVID-19, Normal CXR
ages. In addition, a productive oppositional structure was trained for the DeepCoroNet [26] 1.00 COVID-19, Normal, CXR
Pneumonia
formation and growth of the COVID-19 class. Researchers have been

2
N. Aslan et al. Chemometrics and Intelligent Laboratory Systems 224 (2022) 104539

images together are analyzed to determine the diagnosis of COVID-19

with the help of deep learning methods [6,27–34].

1.2. Our method

In this study, a large-scale method of diagnosing COVID-19 is pre-

sented. This method is consisted of preprocessing, classification using
deep CNN, classification with feature selection INCA and IRF. In the
preprocessing step, the input Chest X-Ray image is converted to RGB
converted Portable Network Graphics (PNG) image with a size of
299x299 and 8 bit depth and resized according to the required input size
of the meshes used for classification. The features produced with the help
of deep CNN are combined, and the distinctive ones are selected using
INCA and IRF. One of the most preferred distance-based feature selection
methods is ReliefF. It is a parametric property selector. 10-fold cross-
validation is used for both feature selections. The features selected by
INCA and IRF are again used as input to deep CNN classifiers.

1.3. Contributions

The main contributions of this study are given below:

 Two different feature selection methods (INCA and IRF) are applied
on the same dataset to increase the efficiency of classification and to
reduce the validation time. It is seen that the use of these feature
selection algorithms gave efficient results.
 With the help of these methods, very high classification accuracies
were calculated for thirteen different CNN. These results are denoted
the overall success of the proposed IRF and INCA method.
 In addition, when the number of features is huge, using feature se-
lection is one of the most important steps for machine learning. To
optimize this step, INCA and IRF feature selectors are used and higher
performance criteria are obtained with this model then the perfor-
mance criteria of unfeatured selection model.

The remainder of this study is organized as follows: In Section 2,

dataset collection and preparation, feature selection and evaluation
metrics are mentioned. Section 3 presents the experimental results and
Fig. 1. Workflow of proposed Deep CNN framework for classifying the COVID-
comparisons. Section 4 presents an extensive discussion with similar
19 status in X-Ray images.
literature studies. Finally, in Section 5, the conclusions and future work
are discussed.
classification of the extracted features as a result of the analysis is per-
2. Material and method formed. Then, these features are selected and reclassified using INCA and
IRF feature selection algorithms. Fig. 2 illustrates a sample of Chest X-ray
2.1. Material images used for this study. All three datasets are updated regularly
whereby hospitals around the world.
RT-PCR testing for the detection of COVID-19 has a very laborious, The entire collections of 15153 Chest X-ray images have proper dis-
long and complex process. Therefore, we selected an evident and publicly tributions to train and test the proposed model. The dataset contains
available dataset containing Chest X-ray images to respond the need for a three classes and a randomly selected set of images from class samples is
fast disease diagnosis system. To obtain a private COVID-19 dataset, shown in Fig. 3. This study is carried out by the deep learning network of
three different datasets, the COVID-19 Chest X-ray dataset [35], the Viral MATLAB R2020a with a personal computer with 16 GB RAM and 3.30
Pneumonia Chest X-ray dataset [36], the Normal Chest X-ray dataset [37] GHz processor.
is merged. The resulting COVID-19 dataset occurs of 15153 Chest X-ray
images for 4961 patient cases. The dataset includes 10192 Normal, 1345 2.2. Method
Viral Pneumonia, and 3616 COVID-19 infection cases. In viral pneu-
monia samples, the diseases are caused by bacterial and non-COVID-19 In this study, a three-class classification of Normal, COVID-19 and
viral effects. The used dataset consists of Portable Network Graphics Viral Pneumonia Lung X-ray images are performed using VGG16, VGG19,
(PNG) image files with a size of 299x299. It is transformed to RGB with Squeezenet, Shufflenet, Resnet101, Resnet50, Resnet18, GoogleNet,
8-bit depth. The workflow of the proposed Deep CNN framework to DarkNet53, DarkNet19, AlexNet, DenseNet201, InceptionResnet-V2
classify the COVID-19 situation in Chest X-ray images is visually pre- deep CNN. Feature vectors generated using thirteen pre-trained net-
sented in Fig. 1. First, lung X-ray images are collected and these images works are shown in Table 2 below.
are labeled as three different lung X-ray images (Normal, Viral Pneu-
monia and COVID-19). After the dataset is labeled in three different 2.2.1. Feature selection
classes, the images in the dataset are converted to RGB format. Data are Feature selection is the process of evaluating the features according to
divided as 80% training, 20% testing. The training dataset is analyzed the algorithm used and selecting the best k features from among the n
separately with the help of thirteen different deep CNN. First, the features in the data set.

3
N. Aslan et al. Chemometrics and Intelligent Laboratory Systems 224 (2022) 104539

Fig. 2. Sample images of Normal (a), Viral Pneumonia (b), and (c) COVID-19 Chest X-rays.

Fig. 3. Training data (a) and test data (b) used in deep learning analysis of the dataset.

and stochastic gradient descent (SGD) are used to update the weights.
Table 2
Informative and redundant weights represent larger and smaller weights,
Generated feature vectors using the thirteen pretrained networks.
respectively. Also, NCA generates only positive weights [39]. NCA has
Network Layer Length two main problems. First, there is no classifier or an optimal feature
VGG16 ‘fc80 1000 vector for a problem. Secondly, since there is no negative weighted
‘drop70 4096 feature, the redundant feature is selected as elimination. The INCA
VGG19 ‘fc80 1000
feature selector is an iterative and increased model of NCA to accomplish
‘drop70 4096
Squeezenet ‘conv 100 1000 these problems [40]. The primary purpose of the INCA feature selector is
‘drop 90 512 to find the optimum number of features. For this reason, an iterative error
Shufflenet ‘node_2020 1000 calculation process is used in INCA. A classifier is chosen as the error/loss
‘node_2000 544 value calculator, the feature vector with the minimum error value is
Resnet101 ‘fc10000 1000
chosen as the optimum feature vector. INCA selects a variant number of
‘pool50 2048
Resnet50 ‘fc10000 1000 features for variant problems [41]. In this study, we selected the feature
‘avg_pool' 2048 range from 50 to 1000. Firstly, data is normalized. We normalized 3030
Resnet18 ‘fc10000 1000 sized feature vectors using min-max normalization.
‘pool50 512
GoogleNet ‘loss3-classifier' 1000 Xð :; iÞ  minðXð :; iÞÞ
‘pool5-drop_7x7_s10 1024 Xð :; iÞN ¼ i ¼ f1; 2; …3030g (1)
DarkNet53 ‘conv 530 1000
maxðXð :; iÞÞ  minðXð :; iÞÞ
‘avg10 1024
DarkNet19 ‘conv 190 1000 where X is the final feature vector with a size of 3030. Therefore, the
‘avg10 1024 normalization process must be applied to obtain optimum results. Then,
AlexNet ‘fc80 1000 sorted indices are obtained by applying NCA to the normalized features.
‘drop70 4096
NCA weights are generated by using Eq. (2).
DenseNet201 ‘fc10000 1000
‘avg_pool' 1920
InceptionResnet-V2 ‘predictions' 1000 weights ¼ NCAðX; targetÞ (2)
‘avg_pool' 1536
In order to calculate error value, k-nearest neighborhood (k-NN) is
used. In this work, k-NN with Manhattan distance is used. Weights are
2.2.1.1. Iterative Neighborhood Component Analysis (INCA). Neighbor- sorted by descending with Eq. (3).
hood Component Analysis (NCA) is one of the weight based feature se-  
lectors used to maximize the prediction accuracy of the most preferred sortedweights endex ¼ sortðweightsÞ (3)
classification algorithms. Weights are calculated for all feature columns
where endex is sorted indices with a length of 3030 and target is the
[38]. First, it assigns a fixed weight to all attribute columns. It usually sets
actual/real outputs.
the value 1. An optimizer such as ADAM, distance-based fitness function,
Besides, this process has high calculation complexity. To reduce this

4
N. Aslan et al. Chemometrics and Intelligent Laboratory Systems 224 (2022) 104539

complexity, a feature range of 50 to 1000 is defined. The iterative feature Table 3

selection process is defined as follows: Evaluation metrics used in the analysis.
TP þ TN (9)
f k ð:; lÞ ¼ Xð :; endexðlÞÞk ¼ f1; 2; …901g (4) Accuracy ¼
TP þ TN þ FP þ FN
TP (10)
Sensitivity ¼
Positive
l ¼ f100; 101; …1000gl ¼ k þ 99 (5) TN (11)
Sensitivity ¼
k Negative
f is selected feature vector in the kth iteration, l defines size of the TP (12)
Precision ¼
selected feature vector. TP þ FP
Calculated error values of f k : ðð1 þ beta2 ÞÞ*ðsensitivity*precisionÞÞ (13)
F score ¼
ððbeta2 Þ*ðprecision þ sensitivityÞÞ

ðTP*TNÞ  ðFP*FNÞ (14)
errorðnÞ ¼ 1NN f k ; 10 (6) MCC ¼
ððTP þ FPÞ*ðTP þ FNÞ*ðTN þ FPÞ*ðTN þ FNÞÞ0:5
10-fold cross-validation is used to calculate errors. After this calcu- ðP*ðTP þ FPÞÞ þ ðN*ðFN þ TNÞÞ (15)
po ¼ acc; pe ¼
lation, the minimum error is calculated. ðTP þ TN þ FP þ FNÞ2
po  pe pe  po (16)
kappa ¼ ;
1  pe 1  po
½minimum; endex ¼ minðerrorÞ (7) Kappa ¼ maxðkappaÞ (17)
ð2*TPÞ (18)
Finally, the best feature (feat final ) is selected by using a calculated Dice ¼
ð2*ðTP þ FP þ FN Þ
index value. dice (19)
Jaccard ¼
2  dice
feat final
ðkÞ ¼ Xð :; endexðkÞÞ; kj ¼ f1; 2; …endex þ 49g (8)
final
The generated feat is forwarded to classifiers. predictions in deep CNN. The dataset is grayscale images. So the images
are replicated three times to create an RGB image and these images are
2.2.2. Iterative ReliefF (IRF) resized as needed for deep CNN. The data is divided into 80% training
Relief is a widely used feature selection algorithm that is feature and 20% testing. Before analyzing the training data of deep CNN, some
precision and finds the weights of the estimator when the output of the training parameters are adjusted. The set training options are shown in
estimator depends on a multiclass variable. By using the relief algorithm, Table 4 below.
feature ranking is made for the sample selected from the data set, taking After adjusting the training options, COVID-19 detection is performed
into account the proximity of other classes in its class and its distance on thirteen different deep CNN separately. After the training, the
from other classes. This feature ranking is run with a model with negative extracted features are analyzed separately in the INCA and IRF feature
and positive weights, and the feature selection process is completed. The selection algorithms and the features are extracted. Again, it is given as
feature reduction method can be used to increase the classification ability input data to deep CNN. The workflow of the proposed deep CNN
[42]. framework to classify the COVID-19 condition in Chest X-ray images is
ReliefF generates both positive and negative weights and uses the shown in Fig. 1. The performance criteria of the obtained results are
Manhattan distance to calculate the weights. ReliefF is an enhanced shown separately in the tables. Thirteen different deep CNN classified
model of Relief. Euclidean distance is used in the relief based feature data collected from three different datasets, and the results are given in
selection method. The Manhattan distance is used to construct the ReliefF Table 5. The CNN with the best accuracy estimate is indicated in bold
weights [43]. black.
ReliefF automatically selects the most prominent features. Therefore, The confusion matrix of highest VGG16 and lowest InceptionResnet-
the iterative ReliefF method is proposed. A loss calculator should be V2 results are shown in Fig. 4 and Fig. 5. Confusion matrix of the VGG16
chosen for optimum properties. Therefore, a k-NN classification method network with dataset in Fig. 4, the Normal class achieves almost perfect
is used as the loss value calculator. In this study, both feature selections classification precision, while the Viral Pneumonia class is classified a
are evaluated and an increase in performance criteria is observed as a sensitivity of 93.31% in 269 test samples, classifying 1 sample as COVID-
result of the analysis. 19 and 2 samples as Normal. In Normal class, it is the best sensitivity of
99.36% in 2038 test samples, and 4 samples are COVID-19 and 1 sample
2.3. Evaluation metrics is Viral Pneumonia. In the COVID-19 class distribution, 4 samples are
classified as Normal. Also, no instances are seen in the Viral Pneumonia
In order to evaluate the performance of the deep learning model, class.
different metrics which are accuracy, specificity, sensitivity, precision, F- Table 6 shows the value of the detailed classification results of the
score, Kappa, Matthews correlation coefficient, Dice and Jaccard are best performing VGG16 for the raw dataset. The results are showed that
applied in this study to measure the correct and/or incorrect classifica- MCC, precision, and F_score values of the COVID-19 class have values
tion of COVID-19 detected in the tested Chest X-ray images. Used eval- such as 98.00%, 98.34%, and 98.48%, respectively. The model with the
uation metrics in this study are given in Table 3 below. raw dataset achieves 97.53%, 98.86%, and 97.74% accuracy values of
where, True Positive, True Negative, False Negative and False Posi- MCC, precision, and F_score of Viral Pneumonia class, respectively. It
tive are TP, TN, FN and FP, respectively. Also value of beta is 1. indicates strong classification performance in Pneumonia and COVID-19,
while usually the Normal class offers the highest accuracy.
3. Results
Table 4
The goal of this study is to predict the COVID-19 viral infection using Parameters used in CNN training.
various deep CNN. For this purpose, three different datasets are collected. Training Options Parameter
This dataset contains 3616 COVID-19, 10192 Normal and 1345 Viral
Optimization algorithm ‘sgdm'
Pneumonia heterogeneous Chest X-ray images. These data are analyzed Mini Batch Size 64
in Matlab R2020a program in thirteen different deep CNN (Vgg16, Max. Epochs 10
Vgg19, Squeezenet, Shufflenet, Resnet101, Resnet50, Resnet18, Goo- Initial Learn Rate 0.0001
gleNet, DarkNet53, DarkNet19, AlexNet, DenseNet201, and Learn Rate Drop Factor 0.6
Learn Rate Drop Period 20
InceptionResnet-V2). Chest X-ray images are preprocessed before making

5
N. Aslan et al. Chemometrics and Intelligent Laboratory Systems 224 (2022) 104539

Table 5
Classification results of various proposed CNN (%).
Acc Sen Spe Prec F_score MCC Kappa Dice Jaccard Time (s)

Vgg16 98.94 98.21 99.22 98.79 98.50 97.76 97.62 98.32 96.70 142963.49
Vgg19 98.42 91.23 98.70 97.98 97.60 96.52 96.44 97.59 95.29 84494.94
Squeezenet 97.23 95.06 97.83 96.96 95.98 94.06 93.76 95.70 91.75 17438.78
Shufflenet 97.82 96.67 98.41 97.40 97.03 95.52 95.10 96.56 93.35 24732.65
Resnet101 97.99 96.43 98.43 97.63 97.02 95.63 95.47 96.89 93.96 112531.98
Resnet50 97.95 96.75 98.36 97.53 97.14 95.68 95.40 96.71 93.62 69306.76
Resnet18 98.09 97.36 98.71 97.22 97.29 95.95 95.69 96.96 94.10 26521.45
GoogleNet 97.52 97.00 98.45 96.35 96.67 94.96 94.43 96.06 92.42 29262.74
DarkNet53 97.92 96.40 98.45 97.09 96.74 95.33 95.32 96.87 93.93 66582.46
DarkNet19 98.68 97.75 98.99 98.60 98.16 97.24 97.03 97.89 95.86 58771.68
AlexNet 98.35 97.69 98.76 97.90 97.79 96.65 96.29 97.38 94.90 9617.79
DenseNet201 97.76 96.40 98.33 97.16 96.77 95.21 94.95 96.50 93.23 92990.08
InceptionResnet-V2 95.45 94.05 96.67 94.35 94.20 90.97 89.75 92.72 86.43 111230.22

Acc: Accuracy, Sen: Sensitivity, Spe: Specificity, Prec: Precision, MCC: Matthews correlation coefficient.

Fig. 4. Confusion matrix of the VGG16 with the raw dataset.

Fig. 5. Confusion matrix of the InceptionResnet-V2 with the raw dataset.

The confusion matrix is shown with the raw dataset of

InceptionResnet-V2 network in Fig. 5. Here, the Normal class achieves Table 6
almost perfect classification precision, while the COVID-19 class has a Performance criteria of the trained Vgg16 in diagnosing the Chest X-rays (%).
sensitivity of 91.84% in 723 test samples, classifying 58 samples as Sen Spe Prec F_score MCC
Normal and 1 sample Viral Pneumonia. In Normal class, it is the best
COVID-19 98.62 99.48 98.34 98.48 98.00
sensitivity of 99.36% in 2038 test samples, 44 samples are COVID-19 and
Normal 99.36 98.29 99.17 99.26 97.75
17 samples are Viral Pneumonia. The Viral Pneumonia class distribution Viral Pneumonia 96.65 99.89 98.86 97.74 97.53
is the best sensitivity of 93.31% in 269 test samples, 4 samples are
COVID-19 and 14 samples are Normal.
In the predictive study, the InceptionResnet-V2 deep CNN belongs to VGG16 deep learning network, which is the highest prediction perfor-
the lowest performance and test results are interpreted in Table 7. The mance value, by 3.45%, 0.54%, 5.61%, 4.53%, 5.00%, respectively. The
prediction values of low COVID-19 are lower than the specificity, predictive values in deep CNN are given in Table 7 range from VGG16
sensitivity, precision, F_score, and MCC performance criteria of the network to the values in InceptionResnet-V2 network.
VGG16 deep learning network, which is the highest prediction perfor- The predictive results are analyzed using INCA and IRF feature se-
mance value 1.56%, 7.07%, 5.16%, 6.03%, 7.92%, respectively. The lection to improve the results obtained in the predictive study using raw
prediction values of low Normal are lower than the sensitivity, speci- data in various deep CNN. As seen in Table 8; the prediction accuracy of
ficity, precision, F_score, and MCC performance criteria of the VGG16 the VGG16 network which has the best prediction value is 99.14% in the
deep learning network, which is the highest prediction performance prediction study using INCA feature selection. Specificity and precision
value, by 2.36%, 5.64%, 2.70%, 2.52%, 7.91%, respectively. The pre- values are showed to improvement 99.56% and 98.80%, respectively. In
diction values of low Viral Pneumonia are lower than the sensitivity, the sensitivity value is observed 0.23% decrease. F_score, MCC, kappa,
specificity, precision, F_score, and MCC performance criteria of the dice and Jaccard are observed to improve by 99.58%, 97.81%, 98.14%,
98.83% and 97.68%, respectively. The highest predictive accuracy on the

6
N. Aslan et al. Chemometrics and Intelligent Laboratory Systems 224 (2022) 104539

Table 7 As seen in Table 9, IRF feature selection; The VGG16 network, which
Performance of the trained InceptionResnet-V2 in diagnosing the Chest X-rays. has the best predictive value, has a predictive result of 98.96% in the
Sen Spe Prec F_score MCC predictive study using IRF feature selection. Specificity and precision
values are showed improvement of 98.85% and 99.01%, respectively.
COVID-19 91.84 97.92 93.26 92.54 90.23
Normal 97.01 92.74 96.49 96.75 90.01 The sensitivity value is decreased by 0.47%. F_score, MCC, kappa, dice
Viral Pneumonia 93.31 99.35 93.31 93.31 92.66 and Jaccard are improved, with 98.89%, 98.05%, 84.28%, 96.57% and
93.36%, respectively. The highest predictive accuracy on the Resnet18,
Resnet50, and Resnet101 deep CNN is 98.30% on the Resnet18 network,
Resnet18, Resnet50, and Resnet101 deep CNN has been 98.76% on the with a forecast time of approximately 2572.103s. According to the
Resnet50 network, with a forecast time of approximately 6530.356s. Resnet18 value is calculated without feature selection, after the IRF
According to the Resnet50 network value is calculated without feature feature selection is made, an increase of 0.21% is seen in the accuracy
selection, after the INCA feature selection is made, an increase of 0.82% rate. This ratio is showed less accuracy than the Resnet50 accuracy es-
is seen in the accuracy rate. The decrease in the prediction time that INCA timate in INCA feature selection. The reduction of the prediction time of
feature selection in the Resnet50 network contributes to improving the the Resnet18 network with IRF feature selection shows that the results of
results. Although there is an increase of 0.64% and 0.22% in sensitivity the analysis are improved. In addition, precision, F_score, MCC, kappa,
and Jaccard values, respectively, it is the lowest value compared to other dice and Jaccard performance metrics are calculated as 97.14%, 97.72%,
performance criteria after feature selection. Although the DenseNet201 96.12%, 95.86%, 97.08% and 94.32%, respectively. We attained speci-
network an accuracy of 97.93%, improvement is observed in all perfor- ficity value of 99.19% in the Resnet50 network. However, it has the
mance criteria. Accuracy, specificity, precision, F_score, MCC, kappa, highest predictive value of 11121.235s in the Resnet group. Although the
dice and Jaccard values are increased by 0.17%, 0.12%, 0.41%, 0.16%, DenseNet201 network has an accuracy of 97.86%, improvement is
0.68%, 0.42%, 0.40% and 0.77%, respectively. The sensitivity value is observed in all performance criteria. Accuracy, specificity, precision,
decreased by 0.08%. GoogleNet, Shufflenet and VGG19 networks are F_score, MCC, kappa, dice and Jaccard values are increased by 0.10%,
observed to have 97.83%, 97.91% and 98.60% accuracy rates, respec- 0.03%, 0.37%, 0.12%, 0.53%, 0.35%, 0.25% and 0.50%, respectively.
tively. Alexnet network is an accuracy of 98.86%, and it has a predictive The sensitivity value is decreased by 0.005%. GoogleNet, Shufflenet, and
time 9513.523s. Squzeenet network has an accuracy of 97.36%. Dar- VGG19 networks have an accuracy of 97.74%, 97.89%, and 98.06%,
knet19 and Darknet53 networks prediction accuracy is 99.02% and respectively. Alexnet network has an accuracy of 98.23%, and it is a
98.26%, respectively. Darknet19 and Darknet53 networks accuracy rates predictive time 9514.632s. Squzeenet network has an accuracy of
increased by 0.34% in both algorithms. Although feature selection is 97.30%. Darknet19 and Darknet53 networks prediction accuracy is
applied, the InceptionResnet-V2 networks is showed a lower perfor- 98.15% and 98.13%, respectively. Although feature selection is applied,
mance value of 96.01% compared to other algorithms. In addition, an the InceptionResnet-V2 network is showed a lower performance value of
increase in accuracy of 0.58% is observed. The lowest kappa and Jaccard 95.60% compared to other algorithms. In addition, an increase in accu-
performance criteria are showed 89.27% and 87.35%, respectively. It is racy of 0.15% is observed. The lowest kappa and Jaccard performance
predictive computational value is faster than other algorithms and has a criteria show 89.14% and 87.17%, respectively. It is predictive
value of 1005.125s.

Table 8
Performance results of some deep CNN calculated with INCA feature selection (%).
Acc Sen Spe Prec F_score MCC Kappa Dice Jaccard Time (s)

Vgg16 99.14 97.98 99.56 98.80 99.58 97.8 98.1 98.83 97.6 12876.553
Vgg19 98.60 91.10 98.88 98.16 97.75 97.21 96.85 98.32 96.69 7652.820
Squeezenet 97.36 95.86 98.23 97.54 96.12 94.58 94.23 95.75 93.47 1753.614
Shufflenet 97.91 96.95 98.62 98.56 97.40 95.93 95.85 96.72 93.64 2373.245
Resnet101 98.12 97.35 99.21 98.16 97.23 96.21 95.82 96.95 94.08 11127.821
Resnet50 98.76 97.38 98.75 97.98 97.56 95.79 95.96 96.82 93.83 6530.356
Resnet18 98.54 97.77 99.12 97.86 97.75 96.15 95.96 97.12 94.40 2586.315
GoogleNet 97.83 96.85 98.75 96.74 96.81 95.18 94.73 96.58 93.38 2815.256
DarkNet53 98.26 96.25 98.73 97.86 96.95 95.73 95.82 97.10 94.36 6585.325
DarkNet19 99.02 97.12 99.17 98.85 98.56 97.87 97.25 98.23 96.52 5625.452
AlexNet 98.86 97.63 98.87 98.23 97.86 96.95 96.72 97.75 95.59 9513.523
DenseNet201 97.93 96.32 98.45 97.56 96.93 95.87 95.36 96.89 93.96 9159.015
InceptionResnet-V2 96.01 93.65 97.20 95.20 94.57 91.25 89.27 93.25 87.35 1005.125

Table 9
Performance results of some deep CNN calculated with IRF feature selection (%).
Acc Sen Spe Prec F_score MCC Kappa Dice Jaccard Time (s)

Vgg16 98.96 97.74 98.85 99.01 98.89 98.05 84.28 96.57 93.36 12885.12
Vgg19 98.06 91.10 98.76 98.10 97.23 97.03 96.17 98.29 96.63 7663.270
Squeezenet 97.30 95.72 98.12 97.65 96.08 94.50 94.13 95.65 91.66 1745.853
Shufflenet 97.89 96.94 98.52 98.46 97.36 95.82 95.71 96.68 93.57 2371.265
Resnet101 98.08 97.32 99.19 98.12 97.13 96.18 95.75 96.86 93.91 11121.23
Resnet50 97.95 97.12 98.64 97.70 97.48 95.68 95.91 96.78 93.76 6528.20
Resnet18 98.30 97.58 99.05 97.14 97.72 96.12 95.86 97.08 94.32 2572.10
GoogleNet 97.74 96.72 98.65 96.62 96.79 95.11 94.65 96.42 93.08 2802.10
DarkNet53 98.13 96.17 98.65 97.75 96.90 95.69 95.78 97.06 94.28 6542.12
DarkNet19 98.15 97.03 99.10 98.75 98.50 97.85 97.05 98.17 96.40 5623.25
AlexNet 98.23 97.61 98.56 98.12 97.80 96.92 96.68 97.71 95.52 9514.63
DenseNet201 97.86 96.25 98.36 97.53 96.89 95.72 95.29 96.75 93.70 9160.00
InceptionResnet-V2 95.60 93.23 97.12 95.12 94.48 91.14 89.14 93.15 87.17 1014.85

7
N. Aslan et al. Chemometrics and Intelligent Laboratory Systems 224 (2022) 104539

Fig. 6. Violin plots of the VGG16 network using accuracy, sensitivity, specificity, precision, F-score, MCC, kappa, dice, Jaccard, with suggested median values and
mean against Un-feature selection, INCA and IRF segmentation methods.

8
N. Aslan et al. Chemometrics and Intelligent Laboratory Systems 224 (2022) 104539

computational value which is faster than other algorithms and has a and IRF feature selections are applied separately, and the graphs of the
value of 1014.857s. algorithm with the best results and the algorithm training losses with the
We selected a violin chart to present comparisons and display each worst results are shown in Fig. 7. Fig. 7a shows the training loss in the
quantitative metric in a single way by calculating the proposed seg- calculation without any feature selection. This graph shows that the
mentation quality measurement tools with the un-feature selection, training loss of InceptionResnet-V2 network with low predictive accu-
INCA, and IRF methods over the Chest X-ray dataset. Violin charts show racy is higher than VGG16 network with high training accuracy. After
the probability of distribution of data at different values. The asymmet- applying feature selection to the data, the highest estimation accuracy is
rical outer shape represents all possible outcomes. Also, as shown in observed in the VGG16 network, the lowest in the InceptionResnet-V2
Fig. 6, the middle (þ) toolbar shows the median value, and the green box network, the INCA and IRF feature selection training loss result graphs
shows the mean value of the data. Red, blue, and purple indicate seg- are shown in Fig. 7b and c.
mentation results of un-feature selection, INCA and IRF, respectively. The
comparison of the VGG16 network, which gave the best results from all 4. Discussions
analyzed deep CNN, with nine segmentation quality metrics is made. In
Fig. 6, each of the nine separate graphs corresponds to three different In this section, we compared between the pre-trained deep CNN
distributions. Interestingly, the means between the three distributions proposed to detect COVID-19 and the models proposed in binary and
and the intervals between the mean are different. Also, the distribution multi-classification. Table 10 shows the highest data obtained in the
patterns are different. Accuracy (Fig. 6a), sensitivity (Fig. 6b), specificity studies mentioned. Seven different deep CNN and modified U-net
(Fig. 6c), precision (Fig. 6d), F-score (Fig. 6e), MCC (Fig. 6f), kappa network algorithms were used in the article [10], which consists of
(Fig. 6g), dice (Fig. 6h) and Jaccard (Fig. 6ı) indicate a better segmen- almost the same dataset in our study. In total, 18479 Chest X-ray images
tation performance. It is clear that the statistics of the accuracy, speci- were used, and 3616 of these images are the same COVID-19 data. As a
ficity, MCC, precision, and kappa metrics provided by INCA result of the analysis, sensitivity of 97.2%, accuracy of 96.29%, F1-score
segmentation are greater than the un-feature selection and IRF methods. 96.28%. Modified U-net network, the accuracy of 98.63%, dice 96.94%
In addition, it is clear that the statistics of Dice, Jaccard, and sensitivity predictive values were observed. Our proposed study is calculated the
metrics provided by un-feature selection segmentation are larger than highest accuracy value as 98.94%, sensitivity 98.21%, and F1-score
INCA and IRF methods. 98.50% in the VGG16 network without any feature selection. It is
Prediction accuracy and performance criteria are calculated without observed that VGG16 network performance criteria performed better
any feature selection in the raw data in various deep CNN. Then the INCA after INCA and IRF feature selection. When compared with this study, it is

Fig. 7. Chart of non-feature selection (a), INCA feature selection (b) and IRF feature selection (c) deep CNN training loss.

9
N. Aslan et al. Chemometrics and Intelligent Laboratory Systems 224 (2022) 104539

Table 10
Comparison of the lung Chest X-ray models for multiclass classification.
Ref. Method Dataset Performance (%)

Covid Total Acc Sen Spe Prec F1_ score Dice MCC Kappa Jaccard

[10] Seven different CNN networks 3616 18479 96.29 97.2 – 96.28
Unet network 3616 18479 98.63 96.94
[44] Inception-V3, Xception, 490 6432 96.00 92.00
ResNet
[11] Bayes-SqueezeNet 76 5949 98.3 98.3 97.4
[45] ResNet101 1765 5982 71.9 77.3 71.8 – – – – – –
[25] ResNet34 203 406 98.3 1 96.67 96.7 98.36 – – – –
[46] VGG19 224 1427 93.48 92.85 98.75
[46] MobileNet v2 224 1427 92.85 99.10 97.09
Our study Thirteen different CNN networks 3616 15153 99.14 97.98 99.56 98.8 99.58 98.83 97.8 98.1 97.6

seen that the INCA and IRF feature selection methods in our proposed increased to 98.96%. The accuracy value of the InceptionResnet-V2
study are improvement on performance criteria. In another study [44], network is increased by 95.60%. Accuracy, specificity, sensitivity, pre-
COVID-19 diagnosis was studied in Inception-V3, Xception, and ResNet cision, F-score, MCC, kappa, dice, Jaccard, with suggested median values
networks. In addition, 6432 Chest X-ray images were used in the study and mean against Un-feature selection, using INCA and IRF segmentation
and 490 of these images were COVID-19 datasets. Compared to our study, methods are presented violin plots. However, training losses are
less data was used as dataset. The highest values obtained are as seen in observed during Un-feature selection, INCA and IRF segmentation. When
Table 10. Uçar and Korkmaz [11] performed a comprehensive analysis the data are analyzed without feature selection, the highest accuracy
on a new Bayes-SqueezeNet network using a total of 5949 Chest X-ray result is 98.94% in VGG16 network. Sensitivity, specificity, precision,
images, 76 of which were COVID-19 data. As a result of the analysis, F_score and Dice values are 98.21%, 99.22%, 98.79%, 98.50% and
accuracy, F1_score and MCC performance criteria values were found 98.32% respectively. MCC, Kappa and Jaccard performance criteria
98.3%, 98.3% and 97.4%, respectively. In the study using the ResNet101 values are 97.76%, 97.62% and 96.70%, respectively. In the analysis
network [45], 5982 Chest X-ray images were used and 1765 of these made with the INCA feature selection method, the VGG16 network is the
images were COVID-19 data. The highest performance criteria were best accuracy of 99.14% performance criteria. Specificity, precision,
specificity, sensitivity and accuracy of 71.8%, 77.3% and 71.9%, F_score, Kappa and Dice values are 99.56%, 98.80%, 99.56%, 98.14%
respectively. In our proposed study; sensitivity, specificity and accuracy and 98.83% respectively. Sensitivity, MCC, Jaccard performance criteria
values of 15153 datasets analyzed without feature selection with the values are 97.98%, 97.81% and 97.68%, respectively. In the analysis
ResNet101 network is calculated as 96.43%, 98.43% and 97.99%, made with the IRF feature selection method, the VGG16 network has the
respectively. In addition, after the feature selection is applied in our best accuracy of 98.96% performance criteria. Specificity, precision,
proposed study, the performance criteria of the ResNet101 network are F_score and MCC performance criteria values are 98.85%, 99.01%,
improved. In the study conducted using the ResNet34 network [25], the 98.89% and 98.05%, respectively. Sensitivity, Kappa, Dice, Jaccard
prediction accuracy value was calculated as 98.3%. 406 Chest X-ray performance criteria values are 97.74%, 84.28%, 96.57% and 93.36%,
images was analyzed, although the sensitivity value was better than the respectively.
proposed study. In Ref. [46], COVID-19 detection of VGG19 and In future, we intend to refine our techniques and suggest new tech-
MobileNet-V2 networks were compared. In the analysis using the VGG19 niques as more real data become available. In addition, different feature
network, the performance criteria of accuracy, recall and specificity were selection or feature extraction methods can be developed and tested on a
observed as 93.48%, 92.85% and 98.75%, respectively. dataset with different features.

5. Conclusion Funding

This study presents a deep CNN approach for the automatic detection The authors state that this work has not received any funding.
of COVID-19 pneumonia. The dataset is consisted of a total of 15153
Chest X-ray images. Thirteen different popular and previously reported CRediT authorship contribution statement
effective deep CNN are trained and tested to classify normal, COVID-19
and viral pneumonia patients using Chest X-ray images. The features Narin Aslan: Visualization, Writing – review & editing, Writing –
obtained from these deep CNN are given as input data to the INCA and original draft, Data curation, Resources, Investigation, Formal analysis,
IRF feature selection algorithms. The features obtained after the analysis Validation, Software, Conceptualization, Methodology. Gonca Ozmen
of thirteen different deep CNN with INCA and IRF feature selection Koca: Project administration, Supervision, Visualization, Writing – re-
methods were given to the deep CNN as input data again. The specified view & editing, Writing – original draft, Data curation, Resources, Formal
performance criteria are calculated. Although the VGG16 network shows analysis, Validation, Methodology, Conceptualization. Mehmet Ali
the best performance criterion with a value of 98.94%, the worst per- Kobat: Writing – original draft, Validation, Formal analysis, Conceptu-
formance criterion is InceptionResnet-V2 network with a value of alization, Writing – review & editing. Sengul Dogan: Project adminis-
95.45%. First of all, the features extracted from the analysis are classified. tration, Supervision, Writing – review & editing, Writing – original draft,
Then, these features are reclassified using INCA and IRF feature selection Formal analysis, Validation, Methodology, Conceptualization.
algorithms. Deep CNN with the best and worst results hasn't changed
after INCA feature selection. After the INCA feature selection, the accu-
racy value of the VGG16 network is increased to 99.14%. The accuracy of Declaration of competing interest
the InceptionResnet-V2 network is increased by 96.01%. Deep CNN with
the best and worst results hasn't changed after IRF feature selection. After The authors declare that they have no known competing financial
the INCA feature selection, the accuracy value of the VGG16 network is interests or personal relationships that could have appeared to influence
the work reported in this paper.

10
N. Aslan et al. Chemometrics and Intelligent Laboratory Systems 224 (2022) 104539

References [23] M.E. Karar, E.E.D. Hemdan, M.A. Shouman, Cascaded deep learning classifiers for
computer-aided diagnosis of COVID-19 and pneumonia diseases in X-ray scans,
Complex & Intelligent Systems 7 (1) (2021) 235–247.
[1] World Health Organization, Pneumonia of unknown cause–China, in: Emergencies
[24] P. Saha, M.S. Sadi, M.M. Islam, EMCNet: automated COVID-19 diagnosis from X-ray
Preparedness, Response, Disease Outbreak News, World Health Organization
images using convolutional neural network and ensemble of machine learning
(WHO), 2020.
classifiers, Informatics in medicine unlocked 22 (2021) 100505.
[2] Y. Li, L. Yao, Jiawei Li, Lei Chen, et al., Stability issues of RT-PCR testing of SARS-
[25] S.R. Nayak, D.R. Nayak, U. Sinha, V. Arora, R.B. Pachori, Application of deep
CoV-2 for hospitalized patients clinically diagnosed with COVID-19, J. Med. Virol.
learning techniques for detection of COVID-19 cases using chest X-ray images: a
(2020) 1–6, https://doi.org/10.1002/jmv.25786. Wiley.
comprehensive study, Biomed. Signal Process Control 64 (2021) 102365.
[3] Y. Wang, H. Kang, X. Liu, Z. Tong, Combination of RT-PCR testing and clinical
[26] F. Demir, DeepCoroNet: a deep LSTM approach for automated detection of COVID-
features for diagnosis of COVID-19 facilitates management of SARSCoV-2 outbreak,
19 cases from chest X-ray images, Appl. Soft Comput. 103 (2021) 107160.
J. Med. Virol. 22 (6) (2020) 538–539, https://doi.org/10.1002/jmv.25721 (Wiley).
[27] A. Saygılı, A new approach for computer-aided detection of coronavirus (COVID-
[4] T. Zebin, S. Rezvy, COVID-19 detection and disease progression visualization: deep
19) from CT and X-ray images using machine learning methods, Appl. Soft Comput.
learning on chest X-rays for classification and coarse localization, Appl. Intell. 51
105 (2021) 107323.
(2) (2021) 1010–1021.
[28] D.M. Ibrahim, N.M. Elshennawy, A.M. Sarhan, Deep-chest: multi-classification deep
[5] M.E. Karar, E.E.D. Hemdan, M.A. Shouman, Cascaded deep learning classifiers for
learning model for diagnosing COVID-19, pneumonia, and lung cancer chest
computer-aided diagnosis of COVID-19 and pneumonia diseases in X-ray scans,
diseases, Comput. Biol. Med. 132 (2021) 104348.
Complex & Intelligent Systems 7 (1) (2021) 235–247.
[29] H.S. Maghdid, A.T. Asaad, K.Z. Ghafoor, A.S. Sadiq, S. Mirjalili, M.K. Khan, April).
[6] H.S. Maghdid, A.T. Asaad, K.Z. Ghafoor, A.S. Sadiq, S. Mirjalili, M.K. Khan, April).
Diagnosing COVID-19 pneumonia from X-ray and CT images using deep learning
Diagnosing COVID-19 pneumonia from X-ray and CT images using deep learning
and transfer learning algorithms, 2021, in: Multimodal Image Exploitation and
and transfer learning algorithms, 2021, in: Multimodal Image Exploitation and
Learning, vol. 11734International Society for Optics and Photonics, 2021,
Learning, vol. 11734International Society for Optics and Photonics, 2021,
p. 117340E.
p. 117340E.
[30] M. To gaçar, B. Ergen, Z. C€
omert, Detection of lung cancer on chest CT images using
[7] M.A. Khan, S. Kadry, Y.D. Zhang, T. Akram, M. Sharif, A. Rehman, T. Saba,
minimum redundancy maximum relevance feature selection method with
Prediction of COVID-19-pneumonia based on selected deep features and one class
convolutional neural networks, Biocybernetics and Biomedical Engineering 40 (1)
kernel extreme learning machine, Comput. Electr. Eng. 90 (2021) 106960.
(2020) 23–39.
[8] A. Tarik, H. Aissa, F. Yousef, Artificial intelligence and machine learning to predict
[31] A. Amyar, R. Modzelewski, H. Li, S. Ruan, Multi-task deep learning based CT
student performance during the COVID-19, Procedia Comput. Sci. 184 (2021)
imaging analysis for COVID-19 pneumonia: classification and segmentation,
835–840.
Comput. Biol. Med. 126 (2020) 104037.
[9] Z. Wu, J.M. McGoogan, Characteristics of and important lessons from the
[32] M. Loey, G. Manogaran, N.E.M. Khalifa, A deep transfer learning model with
coronavirus disease 2019 (COVID-19) outbreak in China: summary of a report of 72
classical data augmentation and cgan to detect covid-19 from chest ct radiography
314 cases from the Chinese Center for Disease Control and Prevention, JAMA 323
digital images, Neural Comput. Appl. (2020) 1–13.
(13) (2020) 1239–1242.
[33] U. Budak, M. Cibuk, Z. Comert, A. Sengur, Efficient COVID-19 segmentation from
[10] T. Rahman, A. Khandakar, Y. Qiblawey, A. Tahir, S. Kiranyaz, S.B.A. Kashem,
CT slices exploiting semantic segmentation with integrated attention mechanism,
M.E. Chowdhury, Exploring the effect of image enhancement techniques on COVID-
J. Digit. Imag. (2021) 1–10.
19 detection using chest X-ray images, Comput. Biol. Med. 132 (2021) 104319.
[34] S. Ahuja, B.K. Panigrahi, N. Dey, V. Rajinikanth, T.K. Gandhi, Deep transfer
[11] F. Ucar, D. Korkmaz, COVIDiagnosis-Net: deep Bayes-SqueezeNet based diagnosis
learning-based automated detection of COVID-19 from lung CT scan slices, Appl.
of the coronavirus disease 2019 (COVID-19) from X-ray images, Med. Hypotheses
Intell. 51 (1) (2021) 571–585.
140 (2020) 109761.
[35] https://sirm.org/category/senza-categoria/covid-19,(Access date:06/03/2021).
[12] T. Mahmud, M.A. Rahman, S.A. Fattah, CovXNet: a multi-dilation convolutional
[36] https://www.kaggle.com/paultimothymooney/chest-xray-pneumonia, (Access
neural network for automatic COVID-19 and other pneumonia detection from chest
date:06/03/2021).
X-ray images with transferable multi-receptive feature optimization, Comput. Biol.
[37] https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/data,(Access
Med. 122 (2020) 103869.
date:06/03/2021).
[13] M. Nour, Z. C€ omert, K. Polat, A novel medical diagnosis model for COVID-19
[38] S. Raghu, N. Sriraam, Classification of focal and non-focal EEG signals using
infection detection based on deep features and Bayesian optimization, Appl. Soft
neighborhood component analysis and machine learning algorithms, Expert Syst.
Comput. 97 (2020) 106580.
Appl. 113 (2018) 18–32.
[14] T. Tuncer, F. Ozyurt, S. Dogan, A. Subasi, A novel Covid-19 and pneumonia
[39] M. Jin, W. Deng, Predication of different stages of Alzheimer's disease using
classification method based on F-transform, Chemometr. Intell. Lab. Syst. 210
neighborhood component analysis and ensemble decision tree, J. Neurosci.
(2021) 104256.
Methods 302 (2018) 35–41.
[15] T. Ozturk, M. Talo, E.A. Yildirim, U.B. Baloglu, O. Yildirim, U.R. Acharya, €
[40] T. Tuncer, S. Dogan, F. Ozyurt, S.B. Belhaouari, H. Bensmail, Novel multi center and
Automated detection of COVID-19 cases using deep neural networks with X-ray
threshold ternary pattern based method for disease detection method using voice,
images, Comput. Biol. Med. 121 (2020) 103792.
IEEE Access 8 (2020) 84532–84540.
[16] F. Ozyurt, T. Tuncer, A. Subasi, An automated COVID-19 detection based on fused
[41] T. Tuncer, S. Dogan, U.R. Acharya, Automated accurate speech emotion recognition
dynamic exemplar pyramid feature extraction and hybrid feature selection using
system using twine shuffle pattern and iterative neighborhood component analysis
deep learning, Comput. Biol. Med. 132 (2021) 104356.
techniques, Knowl. Base Syst. 211 (2021) 106547.
[17] G. Marques, D. Agarwal, I. de la Torre Díez, Automated medical diagnosis of
[42] N. Karabulut, G. Ozmen Koca, Wave height prediction with single input parameter
COVID-19 through EfficientNet convolutional neural network, Appl. Soft Comput.
by using regression methods, Energy Sources, Part A Recovery, Util. Environ. Eff. 42
96 (2020) 106691.
(24) (2020) 2972–2989.
[18] A. Abbas, M.M. Abdelsamea, M.M. Gaber, Classification of COVID-19 in chest X-ray
[43] T. Tuncer, S. Dogan, F. Ozyurt, An automated residual exemplar local binary pattern
images using DeTraC deep convolutional neural network, Appl. Intell. 51 (2) (2021)
and iterative ReliefF based COVID-19 detection method using chest X-ray image,
854–864.
Chemometr. Intell. Lab. Syst. 203 (2020) 104054.
[19] S. Karakanis, G. Leontidis, Lightweight deep learning models for detecting COVID-
[44] R. Jain, M. Gupta, S. Taneja, D.J. Hemanth, Deep learning based detection and
19 from chest X-ray images, Comput. Biol. Med. 130 (2021) 104181.
analysis of COVID-19 on chest X-ray images, Appl. Intell. 51 (3) (2021) 1690–1700.
[20] A.M. Ismael, A. Şengür, Deep learning approaches for COVID-19 detection based on
[45] M.Z. Che Azemin, R. Hassan, M.I. Mohd Tamrin, M.A. Md Ali, COVID-19 deep
chest X-ray images, Expert Syst. Appl. 164 (2021) 114054.
learning prediction model using publicly available radiologist-adjudicated chest X-
[21] T. Zebin, S. Rezvy, COVID-19 detection and disease progression visualization: deep
ray images as training data: preliminary findings, Int. J. Biomed. Imag. (2020),
learning on chest X-rays for classification and coarse localization, Appl. Intell. 51
https://doi.org/10.1155/2020/8828855, 2020.
(2) (2021) 1010–1021.
[46] I.D. Apostolopoulos, T.A. Mpesiana, Covid-19: automatic detection from x-ray
[22] R.A. Al-Falluji, Z.D. Katheeth, B. Alathari, Automatic detection of COVID-19 using
images utilizing transfer learning with convolutional neural networks, Physical and
chest X-ray images and modified ResNet18-based convolution neural networks,
Engineering Sciences in Medicine 43 (2) (2020) 635–640.
Comput. Mater. Continua (CMC) (2021) 1301–1313.

11