Chest X-ray and CT Scan Classification using Ensemble Learning through Transfer Learning

COVID-19 has posed an extraordinary challenge to the entire world. As the number of COVID-19 cases continues to climb around the world, medical experts are facing an unprecedented challenge in correctly diagnosing and predicting the disease. The present research attempts to develop a new and effective strategy for classifying chest X-rays and CT Scans in order to distinguish COVID-19 from other diseases. Transfer learning was used to train various models for chest X-rays and CT Scan, including Inceptionv3, Xception, InceptionResNetv2, DenseNet121, and Resnet50. The models are then integrated using an ensemble technique to improve forecast accuracy. The proposed ensemble approach is more effective in classifying X-ray and CT Scan and forecasting COVID-19.


Introduction
With the rapid transmission of COVID-19, medical professionals are looking for precise and quick automated detection ways to prevent the disease from spreading while also reducing equipment processing needs. With the passage of time, the virus has transmitted globally, posing an extraordinarily high risk of harming millions of lives around the world and becoming the leading cause of death for many individuals. Because of the high fatality rate associated with COVID-19 cases around the world, countries with the highest number of current cases have encouraged residents to stay indoors and announced a total lockdown to avoid disease transmission. COVID-19 infection is fatal since it may spread by direct or indirect contact with an infected person. Symptoms include fever, dry cough, vomiting, diarrhea, and myalgia, according to the World Health Organization and the Centers for Disease Control and Prevention. Deep learning (DL) has lately proven to bring remarkable advantages to several practical systems. Numerous researchers have attempted to use DL method for diverse multi-class classification such as diabetic Eye Disease [24,25]. The management of pandemic is complicated by the daily increase in positive COVID-19 cases and incorrect diagnoses. With a huge number of infected people and fewer test kits, practitioners must rely entirely on automated detection systems to effectively tackle the pandemic at an early stage. These detection technologies could also help identify persons who need to be isolated to prevent the disease from spreading. The COVID-19 detection laboratory tests took a long time and had a high false-negative detection rate, which included nucleic acid reagent detection and viral antigen detection. The current research proposes a novel mechanism for classifying chest X Rays and CT Scans into appropriate categories using transfer learning, followed by ensemble learning to increase accuracy and other assessment metrics. The main contribution of this research is as follows: • Trained numerous state of the art pretrained models using transfer learning.
• Consider two different cases viz. Xray and CT scan for multi class classification.
• Proposes a new technique of weighted average ensembling method.
• Consider open source dataset for both classification tasks.

2
The paper has been divided into sections in a logical manner. Section II discusses earlier literature on this topic, as well as the flaws that lead to the advancements in this effort. Section III delves into the system's technique in depth, with a focus on the many machine learning algorithms used in the system. The findings, as well as an in-depth investigation of the resolution of the most efficient method and the accompanying outcomes, are presented in Section IV. The suggested work is discussed and compared to existing efforts in Section V. Finally, the report closes with a prospective conclusion and possible research applications.

Review of Literature
Multiple researches have been carried out in the field of classification of X-ray and CT Scans using convolutional neural network.
In [1], using chest X-Ray pictures, the research propose a Deep Convolutional Neural Network based approach for detecting COVID-19 +ve patients. The proposed work uses many state-of-the-art CNN models, including DenseNet201, Resnet50V2, and Inceptionv3.
To address an urgent need for precise diagnostic solutions based on deep learning algorithms, such as convolutional neural networks, the paper [2] uses a deep learning method based on an advanced convolutional neural network architecture known as EfficientNet that was pre-trained on the ImageNet dataset to detect COVID-19 from chest X-ray images, to rapidly detect positive COVID-19 cases.
A classification model that can evaluate chest X-rays and aid in the proper diagnosis of COVID-19 is presented in the research [3]. The chest X-rays are divided into four categories by the methodology presented: normal, pneumonia, tuberculosis (TB), and COVID-19. COVID-19related X-rays are also divided into mild, medium, and severe severity levels. Similarly, in [4], using a five-fold cross-validation scheme, the suggested approach was tested on two publicly available pneumonia X-ray datasets provided by Kermany et al. and the Radiological Society of North America (RSNA), respectively. The paper [5] uses deep learning to develop methods for autonomously interpreting x-ray scattering images in this paper. For x-ray scattering image categorization, Convolutional Neural Networks and Convolutional Auto encoders in particular has been used. In [6], the suggested approach may successfully localize infections of diverse shapes and sizes, particularly small infection areas that have received less attention in recent research. Furthermore, with 99.64 % sensitivity and 98.72 % specificity, the suggested approach exhibited satisfactory COVID-19 detection performance.
The paper [7] proposes a novel CNN model named CoroDet that was demonstrated as a method for automatically detecting COVID-19 in raw chest X-ray and CT Scan pictures. CoroDet was designed to be a reliable diagnostic tool for COVID and Normal, COVID, Normal, and Non-COVID Pneumonia, and COVID, Normal, and Non-COVID Pneumonia (COVID, Normal, non-COVID viral pneumonia, and non-COVID bacterial pneumonia). In [8], with 2022 pneumonia, 2161 COVID-19, and 5863 normal chest X-ray pictures collected from previous papers and other online resources, a Convolutional Neural Network (CNN) based accurate and efficient ensemble model employing deep learning is developed whereas in [9], a dataset of 2470 X-ray images, a novel methodology combining well-known pre-processing techniques, feature extraction methods, and dataset balancing approaches led to an exceptional rate of recognition of 98 % for COVID-19 images.
The paper [10] demonstrates how deep learning can be used to detect COVID-19 with high accuracy using chest Xray pictures. The research involved the training of deep learning and machine learning classifiers using publicly available X-ray pictures (1583 healthy, 4292 pneumonia, and 225 verified COVID-19). In [11], the study used pretrained deep learning models to construct a speedy, accurate, and automatic tool for severity screening and follow-up therapeutic treatment in 202 patients diagnosed with COVID-19, with the DenseNet-201 with cubic SVM model achieving the greatest performance. Furthermore, paper [12] demonstrates a study based on Convolutional Neural Network (CNN) to detect and identify the COVID-19 illness automatically. CNN has been used to implement two different classifications: binary and multiclass classification.
Classification of COVID infected patients is a priority in the pandemic. The paper [13] proposes a methodology to classify COVID-19 (positive) and COVID-19 (negative) patients is based on transfer-learning pre-trained models. The development of KarNet, a deep learning framework that uses pre-trained models (DenseNet201, VGG16, ResNet50V2, and MobileNet) has been described as its backbone. Finally, in [14], this research provides an assisted diagnosis approach based on ensemble deep learning to help medical workers diagnose patients with new corona virus pneumonia efficiently and quickly. The method combines Stacked Generalization ensemble learning with VGG16 deep learning to create a cascade classifier, with the information for the cascade classifier coming from multiple subsets of the training set, each of which is used to collect deviant information about the data set's generalization behavior, and this deviant information filling the cascade classifier. The paper [16] utilizes COVID-19 chest X-ray pictures, a Deep Transfer Learning (DTL) method employing Convolutional Neural Network (CNN) 3 frameworks -InceptionV3, ResNet50, and VGG19were implemented using the Apache Spark platform as an extended data framework. In 2 classifications, COVID-19 and normal X-ray pictures, the three models are evaluated with 100% correctness, however, with slightly lower accuracy in 3 classifications. The study [17] used a big public dataset to evaluate the effects of pre-processing on the output of a classification problem. They evaluated five distinct pre-processing techniques and examined their influence on the results of classification. The paper [18] examines all research on chest radiographs that used deep learning before March 2021, categorizing them by task: picture-level prediction (classification and regression), EAI Endorsed Transactions on Scalable Information Systems 08 2022 -10 2022 | Volume 9 | Issue 6 | e8 segmentation, localization, image production, and field adaption.
In paper [23], multiple state-of-the-art CNN performance has been evaluated and then a CNN has been trained from scratch. Its goal is to create a deep learning-based system that can accurately classify COVID-19 and identify it using chest radiography.
In [24] and [25], the research has employed various state of the art Deep learning networks for identification and classification of Diabetic Eye Disease by using fundus images. The experiment was carried out using ImageNet's top two pretrained convolutional neural network (CNN) models. The current research presents an ensemble learning-based approach that employs a pre-trained network trained on publicly available data. By utilising a broad subset of COVID data, this approach overcomes the limitation of a limited number of COVID datasets used in several studies. The ability to distinguish X-ray dataset in COVID, pneumonia, lung opacity, and normal and CT Scan dataset in COVID, normal and pneumonia allows for early disease detection, diagnosis, and therapy. The elaborate comparisons with the previous works are demonstrated in Table 3 with respect to different parametric in Section V.

Methodology
The proposed study focuses on classifying both X-ray and CT Scan images separately. Both Image datasets (X-ray and CT Scans) were first trained separately using state-of-the-art pre-trained networks employing transfer learning in the proposed study. Inceptionv3, Xception, Incep-tionResNetv2, DenseNet121, and Resnet50 were trained, and their performance was evaluated using metrics such as accuracy, precision, recall, and F1 score.
In 2015, GoogleNet, a deep convolutional architecture inspired by Inception, was released. Later, it was developed in a variety of ways, and InceptionV3 [19] was born as a result of extra factorization. Inception v3 as shown in figure 1 has 24 million parameters and 42 layers, and it performed well on the ImageNet dataset. This network computed using a factorization method, resulting in more accurate findings. The Xception network [20], which stands for Extreme version of Inception, is a modified version of the Inception network with 22.9 million parameters that can be used to solve a variety of picture categorization and object identification challenges. It is even superior than Inception-v3 using an updated depth wise separable convolution. This structure (see figure 2) involves 36 convolutional layers that serve as the foundation for information extraction. The 14 modules make up the 36 convolutional layers.

Figure 2. Xception Architecture [20]
InceptionResnetv2 [21] is based on the Inception family, however, contains residual connections. It features 56 million parameters and 164 deep layers. More than a million photos from the ImageNet collection were used to train the algorithm. Figure 3 shows the compressed view of inceptionResnetV2. ResNet50 has 50 residual layers that are designed as the network develops deeper to solve concerns like time consumption. Its method uses identity function skip connections between layers as shown in figure 4 to improve model accuracy while lowering training time.
DenseNet121 is a feed forward network with 121 layers and over 8 million trainable parameters that connects each layer to every other layer. DenseNet [22] needs minimal inputs than standard CNNs of similar sorts, and the 'vanishing gradient' problem develops as the number of layers in the CNN increases, i.e. as they go deeper. DenseNets solves this problem by streamlining the interconnection arrangement between layers and altering the usual CNN design. Each layer in DenseNet architecture is deeply linked to each and every layer (see figure 5).

Dataset Description
The training for X-ray classification was carried by using a large portion of publicly available dataset from the Kaggle platform [23]. COVID, Pneumonia, Lung opacity, and normal classes are included in the X-ray dataset. There are 3005 COVID photos, 2629 lung opacity images, 2849 normal images, and 1150 viral pneumonia images in this collection and can be found at link. Data processing techniques such as contrast boosting were used to see if they had any effect on the model's training accuracy, but there was no meaningful improvement.
Before training, data augmentation was also used; however, it did not produce superior outcomes. Finally, transfer learning was used to train images in their original dimensions (299*299 pixels), and performance measures like as accuracy, precision, recall, and macro average f1score were obtained to estimate the model's performance.
The dataset for CT Scan was similarly collected from the Kaggle portal [24]. From this a subset was used which consist 2400 COVID, 2658 normal, and 2400 pneumonia pictures, all in axial view and can be found at link. Since the dimensions of the dataset images differed, they were scaled to 299*299 pixels to make them uniform. Multiclass classification is used in both X-ray and CT Scan, and the CT Scan trained model performance was tested using the same matrix as used for the X-ray trained model. Both the X-ray and CT Scan datasets were divided into 60:20:20 training, testing, and validation subsets.
One of the parameters for evaluating classification models is accuracy. However, using either accuracy or a sensitivity/specificity criterion is insufficient, especially for unbalanced data; while one metric can produce better scores, other metrics can produce lower scores.
The harmonic mean of Sensitivity and Precision is the F1-score; this metric can provide a value that indicates the overall quality of the technique. For each model, a confusion matrix has also been displayed as shown in figure 7 and 8. The equation for accuracy, precision, specificity, F1 score and sensitivity are represented by (1)-(5), respectively. Accuracy = + + + + * 100 (1) Ensembling was done after training the pre-trained models for both classification task (X-ray and CT Scan) separately and metrics for ensemble technique was calculated as shown in table 1 and 2. Weighted average ensembling technique which is shown in figure 6 has been proposed to be employed for both the classification task. The weighted average ensemble, also known as the weighted sum ensemble, is a machine learning approach that combines predictions from several models, weighting each model's contribution according to its competence or performance. This method favours the model that performs better (having lower validation error).Let's say error is given by: Weight of i th network: Assume the model's outputs are x0, x1, and x2, respectively, where x0, x1, and x2 are probabilities of class 1, 2, and 3, and the model's weights are w1, w2, and w3. All of the previously mentioned pre-trained models were trained using transfer learning for X-ray classification, and then a performance matrix was generated. Following that, the top four performing models as per table 1, viz. InceptionresnetV2, InceptionV3, DenseNet121, and xception, were ensemble using the weighted average technique and their performance was evaluated using the same matrix as indicated in table 1. In figure 7, the confusion matrix for each model, including the ensemble model, has been displayed. The same pretrained models were individually trained for comparison using same performances matrixes (see table 2) for CT Scan classification, and then the top three models were ensemble using weighted average technique to boost performance even more, and the ensemble technique performance matrixes are listed in table 2.  Chest X-ray and CT Scan Classification using Ensemble Learning through Transfer Learning

Results and Analysis
This section analyses the results. An Intel i7 processor with 16 GB of RAM and a graphics processing unit (GPU) was employed as the system infrastructure. The first part of classification task involves classifying Xrays into respective classes by employing the dataset [23] (link) and its analysis is presented in section 4.1. Table 1 shows results of different performance metrics employed for comparison. The confusion matrix, that is used to assess efficiency of the algorithms, has been plotted. It evaluates target values to the deep learning model predictions. Confusion matrix can also be used to estimate other performance metrics like precision and recall. The second part which classifies CT Scans into respective categories by utilizing the above mentioned dataset [24] (link) and its analysis is presented in section 4.2. Table 2 shows the results of different performance metrics. The proposed approach of weighted average ensembling technique outperforms the other state of the art pretrained models for the classification tasks as can be verified from

X-ray Classification
1. InceptionV3 presents the best result, with a mean accuracy of 88.48% and an F1 score of 89%. 2. Figure 7 figure 8 (a), (b), (c), (d), (e), (f) and the training and validation accuracy loss curve can be found at CT Scan 6. Adam optimizer was employed during training and 100 epochs were initialized for training.

Discussion
The present work summarizes the application of deep learning in distinguishing COVID from other diseases using chest X-rays and CT Scan images. Two different machine learning models using X-ray and CT Scan image dataset are employed separately. For both the classification tasks, multiple models were extensively trained and tested on test data as shown in table 1 and 2. The dataset has been divided into 4 and 3 categories for X-ray classification and CT Scan classification respectively. Finally, the proposed ensemble approach was applied for both dataset and it shows improved results as can be verified from the result tables 1 and 2.
The confusion matrices have been plotted in figure 7 and 8 for each model which shows the overall proficiency of the employed models. Matrix compares the true value with the model predicted value.
An accuracy of above 96% for CT scan and 89% for Xray classification with large set of data has been achieved using mentioned approach.
Even though data augmentation and picture enhancement were performed on both datasets, the results prove to be inferior to the one obtained without these approaches that is why these methods are not employed for the classification task (tables 1 and 2).  [23] employed three pre-trained models and CNN frameworks with a small set of data overcoming overfitting and poor generalisation issues of the test data and achieve an accuracy of 100% for binary classification but for multiclass it achieves 83.50% accuracy which further got increase to an 87.5% with fine tuning. In terms of quantity and categories of dataset, the proposed approach in this study has obtained sufficient accuracy when compared to prior studies in both circumstances. The findings show that the work done in this study performed EAI Endorsed Transactions on Scalable Information Systems 08 2022 -10 2022 | Volume 9 | Issue 6 | e8 well in various parameters when compared to other methodologies.

Conclusion
The COVID-19 pandemic has wreaked havoc on people's lives and has turned into a global health emergency. Though the academic and medical community has made significant efforts on various fronts, the virus continues to spread at a rapid pace. Different DL and ML algorithms have been developed to identify the illness at an early stage. Since COVID-19 is extremely infectious, it is critical to appropriately manage its transmission channel in order to prevent the disease from transmission. The proposed work successfully employs four different pre-trained deep neural network architectures (Inceptionv3, Xception,InceptionResNetv2 and DenseNet12) to create a deep ensemble learning architecture for X-rayCOVID-19 identification and 3 pretrained architectures (Xception, InceptionResNetv2 and DenseNet12) for CT Scan ensemble classification. It is a robust model and could prove to be an extremely valuable tool that can assist doctors and researchers in accurately detecting CO-VID-19.