Research Accomplishment

PLOS ONE
RESEARCH ARTICLE
Evaluating the Brexit and COVID-19’s

influence on the UK economy: A data analysis
Raghav Gupta1, Md. Mahadi Hasan2, Syed Zahurul Islam ID3, Tahmina Yasmin4,
Jasim Uddin ID1*
1 Cardiff School of Technologies, Cardiff Metropolitan University, Cardiff, United Kingdom, 2 Department of
Computer Science and Engineering, Asian University of Bangladesh, Ashulia, Dhaka, Bangladesh, 3 Power
Integration System, Faculty of Electrical and Electronic Engineering, Universiti Tun Hussein Onn Malaysia,
Parit Raja, Johor, Malaysia, 4 School of Geography, Earth and Environmental Sciences, The Institute for
a1111111111 Global Innovation, University of Birmingham, Birmingham, United Kingdom
a1111111111
* juddin@cardiffmet.ac.uk
a1111111111
a1111111111
a1111111111
Abstract
The economic landscape of the United Kingdom has been significantly shaped by the inter-
twined issues of Brexit, COVID-19, and their interconnected impacts. Despite the country’s
OPEN ACCESS robust and diverse economy, the disruptions caused by Brexit and the COVID-19 pandemic
Citation: Gupta R, Hasan M.M, Islam SZ, Yasmin have created uncertainty and upheaval for both businesses and individuals. Recognizing
T, Uddin J (2023) Evaluating the Brexit and COVID- the magnitude of these challenges, academic literature has directed its attention toward
19’s influence on the UK economy: A data analysis.
PLoS ONE 18(6): e0287342. https://doi.org/
conducting immediate research in this crucial area. This study sets out to investigate key
10.1371/journal.pone.0287342 economic factors that have influenced various sectors of the UK economy and have broader
Editor: Umer Shahzad, University of Galway,
economic implications within the context of Brexit and COVID-19. The factors under scrutiny
Ireland / Anhui University of Finance and include the unemployment rate, GDP index, earnings, and trade. To accomplish this, a
Economics, CHINA range of data analysis tools and techniques were employed, including the Box-Jenkins
Received: January 3, 2023 method, neural network modeling, Google Trend analysis, and Twitter-sentiment analysis.
Accepted: June 4, 2023
The analysis encompassed different periods: pre-Brexit (2011-2016), Brexit (2016-2020),
the COVID-19 period, and post-Brexit (2020-2021). The findings of the analysis offer intrigu-
Published: June 15, 2023
ing insights spanning the past decade. For instance, the unemployment rate displayed a
Peer Review History: PLOS recognizes the downward trend until 2020 but experienced a spike in 2021, persisting for a six-month
benefits of transparency in the peer review
process; therefore, we enable the publication of
period. Meanwhile, total earnings per week exhibited a gradual increase over time, and the
all of the content of peer review and author GDP index demonstrated an upward trajectory until 2020 but declined during the COVID-19
responses alongside final, published articles. The period. Notably, trade experienced the most significant decline following both Brexit and the
editorial history of this article is available here:
COVID-19 pandemic. Furthermore, the impact of these events exhibited variations across
https://doi.org/10.1371/journal.pone.0287342
the UK’s four regions and twelve industries. Wales and Northern Ireland emerged as the
Copyright: © 2023 Gupta et al. This is an open
regions most affected by Brexit and COVID-19, with industries such as accommodation,
access article distributed under the terms of the
Creative Commons Attribution License, which construction, and wholesale trade particularly impacted in terms of earnings and employ-
permits unrestricted use, distribution, and ment levels. Conversely, industries such as finance, science, and health demonstrated an
reproduction in any medium, provided the original increased contribution to the UK’s total GDP in the post-Brexit period, indicating some posi-
author and source are credited.
tive outcomes. It is worth highlighting that the impact of these economic factors was more
Data Availability Statement: All relevant data are pronounced on men than on women. Among all the variables analyzed, trade suffered the
within the paper and its Supporting information
most severe consequences in the UK. By early 2021, the macroeconomic situation in the
files.
country was characterized by a simple dynamic: economic demand rebounded at a faster
Funding: NO.
PLOS ONE | https://doi.org/10.1371/journal.pone.0287342 June 15, 2023 1 / 33

PLOS ONE Evaluating the Brexit and COVID-19’s influence on the UK economy
Competing interests: The authors have declared pace than supply, leading to shortages, bottlenecks, and inflation. The findings of this
that no competing interests exist. research carry significant value for the UK government and businesses, empowering them
to adapt and innovate based on forecasts to navigate the challenges posed by Brexit and
COVID-19. By doing so, they can promote long-term economic growth and effectively
address the disruptions caused by these interrelated issues.
1 Introduction
Over the past decade, the United Kingdom (UK) has undergone significant transformations
that have shaped its political and economic landscape. Two major events stand out during this
period: the decision to pursue Brexit and the emergence of the COVID-19 pandemic. Brexit,
which concluded on January 31, 2020, marked the end of the UK’s longstanding 47-year alli-
ance with the European Union [1]. Concurrently, the UK was confronted with the first cases
of COVID-19 within its borders, and by March 2020, the pandemic had swept across the
globe, causing widespread fear and discontent. In response to the COVID-19 outbreak, the
UK, like many other affected countries, implemented stringent measures, including social and
economic restrictions such as lockdowns. While these measures aimed to curb the spread of
the virus, they came at a significant cost to both society and the economy [2]. The convergence
of Brexit and COVID-19 has raised concerns among experts, who predict substantial eco-
nomic consequences for the UK, especially during the post-Brexit transitional phase and nego-
tiations with the EU. The combined impact of Brexit and the COVID-19 pandemic is expected
to reverberate across various regions and industries within the UK [3]. Therefore, it is crucial
for the British people to have access to accurate information about the economic ramifications
of these events. The prevailing uncertainty has created unease among investors, resulting in a
decline in consumer confidence. Additionally, the British currency, the pound, is anticipated
to face significant depreciation in the near future, further exacerbating the existing economic
challenges.
In terms of long-run effects, ‘Brexiteers’ argue that leaving the EU will benefit Britain, while
‘Remain’ advocates warn of significant economic harm [4]. This is due to the fact that, by leav-
ing the EU, the UK would, on the one hand, have completed decision-making over its future
laws and policies, as well as the ability to make its own decisions, so opening up countless
growth opportunities. However, when firms experience challenges dealing with the EU, such
as border delays and high administrative costs which may result in more unemployment/relo-
cation of personnel, a drop in trade/investment, and, as a result, lower profitability. This will
lead to an increase in costs and prices, while competitiveness falls [5]. Further, COVID-19
seems to have the possibility to have far-reaching economic and structural consequences for
the United Kingdom economy and its workforce. Recently, the indirect costs of lessening and
suppressing the pandemic have risen, and with the beginning of COVID-19, existing trends
have accelerated [6], for example, the shift towards online shopping to a great extent and the
emergence of more employees who work from home [7, 8]. Moreover, because of national-
lockdowns and travel restrictions imposed by COVID-19; the UK’s domestic and international
commerce has been negatively affected, which has further impacted numerous firms and
organisations.
As a result, both Brexit and COVID-19 possibly cause a long-term re-organisation of the
UK economy, with continuing ramifications, which might be felt in upcoming years. Due to
these discrepancies in forecasts, a detailed examination is required to establish the absolute
economic consequence of these events on the UK. Several economists have already

investigated the economic impact of Brexit and the COVID-19 outbreak in the UK. The analy-
sis demonstrates that the consequences currently have been negative since the UK’s overall
growth, share prices, trade, and exchange rates have mostly deteriorated. Regional disparity is
predicted to be exacerbated by the impacts of COVID-19 and Brexit on the economy. The
most vulnerable places are London, the Northeast, Wales, the Southeast, and the West Mid-
lands, as these coastal communities depend on tourism and cities are reliant on hospitality for
income [9]. In addition, researchers have used sentiment and trend analysis to examine public
reaction in the UK and other impacted nations [10]. The results suggest that the majority of
the public is concerned about the economic consequences of both Brexit and COVID-19 in
the UK. According to the existing studies, the higher government authorities should endeav-
our to help businesses adapt to new trading agreements and keep border costs as low as feasi-
ble. However, the impact of these events on the UK has yet to be documented. COVID-19’s
economic impact is unlikely to be felt in the same areas and industries that are most vulnerable
to Brexit. As a result, this research builds on previous research by evaluating the disparities in
the consequences of recent events among UK regions and industries, as well as the evolution
of the UK economy over the last decade (2011–2021).
This study aims to explore the changes in the UK economy in three key areas: growth, the
people, and trade, as a result of the occurrences of COVID-19 and Brexit. To assess the impact
of these events, various data analysis tools and techniques, including the Box-Jenkins method,
neural-network modeling, Google Trend analysis, and Twitter-sentiment analysis, were
employed. These methods were used to predict and forecast economic factors such as unem-
ployment, gross domestic product (GDP), earnings, and trade in the UK.
The primary objective of this study is to compare the economic levels of the UK before and
after the financial crisis. Additionally, the study analyzes and visualizes economic data by
region and industry in the UK to identify differences in the impacts throughout the country.
Data analytic models were developed and assessed using Python to gain insights into these var-
iations. Furthermore, this study attempts to predict the future consequences of economic fac-
tors such as unemployment, GDP, earnings, and trade. Through social media sentiment
analysis and trend analysis, the study also investigates the reactions of the British public to
these situations.
This study aims to explore the changes in the UK economy in three key areas: growth, the
people, and trade, as a result of the occurrences of COVID-19 and Brexit. To assess the impact
of these events, various data analysis tools and techniques, including the Box-Jenkins method,
neural-network modeling, Google Trend analysis, and Twitter-sentiment analysis, were
employed. These methods were used to predict and forecast economic factors such as unem-
ployment, gross domestic product (GDP), earnings, and trade in the UK.
The primary objective of this study is to compare the economic levels of the UK before and
after the financial crisis. Additionally, the study analyzes and visualizes economic data by
region and industry in the UK to identify differences in the impacts throughout the country.
Data analytic models were developed and assessed using Python to gain insights into these var-
iations. Furthermore, this study attempts to predict the future consequences of economic fac-
tors such as unemployment, GDP, earnings, and trade. Through social media sentiment
analysis and trend analysis, the study also investigates the reactions of the British public to
these situations.
This study provides valuable information on how the UK economy has changed and how
individuals have responded in the context of Brexit and COVID-19. The research emphasizes
the need to consider specific economic factors to control for multicollinearity and endogenous
variables. Multicollinearity occurs when independent variables are highly correlated in a mul-
tiple regression equation, which can undermine the statistical significance of an independent

variable. Therefore, the study focuses on economic indicators that are less influenced by exter-
nal causes to ensure robust analysis. While other factors could have been included, the study
chose to examine key economic indicators that are not significantly affected by external fac-
tors. However, future research could explore additional factors to generate a more comprehen-
sive analysis.
Brexit has had a substantial impact on the UK’s trade with the EU, which was previously
the country’s largest trading partner. Exiting the EU required the UK to renegotiate its trade
agreements with the bloc, resulting in increased tariffs and trade barriers. As a consequence,
trade volume between the UK and the EU has declined, with certain industries, such as the
automotive sector, being particularly affected. Moreover, Brexit has influenced investment in
the UK, leading to many companies relocating their operations to other EU countries. This
has resulted in job losses and reduced economic growth within the country. However, the
changing relationship with the EU has also presented opportunities for investment in sectors
like technology and finance. The impact of Brexit on employment in the UK has been mixed,
with some industries, like finance, experiencing job losses while others, like agriculture, have
witnessed increased employment opportunities. The UK’s ability to attract foreign workers has
also been affected, contributing to skills shortages in specific sectors. Overall, Brexit has had a
significant negative impact on the UK’s economy, with certain sectors being hit harder than
others. While there have been opportunities for growth in select industries, the overall effect of
Brexit on the UK’s economy has been detrimental, resulting in reduced trade, investment, and
economic growth.
The following paper is structured as follows: In section 2 highlighted the existing literature
and the consequence of Brexit and COVID-19 in the UK economy. In section 3 displayed the
different data analysis, models and various methods are described in theoretically for a better
understanding of the data. In section 4 presents the detailed parameters of various results sec-
tion that shows the data visualisation. In conclusion, there will be a brief discussion of how
these findings may affect the outcome and further point to the limitations and future work.
2 Background study
The UK’s European Union (EU) membership was decided through a June 23, 2016, referen-
dum, with 51.9% voting to leave out of 33,551,983 total votes. This outcome has caused uncer-
tainty in the tourism sector, impacting consumers, industry players, and policymakers [11]. In
the de-internationalization perspective [12], it was often perceived as a failure, discouraging
firms from limiting or discontinuing exports to foreign markets. Definitions of de-internation-
alization were mostly limited, but some comprehensive ones acknowledge its partial or com-
plete nature. However, they often associate it with voluntariness rather than a strategic
response to changes in the global business environment. The lack of transparency in formal
policies and their impact creates a growing risk for international business [13]. This risk mani-
fests not only through partial de-internationalization, such as decoupling, but also through
complete withdrawal and exit from the international market. Compliance with minimal regu-
latory requirements poses a significant threat, leading firms to strategically opt for de-interna-
tionalization as a means to limit their involvement in international operations.
Together, with the rising importance of research in the economic repercussions of COVID-
19 as a global concern and Brexit, the experimental investigations of the economic factors
(unemployment, GDP, earnings and trade) of these incidents are limited. Campos et al. [14]
examine the effect on migration and trade between the UK and the EU due to Brexit and uti-
lised a structural gravity model. Further undertaking a quantitative analysis, the authors shows
that there have been severe repercussions for the UK’s trade and migratory patterns. Kaminska

et al. [15] signifies published documents on the consequences of Brexit, such as the House of
Commons Treasury Committee (2018), HM Treasury (2016) (Crafts, 2016) and the Bank of
England (2018). Despite each of these published papers adopting different scenarios and tech-
niques in analysing the consequences, their findings were common and indicate undesirable
effects on the UK economy. The IMF (2018) estimated that production will fall by 2%—8% as
a result of Brexit. Their analysis includes a simple calculable general equilibrium model and a
number of assumptions about the relative sizes of various Brexit transmission channels (Inter-
national Monetary Fund, 2018). In another study, a cross-section of pre- and post-Brexit sur-
veys were conducted to look at the industrial development (IDV) nexus in the UK economy
[16]. The purpose of this research was to look at the primary drivers of industrial development
in the UK before and after the implementation of a financial reform plan. Researchers
employed multiple regression analysis to anticipate changes in the dependent variable (IDV)
induced by independent factors (Brexit). Later, shocks were described, quantified, and
explained using the impulse response function (IRF), vector auto-regression (VAR), and vari-
ance decomposition. The data demonstrated that all independent factors, trade openness,
equity openness, and capital account openness were significant predictors of commercial
growth in pre-Brexit polls. However, regression analysis shows that only equity openness (EO)
and trade openness (TO) had a significant influence on Post-Brexit Polling Industrial Develop-
ment (IDV) at the 5% level of significance. As a result, following Brexit, the authors recom-
mended that the UK should rethink important financial policies in order to boost future
industrial growth [16].
The interdependence of economic and public health has become evident during the pan-
demic [17]. It highlighted that a nation’s economic well-being relies on the health of its citi-
zens, while also emphasizing the global significance of people’s well-being. The imposition of
lockdown measures worldwide demonstrated the need to pause economies to control the
spread of SARS-CoV-2. Although economic health metrics like consumer price index (CPI),
gross domestic product (GDP), and unemployment rate are well-established, public health
measures can be enhanced by considering contemporary indicators such as emotional well-
being and mental health. These measures can have significant implications for human capital
and productivity in the economy, especially during times of crisis like the pandemic.
Unlike past crises such as aviation tragedies, natural disasters, and supranational union
exits [18], the COVID-19 pandemic has had an exceptional impact on the tourism and gam-
bling sectors. Unprecedentedly, widespread lockdowns were implemented in cities and coun-
tries, an unprecedented measure. While these actions are vital for saving lives and demand
global cooperation to contain infectious outbreaks like COVID-19, they also pose a substantial
employment risk, particularly in tourism-dependent cities where it constitutes a primary
source of income.
Researchers have examined the impact of Brexit using a variety of data analytic approaches.
In order to analyse the Sterling Pound’s predictability, [19], used Google Trends data from the
last five years to do a prediction analysis on the Pound’s exchange rates to Euro and Dollar.
The goal of the study is to determine if the pound and Google query data are connected by
examining the relationship between the pound and Google query data on ‘Pound’ keywords
and subjects from the 2016 UK referendum through January 31st, 2020. The findings reveal
that there are statistically significant quantile correlations between Google query data and
pound exchange rates, pointing to one of the field’s most important implications: detecting
whether changes in one economic measure elicit reactions in other economic measures.
Simionescu et al. [20] also look at how Brexit has affected the monthly unemployment rate
since the referendum. This is one of the most important indicators of the country’s long-term
development. This study stands out because it uses microdata to illustrate the political

uncertainty caused by Brexit, with Google Trends serving as the data-collecting tool. Statistics
for the four countries that make up the United Kingdom (Wales, Northern Ireland, England,
and Scotland) are also analysed using a multilayer and panel data structure. Despite the paucity
of data to back this assertion, the findings are consistent with an analysis of important macro-
economic indicators, demonstrating that uncertainty caused by Brexit lowered unemployment
from June 2016 to March 2019. The study suggests that government policies should stimulate
investment in order to aid the UK’s future financial expansion and growth. On the other hand,
Keogh-Brown et al., [7] assess COVID-19’s economic impact in the UK by considering direct
illness consequences, public health interventions, and policies. COVID-19, according to their
findings, may have a £39.6 billion economic impact on the UK (1.73 percent of GDP). The
data reveal that COVID-19 has the potential to wreak havoc on the UK economy, and while
the government attempts to reduce mortality appear to be vital; the length of the school and
corporate closures is crucial in deciding the economic impact. Finally, the study suggests that
the UK government’s first economic assistance measures may need to be enhanced if the pan-
demic is to be adequately handled without triggering the collapse of many enterprises and the
loss of many workers’ jobs. Furthermore, researchers wanted to know how people felt about
the COVID-19 vaccination before it was released in the United States and the United
Kingdom.
The COVID-19 pandemic led to a global quarantine in 2020, affecting economies world-
wide, including Malaysia [21]. The quarantine measures aimed at controlling the public health
crisis had mixed effects on economies. The agricultural sector faced significant challenges due
to movement restrictions, resulting in disruptions in the farm-to-consumer supply chain.
Travel limitations had a notable impact on food delivery and all stages of agricultural produc-
tion. Although panic buying has diminished and logistical issues for fresh produce are being
addressed, there are concerns about the country’s ability to maintain food self-sufficiency dur-
ing future emergencies and potential trade disruptions.
Externalities are external events or shocks that can influence the internal operations of
organizations and industries [22]. The global impact of the COVID-19 pandemic exemplifies
such an externality, affecting a wide range of organizations and industries worldwide. Exter-
nalities often drive research and strategic discussions among academics, professionals, and
policymakers. However, it is important to recognize the limitations of descriptive and associa-
tion-oriented research in accurately predicting and prescribing causal effects, as they may lack
essential factors like boundary conditions. Drawing causal conclusions from non-causal
research designs is inappropriate.
According to another study by Leppeman et al. [10], between July 28 and August 28, 2020,
COVID-19 vaccine-related social media posts from the US and the UK were analysed using
neural linguistic programming with human validation. The sentiment analysis investigated the
polarity of the comments (positive, neutral, and negative) as well as the themes that appeared
in the negative ones. The US and the UK had a net sentiment profile of around 8% negative,
28% positive, and 63% neutral in 243,883 social media posts. Further data revealed that senti-
ments about COVID-19 vaccinations on social media in the two countries varied significantly.
There were detected variations in negative emotion themes. Negative sentiments in the US
stemmed mostly from health and safety concerns, the fear of obligatory vaccination, and the
role of pharmaceutical firms in vaccine distribution. In the UK, the major cause of criticism
was the concern about mandating vaccination (almost doubling the amount of vaccine). In the
third quarter of 2020, the UK exhibited widespread opposition to COVID-19 vaccinations.
According to the authors, authorities in both nations may use the reasons for unpleasant emo-
tions to develop evidence-based initiatives to counteract COVID-19 vaccine rejection. Knipe
et al. [23] also utilised Google Trends data (from January 1, 2020, to June 9, 2020) to examine

the shifting resident anxiety trend in the UK in response to the government operations, as
measured by fluctuations in the search frequency for mental agony, coping, and resilience.
During the outbreak, it analysed how particular themes changed in connection to significant
dates, as well as the most popular phrases. COVID-19 was no longer related, either directly or
indirectly, to the inquiries with the most significant rise over time.
The global COVID-19 pandemic has not only caused a public health crisis resulting in loss
of life and widespread suffering but has also severely strained economies worldwide [24].
Lockdown measures were implemented across the globe, including in Malaysia, leading to a
temporary halt in economic activities and subsequent reorganization. This unprecedented
phenomenon is conceptualized in this chapter as the “quarantine economy.” Although the ini-
tial pause was brief, governments gradually reopened their economies while implementing
new social practices such as remote work, physical distancing, and visitor records. These mea-
sures facilitated the adaptation of economic activities to the ongoing pandemic situation,
resulting in a reconfiguration of how businesses operate.
In regard to the Business to Business (B2B) marketing [25], it has increased focus on mar-
keting issues and their implications in the increasingly turbulent B2B market. It sheds light on
B2B marketing strategies that are relevant during major crises, employing the marketing mix
as an organizing framework and utilizing the COVID-19 pandemic as a natural context for
transformative marketing.
Specifically in the transformative marketing emphasizes “competition” and “superior
value,” which may contradict the goal of delivering “benefits to all stakeholders” due to poten-
tial casualties (e.g., dropouts from competition) or costs (e.g., hidden sacrifices or tradeoffs)
associated with such focus. However, the lessons learned from collective efforts in combating
the challenges of COVID-19 suggest that “collaboration” and “shared prosperity” are crucial
for leveraging limited resources and providing agile and valuable responses. These lessons
ensure that no one is left behind during times of adversity.
The digital economy and advancements in technology have made global business an ever-
present force [26]. Firms now have the ability to automate processes, target customers, and
access larger global markets. The COVID-19 pandemic has further emphasized the importance
of global business, enabling firms not only to survive but also to thrive, contributing to the
overall functioning of the economy and society.
What distinguishes the current period from past global crises is the remarkable speed at
which firms have shifted from survival mode to pursuing growth and success. Industries like
cleaning, delivery, and technology services have showcased exceptional adaptability. The
unprecedented changes brought about by the pandemic have created a new normal, demand-
ing the exploration of new ideas and a reassessment of existing ones. This is vital for forging a
transformative path forward and attaining organizational excellence in the realm of global
business.
In the latest study, Sharma et al. [27] addressed both the public’s concerns and the expand-
ing search trend as the number of COVID-19 cases increases in important nations. Statistics
from eight major nations (Spain, China, the US, Italy, the UK, India, Iran, and France) were
collected for their analysis. In these eight countries, the Google search Trend for “COVID-19”
was analysed. The expanding Google Trend reflected the public’s sadness, anguish, and fear in
response to the epidemic. From March 10th to April 10th, 2020, the trend has grown signifi-
cantly. Throughout the observation period, the Google interest wave in the examined nations
demonstrated a sequence of high values. The average interest amount for Google has been
determined for two additional periods: (01 January to 29 February) and (01st March to 10th
April). This indicates that public response has increased over time. According to the study,
Google Trends may be utilised to discover which regions of the country are least and most

affected. Numerous additional researchers, like Chandio and Sah [28] and del Gobbo et al.
[29], have utilised Twitter sentiment analysis, whilst Garcia and Berton [30] and Georgiadou
et al. [31] integrated big data analysis with sentiment analysis and event study to capture the
global effect of Brexit and COVID-19.
3 Methods
The study has been classified into three essential areas: development, exchange, and individual
(person) of the United Kingdom. It examines how the combined catastrophes affected the UK
economy and how individuals responded by utilising industry and region-specific data.
Numerous data analysis methods and methodologies, such as neural networks and Google
Trend Analysis have been utilized for investigating the future economic variables of the United
Kingdom in order to measure the effects of both Brexit and COVID.
This research obtained data extraction procedure and the methodology utilised for the anal-
ysis and to investigate the economic trajectory and public reaction in the United Kingdom
before and after Brexit and COVID-19. For the study of Brexit and COVID-19, the study has
collected the required information (data) for the past decade from 2011 until 2021. This study
investigates the differences between Pre-Brexit, Brexit, Post-Brexit, and COVID-19. The
required data has been collected from reputable web sources the Office for National Statistics
(ONS), Google Trends, and Twitter. ONS is the UK Government website that provides open
and transparent access to UK economic statistics. The Office for National Statistics (ONS) is
dedicated to being open and transparent about the data they hold. It also shows where that
data has come from and gives the main uses of that data. Various data visualisation and data
analysis technologies are utilized to examine the obtained data that helps to investigate the
impact of Brexit and COVID-19 among the industry sector, and its people of the United King-
dom. In addition, to validate the dataset and forecast the effects of the UK economic condi-
tions on unemployment, GDP, earnings, and trade. The extracting data has been analysed and
measured using statistical numerical software. For data forecasting validation and predictive
analysis, a neural network model has been proposed using Python.
Explicit Statement: In light of the economic trend and public reaction across the United
Kingdom, this study is classified into three key stages: 1) before; 2) during; and 3) after Brexit
and COVID-19, throughout the years between 2011–2021. The extracted data (ONS, Google
Trends, and Twitter) was then analysed using relevant statistical software tools to facilitate
data conversion and measurement comparisons. Python has been employed to construct a
neural network model to perform data validation, and the Box-Jenkins method was imple-
mented for forecasting and predictive analysis. In addition, this study also evaluated Twitter
sentiment using the Twitter API and the Text-Blob method in Python. The relevant studies
were identified from entirely open sources that fully complied with the terms and
conditions.
3.1 Data analysis

The information on the UK’s economic variables originates from the National Statistics
sources (Office of National Statistics, 2020), which allows a comprehensive investigation of
projected changes across different economic outcomes, regions, and businesses in the UK as a
result of Brexit and COVID-19. The data obtained from ONS is a time series of quarterly data
(42 quarters) over the past ten years (2011–2021) and encompasses among each the instances
under analysis, Brexit and COVID-19. This study investigates three different time periods
between 2011 and 2021 for comparing the UK economy prior to, during and following the sit-
uations such as “Pre-Brexit” [Q1 (January to March) 2011 to Q2 (April to June) 2016]; “Brexit”

[Q3 (July to September) 2016 to Q4 (October to December) 2019] and, Post-Brexit & COVID-
19 [Q1(January-March)—Q2 (April-Jun), 2021]. The data has been gathered based on the four
most important economic factors: “Unemployment,” “Gross Domestic Product (GDP) index,”
“Earnings,” and “Trade.” In addition, the data collection was categorized into four regions
namely—England, Scotland, Wales, and North Ireland and the twelve industries including
Agriculture, manufacturing, Construction, Wholesale, Transport, Accommodation, Financial,
Science, Public Admin, Education, Health, and Others. In addition, to represent the impact of
the British populace, gender-specific economic data is produced (men and women).
The unemployment rate varies between 3.8% to 8.4%, with an average of 5.65%, as shown
in Table 1. Additionally, the GDP has varied from 73.3% while 102.5%, average weekly earn-
ings are £595.05, and average net trade stands at 10140.88 million pounds. The table also
shows the Pre-Brexit conditions were the most adverse on average, with the largest unemploy-
ment rate, the lowest GDP growth rate, and the lowest incomes. On the other hand, during the
Brexit transition phase, the economy improved, but net trade was severely impacted. In the
post-Brexit and COVID-19 period, the economy initially endured severe economic catastro-
phe, but subsequently started to recover when GDP and profits began to climb. This demon-
strates that the UK has experienced a transition over the last decade This study begins with a
graphical depiction of the quantitative data using line and bar graphs for each economic conse-
quence utilising various Microsoft Excel and Python tools.
3.1.1 Neural network model. In this section, a neural network regression model has been
developed. It has been used in different disciplines and the regression analysis applied to the
extent of the relationship between a dependent variable and a set of independent variables
[32].
The neural network (NN) is a substantial deep learning technique as it can be trained to
estimate the parameters and incorporate the behaviour provided by the samples. The NN
model consists of an activation functional as well as a network structure with an input layer,
several hidden layers, and an output layer. Based on the significance of each neuron input,
the functions determine whether or not each neuron should be triggered for predictions.
The output of a neuron is the product of the input value times the weight, which is then sent
to the next layer [33]. Deep learning neural network models utilize nonlinear activation
functions, which are crucial for learning and modeling complicated data [34]. Deep learning
neural networks may reuse characteristics gained in one hidden layer in subsequent hidden
layers; hence, NN modeling was utilised for this investigation. This enables a deep neural
network to measure performance and imitate a range of natural tasks with a limited number
of weights and units [35]. This method facilitates the generation of improved regression
results. Python is used to create the NN model using the UK economic dataset collected
earlier.
Table 1. Summarizes the descriptive statistics of the four main UK’s economic factors analysed in this paper in all the 42 quarters between the years 2011–2021.
Model MCC Cohen’s kappa Hamming Loss Precision
Count 42 42 42 42
Mean 5.65 87.97 595.05 -10140.88
Std.Dev 1.58 9.11 40.31 2064.41
Min. 3.8 73.3 540.62 -16400.66
Median 5.1 87.3 585.93 -10108.5
Max. 8.4 102.5 677.001 -4677
https://doi.org/10.1371/journal.pone.0287342.t001

3.2 Box-Jenkins method

This study examines the Box-Jenkins method, the fundamental and most widespread tech-
nique for time series analysis [36, 37]. These univariate models aim to better interpret and pre-
dict future observations for a single time-dependent variable. Box Jenkins has been recognized
as an attractive choice for data sets that are mostly consistent and have moderate volatility. The
fundamental assumption of these models appears the data remain stable. The strategy for
detecting outcomes relies on data point variations. Researchers must account for hysteresis
and reduce any turbulence and periodicity as feasible from older data sets. This approach
enables the model to identify patterns utilising three fundamentals such as autoregression,
differencing, and moving average [38].
This study has estimated the Box-Jenkins model using Python. It has been used in three
consequence steps, a Box-Jenkins time series model is developed. In this stage, the data and
other relevant information are utilised to choose a sub-section of the model that would eventu-
ally summarise the data. The software generates the Autocorrelation Function (ACF) graph,
which displays the association between lag values and an observation. In addition, a partial
autocorrelation function (PACF) graph depicting correlations for observation with lag values
utilising the earlier lagged data are created. There are various approaches have been used to
examine the stationarity features of time series data [39].
The null hypothesis of the test is that the time series is nonstationary due to the presence of a
unit root. The test’s p-value is utilized to evaluate the test result [40]. If the p-value is less than
the threshold (5% or 1%), the null hypothesis is rejected and the time series is stationary. If the
p-value is greater than the threshold, the null hypothesis is not rejected, and the time series has
been deemed non-stationary. Kwiatkowski-Phillips Schmidt-Shin (KPSS) is a sort of Unit root
test or statistical technique used to determine if a series is stationary around a deterministic
trend [41]. To reject the null hypothesis, the test statistic must exceed the basic values provided.
The p-value should be low if the desired critical value is genuinely exceeded. The KPSS statistic
will exceed the critical threshold of 5 percent if the p-value is less than 0.05. The number of lags
shown is the number of lags utilized by the model equation of the KPSS test. Typically, one
must define the requisite p, d, and q variables in a basic auto regressive integrated moving aver-
age (ARIMA) model. To decrease non-stationarity, it is generated the values using statistical
methods by conducting a difference. In Auto ARIMA, the model will determine the ideal p, d,
and q parameters for the data set to provide more accurate forecasts. The last stage of the model
evaluates the fitted model considering the available data and looks for regions where it may be
improved. Two diagnostic factors to consider are overfitting and residual errors. Initially,
assessed if the model is overfitting the data. This often indicates that the model appears more
complex than it is and that output from the training set has been captured. This is a concern in
time series forecasting because it restricts the model’s capacity to generalize, resulting in poor
out-of-sample prediction and forecast performance. Both in-sample and out-of-sample perfor-
mance must be considered, necessitating the development of a comprehensive model evalua-
tion test harness. The residual forecasts are a valuable diagnostic tool and the predicted residual
time series would lack temporal structure in an ideal model. Furthermore, in an ideal model,
the anticipated residual time series would have no temporal structure. There is a serial correla-
tion in the residual errors, indicating that this information may be included in the model.
Following the study of economic data analysis, this study has undertaken a sentiment and
trend analysis of social media to obtain a better understanding of how the British public
reacted to the consequences of Brexit and COVID-19. According to previous studies, several
approaches and procedures, such as Google Trend and Twitter sentiment analysis, are utilised
to determine how individuals reacted to the Brexit and COVID-19 events.

4 Results
4.1 Unemployment rate
This study describes the process here on the key findings of the various analysis conducted to
determine the impact of Brexit and COVID-19 on the UK. Initially, the four economic param-
eters Unemployment, Earnings, GDP, and Trade are examined in relation to the events based
on the timeframe from 2011 to 2021 in a particular region, industry, and gender areas. The
neural network regression model is evaluated. The Box-Jenkins Method is utilised to analyse
the time series data and then anticipate the subsequent years. All the data analysis, exploratory,
visualisation were extracted from ons.gov.uk. Finally, social media analysis is used to assess the
public’s reaction.
The graph in Fig 1 depicts the overall unemployment rate (%) in the UK from 2011 Q1 to
2021 (ONS, 2021). The pattern of the line graphs indicates that the unemployment rate was
greater at the beginning of the decade before it began to decline in 2014. (Pre Brexit). The rates
decreased steadily until 2019(Brexit) when they suddenly increased in 2020 Q1 (Post- Brexit &
COVID-19). By the end of 2020, the UK has taken control of the rate and it began to drop.
Fig 2 is a summary of regional unemployment in the UK. The effects of the two catastrophes
varied throughout all regions. During the past decade, England and Scotland had lower unem-
ployment rates than Wales and Northern Ireland. So, between 2016 and 2020 (Brexit), the
unemployment rates varied, in the post-Brexit and COVID-19 period, all regions have seen an
increase while with England having the highest and Northern Ireland the lowest (2020). By the
beginning of 2021, the regions began to exercise control over their labour markets.
Fig 3 illustrates the average regional unemployment rate across gender in the UK during
the three periods analyzed in this study. It is notable that the unemployment rate has consis-
tently been higher for men compared to women across all periods in the UK. Moreover, when
examining the regions, Wales and Northern Ireland display higher unemployment rates than
England and Scotland. However, the impact of Brexit and COVID-19 has had a more signifi-
cant effect on unemployment in England and Scotland compared to Wales and Northern
Ireland.
Fig 4 depicts the average unemployment rate in the UK’s major industries Pre-Brexit,
Brexit, COVID-19, and post-Brexit. The industries with the highest unemployment rates were
Fig 1. Unemployment rate in UK (2011–2021). Source: ONS.

https://doi.org/10.1371/journal.pone.0287342.g001

Fig 2. Unemployment rate in all regions of UK. Source: ONS.

‘Accommodation’, ‘Transport’, ‘Construction’, ‘Wholesale’, and ‘Other’. During the Brexit

period (2016–2020), it is evident that the unemployment rate decreased, but these industries
remained at the top. Furthermore, COVID-19 and Post-Brexit had the greatest impact on
these and all other industries. Even throughout COVID-19 and the post-Brexit period, indus-
tries like Public Administration and the health sector had low unemployment rates.
Fig 3. Unemployment rate in UK gender-wise. Source: ONS.


Fig 4. Unemployment rate of industries in the UK. Source: ONS.

4.2 Earnings (average weekly)

This study further examines the effect of the two events on the UK’s earning analysis.
Fig 5 aggregates the total and regional incomes (mean average pound per week) of the UK
between 2011 and 2021. The data indicate that England had the greatest incomes, followed by
Scotland, Wales, and Northern Ireland. Considering the period, all regions’ revenue has
increased throughout the Brexit phase. In COVID-19 and post-Brexit periods, incomes for all
areas decreased for two quarters before increasing again in 2021, except for England, where
wages increased continuously.
Fig 6 provides an overview of the average regional earnings for the Pre-Brexit, Brexit,
COVID-19, and Post-Brexit periods. England and Scotland have had more relative wage
increases than Wales and Northern Ireland.
Fig 7 indicates the average percentage change in weekly wages across industries in the UK
throughout the pre-Brexit, Brexit, COVID-19, and post-Brexit periods. During the Brexit,
COVID-19, and post-Brexit periods, the average incomes across all industries increased. ‘Sci-
ence,’ ‘Financial,’ ‘Public Administration,’ and ‘Health’ are the industries with the greatest rise
in profits. On the other hand, the incomes for businesses such as “Accommodation,” “Educa-
tion,” and “Agriculture” were comparatively lower.
4.3 Gross Domestic Product (GDP) index

Gross Domestic Product (GDP) serves as a standard measure to assess the value generated
through the production of goods and services within a country over a specific period. It reflects
the revenue generated by this production and the total expenditure on final products and ser-
vices, excluding imports. However, GDP alone does not consider the impact of inflation or ris-
ing prices, regardless of whether GDP increases or decreases. To address this limitation, the

Fig 5. Mean average earnings per week of the UK and its regions. Source: ONS.
GDP price deflator, also known as the GDP index, comes into play. It evaluates the influence
of price changes on GDP by selecting a base year and comparing current prices to those of the
reference period. The GDP price deflator effectively measures the extent to which price fluctu-
ations impact changes in GDP [42]. It tracks the prices paid by businesses, the government,
and consumers, providing insights into the variations in price levels or inflation within the
Fig 6. Comparison between earnings of different regions of the UK. Source: ONS.

Fig 7. Average earnings in different industries of UK. Source: ONS.

economy. By utilizing the GDP price deflator, economists can compare the real volume of eco-
nomic activity across different years. This comparison is crucial because evaluating the GDP of
two years with distinct price levels can lead to misleading conclusions [42]. Therefore, the
GDP price deflator enables a more accurate analysis of economic activity by considering the
impact of changing price levels over time.
Fig 8 depicts the UK’s GDP price deflator index from 2011 to 2021. It may be interpreted
that the GDP increased modestly until 2020, then declined for two quarters during COVID-19
and post-Brexit before beginning to rise again in 2021.
Fig 9 displays the average GDP index of the various UK industries throughout the pre-
Brexit, Brexit, and post-Brexit periods. It is proven that the contribution of the industries grew
during the Pre-Brexit, Brexit period and decreased during the COVID-19 and Post-Brexit and

Fig 8. GDP index of the UK for the last decade. Source: ONS.
Fig 9. The average contribution of the industries to UK’s GDP index. Source: ONS.

Brexit periods. During the Post-Brexit and COVID-19 period, the accommodation, construc-
tion, education, and agriculture industries experienced the most from COVID-19, while the
science, financial, public administration, and health industries contributed the most to the
UK’s GDP.
4.4 Trade
Since the UK has been a worldwide trading force, it is essential to evaluate the effects of Brexit
and COVID-19 on UK trade.
The commerce, imports, and exports of the UK from 2011 to 2021 are depicted in Fig 10.
Prior to 2019, it is evident that the UK had a negative overall trade value with imports exceed-
ing exports. The trade situation improved in 2019–2020 as total exports rose and total imports
dropped, putting the overall trade value near zero/positive. Beginning in 2021 in COVID-19
and post-Brexit period, the total imports increased while total exports dropped, resulting in a
trade imbalance.
In Fig 11 provides a summary of the UK’s trade relations with European Union (EU)
nations and non-EU countries, individually. In comparison to the UK’s trade with EU nations,
the Non-EU trade has increased and has been greater during the past decade. In addition, with
Brexit and COVID-19, non-EU trade has increased relative to EU trade, but overall, the trade
has dropped due to COVID-19.
Fig 12 depicts the average volume of trade in various regions of the UK throughout the
pre-Brexit, Brexit, and post-Brexit and COVID-19 periods. England is the only country
having the largest trade imbalance. Other regions, like Scotland, Wales, and Northern Ire-
land, have positive trade that is close to zero. During Brexit, trade levels increased in Scot-
land and Northern Ireland, but still, the trade imbalance widened in England, and trade
levels declined but remained positive in Wales. During the Post-Brexit and COVID-19,
Scotland, Wales, and Northern Ireland had a decline in trade, but England’s trade deficit
was reduced.
Fig 10. Total trade of the UK in the last decade. Source: ONS.

Fig 11. UK trade with EU and Non-EU. Source: ONS.

4.5 Neural-network model

Furthermore, utilizing the UK’s economic statistics, this study developed a neural network
regression model and evaluated the appropriate findings. To improve the regression model,
hyperparameter tuning is recommended to use a batch size of 20, a number of epochs of 200,
and an optimizer input of RMSProp. Using the aforementioned parameters in the regression
model, the training and validation loss was computed. Originally, the difference between the
Fig 12. Comparison of the UK’s total trade in three different periods. Source: ONS.

Fig 13. Losses on each epoch and its difference.

two losses was rather significant as shown in Fig 13, where the discrepancy steadily decreased
to zero. The model’s final loss calculation was 0.437, indicating that even after running the
model making a perfect prediction because the loss is nearly zero. It has calculated the mean
absolute error, which becomes 0.448, indicating that the gap between actual and predicted val-
ues is negligible. This demonstrates that the error rate of the training and validation models is
quite low.
4.6 Box-Jenkins method

The Box-Jenkins approach expands on the time series data acquired for the study and forecasts
the economic aspects of the UK. Initially, the data is examined using the decomposition
method.
Fig 14 displays the time series data illustrating the decomposition of unemployment in the
UK. The data reveals a consistent downward trend in the unemployment rate over time. Nota-
bly, there are no discernible repeating patterns within a one-year period, and the residuals are
minimal, indicating the absence of seasonality. However, the declining trend suggests that the
data is non-stationary.
To further analyze the data, exponential moving averages, specifically the rolling mean and
rolling standard deviation, are computed. The calculation of moving averages necessitates cer-
tain assumptions about the data. In this case, it is assumed that the time series is devoid of
both seasonal and trend components. This assumption implies that the time series is station-
ary, with no evident long-term increasing or decreasing trends, and lacks consistent periodic
patterns of seasonality [43].
The initial, moving average, and standard deviation of unemployment are shown in Fig 15.
There is a declining trend in the rolling mean, and the standard deviation is also growing. It
demonstrates that the coefficients are independent of time. The ACF and PACF graphs are
generated to confirm that the dataset is a non-stationary time series. The graph illustrates the

Fig 14. Decomposition graphs.

effect of previous values on time series values. If the time series is stationary, the ACF/PACF
plots will reveal a quick termination after a small number of delays.
Figs 16 and 17 depict the ACF and PACF graphs of the UK’s unemployment rates. It is fea-
sible to conclude that the time series is randomized since the autocorrelation reduces as the
number of delays are increasing. This indicates that the data are not time-dependent and are
consequently non-stationary. In addition, the ADF statistical test may be used to examine the
Fig 15. Rolling-mean & standard deviation graph.


Fig 16. PACF graph.

Fig 17. ACF graph.

stationary properties of time series data and to calculate the number of delays required to
make the data stable. Since the time series has a unit root, the null hypothesis of the test is non-
stationary. If the p-value is above the threshold, the null hypothesis is not rejected, and the
time series is nonstationary [43].
Fig 18 depicts the residuals and data density to determine the model’s validity. The chart
demonstrates that residuals are between (-1,1) and there was a positive spike beyond the year
2020, indicating that unemployment grew during COVID-19 and the post-Brexit era. In addi-
tion, the density graph reveals that residuals are regularly distributed but random about 0
value. This indicates that the data can be utilised to forecast the future.
The graph in Fig 19 compares the actual and expected unemployment rates. It may be
observed that anticipated values mostly overlap actual values; hence, forecasting is feasible.

Fig 18. Residual and density graphs.

Fig 19. Actual vs fitted graph for unemployment.

The only exception is the year 2020, for which the predicted numbers did not match with
actual values due to the unforeseeable pandemic.
It is then projected for the following five years as shown in Fig 20. Undoubtedly, unemploy-
ment rates are projected to decline, but fluctuations may arise if the economy experiences sig-
nificant disruptions such as COVID-19.

Fig 20. Forecast graph for unemployment.

Figs 21–23 show the forecasts of income, gross domestic product (GDP), and trade, respec-
tively. It may be inferred that the GDP (prices) and Earnings are likely to rise in value during
the next five years, while UK trade is expected to decline. This implies that even if the economy
of the UK as a whole strengthens, the trade imbalance may increase and negatively affect
growth in the long run.
Fig 21. Forecast graph for earnings.


Fig 22. Forecast graph for GDP.

Fig 23. Forecast graph for Trade.

4.7 Public reaction

4.7.1 Google trends. To analyse the public’s reaction to Brexit and COVID-19, the total
number of web searches conducted into various categories is depicted below. Fig 24 shows the
all categories of search volume for Brexit and COVID-19. It is obvious that searches for
COVID-19 exceeded those for Brexit in the UK. People were most engaged in Brexit in 2016
and 2020, when the Brexit announcement was made and the transitional phase ended,

Fig 24. Web searches of Brexit and COVID-19 under all categories.
respectively. By the end of 2021, there are no Brexit-related searches and in contrast, COVID-
19 searches began in 2020 and accelerated in 2021.
The regional searches for Brexit and COVID-19 across all categories are displayed in
Table 2. This explains the popularity of Brexit and COVID-19 in various parts of the UK.
England (21%) has the most Brexit-related searches, followed by Northern Ireland (19%),
while Scotland (84%) has the most COVID-19-related searches, followed by Wales (82%).
Fig 25 shows the search trends for Brexit and COVID-19 in the business and industry cate-
gory providing insights into the level of concern among individuals. The results indicate that
during the COVID-19 period, there was an increased focus on the impact of the pandemic on
the UK’s businesses and industries, surpassing the levels seen in previous years. Table 3 further
highlights that individuals in England expressed greater apprehension towards Brexit com-
pared to those in other regions. Moreover, throughout the COVID-19 period, people in Scot-
land and Wales conducted a higher volume of searches related to UK businesses and sectors
when compared to individuals in other areas.
Fig 26 displays the Brexit and COVID-19 searches under the Finance category. According
to the data, Brexit had a high number of searches in the financial category in 2016 and contin-
ued until 2021. In contrast, inquiries for COVID-19 increased in 2020 and surpassed Brexit in
popularity in 2016.
Table 2. Regional popularity of Brexit and COVID-19 under all categories.

Region Brexit Covid
Northern Ireland 19% 81%
Wales 18% 82%
Scotland 16% 84%
England 21% 79%

Fig 25. Web search of the events under business and industrial category.
The geographical searches for Brexit and COVID-19 in the finance category are displayed
in Table 4. The statistics indicate that people in England conducted more searches concerning
Brexit, while those in Wales conducted more searches on COVID-19.
Fig 27 depicts the Brexit and COVID-19 web search under the Jobs and Education category
in the UK. Brexit searches were far lower than COVID19 searches beginning in 2020.
Table 5 displays the regional prevalence of this category. Northern Ireland had the most
Brexit-related searches compared to other areas. In contrast, England and Wales had the high-
est number of COVID-19 searches.
4.7.2 Twitter sentiment analysis. In performing a Google Trend analysis, this study
investigated the public’s sentiment toward Brexit and COVID-19 using Twitter sentiment
analysis.
Fig 28 shows the polarity of the sentiments from the tweets on Brexit. The maximum polar-
ity appears between -0.25 and 0.25. This indicates that the majority of Brexit-related tweets
were neutral. Some tweets had both negative and positive feelings, although the proportion of
positive tweets was greater than negative tweets. This indicates that fewer people were entirely
pleased or entirely depressed because of the incident. In addition, it can be proven that most
individuals in the UK were uncertain about the consequences or impact of Brexit.
Table 3. Regional popularity of Brexit and COVID-19 under business and industrial category.
Region Brexit Covid
Wales 10% 90%
England 11% 89%
Scotland 10% 90%

Fig 26. Web searches of Brexit and COVID-19 under the finance category.
Fig 29 illustrates the sensitivity of Brexit-related tweets. The majority of tweets were unre-
lated to Brexit, with barely a few focusing on the event. This indicates that fewer individuals
tweeted about Brexit.
Fig 30 depicts the polarity of the sentiment expressed in COVID-19 tweets. The graph dem-
onstrates that most tweets were neutral. Some of the remaining tweets were entirely positive,
and the majority of them were negative. This demonstrates that individuals were depressed
and had a detrimental effect on the attitudes of the British population.
Fig 31 illustrates the subjectivity of COVID-19 tweets. It is apparent that the majority of
tweets are related to the epidemic since they are highly subjective. This indicates that individu-
als tweeted often about COVID-19 and the epidemic as a whole. The aforementioned findings
demonstrate that individuals in the UK were more engaged in social media during COVID-19
than during Brexit. Also, people had considerably greater feelings about COVID-19 than they
did over Brexit; as a result, the subjectivity of tweets about COVID-19 were higher than that of
tweets about Brexit.
5 Discussion and conclusion

Understanding the economic consequences of Brexit and the COVID-19 pandemic on the
United Kingdom (UK) is a complex undertaking due to their close temporal proximity,
Table 4. Regional popularity of Brexit and COVID-19 under the finance category.
Region Brexit Covid
Wales 30% 70%
Scotland 33% 67%
England 40% 60%

Fig 27. Web searches of Brexit and COVID-19 under the jobs and education category.
making it challenging to discern their individual impacts on the UK economy. Nonetheless,

both events have undeniably instigated significant changes and will continue to shape the
long-term economic landscape of the UK. This study aims to examine the UK economy during
three distinct periods: Pre-Brexit (2011–2016), Brexit (2016–2020), and COVID-19 & Post-
Brexit (2020–2021), utilizing diverse data analysis methodologies. The primary focus is to ana-
lyze key economic indicators such as the unemployment rate, GDP index, earnings, and trade
across different regions and industries of the UK, providing a comprehensive understanding
of the effects of Brexit and COVID-19. The findings shed light on compelling insights into the
performance of the UK economy over the past decade. Visual representations of the data
unveil several trends. The unemployment rate demonstrated a downward trajectory until 2020
but experienced a spike for a six-month period in 2021. Average weekly earnings exhibited a
gradual increase over time, while the GDP index displayed an upward trend until 2020, fol-
lowed by a decline during the COVID-19 period. Trade experienced the most significant
decline post-Brexit and during the COVID-19 pandemic. Furthermore, the impact of these
events varied among the four regions and twelve industries. Wales and Northern Ireland
emerged as the most affected regions by both Brexit and COVID-19, whereas industries such
as Accommodation, Construction, Wholesale, and Others experienced the most substantial
impact on earnings and employment levels. Conversely, industries such as finance, science,
Table 5. Regional popularity of Brexit and COVID-19 under the jobs and education category.
Region Brexit Covid
England 6% 94%
Wales 6% 94%
Scotland 7% 93%

Fig 28. The subjectivity of Brexit tweets.

and health witnessed an increased contribution to the UK’s total GDP following Brexit and
COVID-19. Gender-wise, the impact was more pronounced on men than women. Among all
the economic factors analyzed, trade stood out as the most affected variable in the UK. In early
2021, the macroeconomic situation in the UK revealed a discernible pattern: economic
Fig 29. The polarity of Brexit tweets.


Fig 30. The polarity of COVID-19 tweets.

demand outpaced supply, resulting in shortages, bottlenecks, and inflation. Various industries
reported a scarcity of labor, partly attributable to the global consequences of the pandemic.
The findings of the neural network model indicated minimal loss between the training and val-
idation models, indicating the model’s effectiveness in predicting values with minimal error.
Fig 31. The subjectivity of COVID-19 tweets.


The Box-Jenkins analysis identified the dataset as non-stationary but with a discernible trend
that could be utilized to forecast coefficients. The results of the ARIMA model projected a
gradual decline in unemployment over the next five years, with the GDP index and earnings
expected to continue ascending. However, trade was anticipated to decline, and net trade was
forecasted to remain negative in the coming years. Finally, the results of the social media analy-
sis confirmed that people displayed greater concern and engagement during the COVID-19
period (starting in 2020) compared to the Brexit period (starting in 2016). Individuals
expressed neutrality towards Brexit and posted fewer tweets on the subject, whereas during
COVID-19, people expressed unhappiness and extensively tweeted about the pandemic. It is
crucial to acknowledge that these results are based on the analysis of recent data, capturing the
immediate and overall impact of Brexit and COVID-19 on the UK economy. As ongoing eco-
nomic fluctuations remain unpredictable, forecast predictions derived from the analysis may
differ from actual outcomes. Moreover, this research relies on time-series data, which imposes
limitations on the scope and accuracy of the analysis. The graphical representation of the time-
series data is confined to line and bar graphs for analytical purposes. The existing literature on
this topic is insufficient to validate the analysis results and support the research framework.
Therefore, further work is needed to enhance the assessment of the impact of Brexit and
COVID-19 on the UK economy. Continuously.
Supporting information
S1 File.
(ZIP)
S1 Dataset.
(ZIP)
Acknowledgments
The author would like to acknowledge the support of work from Cardiff Metropolitan
University.
Author Contributions
Conceptualization: Raghav Gupta.
Data curation: Md. Mahadi Hasan.
Investigation: Syed Zahurul Islam.
Methodology: Raghav Gupta, Md. Mahadi Hasan, Syed Zahurul Islam.
Supervision: Tahmina Yasmin, Jasim Uddin.
Writing – original draft: Raghav Gupta, Md. Mahadi Hasan, Tahmina Yasmin, Jasim Uddin.
Writing – review & editing: Raghav Gupta, Syed Zahurul Islam, Tahmina Yasmin, Jasim
Uddin.
References
1. Durrant T, Pope T, Lilly A, Guerin B, Shepheard M, Nickson S, et al. Whitehall Monitor 2021. Institute
for Government. 2021.
2. Zhan C, Chi KT, Gao Y, Hao T. Comparative Study of COVID-19 Pandemic Progressions in 175
Regions in Australia, Canada, Italy, Japan, Spain, UK and USA Using a Novel Model That Considers

Testing Capacity and Deficiency in Confirming Infected Cases. IEEE Journal of Biomedical and Health
Informatics. 2021 Jun 15; 25(8):2836–47. https://doi.org/10.1109/JBHI.2021.3089577 PMID: 34129512
3. Thomas R, Routledge D, Jenkins J. Cardiff School of Pharmacy & Pharmaceutical Sciences research
abstracts–2021.
4. Busch B, Matthes J. Brexit: the economic impact–a survey. InCESifo Forum 2016 (Vol. 17, No. 2, pp. 37-
44). München: ifo Institut-Leibniz-Institut für Wirtschaftsforschung an der Universität München.
5. De Lyon J, Dhingra S. The impacts of Covid-19 and Brexit on the UK economy: early evidence in 2021.
London: Centre for Economic Performance, London School of Economics and Political Science; 2021
May 6.
6. Hasan M M, Islam M U, Sadeq M J, Fung WK, Uddin J, Review on the Evaluation and Development of
Artificial Intelligence for COVID-19 Containment. Sensors, 2023, 23(1): 527. https://doi.org/10.3390/
s23010527 PMID: 36617124
7. Keogh-Brown MR, Jensen HT, Edmunds WJ, Smith RD. The impact of Covid-19, associated behav-
iours and policies on the UK economy: A computable general equilibrium model. SSM-population
health. 2020 Dec 1; 12:100651. https://doi.org/10.1016/j.ssmph.2020.100651 PMID: 33072839
8. Hasan M M, Murtaz S B, Islam M U, Sadeq M J, Uddin J. Robust and Efficient COVID-19 Detection
Techniques: A Machine Learning Approach. PLOS ONE, 2022, 1–18, 2022. https://doi.org/10.1371/
journal.pone.0274538 PMID: 36107971
9. Elsayed A, Abdelrhim M. The Impact of Brexit on the United Kingdom Stock Market Sectors in Under of
the COVID-19 Crisis. Available at SSRN 3825230. 2021 Apr 12.
10. Lappeman J, Munyai K, Kagina BM. Negative sentiment towards COVID-19 vaccines: A comparative
study of USA and UK social media posts before vaccination rollout. F1000Research. 2021 Jun 15; 10
(472):472. https://doi.org/10.12688/f1000research.52061.1
11. Lim WM. Exiting supranational unions and the corresponding impact on tourism: Some insights from a
rejoinder to Brexit. Current Issues in Tourism. 2018 Jun 13; 21(9):970–4. https://doi.org/10.1080/
13683500.2016.1272555
12. Lim WM, Mandrinos S. A general theory of de-internationalization. Global Business and Organizational
Excellence. 2023 Jan; 42(2):9–15. https://doi.org/10.1002/joe.22186
13. Mandrinos S, Lim WM. De-internationalization: An organizational institutionalism perspective. Global
Business and Organizational Excellence. 2023 Mar; 42(3):58–73. https://doi.org/10.1002/joe.22195
14. Campos RG, Timini J. An estimation of the effects of Brexit on trade and migration. Banco de Espana
Occasional Paper. 2019 May 31 (1912).
15. Kaminska I, Walker M. Submission to the House of Commons Treasury Committee-Digital Currencies
Inquiry. Available at SSRN 3412228. 2018 Apr 12.
16. Jawad M, Maroof Z, Naz M. Development dynamics: pre and Post Brexit analysis of United Kingdom.
Quality & Quantity. 2019 Mar; 53(2):791–811. https://doi.org/10.1007/s11135-018-0789-3
17. Lim WM. History, lessons, and ways forward from the COVID-19 pandemic. Lim, WM (2021). History,
lessons, and ways forward from the COVID-19 pandemic. International Journal of Quality and Innova-
tion. 2021; 5(2):101–8.
18. Lim WM, To WM. The economic impact of a global pandemic on the tourism economy: The case of
COVID-19 and Macao’s destination-and gambling-dependent economy. Current Issues in Tourism.
2022 Apr 18; 25(8):1258–69. https://doi.org/10.1080/13683500.2021.1910218
19. Mavragani A, Gkillas K, Tsagarakis KP. Predictability analysis of the Pound’s Brexit exchange rates
based on Google Trends data. Journal of big Data. 2020 Dec; 7(1):1–9. https://doi.org/10.1186/s40537-
020-00337-2 PMID: 32963933
20. Simionescu M, Streimikiene D, Strielkowski W. What does Google Trends tell us about the impact of
Brexit on the unemployment rate in the UK?. Sustainability. 2020 Jan 31; 12(3):1011. https://doi.org/10.
3390/su12031011
21. Lim WM. The quarantine economy: the case of COVID-19 and Malaysia. COVID-19, Business, and
Economy in Malaysia. 2021 Sep 5:3–23. https://doi.org/10.4324/9781003182740-1
22. Lim WM. Conditional recipes for predicting impacts and prescribing solutions for externalities: The case
of COVID-19 and tourism. Tourism Recreation Research. 2021 Apr 3; 46(2):314–8. https://doi.org/10.
1080/02508281.2021.1881708
23. Knipe D, Evans H, Sinyor M, Niederkrotenthaler T, Gunnell D, John A. Tracking online searches for
emotional wellbeing concerns and coping strategies in the UK during the COVID-19 pandemic: a Goo-
gle Trends analysis. Wellcome Open Research. 2020 Sep 22; 5(220):220.
24. Lim WM, Kaur S, Cheong HF, editors. COVID-19, Business, and economy in Malaysia: retrospective
and prospective perspectives. Routledge; 2021 Sep 6.

25. Lim WM. Transformative marketing in the new normal: A novel practice-scholarly integrative review of
business-to-business marketing mix challenges, opportunities, and solutions. Journal of Business
Research. 2023 May 1; 160:113638. https://doi.org/10.1016/j.jbusres.2022.113638
26. Lim WM. Ushering a new era of Global Business and Organizational Excellence: Taking a leaf out of
recent trends in the new normal. Global Business and Organizational Excellence. 2022 Jul; 41(5):5–13.
27. Sharma M, Sharma S. The rising number of COVID-19 cases reflecting growing search trend and con-
cern of people: a Google Trend analysis of eight major countries. Journal of Medical Systems. 2020 Jul;
44(7):1–3. https://doi.org/10.1007/s10916-020-01588-5 PMID: 32430650
28. Chandio MM, Sah M. Brexit twitter sentiment analysis: Changing opinions about brexit and uk politi-
cians. InInternational Conference on Information, Communication and Computing Technology 2019
Oct 30 (pp. 1-11). Springer, Cham.
29. del Gobbo E, Fontanella S, Sarra A, Fontanella L. Emerging topics in Brexit debate on Twitter around
the deadlines. Social Indicators Research. 2021 Aug; 156(2):669–88. https://doi.org/10.1007/s11205-
020-02442-4
30. Garcia K, Berton L. Topic detection and sentiment analysis in Twitter content related to COVID-19 from
Brazil and the USA. Applied soft computing. 2021 Mar 1; 101:107057. https://doi.org/10.1016/j.asoc.
2020.107057 PMID: 33519326
31. Georgiadou E, Angelopoulos S, Drake H. Big data analytics and international negotiations: Sentiment
analysis of Brexit negotiating outcomes. International Journal of Information Management. 2020 Apr 1;
51:102048. https://doi.org/10.1016/j.ijinfomgt.2019.102048
32. Massaron L, Boschetti A. Regression analysis with Python. Packt Publishing Ltd; 2016 Feb 29.
33. Wang X, Lu H, Wei X, Wei G, Behbahani SS, Iseley T. Application of artificial neural network in tunnel
engineering: A systematic review. IEEE Access. 2020 Jun 25; 8:119527–43. https://doi.org/10.1109/
ACCESS.2020.3004995
34. Shridhar K, Lee J, Hayashi H, Mehta P, Iwana BK, Kang S, et al. Probact: A probabilistic activation func-
tion for deep neural networks. arXiv preprint arXiv:1905.10761. 2019 May 26.
35. Kriegeskorte N, Golan T. Neural network models and deep learning. Current Biology. 2019 Apr 1; 29(7):
R231–6. https://doi.org/10.1016/j.cub.2019.02.034 PMID: 30939301
36. Guarin DL, Kearney RE. Identification of a time-varying, box-jenkins model of intrinsic joint compliance.
IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2016 Oct 19; 25(8):1211–20.
https://doi.org/10.1109/TNSRE.2016.2619162 PMID: 27775527
37. Idrees SM, Alam MA, Agarwal P. A prediction approach for stock market volatility based on time series
data. IEEE Access. 2019 Jan 25; 7:17287–98. https://doi.org/10.1109/ACCESS.2019.2895252
38. Cai W, Yang J, Yu Y, Song Y, Zhou T, Qin J. PSO-ELM: A hybrid learning model for short-term traffic
flow forecasting. IEEE access. 2020 Jan 3; 8:6505–14. https://doi.org/10.1109/ACCESS.2019.
2963784
39. Kwiatkowski D, Phillips PC, Schmidt P, Shin Y. Testing the null hypothesis of stationarity against the
alternative of a unit root: How sure are we that economic time series have a unit root?. Journal of econo-
metrics. 1992 Oct 1; 54(1-3):159–78. https://doi.org/10.1016/0304-4076(92)90104-Y
40. Masrakin K, Rahim HA, Soh PJ, Abdulmalek M, Adam I, Warip MN, et al. Assessment of worn textile
antennas’ exposure on the physiological parameters and well-being of adults. IEEE Access. 2019 Jul
12; 7:98946–58. https://doi.org/10.1109/ACCESS.2019.2928343
41. Chen J, Wang X. Asymmetric Risk Spillovers Between China and ASEAN Stock Markets. IEEE Access.
2021 Oct 14; 9:141479–503. https://doi.org/10.1109/ACCESS.2021.3119932
42. Chowdhury A. Methods explained: the GDP implied deflator. Economic & Labour Market Review. 2008
Jun; 2(6):53–6. https://doi.org/10.1057/elmr.2008.91
43. Hunt B., “Book reviews—Digital foundations of time series analysis, vol. 1, the box-jenkins approach,” in
IEEE Communications Magazine, vol. 19, no. 4, pp. 53–53, July 1981. https://doi.org/10.1109/MCOM.
1981.1090536

SN Computer Science (2023) 4:271
https://doi.org/10.1007/s42979-023-01744-x
ORIGINAL RESEARCH
Advanced Persistent Threat Identification with Boosting

and Explainable AI
Md. Mahadi Hasan1 · Muhammad Usama Islam2 · Jasim Uddin3
Received: 17 October 2022 / Accepted: 4 January 2023 / Published online: 20 March 2023
© The Author(s) 2023
Abstract
Advanced persistent threat (APT) is a serious concern in cyber-security that has matured and grown over the years with the
advent of technology. The main aim of this study is to establish an effective identification model for APT attacks to prevent
and reduce their influence. Machine learning has the potential as well as substantial background to detect and predict cyber-
security threats including APT. This study utilized several boosting-based machine learning methods to predict various types
of APTs that are consistent in cyber-security domain. Furthermore, Explainable Artificial Intelligence (XAI) was coupled
with the predictions to provide actionable insights to the domain stakeholders as well as practitioners in this domain. The
results, particularly XGBoost with weighted F1 score of 0.97 and SHapley Additive exPlanations (SHAP)-based explanation,
prove that boosting methods as well as machine learning models paired with XAI are indeed promising in handling cyber-
security-related dataset problems which can be extrapolated towards new avenues of challenging research by effectively
deploying boosting-based XAI models.
Keywords APT · Machine learning · Boosting · XAI · Explainability · Cyber-security
Introduction mainly but recently has diversified to several domains [2].

An APT is stealthy in nature and remains undetected for a
The advent of information and technology posed great chal- prolonged period of time and is utilized by stealthy threat
lenges pertaining to data security and privacy which lead the actors for political, economic influence or monetary gains
emergence towards the field of cyber-security in this decade [3].
[1]. Advanced persistent threat (APT) being one of the major Machine learning became one of the important tools for
cyber-security issues emerged in this decade which can be detecting security threats in the cyber world domain with the
defined as a long-term cyber-security-related sophisticated explosion of data in this century [4]. Particularly, intrusion
exploitation and hostile situations aimed at governments detection systems (IDS) were developed keeping in mind
the power of machine learning to identify unwanted intru-
sions in cyber world. Furthermore, APTs being one of the
* Jasim Uddin sub-domain of cyber-security threats gained much attention
juddin@cardiffmet.ac.uk
from machine learning practitioners that lead to emergence
Md. Mahadi Hasan of machine learning as a constituent solution towards detect-
mahadihasan@aub.edu.bd
ing APTs [3, 4].
Muhammad Usama Islam Friedberg and colleagues [5] worked extensively on
usamaislam@iut-dhaka.edu
a theoretical framework for anomaly detection which
1
Department of Computer Science and Engineering, Asian learns over time and reports anomaly that differs from the
University of Bangladesh, Ashulia, Dhaka 1341, Bangladesh model. Furthermore, Siddiqui and his team [6] developed
2
School of Computing and Informatics, University a fractal-based anomaly detection system and compared
of Louisiana at Lafayette, 104 E University Ave, Lafayette, against the traditional machine learning methods, thus
LA 70503, USA improving false-positive and false-negative rates while
3
Department of Applied Computing and Engineering, Cardiff retaining better classification accuracy. Siddiqui’s work
School of Technologies, Cardiff Metropolitan University, [6] can be further corroborated with work carried out by
Western Avenue, Cardiff, Wales CF5 2YB, UK
SN Computer Science
Vol.:(0123456789)
271 Page 2 of 9 SN Computer Science (2023) 4:271
Ghafir’s team [3]. Ghafir and his team [3] reported a pre- our research work by providing comprehensive insights of
diction accuracy of 84.8% with true-positive rate (TPR) contribution, and denoting limitations to practitioners on this
of 81.8% and false-positive rate (FPR) of 4.5% with their field to take the research forward.
novel machine learning-based system called MLAPT that
utilized ensemble and support vector machine (SVM)-
based models. Their system provided three major contri- Related Work
butions, including threat detection, alert correlation, and
attack prediction. Similar research work was carried out Intrusion detection systems (IDS) have garnered formidable
in [7] where the researchers developed TerminAPTor, an importance in the cyber-security world with the advent of
Information Flow Tracking (IFT)-based APT detection information and technology [11]. While developments have
system that finds out the chains of traces that were left been made, challenges pertaining to dataset, techniques, and
by attackers through several stages of attack campaign. approaches have attracted researchers in the cyber-security
Although their system had admirable accuracy and TPR, world to advance the research field further [12]. Particularly,
the FPR rate minimization carried out by Ghafir’s team the field of machine learning and deep learning evolved
[3] was more commendable. Issues regarding the existing quickly to aid the tasks of IDS, thus contributing broadly to
solution and their subsequent disadvantage of static data the ever-changing cyber world [13].
with longer training time and the nbeed for complete re- Advanced Persistent Threat (APT), a subset of IDS [14],
training for new APT sample or APT class are discussed can be deduced as a form of a stealthy threat actor which
by Laurenza and his colleagues [8]. Laurenza’s team opted usually is a nation-state or can be originated as a funded
for a solution resulting in precision and accuracy over 90% group from state sponsorship that gains unauthorized access
that included moving from multi-class classification to a to a network and remains undetected for a prolonged period
group of single-class classifiers, thus minimizing the runt- of timespan. APT remains a center of attention and con-
ime significantly and allowing higher level of modularity cern for governments and companies as history has shown
[8]. Neuschmied et al. [9] applied several autoencoder- that unauthorized and undetected access from adversaries
based IDS methods for the detection of such IDS attack has resulted in unwanted outcomes of which political, eco-
patterns where they utilized tools of statistics to converge nomic, and socio-economic shift is noteworthy. Various
into statistical analysis to understand features that supple- detection methods exist on dealing with APT attacks of
mented the anomaly as well as IDS workflow. Explainable which machine learning and deep learning methods have
artificial intelligence (XAI) was explored in [10] where garnered attraction due to the ability of these methods to
their mechanism provided insights and interpretations provide actionable insights from large amounts of data often
for designing the defense strategy and resource alloca- absent in generic algorithms. Apart from MLs, honeypots
tion scheme of the edge defender based on edge Bayesian are utilized by Saud and Islam [15] to detect APTs where,
Stackelberg game and cyber threat intelligence (CTI) to for the architecture, they have utilized KFSensor. Han’s
detect APT. team [16] proposed a novel APT malware identification and
In this work, we have explored vividly into network secu- framework named APTMalInsight based on system call and
rity threats pertaining to advanced persistent threats and ontology knowledge framework. Similarly, Holmes frame-
proposed a novel approach that combines boosting-based work was proposed in [17] by correlating the suspicious
ensemble methods and XAI to detect, identify, report, and information flows. Niu’s team [18] utilized mobile DNS
explain the APT-based attacks. logging for APT detection tasks. Network traffic behavior
Hence, with a strong basis of introductory analysis and can be modeled with ML models for anomaly detection tasks
background in the section “Introduction”, the organization and subsequently reduce the false positives on alarms. and
of our research work is as follows. The section “Related detect and eliminate threats in real-time scenarios. The rea-
Work” provides background on our research work by pro- son ML has attracted researchers note mainly because the
viding related work on this topic. The section “Materials same ML models can also be used to create the attacks [3].
and Methods” dives deep into materials and methods that Contribution to the body of research on APT is done by
were employed in the study including dataset description, Myneni’s team [19] that contributed a benchmark dataset on
theories and practical information regarding data pre-pro- APT named DAPT-2020. The research work also discussed
cessing, boosting-based ensemble methods, and rationale several limitation factors related to generic APT datasets.
behind utilizing them. The section “Results and Discus- The authors through experimentation with semi-supervised
sion” encompasses the results and discussion that includes approach reported having class imbalance in their dataset
the theoretical framework of model evaluation followed by that ultimately led their model to perform poorly to detect
experimental result analysis and visualization through XAI. attacks. The problem of class imbalance was also found in
Finally, the section “Conclusion” summarizes and concludes the contribution of another similar benchmark contribution
SN Computer Science
SN Computer Science (2023) 4:271 Page 3 of 9 271
named SCVIC-APT-2021 [20] where the authors had bet- movement, normal traffic, reconnaissance, and pivoting that
ter luck with ensemble methods and an machine learning- followed the global knowledge base of adversary tactics and
based Attack Centric Method (ACM) is proposed to evalu- approaches, thus serving as the building blocks for their
ate the model performance on contributed dataset. Although selection of common attack techniques.
research in [19] had relatively poor performance, Liu and
his team’s work [20] outperformed the baseline models with Data Pre‑processing
a maximum macro-average F1 score of 82.27% that cor-
responds to 9.4% improvement with respect to the baseline The dataset contained several columns, such as identity,
performance. Work on Liu’s benchmark dataset [20] was ports, and protocols, which apart from being an identifica-
carried out in [21] where the authors proposed a machine tion number did not have any substantial value in classifica-
learning-based model named Prior Knowledge Input (PKI). tion problem that ultimately lead us to drop several columns
PKI utilized unsupervised clustering methodologies to pre- leaving us with 77 features in total to work with. The dataset
classify the original dataset to obtain prior knowledge which contained infinity values and null as well as NaN values
eventually is incorporated onto the supervised model that which were treated by imputing 0 to remove any unwanted
minimizes training complexity. Authors reported having best values in the dataset followed by treating the duplicate val-
macro-average F1 score of 81.37%, which is 10.47% higher ues. Although the dataset has been cleaned through pre-
than the baseline results. ceding approaches, the next data pre-processing challenge
The motivation of our research can be drawn from several poised us with processing the categorical data that lead us
factors related to our contribution to the body of knowledge to opt label encoding techniques. For data scaling, we have
on cyber-security. First, the field of APT has much attention used normalization method through min–max normalization.
with low resources to tackle the challenges as discussed in The formula for min–max normalization is provided in Eq. 1
related work. Second, APT is of much importance in cyber-
security domain. The global anomaly detection market is Xi − min(X)
Xnew = . (1)
said to grow up to 4.47 billion in 2022 at a compound annual max(x) − min(x)
growth rate (CAGR) of 16.7%. Subsequently, this untapped
There remains a substantial debate as to whether to use nor-
market statistically speaking would grow up to 8.0 billion
malization or standardization where we observe that data
in 2026 at a CAGR of 15.7% [22]. Third, applications of
scaling is particularly important for distance-based algo-
machine learning particularly, boosting methods, are good
rithms which in our experimentations will be substantially
at feature understanding and are generally resilient towards
absent. Furthermore, the feature values and its range have
overfitting. The third point coupled with XAI in our view
diverse values with less number of outliers which aided us
would provide the readers a new view at APT that would not
in taking the decision of using a normalization method over
only provide insights but would also provide interpretability
a standardization mechanism.
and explainability which is often absent in the cyber-security
Eventually, we have had 206,055 rows and 77 columns to
domain. At length, our scope of work lies within explor-
work within our dataset which was split into 80–20 standard
ing machine learning models, specifically boosting methods
train-test split to carry out our experiment.
and coupling the results with XAI to provide explainable
insights to the researchers as well as other stakeholders in
this domain. The next section termed Materials and Methods Boosting Algorithms
goes into details explaining the boosting methods and their
usability in detecting APT. Boosting can be defined as a method of converting weak
learners into strong learners. Several boosting-based classi-
fication algorithms of machine learning were utilized in the
Materials and Methods prediction of APT. This section discusses the mathematical
background of several boosting algorithms that were utilized
Dataset Description in our research work. Thus, we have illustrated the math-
ematical descriptions of these algorithms in the following
The dataset SCVIC-APT-2021 [23] is taken from the subsections.
research work carried out in [20] and is one of the latest
benchmark datasets of 2022 for detecting Advanced Persis- AdaBoost
tent Threats (APT) in network traffics. The dataset contains
315,607 rows of data and 84 features in total. As per the AdaBoost defines the weakness by the weak estimator’s
description of dataset, the target label consists of six class error rate [24]. In each iteration, AdaBoost identifies miss-
labels. They are data exfiltration, initial compromise, lateral classified data points and decreases the correct weights
SN Computer Science
as well as increases the incorrect weights, so that the suc- Table 1 Similarities and differences amongst CatBoost vs. Light-
ceeding classifier will have residual extra attention to get GBM vs. XGBoost
them right. The algorithm for adaboost is given below in Variables CatBoost LightGBM XGBoost
Algorithm 1.
Tree symmetry Symmetric Asymmetric Asymmetric
(leaf-wise) (level-
Algorithm 1 Adaboost Algorithm [24]
wise)
Initial weights L : L1 , L2 , . . . , Ln = n1
for i in [1, W ]; W = weak classifiers Type Ordered – –
Feature support Variable Numerical Numerical
fit weak classifier C with sample weights I
i
n Missing value handling Yes Yes Yes

Lj L(C i (Xj )=Yj )
Ei= j=1 n // Error , E
j=1 Lj
(1− E )
i

αi = log + log(K − 1)// coefficient for C i
E
Table 2 Strength and weakness analysis of boosting algorithms
i
i
+l(C i (Xj )=Yj )
Lj = Lj × ea for Lj ∈ I// if wrong, increase weights
Strength Weakness
L = L − mean(L)// normalize weights
Explainable, easy interpretation and good perfor- Outlier sensitivity

i
Prediction: YJ = maxk i=1 αi I C (Xj ) = k
i
// Low errors leads to large
α, which means higher importance in the voting. output : find the class with mance
highest vote.
Inferred feature selection Scalability
Overfitting resilience Training time
Gradient Boosting strengthen our case as to why we have selected boosting, a

pros and cons analysis is provided in Table 2.
Gradient boosting takes the approach in a slightly different As boosting is basically an ensemble-based model, thus it
way. Instead of increasing or decreasing weight and points, is easily explainable, and interpretable with admirable bench-
gradient boosting utilizes the difference between the ground mark performance measures. Usually, they are not suscepti-
value of truth and prediction value [25]. The algorithm for ble to overfitting thus providing them with stronger prediction
gradient boosting is given in Algorithm 2. power making boosting methods usually greater in perfor-
mance from bagging methods. However, boosting methods
Algorithm 2 Gradient Boosting Algorithm [25] are sensitive to outliers and have scalability issues. As our
Fit estimator F E 1 pre-processing unveiled that the dataset does not have many
for i in [1, W ] //W= weak estimators
outliers and the dataset being medium size in nature, we finally
Li = loss in ith iteration opted for boosting methods weighing in all the pros and cons
∂Li
− ∂X = − n2 × Yj − F E i (Xj ) ∀i // calculate negative gradient:
through strength and weakness analysis.
j
Fit a weak estimator W E i on X, ∂X

∂L

Prediction: F E m (X) = F E i (X) + υ × W E i (X) = F E 1 + υ × Results and Discussion

m
i=1 W E i (X)
// υ = variable step size
Model Evaluation
Several performance measures were employed for evaluating

CatBoost vs. LightGBM vs. XGBoost our boosting-based models. Confusion matrix were generated
for this multi-class classification problem which resulted in
CatBoost (Category Boosting), LightGBM (Light Gradi- providing us with the basic derived performance measures,
ent Boosted Machine), and XGBoost (eXtreme Gradient such as true positive (TP), true negative (TN), false positive
Boosting) are all variations of gradient boosting algorithms (FP), and false negative (FN) [27]. From these values, sev-
[26]. Table 1 outlines the basic similarities and differences eral equations pertaining to accuracy scores, precision, recall,
between these boosting methods. and F1 score were used for evaluating the model [28]. The
equations for accuracy, precision, recall, and f-1 scores are
addressed in Eqs. 2, 3, 4, and 5, respectively
Rationale for Using Boosting for Experimentation
TP + TN
Accuracy = , (2)
TP + FP + TN + FN
In our experimentation, we have used the aforementioned
boosting algorithms for experimentation purposes. To
SN Computer Science
TP
Precision = , (3)
TP + FP
TP
Recall = , (4)
TP + FN
2TP
F1 score = . (5)
2TP + FP + FN
Our dataset inherently poses a class imbalance problem,
and while measures were taken in Materials and Methods
to address the imbalance, the performance measure is also
expected to be robust to such irregularities. While F1 score
can be defined as a sufficient metric as it merges precision
and recall in a more interpretable domain, a formidable Fig. 1 Bar chart depicting accuracy of experimented boosting meth-
reliable performance measure is the Matthews Correlation ods
Coefficient (MCC) [29], which is preferred over F1 score
because of its balanced assessment of classifiers irrespective AdaBoost, and LightGBM models performed very poorly,
of class positivity or negativity for that matter. The equa- whereas CatBoost and XGBoost had admirable results. The
tion for MCC is provided in Eq. (6). In addition to that, one same scenario blossomed in Tables 4 and 5 for recall and F1
other robust statistic which is widely used for measuring scores. Where AdaBoost, GradBoost, and LightGBM mod-
the performance is Cohen’s kappa [30], whose equation is els performed very poorly.
provided in Eq. (7) In Table 6, we calculated the weighted precision, recall,
(TP × TN) − (FP × FN) and F1 score to summarize the model’s performance. Where
MCC = √ , GradBoost, AdaBoost, and LightGBM weighted F1 score
(TP + FP)(TP + FN)(TN + FP)(TN + FN) is, respectively, 96%, 82%, and 89%. In contrast, XGBoost
(6) and CatBoost achieved 97 and 99%. GradBoost reached 98
Accuracy − Pe proportions for weighted precision, but it fell by 4% for the
Ck = . (7) weighted recall. The CatBoost achieved the same weighted
1 − Pe
precision and recall score which is 99%. Moreover, Ada-
In our evaluation, we have also incorporated a loss meas- Boost performed 99% for weighted precision; nevertheless,
ure named hamming loss which provides the fraction of the it dramatically went down by 28% for the recall. Besides,
incorrect labels to the total number of labels [31]. The for- XGBoost reached the twin score for weighted precision and
mula for hamming loss is provided with the following Eq. recall which is 97%. In addition, LightGBM gained 96% for
(8). weighted precision and 83% for recall.
m 1 Table 7 shows Matthews correlation coefficient, Cohen’s
1 ∑ |yi Δyi | kappa, and hamming loss scores. According to Matthews
Hl = . (8)
m i=1 Q correlation coefficient, GradBoost, AdaBoost, and Light-
GBM reached, respectively, 35.68%, 19.24%, and 6.17%. On
the other hand, CatBoost and XGBoost achieved 95.40% and
Experimental Results 98.29%. The importance of dealing with class imbalance,
feature utilization resonated fully in the experimental results
Our investigation finds that class imbalance had an overall which can be dealt in future work, where emphasize may be
effect on experimental results. Confusion matrix of experi- given towards sampling techniques to overcome class imbal-
mented ensemble boosting methods is visualized through ance problems.
Fig. 2, whereas the overall accuracy is shown through a
visual barchart in Fig. 1.
Individual class-wise precision, recall, and f1 score were Explainable AI Visualization
constructed through Tables 3, 4 and 5 to visualize in depth
the results attained through our models where DE= data Explainability has added new dimensions to machine learn-
exfiltration, IC=initial compromise, LM=lateral movement, ing and deep learning research. However, ML and DL
NT=normal traffic, R=reconnaissance, and P=pivoting. system consists of several types of architecture which are
Individual precision (Table 3) illustrates GradBoost,
SN Computer Science
Fig. 2 Confusion matrix of

AdaBoost, CatBoost, Gra-
dientBoost, LightGBM, and
XGBoost classifiers
Table 3 Individual precision Boosting method DE IC LM NT P R

result of Data exfiltration
(DE), initial compromise (IC), GradBoost 0.00 0.00 0.20 1.00 0.32 0.07
lateral movement (LM), normal
CatBoost 0.87 0.88 0.86 1.00 0.97 0.92
traffic (NT), reconnaissance (R),
and pivoting (P) experimented AdaBoost 0.20 0.00 0.40 1.00 0.68 0.59
method. XGBoost 0.92 1.00 0.94 1.00 0.98 0.97
LightGBM 0.00 0.02 0.01 0.98 0.04 0.00
Bold indicates CatBoost and XGBoost models for effective visualization
Table 4 Individual recall result Boosting method DE IC LM NT P R

of DE, IC, LM, NT, P and
R experimented methods GradBoost 0.00 0.00 0.02 0.95 0.38 0.72
CatBoost 0.93 0.93 0.89 0.99 0.94 0.91
AdaBoost 0.62 0.10 0.25 0.71 0.44 0.69
XGBoost 0.97 0.93 0.98 0.99 0.98 0.97
LightGBM 0.00 0.02 0.01 0.98 0.04 0.00
SN Computer Science
Table 5 Individual F1 score Boosting method DE IC LM NT P R

result of DE, IC, LM, NT, P and
R experimented methods GradBoost 0.00 0.00 0.04 0.97 0.34 0.13
CatBoost 0.90 0.90 0.88 0.99 0.95 0.91
AdaBoost 0.31 0.00 0.31 0.83 0.53 0.63
XGBoost 0.94 0.97 0.96 0.99 0.98 0.97
LightGBM 0.00 0.04 0.01 0.91 0.07 0.00
Table 6 Weighted precision, recall, and F1 score of our experimented additive feature attributions and Shapley additive explana-
models tion. Where the LIME model provides predictions based
Boosting method Weighted Weighted recall Weighted on local approximation which was an additive feature attri-
precision F1 score bution method [35]. Moreover, DeepLIFT was introduced
as a recursive prediction explanation method. DeepLIFT
GradBoost 0.98 0.94 0.96
used the summation-to-delta property that matches with
CatBoost 0.99 0.99 0.99
additive feature attribution. In addition, layer-wise rel-
AdaBoost 0.99 0.71 0.82
evance propagation (LRP) is equivalent to DeepLIFT.
XGBoost 0.97 0.97 0.97
SHAP utilizes a game theory-based approach to provide
LightGBM 0.96 0.83 0.89
an explanation of the output generated by machine learning
Bold indicates CatBoost and XGBoost models for effective visualiza- model which in our cases are the boosting methods that were
tion utilized in our study and experimentation.
It provides an optimal credit allocation alongside a form
of local explanations by accumulating the calculation per-
Table 7 Matthews Correlation Coefficient (MCC), Cohen’s Kappa formed through the classic Shapley values [35]. Figure 3
(Ck ), and Hamming loss ( Hl ) values of our experimented boosting
methods shows each feature’s importance visually briefly. High val-
ues of the latitude feature have a high contribution and the
Boosting method MCC Ck Hl low values have a low contribution. The variation of color
GradBoost 0.3568 0.3146 0.0612 of features indicates the classes. The Dataset feature “Idle
CatBoost 0.9540 0.9540 0.0022 Max” has high variations and reached a high impact on the
AdaBoost 0.1924 0.0882 0.2929 XGBoost model prediction; on the other hand, the “Fwd
XGBoost 0.9829 0.9829 0.0008 header length” reached a high impact on the CatBoost model
LightGBM 0.0617 0.0405 0.1745 prediction as depicted in the aforementioned figures.
In XGBoost SHAP values, the “Idle Max” feature pro-
Bold indicates CatBoost and XGBoost models for effective visualiza- duced the maximum influence on class 4 then classes 3,
tion
0, 5, 2, and 1. Moreover, the “Fwd Seg Size Min” feature
impressed class 3 more than class 1. Apart from CatBoost
SHAP values, the dataset feature “Fwd Header Length” has
considered as black-box. Scientists are trying to open the a high impact on class 3 than on classes 4, 2, 5, 1, and 0.
black-box and turn it into a transparent system [32]. Sequentially, other features “Fwd Seg Size Min”, “Idle Max”
XAI is now essential, because people who are influ- etc.
enced by AI decisions. It is crucial to understand model SHAP shows the force plot for the XGBoost classifier.
transparency, explainability, and trust in the AI system. Figure 4, where features in red color show risk factors that
XAI has various benefits, including model justification, push up the overall probability. In contrast, the blue color
controlling, debugging, model improvement, and knowl- protective factors push down the probability.
edge discovery [33]. XAI could explain why and how the
model made a decision, and then, developers could easily
upgrade it, improve it, as well as make it smarter. Conclusion
Several methods are available in explaining the results
of which SHapley Additive exPlanations (SHAP) is a well- In this paper, we have proposed a novel advanced persistent
known XAI method [34]. Other XAI techniques include threat identification approach using boosting algorithms
LIME, DeepLIFT, as well as layer-wise relevance propa- with explainable artificial intelligence. We have extensively
gation. The specialty of SHAP is uniquely determined by investigated the whole dataset and effectively preprocessed
SN Computer Science
it. Our study demonstrates the data-driven pre-processing

technique significantly improved algorithms performance by
4–9%. The study examined various ensemble-based boosting
algorithms to avoid bias and variance problems and convert
weak learners into strong learners. This study applied several
boosting algorithms, including gradient, light gradient, cat-
egory, and extreme gradient, and compared the performance
of the models. According to the evaluation metrics, Cat-
Boost and XGBoost algorithms performed better than Grad-
Boost, LightGBM, and AdaBoost. This paper also applied
XAI to demonstrate how much a single feature affected the
output and characterizes model fairness, transparency, and
its subsequent outcomes. Future direction of work will be to
address the class imbalance problem depicted in this dataset
to reduce the excessive FPR rates and optimize class-wise
precision, recall, and accuracy, thus contributing to overall
performance of the system. Furthermore, research work may
be carried out to develop more adaptive, AI-based explain-
able cyber-security techniques that are privacy-preserving.
Availability of Data and Materials The code is available for further

research and repeatability in Github https://github.com/MdMahadiHa
san1/Advanced-persistent-threat-identification.
Declarations
Conflict of Interest The authors declare that there is no conflict of in-
terest.
Open Access This article is licensed under a Creative Commons Attri-

bution 4.0 International License, which permits use, sharing, adapta-
tion, distribution and reproduction in any medium or format, as long
as you give appropriate credit to the original author(s) and the source,
provide a link to the Creative Commons licence, and indicate if changes
were made. The images or other third party material in this article are
included in the article's Creative Commons licence, unless indicated
otherwise in a credit line to the material. If material is not included in
the article's Creative Commons licence and your intended use is not
permitted by statutory regulation or exceeds the permitted use, you will
need to obtain permission directly from the copyright holder. To view a
copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Fig. 3 SHAP-based XAI Plot showing how much a single feature

affected the output. a XGBoost SHAP values; b CatBoost SHAP val-
ues
References
1. Jang-Jaccard J, Nepal S. A survey of emerging threats in cyber-
security. J Comput Syst Sci. 2014;80(5):973–93. https://doi.org/
10.1016/j.jcss.2014.02.005.
2. Daly MK. Advanced persistent threat. Usenix. 2009;4(4):2013–6.
3. Ghafir I, Hammoudeh M, Prenosil V, Han L, Hegarty R, Rabie
K, Aparicio-Navarro FJ. Detection of advanced persistent threat
using machine-learning correlation analysis. Futur Gener Comput
Syst. 2018;89:349–59. https://doi.org/10.1016/j.future.2018.06.
055.
4. Buczak AL, Guven E. A survey of data mining and machine learn-
ing methods for cyber security intrusion detection. IEEE Commun
Fig. 4 Force plot illustrates the outcome of SHAP values on the inter- Surv Tutor. 2015;18(2):1153–76. https://d oi.o rg/1 0.1 109/C
OMST.
action of features and the prediction 2015.2494502.
SN Computer Science
5. Friedberg I, Skopik F, Settanni G, Fiedler R. Combating advanced 20. Liu J, Shen Y, Simsek M, Kantarci B, Mouftah HT, Bagheri M,
persistent threats: from network event correlation to incident Djukic P. A new realistic benchmark for advanced persistent
detection. Comput Secur. 2015;48:35–57. https://doi.org/10. threats in network traffic. IEEE Network Lett. 2022;4(3):162–6.
1016/j.cose.2014.09.006. https://doi.org/10.1109/LNET.2022.3185553.
6. Siddiqui S, Khan MS, Ferens K, Kinsner W. Detecting advanced 21. Shen Y, Simsek M, Kantarci B, Mouftah HT, Bagheri M, Djukic
persistent threats using fractal dimension based machine learn- P. Prior knowledge based advanced persistent threats detection
ing classification. In: Proceedings of the 2016 ACM on Interna- for IoT in a realistic benchmark. arXiv preprint arXiv:2 208.0 5089
tional Workshop on Security and Privacy Analytics, 2016;p. 64–9. 2022; https://doi.org/10.48550/arXiv.2208.05089.
https://doi.org/10.1145/2875475.2875484 22. ReportLinker: Anomaly detection global market report 2022.
7. Brogi G, Tong VVT. Terminaptor: Highlighting advanced per- ReportLinker (2022). https://w ww.g loben ewswi re.c om/n ews-r elea
sistent threats through information flow tracking. In: 2016 8th se/2 022/0 9/3 0/2 52607 4/0/e n/A
nomal y-D
etect ion-G
lobal-M arket-
IFIP International Conference on New Technologies, Mobility and Report-2022.html.
Security (NTMS), 2016;p. 1–5. https://doi.org/10.1109/NTMS. 23. Liu J, Shen Y, Simsek M, Kantarci B, Mouftah HT, Bagheri M,
2016.7792480. IEEE. Djukic P. SCVIC-APT-2021. https://doi.org/10.21227/g2z5-ep97
8. Laurenza G, Lazzeretti R, Mazzotti L. Malware triage for early 24. Hastie T, Rosset S, Zhu J, Zou H. Multi-class adaboost. Stat Inter-
identification of advanced persistent threat activities. Digit face. 2009;2(3):349–60. https://doi.org/10.4310/SII.2009.v2.n3.
Threats. 2020;1(3):1–17. https://doi.org/10.1145/3386581. a8.
9. Neuschmied H, Winter M, Stojanović B, Hofer-Schmitz K, Božić 25. Friedman JH. Greedy function approximation: a gradient boosting
J, Kleb U. Apt-attack detection based on multi-stage autoencod- machine. Ann Stat. 2001;1189–232
ers. Appl Sci. 2022;12(13):6816. https://doi.org/10.3390/app12 26. Al Daoud E. Comparison between xgboost, lightgbm and catboost
136816. using a home credit dataset. Int J Comput Inf Eng. 2019;13(1):6–
10. Li H, Wu J, Xu H, Li G, Guizani M. Explainable intelligence- 10. https://doi.org/10.5281/zenodo.3607805.
driven defense mechanism against advanced persistent threats: a 27. Luque A, Carrasco A, Martín A, de Las Heras A. The impact of
joint edge game and AI approach. IEEE Trans Dependable Secure class imbalance in classification performance metrics based on
Comput. 2021;19(2):757–75. https://d oi.o rg/1 0.1 109/T
DSC.2 021. the binary confusion matrix. Pattern Recogn. 2019;91:216–31.
3130944. https://doi.org/10.1016/j.patcog.2019.02.023.
11. Liao H-J, Lin C-HR, Lin Y-C, Tung K-Y. Intrusion detec- 28. Dalianis H. Evaluation metrics and evaluation. Cham: Springer;
tion system: a comprehensive review. J Netw Comput Appl. 2018. p. 45–53. https://doi.org/10.1007/978-3-319-78503-5_6.
2013;36(1):16–24. https://doi.org/10.1016/j.jnca.2012.09.004. 29. Chicco D, Jurman G. The advantages of the matthews correlation
12. Khraisat A, Gondal I, Vamplew P, Kamruzzaman J. Survey coefficient (mcc) over f1 score and accuracy in binary classifica-
of intrusion detection systems: techniques, datasets and chal- tion evaluation. BMC Genomics. 2020;21(1):1–13. https://d oi.o rg/
lenges. Cybersecurity. 2019;2(1):1–22. https://doi.org/10.1186/ 10.1186/s12864-019-6413-7.
s42400-019-0038-7. 30. Rau G, Shih Y-S. Evaluation of Cohen’s kappa and other measures
13. Javaid A, Niyaz Q, Sun W, Alam M. A deep learning approach of inter-rater agreement for genre analysis and other nominal data.
for network intrusion detection system. In: Proceedings of the 9th J Engl Acad Purp. 2021;53: 101026. https://doi.org/10.1016/j.
EAI International Conference on Bio-inspired Information and jeap.2021.101026.
Communications Technologies (formerly BIONETICS), 2016;p. 31. Yessou H, Sumbul G, Demir B. A comparative study of deep
21–6. https://doi.org/10.4108/eai.3-12-2015.2262516 learning loss functions for multi-label remote sensing image clas-
14. Alshamrani A, Myneni S, Chowdhary A, Huang D. A survey sification. In: IGARSS 2020-2020 IEEE International Geoscience
on advanced persistent threats: techniques, solutions, chal- and Remote Sensing Symposium, 2020;p. 1349–52. https://doi.
lenges, and research opportunities. IEEE Commun Surv Tutor. org/10.1109/IGARSS39084.2020.9323583. IEEE.
2019;21(2):1851–77. https://d oi.o rg/1 0.1 109/C
OMST.2 019.2 8918 32. Xu F, Uszkoreit H, Du Y, Fan W, Zhao D, Zhu J. Explainable AI: a
91. brief survey on history, research areas, approaches and challenges.
15. Saud Z, Islam MH. Towards proactive detection of advanced per- In: CCF International Conference on Natural Language Process-
sistent threat (apt) attacks using honeypots. In: Proceedings of ing and Chinese Computing, p. 563–74 (2019). https://d oi.o rg/1 0.
the 8th International Conference on Security of Information and 1007/978-3-030-32236-6_51. Springer.
Networks, 2015;p. 154–7. https://doi.org/10.1109/COMST.2019. 33. Krajna A, Kovac M, Brcic M, Šarčević A. Explainable artificial
2891891 intelligence: an updated perspective. In: 2022 45th Jubilee Inter-
16. Han W, Xue J, Wang Y, Zhang F, Gao X. Aptmalinsight: iden- national Convention on Information, Communication and Elec-
tify and cognize apt malware based on system call information tronic Technology (MIPRO), 2022;p. 859–64 . https://doi.org/10.
and ontology knowledge framework. Inf Sci. 2021;546:633–64. 23919/MIPRO55190.2022.9803681. IEEE.
https://doi.org/10.1016/j.ins.2020.08.095. 34. Islam MU, Mottalib M, Hassan M, Alam ZI, Zobaed S, Rabby F,
17. Milajerdi SM, Gjomemo R, Eshete B, Sekar R, Venkatakrishnan et al. The past, present, and prospective future of xai: a compre-
V. Holmes: real-time apt detection through correlation of suspi- hensive review. Explain Artif Intell Cyber Secur. 2022. https://
cious information flows. In: 2019 IEEE Symposium on Security doi.org/10.1007/978-3-030-96630-0_1.
and Privacy (SP), 2019;p. 1137–52. https://doi.org/10.1109/SP. 35. Lundberg SM, Lee S-I. A unified approach to interpreting model
2019.00026. IEEE predictions. Adv Neural Inf Process Syst. 2017;30.
18. Niu W, Zhang X, Yang G, Zhu J, Ren Z. Identifying apt malware
domain based on mobile DNS logging. Math Prob Eng. 2017. Publisher's Note Springer Nature remains neutral with regard to
https://doi.org/10.1155/2017/4916953. jurisdictional claims in published maps and institutional affiliations.
19. Myneni S, Chowdhary A, Sabur A, Sengupta S, Agrawal G,
Huang D, Kang M. Dapt 2020-constructing a benchmark data-
set for advanced persistent threats. In: International Workshop
on Deployable Machine Learning for Security Defense, 2020;p.
138–63. https://doi.org/10.1007/978-3-030-59621-7_8. Springer.
SN Computer Science
sensors
Review
Review on the Evaluation and Development of Artificial
Intelligence for COVID-19 Containment
Md. Mahadi Hasan 1 , Muhammad Usama Islam 2 , Muhammad Jafar Sadeq 1 , Wai-Keung Fung 3
and Jasim Uddin 3, *
1 Department of Computer Science and Engineering, Asian University of Bangladesh,

Ashulia 1349, Bangladesh
2 School of Computing and Informatics, University of Louisiana at Lafayette, Lafayette, LA 70504, USA
3 Department of Applied Computing and Engineering, Cardiff School of Technologies,
Cardiff Metropolitan University, Cardiff CF5 2YB, UK
* Correspondence: juddin@cardiffmet.ac.uk
Abstract: Artificial intelligence has significantly enhanced the research paradigm and spectrum with
a substantiated promise of continuous applicability in the real world domain. Artificial intelligence,
the driving force of the current technological revolution, has been used in many frontiers, includ-
ing education, security, gaming, finance, robotics, autonomous systems, entertainment, and most
importantly the healthcare sector. With the rise of the COVID-19 pandemic, several prediction and
detection methods using artificial intelligence have been employed to understand, forecast, handle,
and curtail the ensuing threats. In this study, the most recent related publications, methodologies
and medical reports were investigated with the purpose of studying artificial intelligence’s role in
the pandemic. This study presents a comprehensive review of artificial intelligence with specific
attention to machine learning, deep learning, image processing, object detection, image segmentation,
and few-shot learning studies that were utilized in several tasks related to COVID-19. In particular,
genetic analysis, medical image analysis, clinical data analysis, sound analysis, biomedical data classi-
fication, socio-demographic data analysis, anomaly detection, health monitoring, personal protective
equipment (PPE) observation, social control, and COVID-19 patients’ mortality risk approaches
Citation: Hasan, M.M.; Islam, M.U.; were used in this study to forecast the threatening factors of COVID-19. This study demonstrates
Sadeq, M.J.; Fung, W.-K.; Uddin, J. that artificial-intelligence-based algorithms integrated into Internet of Things wearable devices were
Review on the Evaluation and quite effective and efficient in COVID-19 detection and forecasting insights which were actionable
Development of Artificial Intelligence through wide usage. The results produced by the study prove that artificial intelligence is a promising
for COVID-19 Containment. Sensors arena of research that can be applied for disease prognosis, disease forecasting, drug discovery, and
2023, 23, 527. https://doi.org/ to the development of the healthcare sector on a global scale. We prove that artificial intelligence
10.3390/s23010527
indeed played a significantly important role in helping to fight against COVID-19, and the insightful
Academic Editors: Hemant Ghayvat knowledge provided here could be extremely beneficial for practitioners and research experts in
and Sharnil Pandya the healthcare domain to implement the artificial-intelligence-based systems in curbing the next
pandemic or healthcare disaster.
Received: 28 November 2022
Revised: 23 December 2022
Keywords: COVID-19; artificial intelligence; machine learning; deep learning; few-shot learning
Accepted: 29 December 2022
Published: 3 January 2023
1. Introduction
Copyright: © 2023 by the authors. The severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is an extremely
Licensee MDPI, Basel, Switzerland. contagious disease that was detected for the very first time on 31 December 2019, Wuhan,
This article is an open access article
China [1]. It subsequently spread internationally, causing more than 547 million confirmed
distributed under the terms and
infections and 6.33 million deaths, making it one of the worst diseases in human history.
conditions of the Creative Commons
Pre-COVID-19, there were three major epidemics in the twenty-first century, starting
Attribution (CC BY) license (https://
with severe acute respiratory syndrome coronavirus (SARS-CoV) in 2002–2003. The second
creativecommons.org/licenses/by/
one was swine flu in 2009–2010, and the third one was Middle East respiratory syndrome
4.0/).
Sensors 2023, 23, 527. https://doi.org/10.3390/s23010527 https://www.mdpi.com/journal/sensors

Sensors 2023, 23, 527 2 of 35
coronavirus (MERS-CoV), which started in 2015 [2]. With various warning shots for the
research community [3,4] relevant to coronavirus and its potential to cause human harm, it
was quite inconceivable that another pandemic was imminent. COVID-19 has proven to
be a challenging sickness that could present in a variety of forms and degrees of severity,
ranging from mild to severe (organ failure and death). Multi-organ failure and death
are unusual from a mild, self-limiting respiratory disease, but it can present as a severe,
progressive pneumonia. With the progression of the pandemic and the increasing number
of verified cases and people suffering from severe respiratory failure and cardiovascular
issues, there are strong grounds to be very worried about the effects of this viral infection [5].
A considerable amount of attention has been placed on identifying acceptable strategies
for addressing COVID-19-related challenges. Artificial intelligence has recently attracted
much research effort towards solving the complex issues in a number of fields, including
engineering [6–8], medicine [9,10], economics [4], and psychology [11].
From the molecular level to the most up-to-date data-driven decision-making models,
research literature and digital technologies have had significant impacts [12,13]. They are
based on the technological advancement of artificial intelligence, which is inspired by the
human brain. Artificial intelligence is the use of algorithms, models, and computer tech-
niques to realize human intelligence [14]. Significant advancements in processing power,
virtual (algorithm) dimensions, numerical optimization, and memory have enabled the
creation and implementation of cutting-edge AI solutions to combat COVID-19 throughout
the past decade. Due to their ability to swiftly adapt to ever-changing inputs, artificial
intelligence systems are useful in situations of rapid change.
Machine learning is the subset of artificial intelligence that has the ability to learn
from and make predictions based on data. Deep learning is a subset of machine learning
that uses data, weights, hyperparameters, and complex structures of algorithms modeled
on the human brain. Machine learning and deep learning have reached important mile-
stones in processing, complicated decision making, information analysis, and extremely
organized self-learning.
Several research themes are in spotlight due to COVID-19. Alballa et al. [15] exclusively
examined machine learning strategies for diagnosis, mortality, and severity risk prediction.
Napolitano et al. [12] provided an overview of COVID-19 applications, including molecular
virology, molecular pharmacology and biomarkers, epidemiology, clinical medicine, clinical
imaging, and AI-based healthcare. The authors have not clarified the limitations they
discovered in the previously mentioned areas. El-Rashidy et al. [16] explored applications of
artificial intelligence for COVID-19. This research centered on COVID-19 diagnosis, spread,
features, treatment, and vaccine development; it included a few supporting applications.
Alyasseri et al. [17] reviewed deep learning (DL) and machine learning (ML) techniques for
only COVID-19 diagnosis. Kwekha-Rashid et al. [18] reported their study of coronavirus
illness cases using just ML (supervised and unsupervised) techniques. Current restrictions
and prospective scopes have not been defined by the authors. Bhattacharya et al. [19] and
Roberts et al. [20] highlighted medical image processing applications for COVID-19 that
use deep learning and machine learning methods. We investigated supervised learning
techniques for COVID-19 forecasting, clinical-feature-based COVID-19 prediction, medical-
image-based COVID-19 detection, the immunological landscape of COVID-19 analysis, and
COVID-19 patients’ mortality-risk prediction, among others. For unsupervised learning, we
show exploratory medical image grouping, risk analysis, anomaly detection, differentiation,
patient severity detection, patient screening, and discovery of disease-related genes. Object
detection approaches we found are for COVID-19 detection and screening, infection risk
assessment, abnormality detection, body temperature measurement, personal protective
equipment detection, and social distance monitoring.
The transfer learning-based COVID-19 study we investigated included automatic
analysis of medical pictures, COVID-19 classification, identification of lung-disease severity,
and an automated COVID-19 screening model. Medical images based on COVID-19
infected area segmentation, lung and infection region segmentation, and infected tissue
Sensors 2023, 23, 527 3 of 35
segmentation are key applications in the image segmentation sector. Detection of COVID-19
and medical image analysis are the primary applications in shot learning arena.
Our research carefully examines AI applications’ current status, identifies technical
issues and limitations, and provides instructions to overcome existing challenges. Moreover,
this comprehensive investigation will assist the research community in the upcoming
development of artificial intelligence technology for any pandemic.
The rest of this paper consists of the following sections. Section 2 presents the method-
ology of the research. Section 3 presents the supervised and unsupervised learning ap-
plications for COVID-19. Section 4 presents object detection, transfer learning, image
segmentation, and shot learning applications for COVID-19. Section 5 encompasses dis-
cussion and future work. Section 6 concludes the paper. Figure 1 shows overview of AI
approaches for COVID-19. Figures 2–8 are infographics overviews. Tables 1–4 are tabulated
summaries. Table 5 includes a meta-analysis for specific objectives. Table 6 includes current
challenges and future research directions.
Figure 1. The infographic overview of AI approaches for COVID-19. The percentage indicates the
use of AI models in diverse areas of COVID-19.
2. Methodology
A multi-step scanning strategy [21] was applied for this systematic review. We
searched relevant published literature from 2020 to 2022. Search strings consisting of
twelve keywords—namely, ‘standards’, ‘report criteria’, ‘COVID-19’, ‘artificial intelligence’,
‘machine learning’, ‘deep learning’, ‘supervised learning’, ‘unsupervised learning’, ‘transfer
Sensors 2023, 23, 527 4 of 35
learning’, ‘object detection’, ‘image segmentation’, and ‘few-shot learning’—in several com-
binations were used. We only searched literature written in English. Initially, we selected
302 papers that matched our research criteria and after removing duplicates retained 239.
Retained papers were filtered based on titles, abstracts, contexts, and the scope of our
domain. Only 150 papers were deemed relevant to our review. We scanned the full-text
and removed ineligible papers. In addition, we included 128 papers for review. Figure 2
shows the overview of databases that were used in this review.
Figure 2. Overview of papers selected from different databases. (a) The bar plot illustrates the
numbers of papers selected from Nature, IEEE, Springer, Elsevier, MDPI, PLOS, Wily, Frontiers, and
others. (b) The pie chart shows the distribution of selected articles and conferences.
Eligibility Criteria
The selection and filtering process is summarized as follows:
• The research must have discussed AI-based devices or architectures, or systems for forecast-
ing or detecting or monitoring or managing health care conditions for COVID patients.
• A clear methodology must have been coined in terms of devices used or architectures
discussed in the studies.
• Unique, relevant, important, significant, and informative works were included.
• Relevant papers with historical insights were included in the discussion to find the
current state of research, past historical evidence, and future policy implications for
future endemics.
• Duplicate research works were excluded.
• Research work without IRB approval where a human subject was directly involved in
the study were discarded.
• Media reports, university reports, and reports with ambiguity were excluded due to
lack of clear methodology.
• Journal ranking (Q1 ≥ Q2 ≥ Q3 ≥ Q4 ≥ no − Q), impact factor, JCR, and conference
ranking by ERA and Quails (A ≥ B ≥ C ≥ Unranked) were prioritized. Papers
belonging to predatory journals were usually discarded to the best of our knowledge.
3. Machine Learning
3.1. Supervised Learning
The supervised learning technique examines the training data and develops the hy-
pothesized function to map new instances. The supervised learning-based COVID-19
studies concentrate on several fields, such as COVID-19 forecasting, clinical-feature-based
COVID-19 prediction, medical-image-based COVID-19 detection, the immunological
Sensors 2023, 23, 527 5 of 35
landscape of COVID-19 analysis, and mortality-risk prediction of COVID-19 patients,

among others.
Cabitza et al. [22] developed blood tests that are inexpensive and deliver results
quickly for identifying COVID-19. The authors applied hematochemical results from
1624 patients hospitalized at San Raphael Hospital (52% were COVID-19 positive). For
classification purposes, they employed a variety of machine learning methods, including
logistic regression, naive bayes, k-nearest neighbor (KNN), random forest, and support
vector machine (SVM). KNN had the highest accuracy. Satu et al. [23] created a COVID-19
short-term forecasting system. The authors researched instances of COVID-19 infection in
Bangladesh. The dataset was taken from the Johns Hopkins University Center for Systems
Science and Engineering’s GitHub repository. Several machine learning methods, such
as support vector regression (SVR), polynomial regression, linear regression, polynomial
multilayer perceptron (PMLP), multilayer perceptron (MLP), and prophet, were employed
to anticipate the numbers of infected and fatal cases. Prophet had the lowest error rate for
predicting the following seven days of infection and mortality cases.
Arpaci et al. [24] developed a clinical-characteristics-based method for predicting
COVID-19 infection. The dataset included 114 cases gathered from the hospital in Taizhou,
Zhejiang Province, China. Positive and negative classes are present in the dataset. On the
basis of fourteen clinical characteristics, they utilized six classic classifiers, including a Bayes
classifier, meta-classifier, rule learner, decision tree, lazy classifier, and logistic regression.
The meta-classifier was accurate 84.21% percent of the time. The mortality prediction
approach for COVID-19 patients was examined by Chowdhury et al. [25]. The authors
utilized a dataset of 375 COVID-19-positive patients admitted to Tongji Hospital (China)
between 10 January and 18 February 2020. In addition, they studied the demographics,
clinical features, and patient outcomes of COVID-19-positive patients. They utilized the
XGBoost algorithm with several trees to forecast patient risk. Patterson et al. [26] developed
cytokines of the immunological landscape of COVID-19 utilizing algorithms for machine
learning. The synthetic oversampling approach waws applied to address the imbalance
issue. The model consists of three components: the severe disease binary classifier, the
multi-class predictor, and the post-acute sequelae of COVID-19 (PASC) binary classifier.
The authors classified data using the random forest algorithm. The accuracy of the binary
classifier for severe illness was 95%, the accuracy of the multi-class predictor was 80%, and
the accuracy of the PASC binary classifier was 96%.
To enhance patient care, Karthikeyan et al. [27] presented a COVID-19 mortality-risk
prediction approach. The dataset contained 2779 computerized records of COVID-19-
infected or suspected-to-be-infected patients from Tongji Hospital in Wuhan, China. The
prediction models utilized a combination of lactate dehydrogenase (LDH), neutrophils,
lymphocytes, high-sensitivity C-reactive protein (hs-CRP), and age extracted from blood
test data. For risk prediction purposes, the researcher used neural networks, XGBoost,
SVM, logistic regression, decision trees, and random forests. Compared to XGBoost, SVM,
logistic regression, decision trees, and random forests, the neural network obtained higher
accuracy. Marcos et al. [28], discovered early identifiers for patients who would die or
require mechanical breathing during hospitalization. The dataset consisted of 1260 verified
COVID-19 patients from Spain’s University Hospital of Salamanca and Hospital Clinic of
Barcelona. Their decision-making approach is based on clinical and laboratory characteris-
tics. The authors utilized three different classifiers: logistic regression, random forest, and
XGBoost. The three models attained area under the curve (AUC) ratings of 83 percent, 81
percent, and 82 percent, respectively. The COVID-19 patient mortality prediction system
was developed by Mahdavi et al. [29]. Between 20 February 2020 and 4 May 2020, the
authors analyzed the electronic medical data of 628 patients at Masih Daneshvari Hospital.
The dataset is divided into three sections: clinical, demographic, and laboratory. Three
SVM models were utilized by the authors. The steps were: First, providing clinical and
demographic information. The second step is to enter clinical, demographic, and laboratory
Sensors 2023, 23, 527 6 of 35
information. Third, just enter laboratory data. Three linked models attained the highest
degree of precision.
Li et al. [30] established a COVID-19 patient mortality prediction system. In Wuhan,
China, the authors collected clinical data from COVID-19 patients. They constructed
three classification-based models. Model one was a decision tree and gradient boosting
classifier combination. Model two was a logistic regression classifier. Model three was a
logistic regression classifier with three or five features. The best accuracy was attained by
the decision tree with gradient boosting. The serum-glucose-based COVID-19 prediction
method was introduced by Podder et al. [31]. There are 5644 rows and 111 columns in the
dataset obtained from Hospital Israelita Albert Einstein in Sao Paulo, Brazil. The dataset
contains significant shortcomings, such as an imbalance between classes and missing
values. To balance the dataset, the undersampling approach was utilized. For classification
objectives, XGBoost, random forest, decision tree, and logistic regression were utilized. The
XGBoost algorithm outperformed the competition.
Chandra et al. [32] explored a categorization approach based on chest X-ray images.
COVID-Chestxray set, Montgomery set, and NIH ChestX-ray14 set are three public reposi-
tories with which datasets were compiled. The authors distinguished between nCOVID
and pneumonia, and normal and aberrant. They utilized the artificial neural network,
support vector machine (kernel: RBF, poly, linear), decision tree, k-nearest neighbor, and
naive bayes for classification purposes. Additionally, they utilized the majority voting
algorithm. Among the previously described models, the majority vote algorithm performed
the best. The short-term cumulative COVID-19 case forecasting model was introduced by
Balli et al. [33]. The dataset comprises the weekly confirmed case and cumulative confirmed
case data compiled by the World Health Organization. They utilized linear regression, MLP,
random forest, and SVM to forecast the pandemic trend. SVM has the best trend among
the listed algorithms. Li et al. [34] found unique risk variables for COVID-19 patients.
In order to train the model, the LASSO (least absolute shrinkage and selection operator)
technique was used. The authors have added blood types such as B and AB as protective
variables, and A as a risk factor. In addition to age, gender, temperature, humidity, health
expenditure, social distance, smoking, urbanization level, and race, they found a number
of other characteristics.
Kang et al. [35] established a prediction model based on clinical data for COVID-19
patients with severe symptoms. China’s Tongji Medical College-affiliated Union Hospital’s
Tumor Center created the dataset. This dataset includes 151 instances between 26 January
and 20 March 2020. They created a four-layer ANN model with six nodes in the input layer,
thirteen and thirteen nodes in the two hidden levels, and one node in the output layer. The
ANN model attained an average of 96.9 percent accuracy.
The clinical prognostic evaluation of COVID-19 patients was proposed by Kocadagli et al. [36].
The COVID-19 patient dataset was retrieved from Koc University Hospital in Istanbul, Turkey.
The data collection included symptoms, demographic features, blood test results, and illness
histories from individuals of all ages and genders. For purposes of classification, ANNs, SVMs,
and AdaBoost (weak learner: decision trees) were implemented. The best accuracy was reached by
ANNs. Udhaya Sankar et al. [37] investigated mobile voice analytic applications for COVID-19
detection. The authors did not specify the methods used or the performances of those algorithms.
Gokcen et al. [38] created artificial neural networks for detecting COVID-19 using cough data.
Experiments have utilized available speech data from the Massachusetts Institute of Technology
(MIT). Using a filter, cough sounds were cleaned, and the mel-frequency cepstral coefficient was
applied to extract characteristics. The model is comprised of four layers, with 256, 128, 64, and
1 neurons in each. The accuracy of the model was 79 percent.
Nalini et al. [39] have explored sentiment analysis of COVID-19 from Twitter. The
dataset consists of 3090 tweets in four classes: fear, sad, anger, and joy. They created
four models: bidirectional encoder representations from transformers (BERT), logistic
regression (LR), support vector machines (SVM), and long-short-term memory (LSTM).
Models produced accuracies of 89%, 75%, 74.75%, and 65% respectively.
Sensors 2023, 23, 527 7 of 35
Table 1. The systematic overview of problems and solutions addressed through the supervised
learning methods for COVID-19.
Ref. Problem Definition ML Models Sample Performance

Logistic regresseaturion,
Prediction of COVID-19 RT-PCR test 263,007 records, Accuracy: 94.41%, 94.99%,
[40] Decision tree, SVM, naive
infection 41 features 92.4%, 94.36%, 89.2%
Bayes, ANN
Six features (deaths,
recovered, confirmed,
The number of the positive Nonlinear regression,
[41] amount of testing, MAPE: 0.24%, 0.18%, 0.02%.
cases prediction method decision tree, random forest
lockdown, lockdown
features)
Prediction model for
398 patients (43 expired and Sensitivity: 91%,
[42] mortality in COVID-19 SVM
355 non-expired) specificity: 91%
infection
COVID-19 computed 169 patients positive,
[43] tomography scan dataset – 76 normal patients, and –
for ML 60 patients with CAP
Risk factors analysis of
Decision tree, logistic
COVID-19 patients and 659 COVID-19 patients and Accuracy: 97%, 93%, 92%,
[44] regression, random forest,
ARDS or no-ARDS clinical features 83%, 90%
SVM, DNN
prediction method
Patient intensive care and Socio-demographic, clinical
[45] mechanical ventilation Random Forest data 212 patients (123 males, AUC: 80%, AUC: 82%
prediction method 89 females)
Test records of
Prediction of COVID-19
5183 individual (cough, Sensitivity: 87.30%,
[46] diagnosis based on Gradient-boosting
fever, sore throat, shortness specificity: 71.98%
symptoms
of breath, etc)
Logistic regression, decision Total 198 patients SVM: median 96%. Other
Early risk identification of
[20] tree, random forest, KNN, (135 non-severe, 63 severe model performance result
(SARS-CoV-2) patients
SVM, AdaBoost, MLP COVID-19) unclear in the paper
Chest X-ray images based
KNN, decision tree, random 371 positive, 1341 normal Precision : 98.96%, 94.89%,
[47] COVID-19 infection
forest, L-SVC, SVC chest X-ray images 97.58%, 99.3%, 99.66%
detection
KNN, RUNN-COV, logistic
SARS-CoV-2 pre-miRNAs positive 569 and negative F1 score : 89.86%, 98.26%,
[48] regression, random forest,
detection 999,888 pre-miRNA samples 89.47%, 91.55%, 89.83%
SVM
An LSTM network-based COVID-19 cases and deaths forecast system was designed
by Yogesh [49]. The dataset was collected from the source (https://ourworldindata.org/,
accessed on 20 December 2022). The dataset contains information about cases and deaths
in Italy, France, Brazil, India, Germany, United States, and Nepal. According to the day,
there were two steps: singlestep and multistep. Multistep models had higher error values
than singlestep models. Social media-based COVID-19 sentiment analysis was done by
Mohamed et al. [50]. A bidirectional long short-term memory (Bi-LSTM) model was
employed for classification purposes. The authors used three datasets obtained from
Twitter and Reddit platforms. The dataset collection duration was 2017 to 2020. The
two Twitter datasets contain 505,243 tweets and several labels (negative, neutral, positive,
fun, surprise, etc). On the other hand, the Reddit one contains 563,079 COVID-19-related
comments. It has five labels (very positive, positive, neutral, negative, and very negative).
The proposed Bi-LSTM model has several layers, such as embedding, bidirectional, dropout,
and dense layers. The model produces highly fluctuating F1 scores.
Sensors 2023, 23, 527 8 of 35
Pahar et al. [51] categorized the COVID-19 cough using machine learning. The au-
thors acquired the dataset via an online site. The normalization method was employed
for data preparation. Features were extracted using mel-frequency cepstral coefficients,
log frame energies, zero-crossing rate, and kurtosis. The support vector machine, logis-
tic regression, k-nearest neighbor, multilayer perceptron, convolutional neural network,
long short-term memory, and residual-based neural network architecture (Resnet50) were
utilized for classification applications. The conventional machine learning techniques
(support vector machine, logistic regression, k-nearest neighbor, and multilayer perceptron)
are extremely inefficient.
Figure 3. The percentages of various supervised learning models being used for COVID-19 tasks.
In supervised learning, Figure 3 shows the SVM contributed highest segment. In

addition, random forest, logistic regression, ANN/MLP, and decision tree contributed
15.1%, 11.8%, 11.8%, and 10.8% respectively. Table 1 presents problems and solutions
through the supervised learning methods for COVID-19.
3.2. Unsupervised Learning

Models of unsupervised learning find hidden objects, patterns, and groupings without
data labels or human interaction. Unsupervised models provide a solution for exploratory
medical picture grouping, risk analysis, anomaly detection, differentiation, patient severity
detection, patient screening, and discovery of disease-related genes.
Boussen et al. [52] created a clustering-based COVID-19 patient severity and intubation
monitoring system. Green, orange, and red are the classifications for the patient monitoring
and decision-making sections, respectively. Each group has distinct characteristics, such
as a lesser risk of intubation, a high degree of hypoxia, and the prompt consideration of
intubation for a patient in the red category. Gaussian mixture represents the clustering
model. The model achieved 87.8 percent accuracy. Zhao et al. [53] built an unsupervised
model to identify anomalous changes in PM2.5 in Chinese cities between 2017 and 2020. The
dataset is comprised of 9,721,023 samples. Encoder, decoder, and anomaly assessment make
up the method’s three blocks. This technology can help monitor air quality in response
to abrupt changes. Lai et al. [54] evaluated publicly accessible and pertinent COVID-19
Sensors 2023, 23, 527 9 of 35
data sources, addressed the difficulty of data heterogeneity by clustering, and categorized
counties based on underlying variations. For clustering purposes, the k-means clustering
method was implemented.
Chen et al. [55] investigated the segmentation network for the COVID-19 computed
tomography (CT) infection. Data were compiled from the Italian Society of Medicine,
International Radiology, and prior research. Their dataset consists of 10,200 2D CT scans
synthesized in a laboratory. The authors utilized the unsupervised domain adaptation
learning architecture. The proposed network is divided into multiple modules, including a
feature extractor, pixel-wise classifier, and domain adaption module. Convolutional and
max-pooling layers were employed to extract features. The network’s dice performance
was 86.54 percent, its sensitivity was 85.54 percent, and its specificity was 99.80 percent.
Kurniawan et al. [56] introduced a clustering and correlation matrix-based COVID-19
risk analysis technique for pandemic nations. The authors retrieved information from
Worldometers. For clustering applications, the k-means method has been used. Five
experiments were conducted based on varying numbers of clusters. Five was the optimal
number of clusters.
Zheng et al. [57] investigated unsupervised meta-learning for distinguishing COVID-
19 and pneumonia patients. They presented a dataset consisting of 2696 images of COVID-
19 pneumonia; and 10,155 images of SARS, MERS, influenza, and bacterial pneumonia.
The data augmentation approach was employed to produce images. This framework
was comprised of two modules: one based on network-based learning and the other
on relational models. Utilizing the DenseNet-121 architecture, network-based learning
characteristics were extracted. The relation model is represented by an 8-pooling layer
network architecture. This model performs better than supervised models such DenseNet-
121, DenseNet-161, ResNet-34, and VGG-19. Oniani et al. [58] investigated relationships
between several biological entities and COVID-19. Experiments were conducted using the
CORD-19-on-FHIR dataset. T-distributed stochastic neighbor (t-SNE) and the density-based
clustering method (DBSCAN) were used for clustering purposes.
Ewen et al. [59] suggested online unsupervised learning approach for COVID-19-
CT-scan image classification. The components of online unsupervised learning include
online machine learning and unsupervised learning. In the experiment, the COVID-19-
CT-scan-images dataset from the signal processing grand challenge (SPGC) was utilized.
The dataset includes three categories: healthy, COVID-19, and CAP. The technique of
horizontally flipped data augmentation was used to increase the number of image copies.
The baseline adopted the DenseNet169 architecture. The accuracy of the proposed model
was 86.7%.
Miao et al. [60] created an unsupervised meta-learning model for the screening of
COVID-19 patients. The author gathered three datasets from publicly accessible sources,
including BIMCV-COVID19+, Kaggle-pneumonia, and CC-CXRI. The unsupervised meta-
learning model was composed of both the DL model and gradient-based optimization.
Convolution, max-pooling, and batch normalization are a few of the DL model’s layers.
This model is superior to the LeNet, Alexnet, visual geometry group (VGG), CovXNet
CNN-RNN, and EMCNet models.
Fujisawa et al. [61] examined the COVID-19 disease-related gene identification approach
using unsupervised main-component-analysis-based feature extraction (PCAUFE). PCAUFE
was applied to the RNA expression patterns of 18 healthy individuals and 16 patients with
COVID-19. The expression of RNA yielded the identification of 123 genes. The authors
classified COVID-19 patients and non-patients based on 123 genes identified by PCAUFE
using logistic regression (LR), support vector machine (SVM), and random forest (RF) models.
Three models attained areas under the curve (AUC) in excess of 90%.
King et al. [62] investigated the grouping of COVID-19 chest X-ray images using self-
organizing feature maps (SOFM). The authors obtained the dataset from a freely accessible
source. This dataset consists of two classes: infected and non-infected. They compared
Sensors 2023, 23, 527 10 of 35
and measured the distributions of pixels in the non-infected cluster and the infected cluster
using the overlapping coefficient.
Xu et al. [63] developed an unsupervised technique for lung segmentation and pul-
monary opacity identification using CT scan images. The datasets were obtained from
Osaka University, Zhejiang University, Hangzhou Second People’s Hospital, Jingmen First
People’s Hospital, Taizhou Hospital, and Sir Run Shaw Affiliated Hospital of Zhejiang
University School of Medicine, among others. The image augmentations approach was
utilized to produce training images. The lung segmentation model had U-shaped convo-
lutional neural network (U-Net) architecture and many pre-trained encoders, including
VGG19, MobileNetV2, and ResNet50. For opacity detection, an auto-encoder based on
generative adversarial networks (GAN) was utilized. Classification techniques utilizing
support vector machine, random forest, adaptive boosting, and XGBoost were compared.
Overall, the accuracy of this method was 95.5%.
Figure 4. The percentages of various unsupervised learning models used for COVID-19.
In unsupervised learning, Figure 4 indicates k-means and meta-learning reached

18.2%. However, gaussian mixture, unsupervised domain adaptation, t-SNE, DBSCAN,
PCA, SOFM, and other models contributed 9.1%.
4. Deep Learning
4.1. Object Detection
Every object has distinguishing qualities and unique characteristics. The object de-
tection methods are comprised of mathematical models and millions of parameters that
are used to learn characteristics from objects and discover new instances. The object de-
tection models have demonstrated possible applications in the COVID-19 domain, such
as COVID-19 detection and screening, infection risk assessment, abnormality detection,
body temperature measurement, personal protective equipment detection, and social
distance monitoring.
Huang et al. [64] created a YOLO v4-based object detection method for multiplexed
circular-flow immunoassay test strips that can rapidly measure and distinguish antibodies
that bind the membrane glycoprotein of severe acute respiratory syndrome coronavirus 2.
The lateral flow immunoassays are comprised of many pads, including sample, conjugate,
absorbent pads, and nitrocellulose membrane. To evaluate the model, the authors did not
Sensors 2023, 23, 527 11 of 35
provide any assessment measures. Xu et al. [65] investigated a location-attention technique-

based screening method to differentiate COVID-19 from influenza-A viral pneumonia
(IAVP) and healthy patients using lung CT scans. The collection comprises 618 CT images:
219 with COVID-19, 224 with IAVP, and 175 from healthy individuals. Utilizing random
clipping, up–down flipping, left—right flipping, and mirroring, photos were enhanced.
The location-attention technique was introduced to the ResNet-18 backbone by the authors.
The noisy OR Bayesian function has been utilized to assess infection kind and overall
confidence score. The total accuracy of the method was 86.7 percent.
Bhuiyan et al. [66] presented detection methods for face masks. Web-sourced data
collection included 300 mask and 300 no-mask entries. To recognize face masks, the you
only look once version 3 (YOLOv3) method was implemented. The LabelIMG software tool
was used for image annotation purposes. The model achieved a classification and detection
accuracy of 96%. Face recognition, mask detection, and social distancing monitoring
were adopted by Vergin et al. [67] to decrease the propagation of the virus and human
participation. The dataset comprises 200 face picture samples, 700 individuals wearing
masks, and 700 individuals without masks. The authors further utilized the Microsoft
Common Objects in Context (MS-COCO) and Visual Object Classes Challenge 2007 and
2012 (VOC0712) datasets. Utilizing a binary patterns histograms technique to recognize
the face, MobileNetV2 transfers knowledge architecture to detect masks and a single
shot detector to monitor social distance. The article lacks performance reports. Sandeep
et al. [68] established a mask-detection approach through AlexNet and CNN. However,
the authors used two datasets, the Real-World Masked Face Dataset (RMFD) and CelebA.
The CNN consists of fifteen layers, including convolutional, max-pooling, flatten, dropout,
and dense. This paper did not have a confusion matrix; even though their accuracy and
precision were extraordinary, recall was dramatically worse. Hybrid deep transfer learning
and machine learning-based face mask detection model were developed by Mohamed
et al. [69]. The datasets were collected from three different sources: RMFD, the Simulated
Masked Face Dataset (SMFD), and Labeled Faces in the Wild (LFW). The model has two
blocks: a feature extraction block and a decision-making block. Resnet50 was employed
for extracting features and the rest of them for decision-making (decision trees, SVM). The
model produced remarkable accuracy. Similar studies were done by Jonathan et al. [70],
who created facial recognition systems individuals with and without face masks. The
dataset consists of 13,359 images: 7067 with a face mask and 6292 without a face mask.
MobileNetV2 was used for classification and FaceNet for face recognition purposes. The
model achieved an overall accuracy of 99.52%.
Hou et al. [71] described a method for detecting social separation using the video
frame. The YOLOv3 model was utilized to identify bounding box coordinates, object
confidence, and class label probabilities. There are no assessment metrics presented. Al-
Antari et al. [72] investigated the object detection technique for chest X-radiation (X-ray)
images. COVID-19, masses, effusion, pneumonia, atelectasis, infiltration, pneumothorax,
cardiomegaly, and nodules are the nine distinct items included in the dataset. Utilization of
you only look once (YOLO) architecture for training purposes. The model obtained 90.67
percent detection accuracy and 97.40 percent classification accuracy. Al-Antari et al. [73]
performed work that was similar. The collection is comprised of 326 chest X-ray images
from two distinct sources: Qatar University and the public dataset. They have utilized the
YOLO object detector as well. This computer-aided design (CAD) framework achieved
detection and classification accuracy of 96.31 percent and 97.40 percent, respectively.
Rezaei et al. [74] recommended using closed-circuit television (CCTV) security cameras
for social distancing monitoring and infection risk assessment (COVID-19). The suggested
model has many components, including CSPDarknet53, neck, and head. The CSPDarknet53
consists of two components: CNN (you only look once, version 4) and the pre-trained
Darknet53 model. The YOLOv3 model is present in the spatial pyramid pooling (SPP), path
aggregation network (PAN), spatial attention module (SAM), and head block components.
For training purposes, Microsoft Common Objects in Context (MS COCO) and Google
Sensors 2023, 23, 527 12 of 35
Open Picture datasets were utilized. For assessment reasons, the Oxford Town Centre
dataset was utilized. The model achieved 99.8 percent accuracy. Li et al. [75] created the
COVID-19 categorization and detection method using chest X-ray images. The dataset
was obtained via Kaggle. There are other classifications within the dataset, including
negative for pneumonia, indeterminate, typical, and unusual appearance. You only look
once, version 5 (Yolov5), was utilized for target location detection. The authors evaluated
the precision using Faster RCNN and EfficientDet as comparisons. The suggested Yolov5
model exceeded expectations.
Yoshitsugu et al. [76] identified the symptoms and localized the afflicted region
of COVID-19. The dataset comprises 6334 chest X-ray scans divided into four classes:
pneumonia-free, usual appearance, uncertain appearance, and atypical appearance. Utiliz-
ing an EfficientNetB7 backbone for image classification applications, YOLOv5 was used to
detect and localize symptoms.
Table 2. The overview of applied object detection methods for COVID-19.
Ref. Problem Definition Architecture Sample Performance

4 classes (194 COVID,
1772 bacterial pneumonia,
583 normal, 493 viral
COVID-19 detection Accuracy (97.11 ± 2.71%)
pneumonia cases) 2 classes
[77] through the chest X-ray DarkNet-53 + YOLO-v3 multi-class, 99.81% binary
(2848 non-COVID,
images class
194 positives, samples)
Dataset augmentation
applied
Social distancing monitoring PASCAL-VOC, MS-COCO,
[78] system using mass video Faster R-CNN, YOLO, SSD vision-based social media mAP 86.8%, 84.7%, 44.5%
surveillance event analysis
5327 images (face mask and
Personal protective
[79] YOLOv4 shield, no face mask, hand Precision 78%
equipment detection
gloves)
DeepLesion, 32,120 axial CT mAP 91.28% (internal
slices (liver, lung, bone, testing), 87.83%
Detecting COVID-19 related
[80] RetinaNet abdomen, mediastinum, (External-Set-1), 71.48%
CT abnormalities
kidney, pelvis, and soft (External-Set-2), 83.04%
tissue) (External-Set-3)
1575 (Various scenarios
Social distancing detector while walking, different Accuracy 95.6%, 91.2%,
YOLOv2, Fast R-CNN,
[81] through thermal images or body positions, running, 88.5%, (Dataset II 94.5%,
R-CNN
video streams sneaking, and and different 90.5%, 86.5%)
motion speeds)
Dataset Celeba, Coco, Helen,
Detection of masks and
IMM, Wider, Group Images,
human eye areas.
IIITD and beyond visible
[82] Measurement of body YOLOv5, Resnet-50 Precision 96.65%, 78.7%
spectrum disguise,
temperature through
UL-FMTV, Terravic Facial
thermal cameras
Infrared, IRIS
The indoor distance Accuracy (4FPS10 62.5%,
measurement method DeepSORT, YOLOv3, 4FPS24 93.7 4FPS35 78.9%
[83] MS COCO dataset
through the closed-circuit YOLOv4 4FPS50 83.3%) mAP
television 30.4%, 42.1%
750 CT images (COVID-19 Accuracy (train, validation,
Data labeling and
[84] mask R-CNN positive, COVID-19 and test 99%, 93.1%,
annotation framework
negative) and 0.8%)
Sensors 2023, 23, 527 13 of 35
The performance of the classification model exceeds that of the object detection model.
COVID-19 lesion detection was created by Nurmaini et al. [85] utilizing CT scans. The
dataset contains 419 CT scan images of SARS-CoV-2 infected individuals and 200 CT scan
images of healthy individuals. The architecture is comprised of CNNs and Faster RCNNs.
The CNN model consists of thirteen convolution layers and four maximum polling layers.
The suggested model attained a mean average accuracy of 90.41 percent (mAP). COVID-
19 classification and localization methodology was developed by Rajaraman et al. [86].
The authors used six different datasets, including Pediatric CXR, RSNA CXR, CheXpert
CXR, NIH CXR-14, Twitter-COVID-19 CXR, and Montreal-COVID-19 CXR. The model
has various steps: segmentation block, repeated specific transfer learning models, and
class-selective relevance mapping (CRM)-based region of interest (ROI) localization. The
custom U-net architecture was designed for segmentation purposes. VGG-16, VGG-19,
Inception-V3, Xception, DenseNet-121, NasNet-Mobile, MobileNet-V2, and ResNet-18 were
applied for knowledge transfer purposes. The CRM-based ROI localization have been
applied to interpret predictions of individual convolutional neural networks and compare
against the ground truth.
Using chest X-ray images, Saiz et al. [87] created the COVID-19 position detection algorithm.
They collected data from Kaggle and GitHub. This set possesses two classes: one standard and
the other COVID-19. They used a single-shot detector architecture to detect the position of an item.
VGG-16 is the foundation of the single-shot multi-box detector network (SSD300). The model
achieved a sensitivity of 94.92 percent and a specificity of 92 percent.
Figure 5. The rate of percentages are used in various object detection models for COVID-19. The
most used models are shown in a sub-chart for proper visualization.
Loey et al. [88] suggested a method for detecting medical face masks. The data
collection included 1535 images. To enrich the dataset, the data augmentation approach
Sensors 2023, 23, 527 14 of 35
was utilized. ResNet-50 architecture is used to extract features, and the YOLO v2 method is
used to recognize medical face masks. The model’s average degree of accuracy was 81%.
Li et al. [89] revealed severe instances and minor symptoms based on the identification
of COVID-19 using CT scans of the lungs. The dataset was acquired from Zhongnan
Hospital Wuhan University and the Italian Society of Medical and Interventional Radiology.
The model is composed of the network-in-network technique with instance normalization.
ResNet was used in COVID-19 CT images to extract features. The performed a performance
comparison of domain-adaptive Faster R-CNN, Faster R-CNN, and few-shot adaptive
faster R-CNN architectures. Their proposed model was superior.
In object detection, YOLO, SSD, and R-CNN were the major applied models. Figure 5
shows YOLO base models contributed 50%, then R-CNN 28.9%, SSD and others models
contributed 10.5%. Table 2 presents a systematic overview of object detection methods for
COVID-19.
4.2. Transfer Learning

Transfer learning applies a previously-trained model to a new problem, which is more
accurate, quicker, and requires less training data. The transfer learning-based COVID-19
study includes automatic analysis of medical pictures, COVID-19 classification, identifica-
tion of lung disease severity, and the automated COVID-19 screening model.
De Moura et al. [90] established automated classification methods for chest X-ray
images. There are three groups in the dataset: COVID-19, pneumonia, and healthy patients.
To enhance the visual data, the scaling with horizontal flipping augmentation approach
was used. The authors employed six deep learning architectures for classification, includ-
ing DenseNet-121, DenseNet-161, ResNet-18, ResNet-34, VGG-16, and VGG-19. When
detecting COVID-19 pneumonia lessons, their method earned a 97.44 percent accuracy rate.
Abbas et al. [91] introduced a transfer learning-based CNN-based self-supervised
sample decomposition method. Three datasets from various sources were utilized. The
steps of this approach include sample decomposition, pretext training, and downstream
task. Knowledge-transfer-pre-trained backbones such as ResNet18, GoogleNet, and VGG19
were utilized by the authors. The suggested model with the ResNet18 backbone exceeded
expectations. Rehman et al. [92] investigated an CT and X-ray image-based categorization
technique. The dataset includes 200 COVID-19, 200 viral pneumonia, 200 bacterial pneumonia,
and 200 X-ray and CT images of healthy individuals. This research applied nine deep trans-
fer learning architectures, such as Aexnet, SqueezeNet, GoogLeNet, VGG16, MobileNetv2,
ResNet18, ResNet50, ResNet101, and DenseNet201. The total accuracy of the model was
98.75 percent. Ahuja et al. [93] reported a technique for detecting COVID-19 utilizing CT
scan images and transfer learning. Two classes comprised the dataset: positive and normal.
Data enhancement was utilized to boost the quantity of photographs. To transmit knowl-
edge, ResNet18, ResNet50, ResNet101, and SqueezeNet designs were utilized. ResNet18 was
utilized to determine the localization of abnormalities. ResNet18 performed better overall.
Shamsi et al. [94] created an uncertainty-aware transfer learning framework for de-
tecting COVID-19 utilizing X-ray and CT images. The dataset was compiled from several
sources. Four deep learning architectures, including InceptionResNetV2, VGG16, ResNet50,
and DenseNet121, were utilized to extract features. To identify COVID-19 instances, ex-
tracted features were analyzed by many machine learning algorithms, including k-nearest
neighbors (KNN), SVM, gaussian process, neural network (NN), random forest, adaboost,
and naive bayes. The ResNet50 architecture with SVM and NN beat the competition.
To categorize chest X-ray images, Bassi et al. [95] built dense convolutional networks
with transfer learning. There are three classifications in the dataset: COVID-19, pneumo-
nia, and normal. The model contains two deep learning backbones, DenseNet 201 and
DenseNet 121. They utilized twice transfer learning with output neuron technique to boost
performance. Jaiswal et al. [96] examined the COVID-19 identification approach based on
transfer learning. The authors acquired 2492 CT scan images from Kaggle for the dataset:
1262 are positive; 1230 are negative. To transfer information, four deep learning models,
Sensors 2023, 23, 527 15 of 35
including VGG-16, ResNet152V4, InceptionResnetV2, and DenseNet201, were utilized.

DenseNet201 outperformed VGG-16, ResNet152V2, and InceptionResnetV2 structures.
Table 3. The overview of existing transfer learning applications for COVID-19.

Category (COVID-19, CAP,
DensetNet201, ResNet101, F1 score 95.53%, 96.74%,
[97] COVID-19 classification SPT, HC) total 1164 CCT
CCSHNet 97.04%
images
The deep transfer learning
technique has used to 413 COVID-19 (+), Accuracy 93.01%, sensitivity
[98] (CNN+ ResNet-50)
classify COVID-19 infected 439 normal or pneumonia 91.45%
patients
219 COVID-19 positive,
An automated COVID-19 Accuracy 89.16%, 96.01%,
[99] CNN, VGG-16 ResNet-50 1345 pneumonia infection
screening model 93.29%
and 1341 no infection
Hybrid deep transfer
COVID-19 219, Viral
learning-based COVID-19
[100] AlexNet, BiLSTM Pneumonia 1345, Normal Accuracy 98.14%, 98.70%
positive cases detection
1341
using chest CT X-ray images
Transfer knowledge-based
chest X-ray images ResNet, Inception v2,
COVID-19 108, other
classification. Random (Inception + ResNet-v2), F1 sore 56%, 74%, 96%, 95%,
[101] pneumonia 515, normal 533,
oversampling was applied DenseNet169, 98%
tuberculosis 58
to overcome the class NASNetLarge
imbalance problem
GAN with deep transfer Total 307 X-ray images
learning technique for Alexnet, Googlenet, (COVID-19, normal, Binary classes accuracy
[102]
coronavirus detection in Restnet18 pneumonia bacterial, and (99.6%, 99.9%, 99.8%)
chest X-ray images pneumonia virus)
Two-step transfer learning COVID-19 189, pneumonia
[103] ResNet34 Accuracy 91.08%
for COVID-19 detection 252, Normal 235 images
Deep transfer
learning-based COVID-19 DenseNet201, Resnet50V2 COVID (+) 538, Accuracy 91.11%, 91.11%,
[104]
detection using X-ray and Inceptionv3 COVID (−) 468 90.43%
images
EfficientNet B0, EfficientNet
B1, EfficientNet B2,
13,800 X-ray images, Accuracy 90.0%, 91.8%,
EfficientNet B3, EfficientNet
COVID-19 screening in Healthy, non-COVID-19 90.0%, 93.9%, 93.0%, 92.2%,
[105] B4, EfficientNet B5,
chest X-rays images pneumonia, COVID-19 90.4%, 90.0%, 83.5%, 77.0%,
MobileNet, MobileNet V2,
patients 75.3%
RESNET 50, VGG-16,
VGG-19
Multiple Kernels-Extreme 349 images of COVID-19
AlexNet, GoogleNet,
Learning Machine-based and 397 images of Accuracy 90.34%, 92.86%,
VGG16, MobileNetv2,
[106] DNN system to detect no-findings (data 92.65%, 93.19%, 92.22%,
ResNet18, Inceptionv3
COVID-19 disease from CT augmentation was applied 92.54%, 98.36%
(DenseNet201+ MK-ELM)
scan images to expand the dataset)
Horry et al. [107] developed transfer learning for COVID-19 identification through
X-ray, ultrasound, and CT scan images. To eliminate sampling bias, a normalization func-
tion and the contrast-limited adaptive histogram equalization (N-CLAHE) approach were
utilized. To enrich the data, techniques such as horizontal flip, horizontal and vertical shift,
and rotation were utilized. For training models, the VGG16, VGG19, ResNet50 V2, Incep-
tion V3, Xception, InceptionResNet V2, NasNetLarge, and DenseNet 121 deep learning
backbones were deployed. The VGG19 model attained the highest precision of 86% for
Sensors 2023, 23, 527 16 of 35
X-rays, 84% for CT scans, and 100% for ultrasound. Zhu et al. [108] investigated convo-
lutional neural network (CNN) and knowledge-based transfer (VGG16) to predict lung
disease severity scores. One-hundred and thirty-one chest X-ray images from 84 COVID-19-
positive individuals make up the dataset. Conv2D, BatchNormalize, MaxPool2D, Flatten,
Dropout, and Dense are the 15 CNN layers. The authors compared CNN and VGG16
models’ performances. The VGG16 architecture was superior.
Figure 6. The percentages of various transfer learning models used for COVID-19.
Sensors 2023, 23, 527 17 of 35
Using chest radiographic images, Gianchandani et al. [109] developed a deep trans-
fer learning-based COVID-19, viral pneumonia, and bacterial pneumonia classification
technique. For experimental purposes, two datasets were utilized, one from Kaggle and
the other from the University of Dhaka and Qatar University. For classification objectives,
VGG16, ResNet152V2, InceptionresNetV2, DenseNet201, and the suggested ensemble
(VGG16+DenseNet) were employed as deep learning backbones. Multiple dense, rectified
linear unit (ReLU) activation, and dropout layers were added to the aforementioned DL
models. The accuracy of VGG16+DenseNet was 96.15 percent.
Hira et al. [110] reported a technique based on deep learning to distinguish COVID-
19 illness patients from bacterial pneumonia, viral pneumonia, and healthy cases. To
enhance the image in the pre-processing phase, random horizontal flipping and random
cropping were implemented. For binary and multiclass classification, the pre-trained
models Inception ResNetV2, ResNeXt-50, GoogleNet, ResNet-50, Se-ResNet-50, AlexNet,
Inception V4, DenseNet121, and Se-ResNet-50 were utilized. Among all the pre-trained
models, the Se-ResNeXt-50 architecture had the highest accuracy for binary class and
multi-class classification.
Ibrahim et al. [111] investigated the classification approach for chest X-ray images
using the AlexNet architecture. The dataset includes 11568 images divided into four cate-
gories, including COVID-19, bacterial pneumonia, non-COVID-19 viral pneumonia, and
normal CXR. Binary to multiclass classification was applied for classification purposes.
The classification method with the highest accuracy was binary. Using chest X-ray images,
Kamal et al. [112] constructed eight transfer knowledge architectures for COVID-19 clas-
sification. The dataset consists of 760 images classified into four categories: COVID-19,
pneumonia, healthy, bacterial pneumonia, and viral pneumonia. For detecting purposes,
eight pre-trained models were employed, including ResNet50, ResNet50V2, VGG19, In-
ceptionV3, MobileNet, MobileNetV2, DenseNet121, and NasNetMobile. DenseNet121
outperformed the others.
Deep transfer learning classification techniques using meta-heuristic algorithms were
created by Canayaz et al. [113]. The dataset includes three image classes: COVID-19,
normal, and pneumonia. X-ray image characteristics were extracted utilizing AlexNet,
VGG19, GoogleNet, and ResNet. Effective features were chosen using binary particle swarm
optimization (BPSO) and binary grey wolf optimization (BGWO) metaheuristic methods.
Selected characteristics were classified using the SVM method. The performance of the
suggested model using the VGG19 architecture was superior. Das et al. [114] devised an
algorithm for the automated analysis of chest X-ray images. CNN and deep transfer learning
blocks are included in the suggested approach. CNN has initially extracted characteristics
from chest X-ray scans. They utilized the Xception architecture to communicate knowledge.
The suggested model was compared against SVM, random forest, backpropagation network,
adaptive neuro-fuzzy inference system, CNN, VGGNet, ResNet50, Alexnet, Googlenet, and
Inceptionnet V3. The proposed model gained improved precision.
The COVID-19 detection method was created by Rodriguez et al. [115] using spec-
trograms of coughing, sneezing, and other respiratory sounds. The dataset contains two
labels, sick and not sick, gathered by the pharmaceutical manufacturer Pfizer in the United
States. To train the model, an Xception pre-trained architecture was utilized. As a result of
the overfitting problem, the model achieved poor performance.
Loey et al. [116] created a categorization algorithm of COVID-19 cough sound symp-
toms using scalogram images. This methodology ran its tests using the COUGHVID
dataset. There are 755 COVID-19 and 702 healthy wave cough sounds in the collection.
Data categorization was performed with GoogleNet, ResNet18, ResNet50, ResNet101, Mo-
bileNetv2, and NasNetmobile architectures. The ResNet18 design achieved high precision.
Imran et al. [117] investigated the COVID-19 screening application by analyzing cough
sounds. The authors utilized the ESC-50 sound dataset, which included 96 bronchitis
coughs, 130 pertussis coughs, 70 COVID-19 coughs, and 247 normal coughs. Mel frequency
cepstral coefficients (MFCC) and principal component analysis were used to extract features
Sensors 2023, 23, 527 18 of 35
(PCA). The sound data was transformed into a mel-spectrogram, which represents the
frequency spectrum. The presented model was divided into three parts: a deep trans-
fer learning-based multi-class classifier, a classical machine learning-based multi-class
classifier, and a deep transfer learning-based binary class classifier. This article does not
specify which transfer learning model was used. The random undersampling approach
was utilized to rectify the imbalance; however, information loss is a worry. Support vector
machine classification model for structural data.
For transfer learning, Figure 6 illustrates ResNet based models achieved highest
proportion in COVID-19 context. Nevertheless, VGG, DenseNet, and inception based
models contributed 15.3%, 13.7%, and 9.9% respectively. Table 3 presents summary of
transfer learning applications for COVID-19.
4.3. Image Segmentation

The segmentation approach provides the grouping of similar regions or segment maps
as outputs corresponding to the input. Medical images based on COVID-19 infected area
segmentation, lung and infection region segmentation, and infected tissue segmentation
are key applications in this sector.
Saeedizadeh et al. [118] investigated the segmentation framework for COVID-19 chest
area detection using CT images. The dataset was compiled from several sources. The
authors created a TV U-Net architecture that resembles a U-Net design. Nine hundred
images were utilized for the aim of evaluating models. The TV U-Net model earned a mIoU
rate of 99 percent and a dice score of around 86 percent. Ma et al. [119] in the Corona Cases
Initiative and Radiopaedia compiled a dataset with 300+ illnesses and 1800+ slices. Task
one, three-class segmentation, including lung, infection, or both, was based on restricted
annotations. Task two, two-class segmentation consisting of lung and infection, was based
on non-COVID-19-CT-scan images. The third task, two-class segmentation—healthy lung
and infected lung—was based on COVID-19 and non-COVID-19-CT-scans. Average dice
similarity coefficient (DSC) scores for their model were 97.3%, 97.7%, and 67.33%.
Yazdekhasty et al. [120] demonstrated COVID-19-infected lung segment areas. The
dataset was compiled from two sources open to the general public. The author has created
a U-Net-based, two-dimensional model for two distinct segmentation kinds. One decoder
was for the healthy lung area and another for the diseased lung region. The design
comprised four building blocks: a decoder, two parallel encoders with a split structure,
and a merging encoder. The model’s sensitivity was 74.9 percent, and its specificity was
99.7 percent. Ranjbarzadeh et al. [121] investigated CNN-based COVID-19 CT images that
autonomously partition diseased lung tissues. The dataset was compiled from two sources
open to the general public. They implemented fuzzy c-means clustering and local directed
pattern encoder algorithms for performance enhancement. The model scored 96 percent
accuracy, 97 percent recall, and 97 percent F-score.
Zheng et al. [122] designed the three-dimensional (3D) CU-Net architecture to detect
COVID-19-infected regions in 3D chest-CT-scan images. The model of 3D CU-Net was
built on a 3D U-Net architecture. Rich features were extracted with the aim of performance
enhancement. Three variations of the model exist: 3D U-Net, 3D CU-Net (α = 0.5, β = 0.5),
and 3D CU-Net (α = 0.3, β = 0.7). Chen et al. [123] investigated 3D U-Net architecture for
COVID-19 segmentation using CT images. This work was evaluated using both public and
private datasets. The private dataset contained 89 COVID-19 infection records. The public
dataset had 1700 data points. Combination loss and data augmentation strategies were
introduced to enhance the training impact. In comparison to other methods, the model
yielded amazing results. Yan et al. [124] reported a 3D-based CNN segmentation approach
for COVID-19-infected chest CT images. The collection included 165,667 annotated chest
CT images from 861 COVID-19-positive individuals. The authors created the FV block
that modifies the global parameters of the features for adaptively segmenting COVID-19
infection. The COVID-19 segmentation model received a dice score of 72.6%, whereas lung
segmentation had a die score of 98.7%.
Sensors 2023, 23, 527 19 of 35
Frid-Adar et al. [125] examined the identification, segmentation, and grading of

COVID-19 pneumonia in chest X-ray images. The authors employed a two-stage training
approach for detection and segmentation. The ResNet50 model was employed for detec-
tion purposes. For segmentation purposes, a modified U-Net with pre-trained VGG-16
architectures was employed. Degerli et al. [126] suggested X-ray images based on the
production and detection of the COVID-19 infection map. They are architectures that
integrate segmentation and detection. For segmentation purposes, the U-Net, UNet++,
and deep layer aggregation (DLA) algorithms were utilized. For detection purposes, the
CheXNet, DenseNet-121, Inception-v3, and ResNet-50 algorithms were used. The model
attained a sensitivity of 94.96 percent and a specificity of 99.88 percent.
Table 4. Review of image segmentation applied applications for COVID-19.

Dual path Network
Segmentation (positive 877,
(DPN)-92, Inception-v3,
CT image segmentation and negative 541) Classification Sensitivity 97.4%,
[127] ResNet-50, and Attention
classification (positive 718, negative 70, Specificity 92.2%
ResNet-50 FCN-8s, V-Net,
and other diseases 343)
U-Net, 3D U-Net++
SCOAT-Net, PSPNet,
Automatic segmentation of ESPNetv2, DenseASPP, Two patients scanned at Proposed model (DSC
[128] lung opacification from CT UNet+, DeepLabV3+, different times, and Kaggle 88.99%, Sensitivity 87.85%,
images U-Net, COPLE-Net, CE-Net, dataset PPV 90.28%)
Attention U-Net
Dilated dual attention
COVID-19 lesion Three open-source datasets
[129] U-Net architecture with a Dice 72.98%, recall 70.71%
segmentation in CT slices total 1645 slices
ResNeXt-50
Segment the radiological Superpixel based fuzzy
[130] 115 CT scan images —
images modified flower pollination
ML and DL-based classifier 3D neural network, AUC 93%, sensitivity 90%,
[131] 2446 chest CTs images
with CT image opacity map DenseUnet specificity 83%
Multi-point supervision
Dice 83.25%, sensitivity
network for segmentation of
[132] U-Net based (MPS-Net) 300 CT images 84.06%, specificity 99.88%,
COVID-19 lung infection
IOU 74.2%
using CT image
Binary and multi-class Binary segmentation
detection and labeling of (SegNet) 95%, multi-class
[133] SegNet and U-NET 100 CT images
infected tissues on CT lung (U-NET) 91% mean
images accuracy
A combination of human
Lung and lobar
and animal 3D CT images. Dice coefcient of
[134] segmentation of CT images Seg3DNet
1453 for training, 7998 for 0.985 ± 0.011
in patients with COVID-19
evaluation
The segmentation and
classification of COVID-19 F1-Score (binary 88%,
[135] U-Net 1645 CXR images
using chest X-ray (CXR) multiclass 83%)
images
COVID-19 classification
COVID-19 3616, Normal
[136] using plain and segmented U-Net, Modified U-Net Dice 96.3%, 96.94%
8851, Non-COVID 6012
lung CXRs
Fung et al. [137] described a two-stage deep learning algorithm for segmenting COVID-
19 chest CT images. They implemented three models for segmentation: U-net, single
SInfNet, and SSInfNet. The SSInfNet model consists of one U-net and one SInfNet. The
single SInfNet model fared better. Wu et al. [138] presented a combined classification and
Sensors 2023, 23, 527 20 of 35
segmentation method for the COVID-19 chest CT diagnosis. They produced a big dataset
consisting of 144,167 chest CT scans of 400 COVID-19 patients and 350 uninfected cases.
There were 3855 CT scans from 200 COVID-19 patients. The Res2Net classification model
was utilized for classification purposes. In the encoder block, the VGG-16 model was uti-
lized. Encoder and decoder blocks utilized a U-shaped design. The model achieved 95.0%
classification sensitivity and 93.0% classification specificity. The score on the segmentation
dice was 78.5 percent.
Laradji et al. [139] presented a consistency-based loss function for the segmentation of
COVID-19 in CT images. The model consists of two branches, the first of which encodes,
and the second decodes the altered input. The authors employed three open-source medical
segmentation datasets for assessment purposes. CB (Flip, Rot) + PL identified the suggested
model with superior performance.
Wang et al. [140] suggested a method for learning using a hybrid encoder. The authors
gathered data from four distinct sources, including the Corona Cases Initiative and Radio
Media, Medical Segmentation Decathlon (MSD) Lung Tumor, Structseg Lung Cancer, and
Non-Small Cell Lung Cancer (NSCLC) databases. For encoder–decoder blocks, the 3D
U-Net architecture was utilized. The authors utilized transfer learning techniques, such as
continuous learning, body fine-tuning, and pre-trained lesion representations. The model
attained DSC values of 0.704, normalized surface distance (NSD) values of 0.735, sensitivity
values of 0.682, F1-score values of 0.707, accuracy values of 0.994, and MCC values of 0.716.
Figure 7. The percentages of segmentation models applied for COVID-19.

Sensors 2023, 23, 527 21 of 35
Using a chest CT image, Oulefki et al. [141] created an algorithm for the segmentation
and quantification of COVID-19 lung infection. This approach combines linear and loga-
rithmic stitching with multilayer, parametric thresholding. The author proposed a superior
method to U-Net, Attention-UNet, Gated-UNet, Dense-UNet, U-Net++, Inf-Net GraphCut,
Watershed, and medical image segmentation (MIS) designs.
For COVID-19 diagnosis, Gao et al. [142] developed dual-branch combination net-
work (DCN) classification and lesion segmentation algorithms. For validation purposes,
internal and external datasets were utilized. The external dataset includes CT images
from 1795 people. The concept is comprised of numerous components, including U-net,
ResNet-50, and fully linked three-layer networks. ResNet-34, ResNet50, ResNet101, VGG-
16, and DenseNet-121 were the conventional deep learning models used in the performance
comparison. Their U-net and ResNet-50 hybrid model surpassed the competition.
U-Net-based models were the major driven method for image segmentation. Figure 7 reveals
U-Net based model contributed 55.4% in this domain. However, DeepLabv3/DeepLabv3+, CNN,
FCN-8, SegNet, supervised InfNet and other models contributed rest of the proportion.
Table 4 presents overview of image segmentation applied methods for COVID-19.
4.4. Few/One-Shot Learning

Unlike humans, machine learning and deep learning models require a huge number
of instances in order to tackle novel problems. The shot learning technique can learn items
from one or a small number of samples. Detection of COVID-19 and medical image analysis
are the primary applications in this arena.
Aradhya et al. [143] developed a cluster-based one-shot learning technique for recog-
nizing COVID-19 in chest X-ray images. A generalized regression neural network (GRNN)
and probabilistic neural network (PNN) were used to form the model (PNN). For their
investigation, the authors utilized 306 images divided into four classes. For model assess-
ment, they offered five clusters based on class and image sample variation. The model’s
detection accuracy ranged from 61.84 percent to one hundred percent.
Using Siamese networks, Jadon et al. [144] investigated a few-shot learning approach
for detecting COVID-19. They utilized two datasets, one from various Asian universities
and the other from the University of Montreal. The Siamese networks have identical char-
acteristics and weight distributions. The Siamese networks attained a detection accuracy of
94.6 percent.
Jiang et al. [145] reported a COVID-19 CT diagnosis approach based on supervised
domain adaptation. The authors’ network architecture was Siamese. The suggested method
consists of three branches: the source branch, the target branch, and the prediction branch.
They used five cases with shot numbers 1, 3, 5, 7, and 9 for assessment reasons. The model’s
accuracy was 80.40 percent, and its F1 score was 79.98 percent.
The few-shot-learning-based double-view matching network that identifies COVID-19 in
X-ray images was built by Szűcs et al. [146]. The network for matching is comprised of metric
learning and the lazy learner. The authors utilized 680 data from 15 classes for experimentation
purposes. Their custom model outperformed both the matching network and the KNN.
Aradhya et al. [147] suggested one-shot learning for chest X-ray image categorization.
One-shot learning is comprised of probabilistic neural networks and entropy used to
describe the input image’s texture. Note that 300 and 24,314 images were examined for
training and testing, respectively. The model scored 96.4 percent precision, 96.6 percent
recall, 96.0 percent F-measure, and 96.4 percent accuracy.
Chen et al. [148] investigated the categorization of COVID-19 utilizing chest CT scans
based on a few training sample shots. For random cropping and random cropping with color
distortions, the authors utilized a stochastic data augmentation approach. The model includes
the encoder, projection head, and contrastive loss function, among other components. They
compared its performance to those of conventional deep learning models, such as ResNet-50
and DenseNet-121. The suggested model was 86.8 percent accurate.
Sensors 2023, 23, 527 22 of 35
Shorfuzzaman et al. [149] introduced automated COVID-19 case detection. The model is
composed of metric-based methods, such as the Siamese network. Nearest neighbor approaches
and kernel density estimates make up the components of metric-based methods. Utilizing
a convolutional neural network, the characteristics of two images were extracted, and their
similarity determined. Thirty and six-hundred and forty-eight CXR images were considered
for training and testing, respectively. The performance of the suggested Siamese network
model is superior to that of existing CNN models, including InceptionV3, Xception, Inception,
ResNetV2, and VGG-16.
Sensors 2023, 23, 527 23 of 35
Table 5. Meta-analysis.
Objective Reference Number of Studies Main Reason for Implementation Technical Issue Faced
1. COVID-19 classification and detection.

2. Point-of-care COVID-19 diagnostic.
3. Infected tissues identification.
4. Infection measurement.
5. Abnormalities detection. 1. Overfitting underfitting.
[32,43,47,55,57,59,60,62,63,65,72,73,75– 6. MERS, influenza, pneumonia classi- 2. Hyper-parameter tuning issues.
Medical Image Analysis 77,80,84–87,89,90,92–96,98–114,118– 76 fication. 3. Physical resource limitation.
141,143–150] 7. Lung disease identification. 4. Imbalanced dataset.
8. Infected or not infected classification. 5. Synthetically generated images.
9. Infection map generation.
10. Intelligent healthcare systems.
11. Localization of the affected area.
12. Control spread of COVID-19 infection.
1. COVID-19 detection.
2. COVID-19 case forecasting.
3. COVID-19 infection prediction.
4. Patients mortality risk prediction. 1. Data inconsistencies and noise.
5. Mechanical ventilation management. 2. Insufficient medical information.
Clinical and socio
[20,22–25,27–31,35,36,40,42,44,46,54] 17 6. Risk factor analysis. 3. Lack of external data.
demographic data analysis
7. Patient intensive care prediction. 4. Imbalanced dataset.
5. Bias in distribution.
8. Early risk identification.
9. Clinical prognosis evaluation.
10. Patient severity and intubation
prediction,
Sensors 2023, 23, 527 24 of 35
Table 5. Cont.
Objective Reference Number of Studies Main Reason for Implementation Technical Issue Faced
1. Overfitting.
1. COVID-19 detection. 2. Low performance.
Sound analysis [37,38,51,115–117] 6 2. Cough classification. 3. Lack of evaluation.
3. Symptoms classification. 4. Synthetically generated.
4. Bronchitis, pertussis identification. 5. Limited physical resources.
6. Imbalanced dataset.
1. COVID-19 disease-related genes detection.

Genetic analysis [26,48,58,61] 4 2. Identification of the critical cytokines. 1. Synthetically generated data.
3. Extract associations for coronavirus infec- 2. Imbalanced dataset.
tious diseases.
1. Face mask, face shield, glasses, gloves de- 1. Lack of proper evaluation and perfor-
Protective equipment observation [66–70,79,82,88] 8 tection. mance.
2. Facial recognition. 2. Synthetically generated images.
3. Measurement of body temperature. 3. Overfitting and underfitting.
1. Detection of social distancing violations.

2. Infected person detection.
3. Maintaining a safe distance.
Social control [71,74,78,81,83] 5 1. Synthetically generated images.
4. Public place monitoring.
2. Low speed for data processing.
5. Live object count.
6. Infection risk assessment.
7. Reduction of the spread of the coronavirus.
Sensors 2023, 23, 527 25 of 35
Table 6. Current challenges and future research directions.
No. Challenges Current Limitation Future Research Directions

Developing metric learning,
meta-learning, plug-and-play modules,
Most deep learning models require
1 Physical resource and computing time optimization, and probability-based
more data and training time.
methods to overcome training time and
physical resources challenges.
Applying bias mitigation methods
including optimized preprocessing, fair
Many models are trained or tested by data adaptation, meta-algorithm for
2 Bias the unrepresentative reality or biased fair classification, adversarial
data. debiasing, rich subgroup fairness,
exponentiated gradient reduction, grid
search reduction, etc.
Design sophisticated machine learning
approach combination of low latency,
The embedded machine learning reduced power consumption,
3 Embedded machine learning
approach has still absent. improved environmental performance,
network bandwidth efficiency, and
strong privacy.
Focusing on protein-coding, mRNA
Requires to identify the most relevant sequence design, molecule generation,
4 Drugs and vaccine development biotargets and large-scale training developing general vaccine prototypes,
datasets. and predicting the response of the
immune system.
Implementing segmentation and shot
5 Limited uses of ultrasound data A few studies used ultrasound images. learn methods through the ultrasound
image for the specific task.
Figure 8. The percentage of shot learning models used for COVID-19.
Karnes et al. [150] created a point-of-care COVID-19 ultrasound imaging diagnostic

system. Few-shot learning was applied to create encoded disease state models. A deep neu-
ral network (DNN) was utilized for feature extraction. Calculated Mahalanobis distances
were used for image categorization.
In shot learning, Figure 8 shows siamese network contributed 37.5%, matching net-
work 12.5%, probability based model and others contributed 25%. Table 5 presents the
Sensors 2023, 23, 527 26 of 35
meta-analysis for specific domains and Table 6 provides current challenges, limitations as
well as future research directions.
5. Discussion
The aforementioned sections where we systematically reviewed papers and their
contribution in the field of AI with respect to COVID unequivocally provide us with
futuristic insights related to upcoming endemic and pandemics. It is evident that AI
has been widely used to tackle the COVID pandemic, and the insights generated from
this incident possesses wider implications for forthcoming disasters in health domain.
Before we discuss the policy implications for future pandemics or endemics in general, it is
important to understand the past and the present.
According to research conducted by [2], a number of pandemics and epidemics have
emerged around the globe since 541, when the Justinian disease became the first pandemic
on Earth. With a mortality toll of 100 million in the then-powerful Roman Empire, the
epidemic contributed significantly to the final weakening and consequent collapse of the
Roman and Byzantine empires, as shown by [151]. Until the late 19th century, the second
pandemic, often known as the black death, killed 200 million people and is still regarded
as one of the deadliest plagues in recorded history [152]. According to [2] the Black Death
(1347–1351) wiped out at least 30 percent of Europe’s population and was followed by
waves such as the plague of Milan (1630), the great epidemic of London (1665–1665), and
the plague of Marseille (1720–1722). Faruque et al. [153] also highlight that the pre-digital
and post-industrial revolution eras have seen a number of influenza pandemics, among
which the cholera pandemic in the 19th century and the Spanish flu (1918–1919) were
notable for their nature, scope, and variability, and contributed significantly to the number
of deaths in the industrial revolution period. Over the course of nine months, three different
waves of the 1918–1919 influenza pandemic were seen to spread. In spring and summer
in 1918, the first wave produced significant morbidity and low fatality. Nevertheless, the
second and third waves had high death rates. The Spanish flu infected around 500 million
individuals globally and was linked to 50 million deaths [154].
The reason the history is important is that we can understand how some deaths could be
curbed if history lessons and insights from the past are efficiently implemented in the future.
One of the future implications of research is general-flu type disease detection. The flu
season usually occurs in the fall and winter. It can be modeled for flu forecasting and flu
detection methodologies by utilizing machine learning, deep learning, and AI in general.
AI can also be utilized for modeling cholera, diarrhea, and several water-borne diseases as
well. Our implication shows that, since AI works on the basis of data which may be audio,
video, text, tabular, or image-based (etc.), all of these data which are specific to a disease can
be modeled for forecasting, prediction, and generating policy implications and guidelines
in the public health domain. The source of water may be modeled to understand a good
or bad source of water to limit the water-borne endemics. Similarly, in Asian countries,
particularly Bangladesh and Afghanistan, people are severely prone to mosquito-related
diseases and deaths due to unsanitary conditions that they are exposed to. The images of
mosquitoes can be processed to understand the genera of the mosquitoes, and there can
be models for specific regions to realize their origins and breeding processes. The insights
generated from this AI -ased model can be used by policymakers and stakeholders to limit
the growth of such insects which eventually would curb the dengue, culex, chikungunya,
or AIDS endemic that are prevailing in South Asian countries.
Monkeypox disease can also be considered endemic in general due to its severity.
It came after COVID-19. AI can also play a substantial role in curbing this disease by
modeling the symptoms, causes, and effects of the disease, which was previously done
for COVID-19. Several time series forecasting methods, image processing, and video
processing-based detection methods may be employed to limit the growth of monkeypox.
As we have seen in COVID-19-related research, AI was utilized in developing intelligent
applications that provide information related to treatment, facilities, and procedural guidance
Sensors 2023, 23, 527 27 of 35
for things such as quarantine, vaccine information, and so forth. There are AI applications that
also detect whether a person is wearing a mask or not. Accumulating all these developments
from the COVID-19 era where AI contributed directly can be mapped to future endemics
of the COVID-19 genre, such as Middle Eastern Respiratory Syndrome (MERS) and Severe
Acute Respiratory Syndrome (SARS), to direct policy guidelines and provide insights for future
generations of stakeholders to lessen upcoming pandemics and endemics.
6. Conclusions
The COVID-19 pandemic’s vulnerability might trigger a serious worldwide calamity in
the future. It has received great attention among certain scholars, corporate industries, and
government organizations across the globe, since the epidemic has impacted a significant
section of the world’s population. In this study, a comprehensive review of artificial
intelligence models and various analysis has been demonstrated. Problem definition,
reason for implementation, data source, models, and performance were examined for
supervised, unsupervised, and few-shot learning. A detailed comparison of COVID-19
prediction models and various object detection methods, including SSD, YOLO, R-CNN,
and other models, along with their accuracy and sensitivity performances, was shown.
The existing transfer learning techniques for different architectures, including ResNet,
DenseNet, VGG, Inception, AlexNet, and CNN, were the main applications for COVID-19
that have been studied and discussed in this work. Moreover, this work also showed various
image segmentation techniques and models, and their sensitivity performances. However,
this work also discussed the limitations of physical resources; the lack of proper evaluation;
and the limitation of some technical issues, including data inconsistencies and noise,
bias in distribution, synthetically generated data, and insufficient medical information.
In the future, it is advised to carefully investigate artificial intelligence for COVID-19
prediction methodologies that have yet to be fully realized, and consider of their benefits
and limitations. Several unique strategies against COVID-19 have attracted a significant
deal of interest, even though they are still plagued by difficulties of complexity. To mitigate
the complexity of the classical approaches against COVID-19, we should concentrate
on the fairness of frameworks and large-scale datasets for training; and identifying the
most relevant biotargets and sophisticated machine learning approaches that combine low
latency, reduced power consumption, and strong privacy.
Author Contributions: Conceptualization, M.M.H. and M.U.I.; methodology, M.M.H.; validation,

M.U.I. and J.U.; formal analysis, M.U.I., W.-K.F. and J.U.; investigation, J.U. and M.J.S.; resources,
M.M.H.; writing—original draft preparation, M.M.H.; writing—review and editing, M.U.I. and J.U.;
visualization, M.M.H.; supervision, J.U., W.-K.F. and M.J.S. All authors have read and agreed to the
published version of the manuscript.
Funding: This research received no external funding.
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement: Not applicable.
Acknowledgments: The authors gratefully acknowledge the support of the Asian University of Bangladesh.
Conflicts of Interest: The authors declare no conflict of interest.
Abbreviations
The following abbreviations are used in this manuscript:
AI Artificial intelligence
DL Deep learning
ML Machine learning
KNN K-nearest neighbors
Sensors 2023, 23, 527 28 of 35
SVR Support vector regression

MLP Multi-layer perceptron
PMLP Polynomial multi-layer perceptron
PASC Post-acute sequelae of COVID-19
ANN Artificial neural networks
RT-PCR Reverse transcription polymerase chain reaction
MAPE Mean absolute percentage error
ARDS Acute respiratory distress syndrome
DNN Deep neural network
LSVC Left superior vena cava
CT Computerized tomography
SARS Severe acute respiratory syndrome
MERS Middle east respiratory syndrome
t-SNE T-distributed stochastic neighbor embedding
PM Particulate matter
DBSCAN Density-based spatial clustering of applications with noise
SPGC Signal processing grand challenge
VGG Visual geometry group
RNN Recurrent neural network
PCAUFE Principal-component-analysis-based unsupervised feature extraction
RNA Ribonucleic acid
SOFM Self-organizing feature maps
U-Net U-shaped convolutional neural network
GAN Generative adversarial network
YOLO You only look once
IAVP Influenza-A viral pneumonia
X-ray X-radiation
CAD Computer-aided design
CCTV Closed-circuit television
SAM Spatial attention module
SPP Spatial pyramid pooling
PAN Path aggregation network
MS COCO Microsoft Common Objects in Context
R-CNNs Region-based convolutional neural networks
ROI Region of interest
CRM Class-selective relevance mapping
SSD Single shot multibox detector
VOC Visual object classes
mAP Mean average precision
CelebA CelebFaces attributes dataset
Wider Web image dataset for event recognition
NN Neural networks
N-CLAHE Normalization function and the contrast limited adaptive histogram equalization
BPSO Binary particle swarm optimization
BGWO Binary gray wolf optimization
ESC Environmental sound classification
DLA Deep layer aggregation
3D Three dimensions
NSD Normalised surface distance
DSC Dice similarity coefficient
PPV Positive predictive value
IoU Intersection over union
CXR Chest X-ray
GRNN Generalized regression neural network
PNN Probabilistic neural network
Sensors 2023, 23, 527 29 of 35
References
1. Zu, Z.Y.; Jiang, M.D.; Xu, P.P.; Chen, W.; Ni, Q.Q.; Lu, G.M.; Zhang, L.J. Coronavirus disease 2019 (COVID-19): A perspective
from China. Radiology 2020, 296, E15–E25. [CrossRef] [PubMed]
2. Cunha, C.B.; Cunha, B.A. Great plagues of the past and remaining questions. Paleomicrobiology 2008, 1–20. [CrossRef]
3. Menachery, V.D.; Yount, B.L.; Debbink, K.; Agnihothram, S.; Gralinski, L.E.; Plante, J.A.; Graham, R.L.; Scobey, T.; Ge, X.Y.;
Donaldson, E.F.; et al. A SARS-like cluster of circulating bat coronaviruses shows potential for human emergence. Nat. Med.
2015, 21, 1508–1513. [CrossRef] [PubMed]
4. Sun, H.; Xiao, Y.; Liu, J.; Wang, D.; Li, F.; Wang, C.; Li, C.; Zhu, J.; Song, J.; Sun, H.; et al. Prevalent Eurasian avian-like H1N1 swine
influenza virus with 2009 pandemic viral genes facilitating human infection. Proc. Natl. Acad. Sci. USA 2020, 117, 17204–17210.
[CrossRef] [PubMed]
5. Giacca, M.; Shah, A.M. The pathological maelstrom of COVID-19 and cardiovascular disease. Nat. Cardiovasc. Res. 2022,
1, 200–210. [CrossRef]
6. He, Q.; Zheng, H.; Ma, X.; Wang, L.; Kong, H.; Zhu, Z. Artificial intelligence application in a renewable energy-driven desalination
system: A critical review. Energy AI 2022, 7, 100123. [CrossRef]
7. Yang, Y.; Yang, Y. Artificial Intelligence Signal Processing Technology in Civil Aviation Electronic Information Engineering.
In Proceedings of the International Conference on Multi-Modal Information Analytics, Huhehaote, China, 22–23 April 2022;
pp. 587–593.
8. Hasan, M.M.; Islam, M.U.; Sadeq, M.J. Towards the Technological Adaptation of Advanced Farming through Artificial Intelligence,
the Internet of Things, and Robotics: A Comprehensive Overview. In Artificial Intelligence and Smart Agriculture Technology;
Auerbach Publications: Boca Raton, FL, USA, 2022; pp. 21–42.
9. van de Sande, D.; Van Genderen, M.E.; Smit, J.M.; Huiskens, J.; Visser, J.J.; Veen, R.E.; van Unen, E.; Hilgers, O.; Gommers, D.; van
Bommel, J. Developing, implementing and governing artificial intelligence in medicine: A step-by-step approach to prevent an
artificial intelligence winter. BMJ Health Care Inform. 2022, 29, e100495. [CrossRef]
10. Bazoukis, G.; Hall, J.; Loscalzo, J.; Antman, E.M.; Fuster, V.; Armoundas, A.A. The inclusion of augmented intelligence in
medicine: A framework for successful implementation. Cell Rep. Med. 2022, 3, 100485. [CrossRef]
11. Zhang, X.; Wang, R.; Sharma, A.; Deverajan, G.G. Artificial intelligence in cognitive psychology—Influence of literature based on
artificial intelligence on children’s mental disorders. Aggress. Violent Behav. 2021, 101590. [CrossRef]
12. Napolitano, F.; Xu, X.; Gao, X. Impact of computational approaches in the fight against COVID-19: An AI guided review of 17,000
studies. Brief. Bioinform. 2022, 23, bbab456. [CrossRef]
13. Bayes-Genis, A.; Iborra-Egea, O.; Spitaleri, G.; Domingo, M.; Revuelta-López, E.; Codina, P.; Cediel, G.; Santiago-Vacas, E.;
Cserkóová, A.; Pascual-Figal, D.; et al. Decoding empagliflozin’s molecular mechanism of action in heart failure with preserved
ejection fraction using artificial intelligence. Sci. Rep. 2021, 11, 12025. [CrossRef]
14. Zhao, W. Inspired, but not mimicking: A conversation between artificial intelligence and human intelligence. Natl. Sci. Rev. 2022,
9, nwac068. [CrossRef]
15. Alballa, N.; Al-Turaiki, I. Machine learning approaches in COVID-19 diagnosis, mortality, and severity risk prediction: A review.
Inform. Med. Unlocked 2021, 24, 100564. [CrossRef]
16. El-Rashidy, N.; Abdelrazik, S.; Abuhmed, T.; Amer, E.; Ali, F.; Hu, J.W.; El-Sappagh, S. Comprehensive survey of using machine
learning in the COVID-19 pandemic. Diagnostics 2021, 11, 1155. [CrossRef]
17. Alyasseri, Z.A.A.; Al-Betar, M.A.; Doush, I.A.; Awadallah, M.A.; Abasi, A.K.; Makhadmeh, S.N.; Alomari, O.A.; Abdulkareem,
K.H.; Adam, A.; Damasevicius, R.; et al. Review on COVID-19 diagnosis models based on machine learning and deep learning
approaches. Expert Syst. 2022, 39, e12759. [CrossRef]
18. Kwekha-Rashid, A.S.; Abduljabbar, H.N.; Alhayani, B. Coronavirus disease (COVID-19) cases analysis using machine-learning
applications. In Applied Nanoscience; Springer: Berlin/Heidelberg, Germany, 2021; pp. 1–13.
19. Bhattacharya, S.; Maddikunta, P.K.R.; Pham, Q.V.; Gadekallu, T.R.; Chowdhary, C.L.; Alazab, M.; Piran, M.J. Deep learning and
medical image processing for coronavirus (COVID-19) pandemic: A survey. Sustain. Cities Soc. 2021, 65, 102589. [CrossRef]
20. Roberts, M.; Driggs, D.; Thorpe, M.; Gilbey, J.; Yeung, M.; Ursprung, S.; Aviles-Rivero, A.I.; Etmann, C.; McCague, C.; Beer, L.;
et al. Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest
radiographs and CT scans. Nat. Mach. Intell. 2021, 3, 199–217. [CrossRef]
21. Selçuk, A.A. A guide for systematic reviews: PRISMA. Turk. Archiv. Otorhinolaryngol. 2019, 57, 57. [CrossRef]
22. Cabitza, F.; Campagner, A.; Ferrari, D.; Di Resta, C.; Ceriotti, D.; Sabetta, E.; Colombini, A.; De Vecchi, E.; Banfi, G.; Locatelli, M.;
et al. Development, evaluation, and validation of machine learning models for COVID-19 detection based on routine blood tests.
Clin. Chem. Lab. Med. CCLM 2021, 59, 421–431. [CrossRef]
23. Satu, M.; Howlader, K.C.; Mahmud, M.; Kaiser, M.S.; Shariful Islam, S.M.; Quinn, J.M.; Alyami, S.A.; Moni, M.A. Short-term
prediction of COVID-19 cases using machine learning models. Appl. Sci. 2021, 11, 4266. [CrossRef]
24. Arpaci, I.; Huang, S.; Al-Emran, M.; Al-Kabi, M.N.; Peng, M. Predicting the COVID-19 infection with fourteen clinical features
using machine learning classification algorithms. Multimed. Tools Appl. 2021, 80, 11943–11957. [CrossRef] [PubMed]
25. Chowdhury, M.E.; Rahman, T.; Khandakar, A.; Al-Madeed, S.; Zughaier, S.M.; Hassen, H.; Islam, M.T. An early warning tool for
predicting mortality risk of COVID-19 patients using machine learning. In Cognitive Computation; Springer: Berlin/Heidelberg,
Germany, 2021; pp. 1–16.
Sensors 2023, 23, 527 30 of 35
26. Patterson, B.K.; Guevara-Coto, J.; Yogendra, R.; Francisco, E.B.; Long, E.; Pise, A.; Rodrigues, H.; Parikh, P.; Mora, J.; Mora-
Rodríguez, R.A. Immune-based prediction of COVID-19 severity and chronicity decoded using machine learning. Front. Immunol.
2021, 12, 2520. [CrossRef] [PubMed]
27. Karthikeyan, A.; Garg, A.; Vinod, P.; Priyakumar, U.D. Machine learning based clinical decision support system for early
COVID-19 mortality prediction. Front. Public Health 2021, 9, 626697. [CrossRef] [PubMed]
28. Marcos, M.; Belhassen-García, M.; Sánchez-Puente, A.; Sampedro-Gomez, J.; Azibeiro, R.; Dorado-Díaz, P.I.; Marcano-Millán, E.;
García-Vidal, C.; Moreiro-Barroso, M.T.; Cubino-Bóveda, N.; et al. Development of a severity of disease score and classification
model by machine learning for hospitalized COVID-19 patients. PLoS ONE 2021, 16, e0240200. [CrossRef] [PubMed]
29. Mahdavi, M.; Choubdar, H.; Zabeh, E.; Rieder, M.; Safavi-Naeini, S.; Jobbagy, Z.; Ghorbani, A.; Abedini, A.; Kiani, A.; Khan-
larzadeh, V.; et al. A machine learning based exploration of COVID-19 mortality risk. PLoS ONE 2021, 16, e0252384. [CrossRef]
30. Li, S.; Lin, Y.; Zhu, T.; Fan, M.; Xu, S.; Qiu, W.; Chen, C.; Li, L.; Wang, Y.; Yan, J.; et al. Development and external evaluation of
predictions models for mortality of COVID-19 patients using machine learning method. In Neural Computing and Applications;
Springer: Berlin/Heidelberg, Germany, 2021; pp. 1–10.
31. Podder, P.; Bharati, S.; Mondal, M.R.H.; Kose, U. Application of Machine Learning for the Diagnosis of COVID-19. In Data Science
for COVID-19; Elsevier: Amsterdam, The Netherlands, 2021; pp. 175–194.
32. Chandra, T.B.; Verma, K.; Singh, B.K.; Jain, D.; Netam, S.S. Coronavirus disease (COVID-19) detection in chest X-ray images
using majority voting based classifier ensemble. Expert Syst. Appl. 2021, 165, 113909. [CrossRef]
33. Ballı, S. Data analysis of COVID-19 pandemic and short-term cumulative case forecasting using machine learning time series
methods. Chaos Solitons Fractals 2021, 142, 110512. [CrossRef]
34. Li, M.; Zhang, Z.; Cao, W.; Liu, Y.; Du, B.; Chen, C.; Liu, Q.; Uddin, M.N.; Jiang, S.; Chen, C.; et al. Identifying novel factors
associated with COVID-19 transmission and fatality using the machine learning approach. Sci. Total Environ. 2021, 764, 142810.
[CrossRef]
35. Kang, J.; Chen, T.; Luo, H.; Luo, Y.; Du, G.; Jiming-Yang, M. Machine learning predictive model for severe COVID-19. Infect.
Genet. Evol. 2021, 90, 104737. [CrossRef]
36. Kocadagli, O.; Baygul, A.; Gokmen, N.; Incir, S.; Aktan, C. Clinical prognosis evaluation of COVID-19 patients: An interpretable
hybrid machine learning approach. Curr. Res. Transl. Med. 2022, 70, 103319. [CrossRef]
37. SM, U.S.; Ganesan, R.; Katiravan, J.; Ramakrishnan, M. Mobile application based speech and voice analysis for COVID-19
detection using computational audit techniques. Int. J. Pervasive Comput. Commun. 2020, 18, 508–517. [CrossRef]
38. Gökcen, A.; Karadağ, B.; Riva, C.; Boyacı, A. Artificial intelligence–Based COVID-19 detection using cough records. Electrica 2021,
21, 203–208. [CrossRef]
39. Chintalapudi, N.; Battineni, G.; Amenta, F. Sentimental analysis of COVID-19 tweets using deep learning models. Infect. Disease
Rep. 2021, 13, 329–339. [CrossRef]
40. Muhammad, L.; Algehyne, E.A.; Usman, S.S.; Ahmad, A.; Chakraborty, C.; Mohammed, I.A. Supervised machine learning
models for prediction of COVID-19 infection using epidemiology dataset. SN Comput. Sci. 2021, 2, 1–13. [CrossRef]
41. Majhi, R.; Thangeda, R.; Sugasi, R.P.; Kumar, N. Analysis and prediction of COVID-19 trajectory: A machine learning approach. J.
Public Aff. 2021, 21, e2537. [CrossRef]
42. Booth, A.L.; Abels, E.; McCaffrey, P. Development of a prognostic model for mortality in COVID-19 infection using machine
learning. Mod. Pathol. 2021, 34, 522–531. [CrossRef]
43. Afshar, P.; Heidarian, S.; Enshaei, N.; Naderkhani, F.; Rafiee, M.J.; Oikonomou, A.; Fard, F.B.; Samimi, K.; Plataniotis, K.N.;
Mohammadi, A. COVID-CT-MD, COVID-19 computed tomography scan dataset applicable in machine learning and deep
learning. Sci. Data 2021, 8, 1–8. [CrossRef]
44. Xu, W.; Sun, N.N.; Gao, H.N.; Chen, Z.Y.; Yang, Y.; Ju, B.; Tang, L.L. Risk factors analysis of COVID-19 patients with ARDS and
prediction based on machine learning. Sci. Rep. 2021, 11, 1–12. [CrossRef]
45. Patel, D.; Kher, V.; Desai, B.; Lei, X.; Cen, S.; Nanda, N.; Gholamrezanezhad, A.; Duddalwar, V.; Varghese, B.; Oberai, A.A.
Machine learning based predictors for COVID-19 disease severity. Sci. Rep. 2021, 11, 1–7. [CrossRef]
46. Zoabi, Y.; Deri-Rozov, S.; Shomron, N. Machine learning-based prediction of COVID-19 diagnosis based on symptoms. NPJ Digit.
Med. 2021, 4, 1–5. [CrossRef]
47. Singh, A.K.; Kumar, A.; Mahmud, M.; Kaiser, M.S.; Kishore, A. COVID-19 infection detection from chest X-ray images using
hybrid social group optimization and support vector classifier. In Cognitive Computation; Springer: Berlin/Heidelberg, Germany,
2021; pp. 1–13.
48. Hasan, M.M.; Murtaz, S.B.; Islam, M.U.; Sadeq, M.J.; Uddin, J. Robust and efficient COVID-19 detection techniques: A machine
learning approach. PLoS ONE 2022, 17, e0274538. [CrossRef] [PubMed]
49. Gautam, Y. Transfer Learning for COVID-19 cases and deaths forecast using LSTM network. ISA Trans. 2022, 124, 41–56.
[CrossRef] [PubMed]
50. Arbane, M.; Benlamri, R.; Brik, Y.; Alahmar, A.D. Social media-based COVID-19 sentiment classification model using Bi-LSTM.
Expert Syst. Appl. 2023, 212, 118710. [CrossRef] [PubMed]
51. Pahar, M.; Klopper, M.; Warren, R.; Niesler, T. COVID-19 cough classification using machine learning and global smartphone
recordings. Comput. Biol. Med. 2021, 135, 104572. [CrossRef] [PubMed]
Sensors 2023, 23, 527 31 of 35
52. Boussen, S.; Cordier, P.Y.; Malet, A.; Simeone, P.; Cataldi, S.; Vaisse, C.; Roche, X.; Castelli, A.; Assal, M.; Pepin, G.; et al. Triage and
monitoring of COVID-19 patients in intensive care using unsupervised machine learning. Comput. Biol. Med. 2022, 142, 105192.
[CrossRef]
53. Zhao, Y.; Wang, L.; Huang, T.; Tao, S.; Liu, J.; Gao, H.; Luo, J.; Huang, Y.; Liu, X.; Chen, K.; et al. Unsupervised PM2.5 anomalies
in China induced by the COVID-19 epidemic. Sci. Total Environ. 2021, 795, 148807. [CrossRef]
54. Lai, Y.; Charpignon, M.L.; Ebner, D.K.; Celi, L.A. Unsupervised learning for county-level typological classification for COVID-19
research. Intell.-Based Med. 2020, 1, 100002. [CrossRef]
55. Chen, H.; Jiang, Y.; Loew, M.; Ko, H. Unsupervised domain adaptation based COVID-19 CT infection segmentation network.
Appl. Intell. 2022, 52, 6340–6353. [CrossRef]
56. Kurniawan, R.; Abdullah, S.N.H.S.; Lestari, F.; Nazri, M.Z.A.; Mujahidin, A.; Adnan, N. Clustering and correlation methods for
predicting coronavirus COVID-19 risk analysis in pandemic countries. In Proceedings of the 2020 8th International Conference
on Cyber and IT Service Management (CITSM), Pangkal Pinang, Indonesia, 23–24 October 2020; pp. 1–5.
57. Zheng, W.; Yan, L.; Gou, C.; Zhang, Z.C.; Zhang, J.J.; Hu, M.; Wang, F.Y. Learning to learn by yourself: Unsupervised meta-
learning with self-knowledge distillation for COVID-19 diagnosis from pneumonia cases. Int. J. Intell. Syst. 2021, 36, 4033–4064.
[CrossRef]
58. Oniani, D.; Jiang, G.; Liu, H.; Shen, F. Constructing co-occurrence network embeddings to assist association extraction for
COVID-19 and other coronavirus infectious diseases. J. Am. Med. Inform. Assoc. 2020, 27, 1259–1267. [CrossRef]
59. Ewen, N.; Khan, N. Online unsupervised Learning for domain shift in COVID-19 CT scan datasets. In Proceedings of the 2021
IEEE International Conference on Autonomous Systems (ICAS), Montréal, QC, Canada, 11–13 August 2021; pp. 1–5.
60. Miao, R.; Dong, X.; Xie, S.L.; Liang, Y.; Lo, S.L. UMLF-COVID: An unsupervised meta-learning model specifically designed to
identify X-ray images of COVID-19 patients. BMC Med. Imaging 2021, 21, 174. [CrossRef]
61. Fujisawa, K.; Shimo, M.; Taguchi, Y.H.; Ikematsu, S.; Miyata, R. PCA-based unsupervised feature extraction for gene expression
analysis of COVID-19 patients. Sci. Rep. 2021, 11, 17351. [CrossRef]
62. King, B.; Barve, S.; Ford, A.; Jha, R. Unsupervised clustering of COVID-19 chest X-ray images with a self-organizing feature map.
In Proceedings of the 2020 IEEE 63rd International Midwest Symposium on Circuits and Systems (MWSCAS), Springfield, MA,
USA, 9–12 August 2020; pp. 395–398.
63. Xu, R.; Cao, X.; Wang, Y.; Chen, Y.W.; Ye, X.; Lin, L.; Zhu, W.; Chen, C.; Xu, F.; Zhou, Y.; et al. Unsupervised detection of
pulmonary opacities for computer-aided diagnosis of COVID-19 on CT images. In Proceedings of the 2020 25th International
Conference on Pattern Recognition (ICPR), Milan, Italy, 10–15 January 2021; pp. 9007–9014.
64. Huang, R.Y.; Herr, D.R. Quantitative circular flow immunoassays with trained object recognition to detect antibodies to
SARS-CoV-2 membrane glycoprotein. Biochem. Biophys. Res. Commun. 2021, 565, 8–13. [CrossRef]
65. Xu, X.; Jiang, X.; Ma, C.; Du, P.; Li, X.; Lv, S.; Yu, L.; Ni, Q.; Chen, Y.; Su, J.; et al. A deep learning system to screen novel
coronavirus disease 2019 pneumonia. Engineering 2020, 6, 1122–1129. [CrossRef]
66. Bhuiyan, M.R.; Khushbu, S.A.; Islam, M.S. A deep learning based assistive system to classify COVID-19 face mask for human
safety with YOLOv3. In Proceedings of the 2020 11th International Conference on Computing, Communication and Networking
Technologies (ICCCNT), Kharagpur, India, 1–3 July 2020; pp. 1–5.
67. RS, V.; Anbarasi, L.; Rukmani, P.; Sruti, V.; Ram, G.; Shaumaya, O.; Gurudaas, N. AI-based COVID-19 Airport Preventive
Measures (AI-CAPM). Int. J. Curr. Res. Rev. Vol. 2021, 13, 115. [CrossRef]
68. Gupta, S.; Sreenivasu, S.; Chouhan, K.; Shrivastava, A.; Sahu, B.; Potdar, R.M. Novel face mask detection technique using machine
learning to control COVID’19 pandemic. Mater. Today Proc. 2021. [CrossRef]
69. Loey, M.; Manogaran, G.; Taha, M.H.N.; Khalifa, N.E.M. A hybrid deep transfer learning model with machine learning methods
for face mask detection in the era of the COVID-19 pandemic. Measurement 2021, 167, 108288. [CrossRef]
70. Talahua, J.S.; Buele, J.; Calvopiña, P.; Varela-Aldás, J. Facial recognition system for people with and without face mask in times of
the COVID-19 pandemic. Sustainability 2021, 13, 6900. [CrossRef]
71. Hou, Y.C.; Baharuddin, M.Z.; Yussof, S.; Dzulkifly, S. Social distancing detection with deep learning model. In Proceedings of the
2020 8th International Conference on Information Technology and Multimedia (ICIMU), Selangor, Malaysia, 24–25 August 2020;
pp. 334–338.
72. Al-Antari, M.A.; Hua, C.H.; Bang, J.; Lee, S. Fast deep learning computer-aided diagnosis of COVID-19 based on digital chest
X-ray images. Appl. Intell. 2021, 51, 2890–2907. [CrossRef]
73. Al-Antari, M.A.; Hua, C.H.; Bang, J.; Choi, D.J.; Kang, S.M.; Lee, S. A Rapid Deep Learning Computer-Aided Diagnosis to
Simultaneously Detect and Classify the Novel COVID-19 Pandemic. In Proceedings of the 2020 IEEE-EMBS Conference on
Biomedical Engineering and Sciences (IECBES), Langkawi, Malaysia, 1–3 March 2021; pp. 585–588.
74. Rezaei, M.; Azarmi, M. Deepsocial: Social distancing monitoring and infection risk assessment in COVID-19 pandemic. Appl. Sci.
2020, 10, 7514. [CrossRef]
75. Li, Y.; He, X. COVID-19 Detection in Chest Radiograph Based on YOLO v5. In Proceedings of the 2021 IEEE International
Conference on Computer Science, Electronic Information Engineering and Intelligent Control Technology (CEI), Fuzhou, China,
24–26 September 2021; pp. 344–347.
Sensors 2023, 23, 527 32 of 35
76. Yoshitsugu, K.; Nakamoto, Y. COVID-19 Diagnosis Using Chest X-ray Images via Classification and Object Detection. In
Proceedings of the 2021 4th Artificial Intelligence and Cloud Computing Conference, Kyoto Japan, 17–19 December 2021;
pp. 62–67.
77. Joshi, R.C.; Yadav, S.; Pathak, V.K.; Malhotra, H.S.; Khokhar, H.V.S.; Parihar, A.; Kohli, N.; Himanshu, D.; Garg, R.K.; Bhatt, M.L.B.;
et al. A deep learning-based COVID-19 automatic diagnostic framework using chest X-ray images. Biocybernet. Biomed. Eng.
2021, 41, 239–254. [CrossRef] [PubMed]
78. Shorfuzzaman, M.; Hossain, M.S.; Alhamid, M.F. Towards the sustainable development of smart cities through mass video
surveillance: A response to the COVID-19 pandemic. Sustain. Cities Soc. 2021, 64, 102582. [CrossRef] [PubMed]
79. Protik, A.A.; Rafi, A.H.; Siddique, S. Real-time Personal Protective Equipment (PPE) Detection Using YOLOv4 and TensorFlow.
In Proceedings of the 2021 IEEE Region 10 Symposium (TENSYMP), Grand Hyatt, Jeju, Republic of Korea, 23–25 August 2021;
pp. 1–6.
80. Dou, Q.; Thus, T.Y.; Jiang, M.; Liu, Q.; Vardhanabhuti, V.; Kaissis, G.; Li, Z.; Si, W.; Lee, H.H.; Yu, K.; et al. Federated deep learning
for detecting COVID-19 lung abnormalities in CT: A privacy-preserving multinational validation study. NPJ Digit. Med. 2021,
4, 1–11. [CrossRef] [PubMed]
81. Saponara, S.; Elhanashi, A.; Gagliardi, A. Implementing a real-time, AI-based, people detection and social distancing measuring
system for COVID-19. J. Real-Time Image Process. 2021, 18, 1937–1947. [CrossRef] [PubMed]
82. Melo, C.; Dixe, S.; Fonseca, J.C.; Moreira, A.H.; Borges, J. AI Based Monitoring of Different Risk Levels in COVID-19 Context.
Sensors 2021, 22, 298. [CrossRef]
83. Shin, M.; Moon, N. Indoor Distance Measurement System COPS (COVID-19 Prevention System). Sustainability 2021, 13, 4738.
[CrossRef]
84. Arunachalam, A.; Ravi, V.; Acharya, V.; Pham, T.D. Toward data-model-agnostic autonomous machine-generated data labeling and
annotation platform: COVID-19 autoannotation use case. IEEE Trans. Eng. Manag. 2021. [CrossRef]
85. Nurmaini, S.; Tondas, A.E.; Partan, R.U.; Rachmatullah, M.N.; Darmawahyuni, A.; Firdaus, F.; Tutuko, B.; Hidayat, R.; Sapitri, A.I.
Automated Detection of COVID-19 Infected Lesion on Computed Tomography Images Using Faster-RCNNs. Eng. Lett. 2020,
28, 1287–1294.
86. Rajaraman, S.; Sornapudi, S.; Alderson, P.O.; Folio, L.R.; Antani, S.K. Analyzing inter-reader variability affecting deep ensemble
learning for COVID-19 detection in chest radiographs. PLoS ONE 2020, 15, e0242301. [CrossRef]
87. Saiz, F.A.; Barandiaran, I. COVID-19 Detection in Chest X-ray Images using a Deep Learning Approach. Int. J. Interact. Multim.
Artif. Intell. 2020, 6, 1–4. [CrossRef]
88. Loey, M.; Manogaran, G.; Taha, M.H.N.; Khalifa, N.E.M. Fighting against COVID-19: A novel deep learning model based on
YOLO-v2 with ResNet-50 for medical face mask detection. Sustain. Cities Soc. 2021, 65, 102600. [CrossRef]
89. Li, W.; Chen, J.; Chen, P.; Yu, L.; Cui, X.; Li, Y.; Cheng, F.; Ouyang, W. NIA-Network: Towards improving lung CT infection
detection for COVID-19 diagnosis. Artific. Intell. Med. 2021, 117, 102082. [CrossRef]
90. de Moura, J.; Novo, J.; Ortega, M. Fully automatic deep convolutional approaches for the analysis of COVID-19 using chest X-ray
images. Appl. Soft Comput. 2022, 115, 108190. [CrossRef]
91. Abbas, A.; Abdelsamea, M.M.; Gaber, M.M. 4S-DT: Self-supervised super sample decomposition for transfer learning with
application to COVID-19 detection. IEEE Trans. Neural Netw. Learn. Syst. 2021, 32, 2798–2808. [CrossRef]
92. Rehman, A.; Naz, S.; Khan, A.; Zaib, A.; Razzak, I. Improving coronavirus (COVID-19) diagnosis using deep transfer learning.
MedRxiv 2020, 350.
93. Ahuja, S.; Panigrahi, B.K.; Dey, N.; Rajinikanth, V.; Gandhi, T.K. Deep transfer learning-based automated detection of COVID-19
from lung CT scan slices. Appl. Intell. 2021, 51, 571–585. [CrossRef]
94. Shamsi, A.; Asgharnezhad, H.; Jokandan, S.S.; Khosravi, A.; Kebria, P.M.; Nahavandi, D.; Nahavandi, S.; Srinivasan, D. An
uncertainty-aware transfer learning-based framework for COVID-19 diagnosis. IEEE Trans. Neural Netw. Learn. Syst. 2021,
32, 1408–1417. [CrossRef]
95. Bassi, P.R.; Attux, R. A deep convolutional neural network for COVID-19 detection using chest X-rays. Res. Biomed. Eng. 2022,
38, 139–148. [CrossRef]
96. Jaiswal, A.; Gianchandani, N.; Singh, D.; Kumar, V.; Kaur, M. Classification of the COVID-19 infected patients using DenseNet201
based deep transfer learning. J. Biomol. Struct. Dyn. 2021, 39, 5682–5689. [CrossRef]
97. Wang, S.H.; Nayak, D.R.; Guttery, D.S.; Zhang, X.; Zhang, Y.D. COVID-19 classification by CCSHNet with deep fusion using
transfer learning and discriminant correlation analysis. Inf. Fusion 2021, 68, 131–148. [CrossRef]
98. Pathak, Y.; Shukla, P.K.; Tiwari, A.; Stalin, S.; Singh, S. Deep transfer learning based classification model for COVID-19 disease.
Irbm 2022, 43, 87–92. [CrossRef] [PubMed]
99. Das, A.K.; Kalam, S.; Kumar, C.; Sinha, D. TLCoV-An automated COVID-19 screening model using Transfer Learning from chest
X-ray images. Chaos Solitons Fractals 2021, 144, 110713. [CrossRef] [PubMed]
100. Aslan, M.F.; Unlersen, M.F.; Sabanci, K.; Durdu, A. CNN-based transfer learning–BiLSTM network: A novel approach for
COVID-19 infection detection. Appl. Soft Comput. 2021, 98, 106912. [CrossRef] [PubMed]
101. Punn, N.S.; Agarwal, S. Automated diagnosis of COVID-19 with limited posteroanterior chest X-ray images using fine-tuned
deep neural networks. Appl. Intell. 2021, 51, 2689–2702. [CrossRef] [PubMed]
Sensors 2023, 23, 527 33 of 35
102. Loey, M.; Smarandache, F.; Khalifa, N.E.M. Within the lack of chest COVID-19 X-ray dataset: A novel detection model based on
GAN and deep transfer learning. Symmetry 2020, 12, 651. [CrossRef]
103. Zhang, R.; Guo, Z.; Sun, Y.; Lu, Q.; Xu, Z.; Yao, Z.; Duan, M.; Liu, S.; Ren, Y.; Huang, L.; et al. COVID19XrayNet: A two-step
transfer learning model for the COVID-19 detecting problem based on a limited number of chest X-ray images. Interdiscip. Sci.
Comput. Life Sci. 2020, 12, 555–565. [CrossRef]
104. Das, A.K.; Ghosh, S.; Thunder, S.; Dutta, R.; Agarwal, S.; Chakrabarti, A. Automatic COVID-19 detection from X-ray images
using ensemble learning with convolutional neural network. Pattern Anal. Appl. 2021, 24, 1111–1124. [CrossRef]
105. Luz, E.; Silva, P.; Silva, R.; Silva, L.; Guimarães, J.; Miozzo, G.; Moreira, G.; Menotti, D. Towards an effective and efficient deep
learning model for COVID-19 patterns detection in X-ray images. Res. Biomed. Eng. 2021, 1–14. [CrossRef]
106. Turkoglu, M. COVID-19 detection system using chest CT images and multiple kernels-extreme learning machine based on deep
neural network. IRBM 2021, 42, 207–214. [CrossRef]
107. Horry, M.J.; Chakraborty, S.; Paul, M.; Ulhaq, A.; Pradhan, B.; Saha, M.; Shukla, N. COVID-19 detection through transfer learning
using multimodal imaging data. IEEE Access 2020, 8, 149808–149824. [CrossRef]
108. Zhu, J.; Shen, B.; Abbasi, A.; Hoshmand-Kochi, M.; Li, H.; Duong, T.Q. Deep transfer learning artificial intelligence accurately
stages COVID-19 lung disease severity on portable chest radiographs. PLoS ONE 2020, 15, e0236621. [CrossRef]
109. Gianchandani, N.; Jaiswal, A.; Singh, D.; Kumar, V.; Kaur, M. Rapid COVID-19 diagnosis using ensemble deep transfer learning
models from chest radiographic images. J. Ambient. Intell. Hum. Comput. 2020, 1–13. [CrossRef]
110. Hira, S.; Bai, A.; Hira, S. An automatic approach based on CNN architecture to detect COVID-19 disease from chest X-ray images.
Appl. Intell. 2021, 51, 2864–2889. [CrossRef]
111. Ibrahim, A.U.; Ozsoz, M.; Serte, S.; Al-Turjman, F.; Yakoi, P.S. Pneumonia classification using deep learning from chest X-ray
images during COVID-19. In Cognitive Computation; Springer: Berlin/Heidelberg, Germany, 2021; pp. 1–13.
112. Kc, K.; Yin, Z.; Wu, M.; Wu, Z. Evaluation of deep learning-based approaches for COVID-19 classification based on chest X-ray
images. Signal Image Video Process. 2021, 15, 959–966. [CrossRef]
113. Canayaz, M. MH-COVIDNet: Diagnosis of COVID-19 using deep neural networks and meta-heuristic-based feature selection on
X-ray images. Biomed. Signal Process. Control 2021, 64, 102257. [CrossRef]
114. Das, N.N.; Kumar, N.; Kaur, M.; Kumar, V.; Singh, D. Automated deep transfer learning-based approach for detection of
COVID-19 infection in chest X-rays. Irbm 2020. [CrossRef]
115. Rodriguez, C.R.; Angeles, D.; Chafloque, R.; Kaseng, F.; Pandey, B. Deep learning audio spectrograms processing to the early
COVID-19 detection. In Proceedings of the 2020 12th International Conference on Computational Intelligence and Communication
Networks (CICN), Bhimtal, India, 25–26 September 2020; pp. 429–434.
116. Loey, M.; Mirjalili, S. COVID-19 cough sound symptoms classification from scalogram image representation using deep learning
models. Comput. Biol. Med. 2021, 139, 105020. [CrossRef]
117. Imran, A.; Posokhova, I.; Qureshi, H.N.; Masood, U.; Riaz, M.S.; Ali, K.; John, C.N.; Hussain, M.I.; Nabeel, M. AI4COVID-19: AI
enabled preliminary diagnosis for COVID-19 from cough samples via an app. Inform. Med. Unlocked 2020, 20, 100378. [CrossRef]
118. Saeedizadeh, N.; Minaee, S.; Kafieh, R.; Yazdani, S.; Sonka, M. COVID TV-Unet: Segmenting COVID-19 chest CT images using
connectivity imposed Unet. Comput. Methods Programs Biomed. Update 2021, 1, 100007. [CrossRef]
119. Ma, J.; Wang, Y.; An, X.; Ge, C.; Yu, Z.; Chen, J.; Zhu, Q.; Dong, G.; He, J.; He, Z.; et al. Toward data-efficient learning: A
benchmark for COVID-19 CT lung and infection segmentation. Med. Phys. 2021, 48, 1197–1210. [CrossRef]
120. Yazdekhasty, P.; Zindari, A.; Nabizadeh-ShahreBabak, Z.; Roshandel, R.; Khadivi, P.; Karimi, N.; Samavi, S. Bifurcated autoencoder
for segmentation of COVID-19 infected regions in CT images. In Proceedings of the International Conference on Pattern Recognition;
Springer: Cham, Switzerland, 2021; pp. 597–607.
121. Ranjbarzadeh, R.; Jafarzadeh Ghoushchi, S.; Bendechache, M.; Amirabadi, A.; Ab Rahman, M.N.; Baseri Saadi, S.; Aghamoham-
madi, A.; Kooshki Forooshani, M. Lung infection segmentation for COVID-19 pneumonia based on a cascade convolutional
network from CT images. BioMed Res. Int. 2021, 2021. [CrossRef] [PubMed]
122. Zheng, R.; Zheng, Y.; Dong-Ye, C. Improved 3D U-Net for COVID-19 Chest CT Image Segmentation. Sci. Programm. 2021,
2021, 9999368. [CrossRef]
123. Chen, C.; Zhou, K.; Zha, M.; Qu, X.; Guo, X.; Chen, H.; Wang, Z.; Xiao, R. An effective deep neural network for lung lesions
segmentation from COVID-19 CT images. IEEE Trans. Ind. Inform. 2021, 17, 6528–6538. [CrossRef]
124. Yan, Q.; Wang, B.; Gong, D.; Luo, C.; Zhao, W.; Shen, J.; Ai, J.; Shi, Q.; Zhang, Y.; Jin, S.; et al. COVID-19 chest CT image
segmentation network by multi-scale fusion and enhancement operations. IEEE Trans. Big Data 2021, 7, 13–24. [CrossRef]
125. Frid-Adar, M.; Amer, R.; Gozes, O.; Nassar, J.; Greenspan, H. COVID-19 in CXR: From detection and severity scoring to patient
disease monitoring. IEEE J. Biomed. Health Inform. 2021, 25, 1892–1903. [CrossRef]
126. Degerli, A.; Ahishali, M.; Yamac, M.; Kiranyaz, S.; Chowdhury, M.E.; Hameed, K.; Hamid, T.; Mazhar, R.; Gabbouj, M. COVID-19
infection map generation and detection from chest X-ray images. Health Inf. Sci. Syst. 2021, 9, 1–16. [CrossRef]
127. Wang, B.; Jin, S.; Yan, Q.; Xu, H.; Luo, C.; Wei, L.; Zhao, W.; Hou, X.; Ma, W.; Xu, Z.; et al. AI-assisted CT imaging analysis for
COVID-19 screening: Building and deploying a medical AI system. Appl. Soft Comput. 2021, 98, 106897. [CrossRef]
128. Zhao, S.; Li, Z.; Chen, Y.; Zhao, W.; Xie, X.; Liu, J.; Zhao, D.; Li, Y. SCOAT-Net: A novel network for segmenting COVID-19 lung
opacification from CT images. Pattern Recognit. 2021, 119, 108109. [CrossRef]
Sensors 2023, 23, 527 34 of 35
129. Zhao, X.; Zhang, P.; Song, F.; Fan, G.; Sun, Y.; Wang, Y.; Tian, Z.; Zhang, L.; Zhang, G. D2A U-Net: Automatic segmentation of
COVID-19 CT slices based on dual attention and hybrid dilated convolution. Comput. Biol. Med. 2021, 135, 104526. [CrossRef]
130. Chakraborty, S.; Mali, K. SuFMoFPA: A superpixel and meta-heuristic based fuzzy image segmentation approach to explicate
COVID-19 radiological images. Expert Syst. Appl. 2021, 167, 114142. [CrossRef]
131. Mortani Barbosa, E.J.; Georgescu, B.; Chaganti, S.; Aleman, G.B.; Cabrero, J.B.; Chabin, G.; Flohr, T.; Grenier, P.; Grbic, S.; Gupta,
N.; et al. Machine learning automatically detects COVID-19 using chest CTs in a large multicenter cohort. Eur. Radiol. 2021,
31, 8775–8785. [CrossRef]
132. Pei, H.Y.; Yang, D.; Liu, G.R.; Lu, T. MPS-net: Multi-point supervised network for ct image segmentation of COVID-19. IEEE
Access 2021, 9, 47144–47153. [CrossRef]
133. Saood, A.; Hatem, I. COVID-19 lung CT image segmentation using deep learning methods: U-Net versus SegNet. BMC Med.
Imaging 2021, 21, 19. [CrossRef]
134. Gerard, S.E.; Herrmann, J.; Xin, Y.; Martin, K.T.; Rezoagli, E.; Ippolito, D.; Bellani, G.; Cereda, M.; Guo, J.; Hoffman, E.A.; et al.
CT image segmentation for inflamed and fibrotic lungs using a multi-resolution convolutional neural network. Sci. Rep. 2021,
11, 1455. [CrossRef]
135. Teixeira, L.O.; Pereira, R.M.; Bertolini, D.; Oliveira, L.S.; Nanni, L.; Cavalcanti, G.D.; Costa, Y.M. Impact of lung segmentation on
the diagnosis and explanation of COVID-19 in chest X-ray images. Sensors 2021, 21, 7116. [CrossRef]
136. Rahman, T.; Khandakar, A.; Qiblawey, Y.; Tahir, A.; Kiranyaz, S.; Kashem, S.B.A.; Islam, M.T.; Al Maadeed, S.; Zughaier, S.M.;
Khan, M.S.; et al. Exploring the effect of image enhancement techniques on COVID-19 detection using chest X-ray images.
Comput. Biol. Med. 2021, 132, 104319. [CrossRef]
137. Fung, D.L.; Liu, Q.; Zammit, J.; Leung, C.K.S.; Hu, P. Self-supervised deep learning model for COVID-19 lung CT image
segmentation highlighting putative causal relationship among age, underlying disease and COVID-19. J. Transl. Med. 2021,
19, 1–18. [CrossRef]
138. Wu, Y.H.; Gao, S.H.; Mei, J.; Xu, J.; Fan, D.P.; Zhang, R.G.; Cheng, M.M. Jcs: An explainable COVID-19 diagnosis system by joint
classification and segmentation. IEEE Trans. Image Process. 2021, 30, 3113–3126. [CrossRef]
139. Laradji, I.; Rodriguez, P.; Manas, O.; Lensink, K.; Law, M.; Kurzman, L.; Parker, W.; Vazquez, D.; Nowrouzezahrai, D. A weakly
supervised consistency-based learning method for COVID-19 segmentation in ct images. In Proceedings of the IEEE/CVF Winter
Conference on Applications of Computer Vision, Waikoloa, HI, USA, 4–8 January 2021; pp. 2453–2462.
140. Wang, Y.; Zhang, Y.; Liu, Y.; Tian, J.; Zhong, C.; Shi, Z.; Zhang, Y.; He, Z. Does non-COVID-19 lung lesion help? investigating
transferability in COVID-19 CT image segmentation. Comput. Methods Prog. Biomed. 2021, 202, 106004. [CrossRef]
141. Oulefki, A.; Agaian, S.; Trongtirakul, T.; Laouar, A.K. Automatic COVID-19 lung infected region segmentation and measurement
using CT-scans images. Pattern Recognit. 2021, 114, 107747. [CrossRef]
142. Gao, K.; Su, J.; Jiang, Z.; Zeng, L.L.; Feng, Z.; Shen, H.; Rong, P.; Xu, X.; Qin, J.; Yang, Y.; et al. Dual-branch combination network
(DCN): Towards accurate diagnosis and lesion segmentation of COVID-19 using CT images. Med. Image Anal. 2021, 67, 101836.
[CrossRef] [PubMed]
143. Aradhya, V.; Mahmud, M.; Guru, D.; Agarwal, B.; Kaiser, M.S. One-shot cluster-based approach for the detection of COVID–19
from chest X–ray images. Cognit. Comput. 2021, 13, 873–881. [CrossRef] [PubMed]
144. Jadon, S. COVID-19 detection from scarce chest X-ray image data using few-shot deep learning approach. In Proceedings of the
Medical Imaging 2021: Imaging Informatics for Healthcare, Research, and Applications; International Society for Optics and Photonics:
Bellingham, WA, USA, 2021; Volume 11601, p. 116010X.
145. Jiang, Y.; Chen, H.; Ko, H.; Han, D.K. Few-shot learning for ct scan based COVID-19 diagnosis. In Proceedings of the
ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Virtual, 6–12 June 2021;
pp. 1045–1049.
146. Szűcs, G.; Németh, M. Double-view matching network for few-shot learning to classify COVID-19 in X-ray images. Infocommun.
J. 2021, 13, 26–34. [CrossRef]
147. Aradhya, V.M.; Mahmud, M.; Chowdhury, M.; Guru, D.; Kaiser, M.S.; Azad, S. Learning Through One Shot: A Phase by
Phase Approach for COVID-19 Chest X-ray Classification. In Proceedings of the 2020 IEEE-EMBS Conference on Biomedical
Engineering and Sciences (IECBES), Langkawi, Malaysia, 1 December 2020–3 March 2021; pp. 241–244.
148. Chen, X.; Yao, L.; Zhou, T.; Dong, J.; Zhang, Y. Momentum contrastive learning for few-shot COVID-19 diagnosis from chest CT
images. Pattern Recognit. 2021, 113, 107826. [CrossRef]
149. Shorfuzzaman, M.; Hossain, M.S. MetaCOVID: A Siamese neural network framework with contrastive loss for n-shot diagnosis
of COVID-19 patients. Pattern Recognit. 2021, 113, 107700. [CrossRef]
150. Karnes, M.; Perera, S.; Adhikari, S.; Yilmaz, A. Adaptive Few-Shot Learning PoC Ultrasound COVID-19 Diagnostic System. In
Proceedings of the 2021 IEEE Biomedical Circuits and Systems Conference (BioCAS), Virtual, 7–9 October 2021; pp. 1–6.
151. Piret, J.; Boivin, G. Pandemics throughout history. Front. Microbiol. 2021, 11, 631736. [CrossRef]
152. Zietz, B.P.; Dunkelberg, H. The history of the plague and the research on the causative agent Yersinia pestis. Int. J. Hyg. Environ.
Health 2004, 207, 165–178. [CrossRef]
Sensors 2023, 23, 527 35 of 35
153. Faruque, S.M.; Albert, M.J.; Mekalanos, J.J. Epidemiology, genetics, and ecology of toxigenic Vibrio cholerae. Microbiol. Mol. Biol.
Rev. 1998, 62, 1301–1314. [CrossRef]
154. Johnson, N.P.; Mueller, J. Updating the accounts: Global mortality of the 1918–1920 “Spanish” influenza pandemic. Bull. Hist.
Med. 2002, 76, 105–115. [CrossRef]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual
author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to
people or property resulting from any ideas, methods, instructions or products referred to in the content.
PLOS ONE
RESEARCH ARTICLE
Robust and efficient COVID-19 detection

techniques: A machine learning approach
Md. Mahadi Hasan1, Saba Binte Murtaz1, Muhammad Usama Islam2, Muhammad
Jafar Sadeq1, Jasim Uddin ID3*
1 Department of Computer Science and Engineering, Asian University of Bangladesh, Ashulia, Dhaka,
Bangladesh, 2 School of Computing and Informatics, University of Louisiana at Lafayette, Lafayette,
Louisiana, United States of America, 3 Department of Applied Computing and Engineering, Cardiff School of
Technologies, Cardiff Metropolitan University, Cardiff, Wales, United Kingdom
* juddin@cardiffmet.ac.uk
a1111111111 Abstract
a1111111111
a1111111111 The devastating impact of the Severe Acute Respiratory Syndrome-Coronavirus 2 (SARS-
a1111111111 CoV-2) pandemic almost halted the global economy and is responsible for 6 million deaths
a1111111111
with infection rates of over 524 million. With significant reservations, initially, the SARS-
CoV-2 virus was suspected to be infected by and closely related to Bats. However, over the
periods of learning and critical development of experimental evidence, it is found to have
some similarities with several gene clusters and virus proteins identified in animal-human
OPEN ACCESS
transmission. Despite this substantial evidence and learnings, there is limited exploration
Citation: Hasan M.M, Murtaz SB, Islam MU, Sadeq regarding the SARS-CoV-2 genome to putative microRNAs (miRNAs) in the virus life cycle.
MJ, Uddin J (2022) Robust and efficient COVID-19
In this context, this paper presents a detection method of SARS-CoV-2 precursor-miRNAs
detection techniques: A machine learning
approach. PLoS ONE 17(9): e0274538. https://doi. (pre-miRNAs) that helps to identify a quick detection of specific ribonucleic acid (RNAs).
org/10.1371/journal.pone.0274538 The approach employs an artificial neural network and proposes a model that estimated
Editor: Sathishkumar V E, Hanyang University, accuracy of 98.24%. The sampling technique includes a random selection of highly unbal-
REPUBLIC OF KOREA anced datasets for reducing class imbalance following the application of matriculation artifi-
Received: June 16, 2022 cial neural network that includes accuracy curve, loss curve, and confusion matrix. The
classical approach to machine learning is then compared with the model and its perfor-
Accepted: August 30, 2022
mance. The proposed approach would be beneficial in identifying the target regions of RNA
Published: September 15, 2022
and better recognising of SARS-CoV-2 genome sequence to design oligonucleotide-based
Copyright: © 2022 Hasan et al. This is an open drugs against the genetic structure of the virus.
access article distributed under the terms of the
Creative Commons Attribution License, which
permits unrestricted use, distribution, and
reproduction in any medium, provided the original
author and source are credited.
Data Availability Statement: The data is publicly

available and has been used from the following 1 Introduction
resource: https://sourceforge.net/projects/ In late 2019, few patients were affected in pneumonia with a nescient symptoms known as
sourcesinc/files/aicovid/dataset.tar.gz.
respiratory syndrome coronavirus 2 (SARS-CoV-2) and later named as coronavirus disease
Funding: The author(s) received no specific 2019 (COVID-19). There is still in debate exactly where it grown up, but Epidemiological evi-
funding for this work. dence shown that the virous spread from Wuhan, Hubei province from local sea food market.
Competing interests: The authors have declared It is also confirmed the gene sequence was identified from Bats. According to World Health
that no competing interests exist. Organization (WHO), the virus was rapidly spread in worldwide over 6 million deaths and
PLOS ONE | https://doi.org/10.1371/journal.pone.0274538 September 15, 2022 1 / 21

PLOS ONE COVID-19 detection techniques using machine learning
still growing continuously. The special attention of this virus was to spread very fast, adapt rap-
idly and affected with the infection of the major symptoms in fever, cough, muscle pain, and
diarrhea. The similar symptoms can also be seen in mice, dogs, cats, camels, pigs, chickens,
and bats [1].
SARS-Cov-2 are an encapsulated and carrying a positive sense of single-stranded RNA
genome that belongs to subfamily Coronavirdiae. However, Micro ribonucleic acid (miRNAs)
were initially identified in 1993 which controlls the timing of nematode Caenorhabditis ele-
gans (Lee, Feinbaum and Ambros, 1993). Micro ribonucleic acid are literally quite small with
an average 22 nucleotides in length available in plants, animals and some viruses including
HSV, HIV-1, Dengue, Influenza, and SARS-COV-2 involved in biological processes [2].
According to the literature [3–7], micro ribonucleic acid exploration is crucial due to
around 30 percentage of human genes are regulated by micro ribonucleic acid, influencing
diverse biological processes including development, proliferation, cell differentiation, and
metabolism across the various cell types.
Considering the various conditions to investigate how micro ribonucleic acid regulated
under various conditions to comprehend the gene expression and disease phenotypes. The
miRNAs can be produced by the most deoxyribonucleic acid (DNA) viruses, but miRNA
expression is controversial in the case of RNA viruses because of their cytoplasmic replication
and insufficient knowledge of the nuclear miRNA complex structure [8]. Therefore the exact
mechanism of viral and cellular miRNAs are not adequately realised in viral infections. How-
ever, miRNAs have recently emerged as antiviral regulators of viral genes triggered by a coro-
navirus [9]. Gene silencing, miRNAs indeed can play a crucial role controlling the expression
of transcription factors [10]. Therefore, using miRNAs to defeat COVID-19 can be
groundbreaking.
MicroRNAs are consequent from pri-miRNAs more than 1000 nt in length and pri-miR-
NAs comprises a hairpin structure that realises from 60–120 nt [11]. The structural properties
of these hairpins are characterised by pri-miRNAs thriving from the other RNA stem loop
similar to the structures establish in the nucleus. In addition, the hairpin is cut off from the
pri-miRNA to comprise the predecessor of miRNA (pre-miRNA) [12].
In order to detect miRNAs, it is important to distinguish pre-miRNAs from other hairpin-
like sequences [13]. In order to the consideration of miRNA biogenesis and small interfering
RNA design, the pre-miRNA prediction has lately become an exciting relevant area in miRNA
research [14].
According to the literature [15], SARS-CoV-2 pre-miRNA identification desires the neces-
sary equipment and real life oriented physical environment setup which resembles very expen-
sive as laborious and burdensome. Instead, Machine learning (ML) can be an alternative
approach in the way to lead in the research specifically in miRNA biology, and focusing on
biomarkers for potential diseases [16].
The major issue with using machine learning to detect pre-miRNAs is that the number of
well-known pre-miRNAs is typically few in comparison to the hundreds of thousands of can-
didate sequences in a genome, making this a high-class imbalanced classification challenge
[17]. H. sapiens genome is an example that has 1710 well-known pre-miRNAs but over 400
million hairpin-like sequences resulting in a 1:28128 imbalance [18]. ML algorithms are gener-
ally representing with balanced data sets but in a supervised classifier, imbalanced data tend to
produce a model biased towards the majority class, with low performance in the minority one
yielding false positives [19]. Many computational approaches, including homologous search,
comparative genomics, and machine learning, have been developed in recent decades to locate
pre-miRNAs and to overcome the imbalanced miRNA (positive) and non-miRNA (negative)
samples problems [20]. Specifically, some machine learning-based computational approaches

such as DIANA-microT [21], TargetScan [22], TargetScanS [23], miRanda [24], mirSVR [25],
RNA22 [26] and RNAhybird [27] have introduced a significant progress improving the perfor-
mance of ML based pre-miRNA detection.
Performance is the main research gap in SARS-CoV-2 pre-miRNA identification. Other
relevant limitations include artificial negative class. In this research, the RUNN-COV (Ran-
dom Under sampling with Neural Network for COVID detection) models are presented. The
proposed model performance was established, and their significant comparison, limitation
and major challenges are introduced along with other existing methods. It is anticipated that
this model will contribute to the fight against COVID-19 by improving its detection and subse-
quent study of the biological functions of SARS-CoV-2 pre-miRNAs, leading to effective and
robust treatments.
One of the major contributions of our research lies in data visualization through explor-
atory data analysis and substantial research to understand the high-class imbalance problem
and through investigation and experimentation finding the best technique which in our case is
random undersampling to solve this tenacious problem thus decreasing the likelihood of over-
fitting and increasing classifier performance in the process. While the main contribution lies
in fine-tuning the model performance, but associated experimentation’s of exploratory data
analysis specifically t-distributed stochastic neighbor embedding (t-SNE) were investigated
thoroughly to understand the data loss patterns as well as separation pattern between negative
and positive data points which ultimately aided us in identifying a more robust, decision
boundary that generated better model performance. All these exploratory data analysis mecha-
nisms, aided us in selecting the best parameters and algorithms, which when fed into our own
model produced a substantially superior performance outperforming several limitations dis-
cussed above. The final contribution of our research is performance comparison, where we
compared our approach and results with existing literature and their performance that would
aid the researchers in understanding the latest state of research in this field and where to go
next.
To summarize, this paper presents a random undersampling technique that deals with the
high-class imbalance problem. In addition, several techniques including correlation matrix, t-
SNE are investigated for data loss visualization, data point visualization, and identifying hid-
den patterns. The RUNN-COV model has represented with an extraordinary result which
compared to the other existing techniques and possible recommendation of their limitations.
This paper also shows a performance comparison with other relevant machine learning mod-
els. Finally, the results were systematically evaluated using nine evaluation metrics.
The rest of the paper is organized as follows. Section 2 introduces the inspiration of the
SAR-CoV-2 pre-miRNA identification. Section 3 presents a survey of the literature for using
computational approaches in the COVID-19 and relevant miRNA context. Section 4 presents
a description of the SARS-CoV-2 dataset. Section 5 presents sampling strategy, clustering
analysis, and RUNN-COV model architecture. Section 6 presents a detailed evaluation strat-
egy, performance analysis, and statistical investigation. Finally, the paper concluded in Sec-
tion 7.
2 Motivation
SARS-CoV-2 has had a tremendous impact in the world, not just in terms of health care but
also in others including agriculture and food security, economic and financial, educational,
industrial, power and energy, oil market, employment, and environmental [28]. Therefore,
select the effective ways to diagnose the infection to control the spread of COVID-19 and gen-
erate a better treatment prospects are crucial.

As mentioned above, MicroRNAs (miRNAs) are the most powerful regulators of gene
expression that play a role in practically all forms of gene regulation. Cellular miRNAs can be
applied as therapeutic options for COVID-19 [8] as well as many other viral infections, such as
Dengue [29], Influenza [30], Human Immunodeficiency Virus (HIV) [31], Herpes Simplex
Viruses [32], and Hepatitis C Virus [33]. Viruses are incapable of self-replication without the
machinery and metabolism of a host cell. Consequently, viruses employ various tactics, one of
them being the modification of host cell miRNA to their advantage [30]. A disorder in the
organism’s internal environment is generally accompanied by aberrant miRNA production or
secretion in the cells or blood, which has become the key indicator to recognize deadly diseases
like cancers [34], diabetes [35], cardiovascular diseases [36], and virus-caused diseases [37].
MicroRNAs can also interfere with the heart [38] and lung [39] disease caused by COVID-19.
Because of the discovery of this link, there may be a great benefit in targeting miRNA-interac-
tion genes to treat COVID-19. Also, nanobased miRNA vaccines can be utilized as nasal spray
or drops to activate the immune response in the respiratory tract, which is the common initial
location for SARS-CoV-2 viral entrance [8].
Unfortunately, currently there is one Food and Drug Administration (FDA) approved anti-
viral drug, Veklury (Remdesivir), for the treatment of COVID-19 under Emergency Use
Authorization (EUA) [40] along with treatments that are under research ranging from other
anti-viral drugs to plasma therapy, vaccines and antibody drugs. In the inadequacy of COVID-
19 treatments and vaccines, miRNA-based therapeutic approaches may be an intriguing
option for regulating the SARs-CoV-2 replication.
One possible avenue of attack is the design and synthesis of oligonucleotides against the
genetic structure of SARS-CoV-2 with the aim to impede its replication or to degrade its
genome [9], This is similar to the possibility of designing therapeutic oligonucleotides on the
basis of the human genome [41].
The proposed RUNN-COV model will assist to detect of SARs-CoV-2 and potentially
many other relevant RNAs as of interest. The findings demonstrate that contemporary
machine learning technologies can be used to assist in responding to public health emergencies
by helping to discover the characteristics of any viral agent and in devising novel therapeutic
approaches.
3 Related works
Being able to reliably test for SARS-CoV-2 is essential to stopping its spread. Several types of
tests exist to identify SARS-CoV-2, varying in how rapidly they give results, how sensitive they
are, and how often they can be performed [42]. The Nucleic Acid Amplification Test (NAAT)
is a high-sensitivity, high-specificity viral diagnostic test for SARS-CoV-2 [43] that can identify
more than one viral RNA gene and specify whether the infection is current or recent. Antigen
tests, which are low cost and are able to provide results rapidly, can find the presence of a spe-
cific viral antigen [44].
Among ML methods, support vector machine (SVM) was used to classify real human pre-
miRNAs from pseudo pre-miRNAs with 90% accuracy [13]. When feature extraction methods
were employed, the accuracy improved to 94.83% [45]. However, it is unclear whether the use
of pseudo pre-microRNAs approximates the real scenarios that will be faced by actual testing
equipment.
Human pre-miRNA classification was also attempted using deep learning, and contrasted
with other machine learning techniques such as naive Bayes classifiers, k-nearest neighbors
and random forest [46]. The under-sampling approach was used to overcome the class imbal-
ance problem, and the model outperformed traditional machine learning models.

Plant miRNA detection has also been demonstrated, achieving 97.54% identification accu-
racy [47]. Human mirtrons and canonical miRNAs have been classified using convolutional
neural networks (CNN) and long short-term memory networks (LSTMN) with 94.3% accu-
racy and 92.5% F1 score [48]. Human miRNA classification for gene prediction was performed
with results of 90.02% sensitivity and 97.28% specificity [49]. The SVM-based porcine pre-
microRNAs prediction method was proposed by [50], achieving a prediction accuracy of
95.6%.
The majority pre-miRNAs detection models are SVM classifier-based [51, 52]. SVM was
used to detect animals and plant miRNAs and pre-miRNAs detection methods [53, 54]. Rice
pre-miRNAs detection was done using random forest, achieving prediction accuracy of
93.48% [55].
When performing machine learning, most algorithms require both positive and negative
examples. Databases of positive examples are readily available, but negative examples are
scarce, forcing researchers to employ tactics such as creating negative examples through vari-
ous means. This has various problems, among which is that there is no guarantee that an
example that has been generated to be negative is not actually an undiscovered positive exam-
ple [56].
This problem of class imbalance in SARS-CoV-2 pre-miRNA detection was demonstrated
using various algorithms such as one-class SVM (OC-SVM), deeSOM, and mirDNN [57]. The
imbalance ratio in the dataset was varied from 1:50 to 1:200, with decreasing performance as
imbalance ratio increased. At the best imbalance ratio of 1:50, OC-SVM, deeSOM, mirDNN
achieved F1 scores of 39%, 51% and 74% respectively. The focal loss function [58] was used to
handle class imbalance, where larger weights are given to the more difficult-to-classify exam-
ples so that the problem of imbalance is ameliorated. Albahri OS et al. [59] reviewed AI-driven
COVID-19 detection and classification using medical images. The research challenges and
critical gaps had highlighted by the authors. Albahri AS et al. [60] provided a systematic review
of AI-based data mining and machine learning algorithms for detecting and diagnosing
COVID-19. This study analyzed the nature of the application, algorithms evaluation methods,
and accuracy for COVID-19.
The RUNN-COV method presented in this work attempts to overcome the class imbalance
problem through undersampling rather than negative example generation or weight
adjustment.
4 Dataset
The dataset was used based upon pre-miRNA detection using machine learning techniques
(Bugnon et al., 2021). It was derived by applying various techniques the SARS-CoV-2 genome
from National Center for Biotechnology Information (NCBI) Reference Sequence
NC_045512.2, resulting in 569 pre-miRNA samples with 73 features. The dataset also included
999888 hairpin-like sequences from the human genome as the negative class. The dataset
detailed can be found in [61] where the positive samples are identified labeled as 1 and the neg-
ative samples indicated as 0.
Pearson correlation was applied to the features in the dataset. It was found that many of the
features are uncorrelated. Fig 1 shows the comparison between positive and negative samples.
The central mark in the box indicates the median value, the edges of the box are the lower
quartile and upper quartile values, and whiskers are goes to the minimum and maximum val-
ues. The outlier or single data point is depicted as the black dot. The sequence length, ensemble
frequency, dQ, triplets0, mfe, mfei1, mfei2, etc are features of the dataset.

Fig 1. Box plots illustrate the distribution of numerical data with the pre-miRNA label class.
5 RUNN-COV model
5.1 Random undersampling
Random undersampling and oversampling are two techniques that are used to overcome the
problem of class imbalance. Random oversampling involves duplicating the examples in the
minority class, but this increases the likelihood of overfitting, decreases the classifier perfor-
mance, and increases the computational effort [62]. Random undersampling instead removes
examples randomly from the majority class, and was demonstrated experimentally to signifi-
cantly improve classification performance [63]. Therefore, random undersampling was
applied in this work to make the class ratio 1:1.
One concern with random undersampling is information loss when samples are removed.
To demonstrate the effect of random undersampling, heatmaps of correlation matrices of the
dataset before and after the procedure were taken. A random instance of from various runs of
the experiment is given in Fig 2, where it can be seen visually that the heatmaps remain virtu-
ally the same before and after the random undersampling procedure. The variation of color
depends on the intensity of the dataset feature. The correlation matrix before (top) and after
(bottom) random undersampling.
After preparing the data through random undersampling, a visual representation is
required to obtain a high-level understanding of the distribution of the positive and negative
classes. In fact, visual understanding of high-dimensional data is crucial in many areas, such as
the detailed analysis of single-cell datasets [64].
In this paper, t-distributed stochastic neighbor embedding (t-SNE), a nonlinear algorithm
[65] for high dimensional data exploring, data point visualization, and identifying hidden pat-
terns, was used to prepare a visual representation of the data as shown in Fig 3. The t-SNE was
chosen because it outperforms a wide range of nonparametric visualization approaches [66]. It
has become popular in the machine learning field because of its exceptional ability to generate
two-dimensional (2D) maps from data with thousands of dimensions. The t-SNE is extremely
flexible and can often identify structure where other dimensionality-reduction techniques can-
not [67].

Fig 2. The heat map shows the patterns, similarities, associations, correlations and expression of pre-miRNA.
5.2 Neural network architecture

A neural network model was developed and applied to the data to determine if the positive
and negative classes could be accurately identified. The model has 18 layers consisting of dense
layers and batch normalization layers.

Fig 3. Overview of t-SNE-driven clustering analysis strategy. The landscape of the gene expression profiles represented high dimensional data in this
two-dimensional map. The t-SNE projection shows the separation between positive and negative data points. In addition, the results of the pre-miRNAs
cluster analysis are represented in two colors. The maroon dot indicates positive data point and the blue dot indicates the negation data point.
The rectified linear unit (ReLu) activation function was used in the dense layers, since it has
various benefits such as computational simplicity, sparsity and linear behavior [68].
The batch normalization layers reduce training time and generalization error, minimize the
over-fitting problem, increase stability, and smoothen the loss function [69]. The output layer
uses the sigmoid function for binary classification.
The complete architecture has 1,128,994 parameters, of which 1,127,042 are trainable and
1,952 are non-trainable. The Adam optimizer was used to update network weights [70]. Other
hyperparameter details include the use of the binary entropy loss function, 90 epochs, batch
size 20, and data shuffling. The details of the neural network layers are shown in Table 1. This
architecture was arrived at after extensive experimentation using different neural network
architectures until one was found that produced high performance in identifying the examples
of the dataset.

Table 1. Neural network structure.

Layer Number Layer Type Output Shape Parameters
1 Dense (None, 1024) 75776
2 Dense (None, 512) 524800
3 Batch normalization (None, 512) 2048
4 Dense (None, 512) 262656
5 Dense (None, 256) 131328
7 Dense (None, 256) 65792
8 Dense (None, 128) 32896
10 Dense (None, 128) 16512
11 Dense (None, 64) 8256
13 Dense (None, 64) 4160
14 Dense (None, 32) 2080
15 Dense (None, 16) 528
17 Dense (None, 16) 272
18 Dense (None, 2) 34
6 Results and discussion

6.1 Evaluation metrics
Various measures were chosen for evaluating RUNN-COV. The basic performance measures
that are derived from true positive (TP), true negative (TN), false positive (FP) and false nega-
tive (FN): accuracy scores, along with precision, recall and F1 score, are shown in Eqs 1–4.
TP þ TN
Accuracy ¼ ð1Þ
TP þ TN þ FP þ FN
TP
Precision ¼ ð2Þ
TP þ FP
TP
Recall ¼ ð3Þ
TP þ FN
Precision � Recall
F1 Score ¼ 2 � ð4Þ
Precision þ Recall
Another more reliable derivative performance measure is the Matthews correlation coeffi-
cient (MCC), which only produces a high score if the prediction gained good results in TP,
FN, TN, and FP [71].
ðTP � TNÞ ðFP � FNÞ

MCC ¼ pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ð5Þ
ðTP þ FPÞðTP þ FNÞðTN þ FPÞðTN þ FNÞ

The Cohen’s kappa (CK) is a robust statistic widely used to measure the algorithm perfor-
mance [71].
Accuracy Pe
CK ¼ ð6Þ
1 Pe
Hamming loss gives the fraction of all the labels that have been incorrectly identified.
1X m
jyi Dyi1 j
HL ¼ ð7Þ
m i¼1 Q
F-Beta (beta = 1.0) is the weighted harmonic mean between recall and precision.
The dataset was split into 80% training data and 20% test data. Models were run with 90
epochs and the model scored well with the above performance metrics: accuracy 98.24%,
Cohen’s kappa score 96.49%, Matthews correlation coefficient 96.50%, hamming loss 0.0175,
precision 98%, recall 98%, f1-score 98%, area under the receiver operating characteristic (ROC
AUC) score 98.24%, F-beta(beta = 1) score 98.26%. The accuracy curve, loss curve and confu-
sion matrix are shown in Fig 4. The top left plot indicates the accuracy curve and the top-right
plot the loss curve. Each plot x-axis shows time or epoch and y-axis learning or loss. The learn-
ing curve has improve over time. The bottom center plot illustrates the confusion matrix to
Fig 4. Accuracy curve, loss curve, and confusion matrix.


observe the proposed model performance. The confusion matrix reflect the true positive rate
at the top left corner, the false-positive rate at the top right corner, the false-negative rate at the
bottom left corner, and the true-negative rate at the bottom right corner.
6.2 Comparison with other standard algorithms

The model’s performance was compared to the following popular machine learning
algorithms.
• Logistic Regression is a popular algorithm used to used to solve classification problems [73],
using gradient descent to reduce the cost. The logistic regression algorithm achieved 89.47%
detection accuracy.
• The k-nearest-neighbours (KNN) classifies based on similarity of examples. It can help
reduce the computational cost while maintaining classification accuracy [74]. The KNN clas-
sifier achieved 89.91% detection accuracy.
• Support Vector Machines (SVM) uses hyperplanes to separate classes [75]. The SVM classi-
fier achieved 89.47% detection accuracy.
• Random Forest provides accurate results most of the time without hyper-parameter tuning
[76]. The random forest creates several decision trees and merges them to get more accurate
results. The random forest classifier achieved best 91.66% detection accuracy.
By contrasting the confusion matrices (Figs 4 and 5) and from the performance metrics in
Table 2, it was found that RUNN-COV performs better than these standard machine learning
algorithms.
Table 2 shows that RUNN-COV achieved detection accuracy 98.24%, faring better than
logistic regression (89.47%), k-nearest neighbors (89.91%), support vector machines (89.47%),
and random forest (91.66%). This is visually represented in Fig 6.
6.3 Performance comparison with existing approaches

Table 3 shows the comparison of RUNN-COV against various approaches in literature.
Among them, the only one that used the same dataset achieved an accuracy of 51% [57] while
RUNN-COV achieves 98.26% accuracy. RUNN-COV also has comparable results to other
models that were run on different datasets.
6.4 Statistical analysis

We used the statistical investigation to compare performance and novelty against previous
studies. To compare the difference between the previous and proposed studies, we used a
paired samples t-test. The statistical paired t-test is appropriate to compare statistical signifi-
cance’s and differences [77]. To investigate performance used evaluation metrics including the
F1 score and precision score.
x� m
t¼ p ð8Þ
S= N
In Eq 8, x� is a sample mean μ is the constant for the population mean, N is the number of
p
observations, and S= N is the estimated standard error of the mean. We structured the fol-
lowing six null hypotheses:
x1H0: The deesom (1:50) Vs. RUNN-COV model performance have no significant difference.

Fig 5. The confusion matrix shows the accuracy of traditional machine learning algorithms. The correctly classified data is reflected along the
diagonal regions. The misclassified is reflected in the off-diagonal regions. Top-left plot logistic regression confusion matrix, top-right plot k-nearest-
neighbors confusion matrix, bottom-left plot support vector machines confusion matrix, and bottom-right plot random forest confusion matrix.
x2H0: The OC-SVM (1:50) Vs. RUNN-COV model performance have no significant
difference.
x3H0: The deesom (1:100) Vs. RUNN-COV model performance have no significant
difference.
difference.
Table 2. Performance comparison with traditional ML models.

Model MCC Cohen’s kappa Hamming Loss Precision Recall F1 score Accuracy ROC AUC
RUNN-COV 96.50% 96.49% 0.0175 97.41% 99.12% 98.26% 98.24% 98.24%
RF 83.33% 83.33% 0.0833 91.15% 91.96% 91.55% 91.66% 91.67%
LR 79.00% 78.95% 0.1052 87.93% 91.07% 89.47% 89.47% 89.50%
SVM 79.41% 78.97% 0.1052 85.48% 94.64% 89.83% 89.47% 89.56%
KNN 79.85% 79.82% 0.1008 88.69% 91.07% 89.86% 89.91% 89.93%

Fig 6. The radar chart illustrates the differences in performance metrics of KNN, RUNN-COV, logistic regression, random forest, and SVM
algorithms. In this visual analysis, the different vertices show where each algorithm performs well and where each performs poorly.
x5H0: The deesom (1:200) Vs. RUNN-COV model performance have no significant
difference.
difference.
Table 4 shows the paired t-test results at 0.05 significance level and justifies the significant
difference between the performance of previous studies and proposed studies. The paired t-
test results that the p-value is less than 0.05 hence all six hypotheses x1H0, x2H0, x3H0, x4H0,
x5H0, and x6H0 are rejected.
Table 3. Performance comparison proposed model with relevant existing approaches.

Ref. Models MCC Precision Sensitivity/Recall F1 score Accuracy
[45] MicroRNA-NHPred 89.65% - - - 94.83%
[48] CNN-LSTM 88.00% - 94.80% 92.50% 94.30%
[48] SVM - - - - 90.00%
[47] PlantMirP2 - - 96.75% - 97.54%
[55] Plantmirp-rice: 87.10% - 87.91% - 93.48%
[46] DP-miRNA - - 97.30% - 96.80%
[54] PlantMiRNAPred - - 90.31% - 92.06%
[57] deeSOM - - - 51% -
[57] OC-SVM - - - 39% -
This paper RUNN-COV 96.50% 97.41% 99.12% 98.26% 98.24%

Table 4. RUNN-COV model against previous studies paired sample t-test (significance level of 0.05).
Model Pair p-value
deesom (1:50) Vs. RUNN-COV 0.026
OC-SVM (1:50) Vs. RUNN-COV 0.021
Table 5. Number of experiments, sampling strategy, variation of models and layers, and performance against previous and proposed approaches.
Experiment Strategy (Dataset) Models F1 score
Exp1 Without undersampling RF 13.33%
Exp2 Without undersampling LR 20.14%
Exp3 Without undersampling SVM 05.21%
Exp4 Without undersampling KNN 08.47%
Exp5 Without undersampling Without Batch Normalization layers 47.17%
Exp6 Undersampling RF 91.55%
Exp7 Undersampling LR 89.47%
Exp8 Undersampling SVM 89.83%
Exp9 Undersampling KNN 89.86%
Exp10 Undersampling Without Batch Normalization layers 92.10%
Exp11 Undersampling With Dropout layers 92.10%
Exp12 Undersampling RUNN-COV 98.26%
Previous literature Ratio(1:50) deeSOM 51.00%
Previous literature Ratio(1:50) OC-SVM 39.00%
With p-value less than 0.05 for all tests, with 95% confidence our model can be viewed as
novel in nature and performs significantly better than existing approaches.
Table 5 displays our number of experiments, novelty, sampling strategy, variation of models
and layers, as well as performance. Table 6 shows the p-values are less than 0.05, which indi-
cates the significant difference between standard machine learning models against the pro-
posed model.
Table 6. Paired sample t-test. (significance level of 0.05).

Model Pair p-value
RUNN-COV Vs. Exp7 0.00040

6.5 Discussion
In this research work, we have extensively explored the detection method of SARS-CoV-2 pre-
cursor-miRNAs through Artificial Neural networks. The initial dataset through experimenta-
tion provided a moderate performance which through dataset investigation, we unearthed
existing class imbalance problem. Our research on solving high class imbalance problem, led
us to further investigate solution manuals regarding class imbalancce problems and opted for
random undersampling techniques. Similarly, As the dataset is noisy, it was handy for practi-
tioners to cluster the dataset for which t-SNE was employed. t-SNE uncovers inexact contigu-
ity in an basic high-dimensional complex, so clusters on the low-dimensional representation
of the high-dimensional space maximize the probability that bordering data points will not be
within the same cluster. Before employing t-SNE, we undertook the tradeoff that, t-SNE does
not preserve distances nor density but preserves some form of nearest neighbours. t-SNE gave
us 2D maps for the visualization of bias and variance from high-dimensional data. While the
difference is subtle, but it affects any distance based algorithms at its core which led us to select
distance-exclusionary algorithms that achieved extremely high- performance scores on various
measures, outperforming traditional machine learning models and the other existing work on
the same dataset. One might argue, why not use random oversampling instead of undersam-
pling. We have investigated these as well albeit not reported in the results. We have realized
that duplicating the instances in the minority class although solved class imbalance problem
but it increased the likelihood of overfitting, as instances were increased gradually. Eventually,
random oversampling contributed to a substantial decrease of classifier performance and
increased computational error manifold that led us to stick to undersampling techniques.
An interesting instance of our neural network structure is susbsequent utilization of batch
normalization. Although, using undersampling instead of oversampling minimized overfitting
problem, in our experimentation, we still received anomaly through overfitting which was ulti-
mately solved using batch normalization. Furthermore, this helped the network in reducing
training time, smoothing the loss function and increased overall stability by reducing the gen-
eralization error. While we proposed the 18-layer exhaustive neural network, we have experi-
mented the model with several variants of layers by including and excluding normalization
and dropout layer as well. However, in our investigation, while including these two we have
realized some interesting insights about neural networks in general. When using dropouts
during training, activations are scaled to maintain the average after the dropout shift. How-
ever, the difference is not preserved. Traversing a non-linear slice translates this dispersion
shift into an activation average shift, transitioning to the final linear projection slice. The final
prediction is trained to fit the training time stats, so if dropout is off, it will fail during valida-
tion. This behavior is not an issue for tasks where only relative scaling of the output is impor-
tant (such as softmax classification). In our case, if the output represents an absolute quantity,
this leads to poor inference time performance. This architecture was arrived at after extensive
experimentation using different neural network models.
The novelty of this research stands at experimentation of exploratory data analysis (EDA)
including correlation matrix and t-distributed stochastic neighbor embedding. Efficient
exploratory data analysis including data visual representation and data point analysis mecha-
nisms aided us in selecting the best hyperparameters and models. RUNN-COV (Random
Under sampling with Neural Network for COVID detection) models was presented based on
previous EDA. Our designed model has produced a substantially superior performance which
we have shown through experimentation. Statistical analysis was conducted against previous
studies to understand the objective statistical significance of our research work that concluded
our work is significant in nature.

While Tables 2 and 3 shows our model outperforming the previous literature as well as tra-
ditional machine learning approaches, the neural network models always hankers for space
and time. The research can be continued further in evaluating this detection through more
explainable mechanisms, which would aid us in excluding or including layers, and hyperpara-
meters from architecture to make the model more robust and modular for daily usage. Semi-
supervised learning such as active learning, sub-modular optimization, reinforcement learning
can also be employed to attack the challenges again that were investigated in our research and
reported in subsequent discussion to advance this field of research. The performance measures
were also chosen to address the high class imbalance problem. We have observed that albeit
having a greater accuracy, the accuracy metrics itself is not a good measure of performance in
these highly imbalanced set of data. This lead us to experiment with precision and recall where
precision provided us with an insight on how good the model was at predicting a specific type
of target. Recall provides an insight on how many times the model detected a specific target.
Overall, experimentation with performance measures such as MCC, F1- score were also added
to the list for understanding the actual performance of our model.
7 Conclusion
COVID research necessitates an all-hands-on-deck strategy in order to eradicate the virus’s
impact on the planet in a manner that is environmentally responsible. Among the various
research domain specialists, computer scientists play a significant role in developing, analys-
ing, and deploying cutting-edge research to continue the decent battle. In this study, a strategy
is suggested that combines deep learning-based efficient pre-processing with neural network-
inspired classification structures to identify SARS-CoV-2 pre-miRNAs effectively, therefore
enhancing their performance. Our study demonstrates the effective processing and visualisa-
tion tools to generate insights, such as random undersampling, t-SNE, and correlation matrix,
which gave insightful information that eventually increased the current research performance.
The study examines various approaches to the class imbalance, adopted a random undersam-
pling approaches and visualised data with t-SNE to generate better performances. Later, it is
compared with existing approaches to classical Machine learning algorithms to provide an
understanding of the contribution throughout the study.
This study has presented the RUNN-COV model, a neural network model with dense layers
and batch normalisation layers for SARS-CoV-2 pre-miRNAs identification based on seventy-
three features that enable the rapid detection of COVID. The model is comprised of using ran-
dom undersampling techniques to the extremely imbalanced dataset in order to decrease class
imbalance, followed by the use of a precisely designed artificial neural network. With Mat-
thew’s correlation coefficient (MCC) score of 96.50 percent and an F1 score of 98.26 percent,
the model outperformed typical machine learning models and other current work on the same
dataset on a variety of other performance metrics. Additionally, the model performance is sim-
ilar to that of other models that have been performed on distinct datasets.
It is believed that RUNN-COV will aid in the sequencing of the SARS-CoV-2 genome and
the identification of target sites in an RNA in order to create oligonucleotide-based medicines
against the genetic structure of the virus. Comparing this study with previous research, a com-
prehensive statistical analysis was undertaken to determine the objective statistical significance
of the study.
Future work will include the development of a pre-miRNA detection technique based on
raw RNA sequence data and the application of RUNN-COV to additional organisms’ datasets.
Practitioners are also required to research to create more COVID-related data so that the issue
of class imbalance may be resolved from the outset. In addition, prospects for study may be

identified in the interpretation of results using explainable artificial intelligence, since the phy-
sicians, nurses, and patients are the most probable layperson consumers and stakeholders.
The whole research work including data visualization, exploratory data analysis, COVID
detection task are available in github https://github.com/MdMahadiHasan1/SARS-CoV-
2-pre-miRNA for repeatability and seamless replication.
Supporting information
S1 File.
(DOCX)
Acknowledgments
Author would like to acknowledge the support of work from Asian University of Bangladesh.
Author Contributions
Conceptualization: Md. Mahadi Hasan.
Data curation: Md. Mahadi Hasan.
Investigation: Muhammad Jafar Sadeq, Jasim Uddin.
Methodology: Md. Mahadi Hasan, Muhammad Jafar Sadeq, Jasim Uddin.
Supervision: Muhammad Jafar Sadeq, Jasim Uddin.
Writing – original draft: Md. Mahadi Hasan, Saba Binte Murtaz, Muhammad Usama Islam,
Muhammad Jafar Sadeq, Jasim Uddin.
Writing – review & editing: Saba Binte Murtaz, Muhammad Usama Islam, Muhammad Jafar
Sadeq, Jasim Uddin.
References
1. Pal M, Berhanu G, Desalegn C, Kandi V. Severe acute respiratory syndrome coronavirus-2 (SARS-
CoV-2): an update. Cureus. 2020 Mar 26; 12(3). https://doi.org/10.7759/cureus.7423 PMID: 32337143
2. Grundhoff A, Sullivan CS. Virus-encoded microRNAs. Virology. 2011 Mar 15; 411(2):325–43. https://
doi.org/10.1016/j.virol.2011.01.002 PMID: 21277611
3. Li M, Marin-Muller C, Bharadwaj U, Chow KH, Yao Q, Chen C. MicroRNAs: control and loss of control in
human physiology and disease. World journal of surgery. 2009 Apr; 33(4):667–84. https://doi.org/10.
1007/s00268-008-9836-x PMID: 19030926
4. Zhao Y, Samal E, Srivastava D. Serum response factor regulates a muscle-specific microRNA that tar-
gets Hand2 during cardiogenesis. Nature. 2005 Jul; 436(7048):214–20. https://doi.org/10.1038/
nature03817 PMID: 15951802
5. Chen JF, Mandel EM, Thomson JM, Wu Q, Callis TE, Hammond SM, et al. The role of microRNA-1 and
microRNA-133 in skeletal muscle proliferation and differentiation. Nature genetics. 2006 Feb; 38
(2):228–33. https://doi.org/10.1038/ng1725 PMID: 16380711
6. Naguibneva I., Ameyar-Zazoua M., Polesskaya A., Ait-Si-Ali S., Groisman R., Souidi M., et al., 2006.
The microrna mir-181 targets the homeobox protein hox-a11 during mammalian myoblast differentia-
tion. Nature cell biology 8, 278–284. https://doi.org/10.1038/ncb1373 PMID: 16489342
7. Filipowicz W., 2005. Rnai: the nuts and bolts of the risc machine. Cell 122, 17–20. https://doi.org/10.
1016/j.cell.2005.06.023 PMID: 16009129
8. Fani M, Zandi M, Ebrahimi S, Soltani S, Abbasi S. The role of miRNAs in COVID-19 disease. Future
Virology. 2021 Apr; 16(4):301–6. https://doi.org/10.2217/fvl-2020-0389
9. Ying H, Ebrahimi M, Keivan M, Khoshnam SE, Salahi S, Farzaneh M. miRNAs; a novel strategy for the
treatment of COVID-19. Cell biology international. 2021 Oct; 45(10):2045–53. https://doi.org/10.1002/
cbin.11653 PMID: 34180562

10. Jonas S, Izaurralde E. Towards a molecular understanding of microRNA-mediated gene silencing.

Nature reviews genetics. 2015 Jul; 16(7):421–33. https://doi.org/10.1038/nrg3965 PMID: 26077373
11. Lee Y., Kim M., Han J., Yeom K.H., Lee S., Baek S.H., et al, 2004. Microrna genes are transcribed by
rna polymerase ii. The EMBO journal 23, 4051–4060. https://doi.org/10.1038/sj.emboj.7600385 PMID:
15372072
12. Han J, Lee Y, Yeom KH, Nam JW, Heo I, Rhee JK, et al. Molecular basis for the recognition of primary
microRNAs by the Drosha-DGCR8 complex. cell. 2006 Jun 2; 125(5):887–901. https://doi.org/10.1016/
j.cell.2006.03.043 PMID: 16751099
13. Xue C, Li F, He T, Liu GP, Li Y, Zhang X. Classification of real and pseudo microRNA precursors using
local structure-sequence features and support vector machine. BMC bioinformatics. 2005 Dec; 6(1):1–
7. https://doi.org/10.1186/1471-2105-6-310 PMID: 16381612
14. Fu X, Zhu W, Cai L, Liao B, Peng L, Chen Y, et al. Improved pre-miRNAs identification through mutual
information of pre-miRNA sequences and structures. Frontiers in genetics. 2019 Feb 25; 10:119.
https://doi.org/10.3389/fgene.2019.00119 PMID: 30858864
15. Li L, Xu J, Yang D, Tan X, Wang H. Computational approaches for microRNA studies: a review. Mam-
malian Genome. 2010 Feb; 21(1):1–2. https://doi.org/10.1007/s00335-009-9241-2 PMID: 20012966
16. Zheng K, You ZH, Wang L, Zhou Y, Li LP, Li ZW. MLMDA: a machine learning approach to predict and
validate MicroRNA–disease associations by integrating of heterogenous information sources. Journal
of translational medicine. 2019 Dec; 17(1):1–4. https://doi.org/10.1186/s12967-019-2009-x PMID:
31395072
17. Stegmayer G, Di Persia LE, Rubiolo M, Gerard M, Pividori M, Yones C, et al. Predicting novel micro-
RNA: a comprehensive comparison of machine learning approaches. Briefings in bioinformatics. 2019
Sep; 20(5):1607–20. https://doi.org/10.1093/bib/bby037 PMID: 29800232
18. Bugnon LA, Yones C, Raad J, Milone DH, Stegmayer G. Genome-wide hairpins datasets of animals
and plants for novel miRNA prediction. Data in brief. 2019 Aug 1; 25:104209. https://doi.org/10.1016/j.
dib.2019.104209 PMID: 31453279
19. Bugnon LA, Yones C, Milone DH, Stegmayer G. Deep neural architectures for highly imbalanced data
in bioinformatics. IEEE Transactions on Neural Networks and Learning Systems. 2019 Jun 3; 31
(8):2857–67. https://doi.org/10.1109/TNNLS.2019.2914471 PMID: 31170082
20. Yu T, Xu N, Haque N, Gao C, Huang W, Huang Z. Popular computational tools used for miRNA predic-
tion and their future development prospects. Interdisciplinary Sciences: Computational Life Sciences.
2020 Dec; 12(4):395–413. PMID: 32959233
21. Maragkakis M, Alexiou P, Papadopoulos GL, Reczko M, Dalamagas T, Giannopoulos G, et al. Accurate
microRNA target prediction correlates with protein repression levels. BMC bioinformatics. 2009 Dec; 10
(1):1–0. https://doi.org/10.1186/1471-2105-10-295 PMID: 19765283
22. Lewis BP, Shih IH, Jones-Rhoades MW, Bartel DP, Burge CB. Prediction of mammalian microRNA tar-
gets. Cell. 2003 Dec 26; 115(7):787–98. https://doi.org/10.1016/S0092-8674(03)01018-3 PMID:
14697198
23. Lewis BP, Burge CB, Bartel DP. Conserved seed pairing, often flanked by adenosines, indicates that
thousands of human genes are microRNA targets. cell. 2005 Jan 14; 120(1):15–20. https://doi.org/10.
1016/j.cell.2004.12.035 PMID: 15652477
24. John B, Enright AJ, Aravin A, Tuschl T, Sander C, Marks DS, et al. Human microRNA targets. PLoS
biology. 2004 Nov; 2(11):e363. https://doi.org/10.1371/journal.pbio.0020363 PMID: 15502875
25. Betel D, Koppal A, Agius P, Sander C, Leslie C. Comprehensive modeling of microRNA targets predicts
functional non-conserved and non-canonical sites. Genome biology. 2010 Aug; 11(8):1–4. https://doi.
org/10.1186/gb-2010-11-8-r90 PMID: 20799968
26. Miranda KC, Huynh T, Tay Y, Ang YS, Tam WL, Thomson AM, et al. A pattern-based method for the
identification of MicroRNA binding sites and their corresponding heteroduplexes. Cell. 2006 Sep 22;
126(6):1203–17. https://doi.org/10.1016/j.cell.2006.07.031 PMID: 16990141
27. Rehmsmeier M, Steffen P, Höchsmann M, Giegerich R. Fast and effective prediction of microRNA/tar-
get duplexes. Rna. 2004 Oct 1; 10(10):1507–17. https://doi.org/10.1261/rna.5248604 PMID: 15383676
28. Kumar S, Viral R, Deep V, Sharma P, Kumar M, Mahmud M, et al. Forecasting major impacts of
COVID-19 pandemic on country-driven sectors: challenges, lessons, and future roadmap. Personal
and Ubiquitous Computing. 2021 Mar 26:1–24. https://doi.org/10.1007/s00779-021-01530-7 PMID:
33815032
29. Ouyang X, Jiang X, Gu D, Zhang Y, Kong SK, Jiang C, et al. Dysregulated serum MiRNA profile and
promising biomarkers in dengue-infected patients. International journal of medical sciences. 2016; 13
(3):195. https://doi.org/10.7150/ijms.13996 PMID: 26941580

30. Scaria V, Hariharan M, Maiti S, Pillai B, Brahmachari SK. Host-virus interaction: a new role for micro-
RNAs. Retrovirology. 2006 Dec; 3(1):1–9. https://doi.org/10.1186/1742-4690-3-68 PMID: 17032463
31. Omoto S, Fujii YR. Regulation of human immunodeficiency virus 1 transcription by nef microRNA. Jour-
nal of General Virology. 2005 Mar 1; 86(3):751–5. https://doi.org/10.1099/vir.0.80449-0 PMID:
15722536
32. Gupta A, Gartner JJ, Sethupathy P, Hatzigeorgiou AG, Fraser NW. Anti-apoptotic function of a micro-
RNA encoded by the HSV-1 latency-associated transcript. Nature. 2006 Jul; 442(7098):82–5. https://
doi.org/10.1038/nature04836 PMID: 16738545
33. Ura S, Honda M, Yamashita T, Ueda T, Takatori H, Nishino R, et al. Differential microRNA expression
between hepatitis B and hepatitis C leading disease progression to hepatocellular carcinoma. Hepatol-
ogy. 2009 Apr; 49(4):1098–112. https://doi.org/10.1002/hep.22749 PMID: 19173277
34. Rupaimoole R, Slack FJ. MicroRNA therapeutics: towards a new era for the management of cancer and
other diseases. Nature reviews Drug discovery. 2017 Mar; 16(3):203–22. https://doi.org/10.1038/nrd.
2016.246 PMID: 28209991
35. Regazzi R. MicroRNAs as therapeutic targets for the treatment of diabetes mellitus and its complica-
tions. Expert opinion on therapeutic targets. 2018 Feb 1; 22(2):153–60. https://doi.org/10.1080/
14728222.2018.1420168 PMID: 29257914
36. Zhou SS, Jin JP, Wang JQ, Zhang ZG, Freedman JH, Zheng Y, et al. miRNAS in cardiovascular dis-
eases: potential biomarkers, therapeutic targets and challenges. Acta Pharmacologica Sinica. 2018 Jul;
39(7):1073–84. https://doi.org/10.1038/aps.2018.30 PMID: 29877320
37. Otsuka M, Kishikawa T, Yoshikawa T, Yamagami M, Ohno M, Takata A, et al. MicroRNAs and liver dis-
ease. Journal of human genetics. 2017 Jan; 62(1):75–80. https://doi.org/10.1038/jhg.2016.53 PMID:
27225852
38. Fulzele S, Sahay B, Yusufu I, Lee TJ, Sharma A, Kolhe R, et al. COVID-19 virulence in aged patients
might be impacted by the host cellular microRNAs abundance/profile. Aging and disease. 2020 Jun; 11
(3):509. https://doi.org/10.14336/AD.2020.0428 PMID: 32489698
39. Guterres A, de Azeredo Lima CH, Miranda RL, Gadelha MR. What is the potential function of micro-
RNAs as biomarkers and therapeutic targets in COVID-19?. Infection, Genetics and Evolution. 2020
Nov 1; 85:104417. https://doi.org/10.1016/j.meegid.2020.104417 PMID: 32526370
40. Lamb YN. Remdesivir: first approval. Drugs. 2020 Sep; 80(13):1355–63. https://doi.org/10.1007/
s40265-020-01378-w PMID: 32870481
41. Levin AA. Treating disease at the RNA level with oligonucleotides. New England Journal of Medicine.
2019 Jan 3; 380(1):57–70. https://doi.org/10.1056/NEJMra1705346 PMID: 30601736
42. Mina MJ, Andersen KG. COVID-19 testing: One size does not fit all. Science. 2021 Jan 8; 371
(6525):126–7. https://doi.org/10.1126/science.abe9187 PMID: 33414210
43. Caturegli G, Materi J, Howard BM, Caturegli P. Clinical Validity of Serum Antibodies to SARS-CoV-2: A
Case–Control Study. Annals of internal medicine. 2020 Oct 20; 173(8):614–22. https://doi.org/10.7326/
M20-2889 PMID: 32628534
44. Liu G, Rusling JF. COVID-19 antibody tests and their limitations. ACS sensors. 2021 Feb 5; 6( 3):593–
612. https://doi.org/10.1021/acssensors.0c02621 PMID: 33544999
45. Ma Y, Yu Z, Han G, Li J, Anh V. Identification of pre-microRNAs by characterizing their sequence order
evolution information and secondary structure graphs. BMC bioinformatics. 2018 Dec; 19(19):25–35.
https://doi.org/10.1186/s12859-018-2518-2 PMID: 30598066
46. Thomas J, Thomas S, Sael L. DP-miRNA: An improved prediction of precursor microRNA using deep
learning model. In2017 IEEE International Conference on Big Data and Smart Computing (BigComp)
2017 Feb 13 (pp. 96-99). IEEE.
47. Fan D, Yao Y, Yi M. PlantMirP2: An Accurate, Fast and Easy-To-Use Program for Plant Pre-miRNA
and miRNA Prediction. Genes. 2021 Aug; 12(8):1280. https://doi.org/10.3390/genes12081280 PMID:
34440454
48. Tasdelen A, Sen B. A hybrid CNN-LSTM model for pre-miRNA classification. Scientific reports. 2021
Jul 8; 11(1):1–9. https://doi.org/10.1038/s41598-021-93656-0 PMID: 34239004
49. Batuwita R, Palade V. microPred: effective classification of pre-miRNAs for human miRNA gene predic-
tion. Bioinformatics. 2009 Apr 15; 25(8):989–95. https://doi.org/10.1093/bioinformatics/btp107 PMID:
19233894
50. Wang Z, He K, Wang Q, Yang Y, Pan Y. The prediction of the porcine pre-microRNAs in genome-wide
based on support vector machine (SVM) and homology searching. BMC genomics. 2012 Dec; 13(1):1–
8. https://doi.org/10.1186/1471-2164-13-729 PMID: 23268561

51. Meng J, Liu D, Sun C, Luan Y. Prediction of plant pre-microRNAs and their microRNAs in genome-
scale sequences using structure-sequence features and support vector machine. BMC bioinformatics.
2014 Dec; 15(1):1–4. https://doi.org/10.1186/s12859-014-0423-x PMID: 25547126
52. Yao Y, Ma C, Deng H, Liu Q, Zhang J, Yi M. plantMirP: an efficient computational program for the pre-
diction of plant pre-miRNA by incorporating knowledge-based energy features. Molecular BioSystems.
2016; 12(10):3124–31. https://doi.org/10.1039/C6MB00295A PMID: 27472470
53. Xuan P, Guo M, Huang Y, Li W, Huang Y. MaturePred: efficient identification of microRNAs within novel
plant pre-miRNAs. PloS one. 2011 Nov 16; 6(11):e27422. https://doi.org/10.1371/journal.pone.
0027422 PMID: 22110646
54. Xuan P, Guo M, Liu X, Huang Y, Li W, Huang Y. PlantMiRNAPred: efficient classification of real and
pseudo plant pre-miRNAs. Bioinformatics. 2011 May 15; 27(10):1368–76. https://doi.org/10.1093/
bioinformatics/btr153 PMID: 21441575
55. Zhang H, Wang H, Yao Y, Yi M. PlantMirP-Rice: An Efficient Program for Rice Pre-miRNA Prediction.
Genes. 2020 Jun; 11(6):662. https://doi.org/10.3390/genes11060662 PMID: 32570706
56. Allmer J, Yousef M. Computational methods for ab initio detection of microRNAs. Frontiers in genetics.
2012 Oct 10; 3:209. https://doi.org/10.3389/fgene.2012.00209 PMID: 23087705
57. Bugnon LA, Raad J, Merino GA, Yones C, Ariel F, Milone DH, et al. Deep Learning for the discovery of
new pre-miRNAs: Helping the fight against COVID-19. Machine Learning with Applications. 2021 Dec
15; 6:100150. https://doi.org/10.1016/j.mlwa.2021.100150 PMID: 34939043
58. Lin TY, Goyal P, Girshick R, He K, Dollár P. Focal loss for dense object detection. InProceedings of the
IEEE international conference on computer vision 2017 (pp. 2980-2988).
59. Albahri OS, Zaidan AA, Albahri AS, Zaidan BB, Abdulkareem KH, Al-Qaysi ZT, et al. Systematic review
of artificial intelligence techniques in the detection and classification of COVID-19 medical images in
terms of evaluation and benchmarking: Taxonomy analysis, challenges, future solutions and methodo-
logical aspects. Journal of infection and public health. 2020 Oct 1; 13(10):1381–96. https://doi.org/10.
1016/j.jiph.2020.06.028 PMID: 32646771
60. Albahri AS, Hamid RA, Al-qays ZT, Zaidan AA, Zaidan BB, Albahri AO, et al. Role of biological data min-
ing and machine learning techniques in detecting and diagnosing the novel coronavirus (COVID-19): a
systematic review. Journal of medical systems. 2020 Jul; 44(7):1–1. https://doi.org/10.1007/s10916-
020-01582-x PMID: 32451808
61. https://sourceforge.net/projects/sourcesinc/files/aicovid/dataset.tar.gz.
62. Branco P, Torgo L, Ribeiro RP. A survey of predictive modeling on imbalanced domains. ACM Comput-
ing Surveys (CSUR). 2016 Aug 13; 49(2):1–50. https://doi.org/10.1145/2907070
63. Prusa J, Khoshgoftaar TM, Dittman DJ, Napolitano A. Using random undersampling to alleviate class
imbalance on tweet sentiment data. In2015 IEEE international conference on information reuse and
integration 2015 Aug 13 (pp. 197-202). IEEE.
64. Belkina AC, Ciccolella CO, Anno R, Halpert R, Spidlen J, Snyder-Cappione JE. Automated optimized
parameters for T-distributed stochastic neighbor embedding improve visualization and analysis of large
datasets. Nature communications. 2019 Nov 28; 10(1):1–2. https://doi.org/10.1038/s41467-019-13055-
y PMID: 31780669
65. Van der Maaten L, Hinton G. Visualizing data using t-SNE. Journal of machine learning research. 2008
Nov 1; 9(11).
66. Zhou B, Jin W. Visualization of single cell RNA-Seq data using t-SNE in R. InStem Cell Transcriptional
Networks 2020 (pp. 159–167). Humana, New York, NY.
67. Wattenberg M, Viégas F, Johnson I. How to use t-SNE effectively. Distill. 2016 Oct 13; 1(10):e2. https://
doi.org/10.23915/distill.00002
68. Glorot X, Bordes A, Bengio Y. Deep sparse rectifier neural networks. InProceedings of the fourteenth
international conference on artificial intelligence and statistics 2011 Jun 14 (pp. 315-323). JMLR Work-
shop and Conference Proceedings.
69. Ioffe S, Szegedy C. Batch normalization: Accelerating deep network training by reducing internal covari-
ate shift. InInternational conference on machine learning 2015 Jun 1 (pp. 448-456). PMLR.
70. Kingma DP, Ba J. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. 2014
Dec 22.
71. Chicco D, Jurman G. The advantages of the Matthews correlation coefficient (MCC) over F1 score and
accuracy in binary classification evaluation. BMC genomics. 2020 Dec; 21(1):1–3. https://doi.org/10.
1186/s12864-019-6413-7 PMID: 31898477
72. McHugh ML. Interrater reliability: the kappa statistic. Biochemia medica. 2012 Oct 15; 22(3):276–82.
https://doi.org/10.11613/BM.2012.031 PMID: 23092060

73. Hosmer DW Jr, Lemeshow S, Sturdivant RX. Applied logistic regression. John Wiley Sons; 2013 Apr 1.
74. Wu X, Kumar V, Ross Quinlan J, Ghosh J, Yang Q, Motoda H, et al. Top 10 algorithms in data mining.
Knowledge and information systems. 2008 Jan; 14(1):1–37.
75. Cortes C, Vapnik V. Support-vector networks. Machine learning. 1995 Sep; 20(3):273–97. https://doi.
org/10.1007/BF00994018
76. Ho TK. Random decision forests. InProceedings of 3rd international conference on document analysis
and recognition 1995 Aug 14 (Vol. 1, pp. 278-282). IEEE.
77. Kim TK. T test as a parametric statistic. Korean journal of anesthesiology. 2015 Dec 1; 68(6):540–6.

Research Accomplishment

Uploaded by

Copyright:

Available Formats

You might also like

Research Accomplishment

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Research Accomplishment

Uploaded by

Copyright:

Available Formats

PLOS ONE

Evaluating the Brexit and COVID-19’s

PLOS ONE | https://doi.org/10.1371/journal.pone.0287342 June 15, 2023 1 / 33

PLOS ONE | https://doi.org/10.1371/journal.pone.0287342 June 15, 2023 2 / 33

PLOS ONE | https://doi.org/10.1371/journal.pone.0287342 June 15, 2023 3 / 33

PLOS ONE | https://doi.org/10.1371/journal.pone.0287342 June 15, 2023 4 / 33

PLOS ONE | https://doi.org/10.1371/journal.pone.0287342 June 15, 2023 5 / 33

PLOS ONE | https://doi.org/10.1371/journal.pone.0287342 June 15, 2023 6 / 33

PLOS ONE | https://doi.org/10.1371/journal.pone.0287342 June 15, 2023 7 / 33

3.1 Data analysis

PLOS ONE | https://doi.org/10.1371/journal.pone.0287342 June 15, 2023 8 / 33

PLOS ONE | https://doi.org/10.1371/journal.pone.0287342 June 15, 2023 9 / 33

3.2 Box-Jenkins method

PLOS ONE | https://doi.org/10.1371/journal.pone.0287342 June 15, 2023 10 / 33

Fig 1. Unemployment rate in UK (2011–2021). Source: ONS.

PLOS ONE | https://doi.org/10.1371/journal.pone.0287342 June 15, 2023 11 / 33

Fig 2. Unemployment rate in all regions of UK. Source: ONS.

‘Accommodation’, ‘Transport’, ‘Construction’, ‘Wholesale’, and ‘Other’. During the Brexit

Fig 3. Unemployment rate in UK gender-wise. Source: ONS.

PLOS ONE | https://doi.org/10.1371/journal.pone.0287342 June 15, 2023 12 / 33

Fig 4. Unemployment rate of industries in the UK. Source: ONS.

4.2 Earnings (average weekly)

4.3 Gross Domestic Product (GDP) index

PLOS ONE | https://doi.org/10.1371/journal.pone.0287342 June 15, 2023 13 / 33

PLOS ONE | https://doi.org/10.1371/journal.pone.0287342 June 15, 2023 14 / 33

Fig 7. Average earnings in different industries of UK. Source: ONS.

PLOS ONE | https://doi.org/10.1371/journal.pone.0287342 June 15, 2023 15 / 33

PLOS ONE | https://doi.org/10.1371/journal.pone.0287342 June 15, 2023 16 / 33

PLOS ONE | https://doi.org/10.1371/journal.pone.0287342 June 15, 2023 17 / 33

Fig 11. UK trade with EU and Non-EU. Source: ONS.

4.5 Neural-network model

PLOS ONE | https://doi.org/10.1371/journal.pone.0287342 June 15, 2023 18 / 33

Fig 13. Losses on each epoch and its difference.

4.6 Box-Jenkins method

PLOS ONE | https://doi.org/10.1371/journal.pone.0287342 June 15, 2023 19 / 33

Fig 14. Decomposition graphs.

Fig 15. Rolling-mean & standard deviation graph.

PLOS ONE | https://doi.org/10.1371/journal.pone.0287342 June 15, 2023 20 / 33

Fig 16. PACF graph.

Fig 17. ACF graph.

PLOS ONE | https://doi.org/10.1371/journal.pone.0287342 June 15, 2023 21 / 33

Fig 18. Residual and density graphs.

Fig 19. Actual vs fitted graph for unemployment.

PLOS ONE | https://doi.org/10.1371/journal.pone.0287342 June 15, 2023 22 / 33

Fig 20. Forecast graph for unemployment.

Fig 21. Forecast graph for earnings.

PLOS ONE | https://doi.org/10.1371/journal.pone.0287342 June 15, 2023 23 / 33

Fig 22. Forecast graph for GDP.

Fig 23. Forecast graph for Trade.

4.7 Public reaction

PLOS ONE | https://doi.org/10.1371/journal.pone.0287342 June 15, 2023 24 / 33

Table 2. Regional popularity of Brexit and COVID-19 under all categories.

PLOS ONE | https://doi.org/10.1371/journal.pone.0287342 June 15, 2023 25 / 33

PLOS ONE | https://doi.org/10.1371/journal.pone.0287342 June 15, 2023 26 / 33

5 Discussion and conclusion

PLOS ONE | https://doi.org/10.1371/journal.pone.0287342 June 15, 2023 27 / 33

making it challenging to discern their individual impacts on the UK economy. Nonetheless,

PLOS ONE | https://doi.org/10.1371/journal.pone.0287342 June 15, 2023 28 / 33

Fig 28. The subjectivity of Brexit tweets.

n Missing value handling Yes Yes Yes