Research Proposal

Research Area: Classification and Regression Using Machine Learning and Deep Learning Approaches
Author: Muhammad Azeem {azeemchaudharyg@gmail.com, +92-3454488058}

Introduction

Machine learning has made great strides in the last two decades, from a laboratory eagerness to a real
technology with commercial applications use. Machine learning has emerged as the most popular method
for producing realistic software for computer vision (Seng, Ang, Schmidtke, & Rogiers, 2018), speech
recognition (Nassif, Shahin, Attili, Azzeh, & Shaalan, 2019), natural language processing (Peng, Harris, &
Sawa, 2018), robot control (Mosavi & Varkonyi-Koczy, 2017), and other applications in artificial intelligence
(AI). Several AI professionals now acknowledge that, among many purposes, training a system by providing
it samples of desired input-output behavior can be significantly easier than programming it explicitly by
predicting the required response for all potential inputs. Supervised learning methods are among the most
extensively utilized machine-learning approaches.

Supervised learning applications, such as e-mail spam classifiers (Panigrahi, 2012), image face recognizers
(Tripathi, 2017), and patient medical diagnosis systems (Picardi, Hawkins, Paterson, & Habli, 2019). In most
cases, supervised teaching methods make predictions using a learned mapping f (x), which generates an
output y for every input x (or an estimate over y given x). Decision trees (Blanco-Justicia & Domingo-Ferrer,
2019), decision forests (Rahman & Islam, 2022), logistic regression (Lee et al., 2018), support vector
machines (Devikanniga, Ramu, & Haldorai, 2020), neural networks (Jiang, Chen, & Fan, 2020), kernel
machines (Tonin, Patrinos, & Suykens, 2021), and Bayesian classifiers (Agrawal & Trivedi, 2021) are all
examples of mapping f. Deep networks, which are multilayer networks of threshold units, each of which
evaluates some fundamental parametric function of its inputs, have been an elevated area of research in
supervised learning in recent times (Gurney, 2018).

Gradient-based optimization algorithms are used by deep learning systems to optimize features across a
whole multilayered network depending on errors at its output. In recent times, large-scale deep learning
systems have had a substantial impact on computer vision (Luo et al., 2018) and speech recognition (Han
et al., 2020), where they have generated significant performance increases over previous techniques. From
natural language translation to collaborative filtering, deep network technologies are now being actively
researched in a range of other fields. In the further sections, different areas will be discussed in which
machine learning and deep learning are taking their parts. Moreover, a brief literature review will also be
presented as well as the proposed area of research. At last, a conclusion will be made to discuss some
future guidance as well.

Background

Medical specialists are usually the ones who examine the assertions of medical data. Due to uncertainty
and the sophistication of the images, a medical expert's ability to analyze images is severely constrained;
large differences occur amongst specialists, and weariness comes in as a result of their excessive job. Deep
learning is considered as giving innovative and reliable methods for medical imaging, which is seen as a key
method for future applications in the medical sector, after its success in other real-world applications. The
technologies such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), long short
term memory (LSTMs), extreme learning machines (ELMs), and generative adversarial networks (GANs) can
be given raw data, have automated attributes, and can train quickly. These algorithms attempt to learn
data several levels of information, presentation, and information from a huge range of images that
demonstrate the required data behavior.

In recent times, image capturing tools have improved significantly; X-ray technology, as well as computer
tomography (CT) and magnetic resonance imaging (MRI) scans, can now give radiological images with much
higher definition. Deep learning is of tremendous interest in every research sector, particularly medical
image analysis; the medical imaging industry is estimated to be worth $300 million by 2021. As a result, by
2021, medical imaging will have received more funding than the entire analysis market did in 2016. It is the
most efficient and supervised machine learning method available. This method employs deep neural
network models, which are a type of neural network that has a lot of similarities to the human brain and
uses more advanced mechanisms than simple neural networks (Razzak, Naz, & Zaib, 2018).

Deep Learning is a major development trend of automated systems, and it was named one of the top ten
advancements of the year in 2013. Several deep learning-based computer vision systems now outperform
humans in identifying cancer signs in blood and tumors in MRI scans. Deep Neural Networks are a type of
artificial neural network (ANN) that has more hidden layers than traditional neural networks, allowing for
a greater abstraction level and better image processing. Due to the remarkable unmatched achievements
in various applications, including object detection (Zhao, Zheng, Xu, & Wu, 2019), speech recognition (Zhang
et al., 2019), face recognition (Duong, Quach, Jalata, Le, & Luu, 2019), and medical imaging (Gibson et al.,
2018), it has become a widely used technology. Different kinds of deep learning models are in use in
research such as CNNs, RNNs, deep belief networks (DBNs), deep Boltzmann machines (DBNs), and deep
conventional extreme machine learning (DC-ELM), etc.

Proposed Research

Convolutional Neural Network (CNN) is the subject of consideration in this proposal. It is to propose that
CNN can perform relatively well for image classification as compared to other ANNs and can be considered
as prosper area of research for digital image processing and vision. CNN captures highly variable
characteristics and patterns all with no preprocessing, according to several research. On each layer, the
CNN features shared weights and replicated filters, as well as local connections. These characteristics help
the models learn more effectively. This would be a great field of research for higher levels and can be
explored with different real-world datasets.

Related Work

Deep learning applied to medical imaging has the potential to be the most revolutionary technology in
radiography since the introduction of digital imaging. Most scholars expect that deep learning-based
systems will substitute people during the next 15 years and that intelligent robots will not only do the
majority of diagnoses, but will also accurately predict disease, prescribe medicine, and lead therapy. In (Vu,
Ho, Yang, Kim, & Song, 2018), an automated and reliable deep learning model based on the CNN method
is presented for detecting and identifying Alzheimer's disease using magnetic resonance imaging (MRI) and
Positron Emission Tomography (PET) pictures dataset which is categorized into three groups such as normal
control, moderate cognitive impairment, and Alzheimer's disease. The model has produced 94.48% average
accuracy. In another work (Islam & Zhang, 2017), an ensemble deep CNN model is developed for detecting
Alzheimer's disease using the Open Access Series of Imaging Studies (OASIS) dataset which acquired 93.18%
of accuracy.

In (Z. Wang et al., 2019) a CAD approach based on deep features with CNN is proposed for the detection of
breast cancer using mammograms images dataset. The proposed model has achieved 81.75% of accuracy
in evaluation. In (Alakwaa, Nassef, & Badr, 2017) a novel CNN-based model is proposed for the detection
of lung cancer using computer tomography (CT) scan images, the proposed model has achieved 86.6% of
accuracy on the given dataset. In (Abiwinanda, Hanif, Hesaputra, Handayani, & Mengko, 2019) a novel CNN
model is proposed to classify images containing normal and abnormal pixels for brain tumor detection. The
proposed model gained 97.87% accuracy, which is very convincing compare to other traditional models. In
(Kalaiselvi, Padmapriya, Sriramakrishnan, & Somasundaram, 2020) a CNN model for classifying benign
tumors is proposed, then trained the model using the BraTS2013 dataset and tested on the WBA dataset,
the proposed model has attained 96-99% accuracy. In (Rajagopalan, Narasimhan, Vinjimoor, & Aiyer, 2020)
a novel CNN model is proposed for accurate detection and classification into normal, drusen macular
degeneration, and diabetic macular edema using Optical Coherence Tomography (OCT) imaging dataset.

Moreover, to reduce intrinsic speckle noise, the Kuan filter is used to input OCT pictures first. Furthermore,
hyperparameters optimization techniques are used to optimize the CNN network and K-fold validation is
also performed to assure that the dataset is fully utilized. The presented model achieved 95.7% of accuracy.
In (Akil, Elloumi, & Kachouri, 2021) CNN-based approach is used to locate, identify, and quantify abnormal
features in eye retinal using OCT images dataset. The proposed model achieved an accuracy of 95.8%. . In
(X. Wang, Lu, Wang, & Chen, 2018) developed a unique retinal disease classification technique based on
CNN using OCT images with noise exposure, however, to reduce noise, the prevalent filter is used, which
delivers inferior consumption and the proposed method has achieved 98.73% accuracy. Different deep
learning models are listed in Table 1 with their specifications, pros, and cons.

Table 1: Comparison of Different Deep Learning Models

Network Type Description Specification Limitation

Deep Neural Network There are multiple layers, It is frequently utilized The training procedure is
(DNN) allowing for complicated and has a high level of not simple because the
non-linear connections. accuracy. error is propagated back
It's utilized for both to the previous
classification and individual layers,
regression. resulting in very minor
errors. The model's
learning process is
likewise far too slow.
Convolutional Neural This network performs The model learns quickly For categorization, it
Network (CNN) exceptionally well with and performs admirably. requires a large amount
two-dimensional data. It is of labeled data.
made up of convolutional
filters that convert 2D to
3D.
Recurrent Neural It is capable of learning There are many Due to a shrinking
Network (RNN) sequences. All stages and variations: learn gradient and the
neurons retain the same sequential events, can necessity for large
weights. represent time datasets, there are
dependencies, and numerous difficulties.
there are many
variations: LSTM,
BLSTM, MDL-STM, and
HLSTM are trademarks
of the LSTM, BLSTM,
MDL-STM, and HLSTM
These offer cutting-edge
accuracy in speech
recognition, character
identification, and a
variety of other NLP
tasks.
Deep Belief Network DBN is utilized in both The greedy technique The training method is
(DBN) supervised and (applied in each layer) computationally
unsupervised machine and the amenable intensive due to the
learning and features a inference maximize the initialization.
unidirectional connection. likelihood directly.
Each sub-hidden
network's layers function
as the feature map for the
next layer.
Deep Conventional For a sample of the local It is frequently utilized If the learning function is
Extreme Learning cuisine, this network's and has a high level of basic and the amount of
Machine (DC-ELM) connections employ a precision. It is a rapid labeled data is minimal,
Gaussian probability and computationally initialization may be
distribution function. efficient training beneficial.
process. It works well
with random distortion.
Methodology

In this research, CNN and RNN deep neural network models will be used to classify medical images. Adam
optimizer will be used to minimize error rate and rectified linear unit (ReLU) will be used as activation
function. Batch normalization technique will be used to reduce gradient vanishing problem will early
stopping techniques as well to overcome time complexity of the model. Radiography images or computed
tomography (CT) images datasets will be used. Python programming will be used with Tensorflow and Keras
frameworks as backend. Dataset is separated into two sets, training, and testing, each with a similar number
of samples from each class. The classification algorithm is validated using two-layer stacked cross-
validation. The ideal classifier is defined as the one that performs best in two-layer nested cross-validation.
The comparison with other state-of-the-art models will also be employed.

Implementation Flow
Step-1: Data Preprocessing

Step-2: Proposed Model

Step-3: Model Training

Step-4: Model Evaluation

Step-5: Model Tuning

Evaluation Criteria

The classifier's performance is evaluated on three components: sensitivity, specificity, and accuracy.
Sensitivity is a metric that assesses the projected output with a change in input. In other words, sensitivity
is the ratio of correctly detected true positives to false positives. This is in contrast to specificity, which is
the ratio of correctly identified true negatives. Accuracy refers to the relationship between the predicted
and actual values. The accuracy of a prediction refers to how near the predicted value is to the actual value.

The following formulas are used to calculate the three components:

𝑇𝑟𝑢𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒𝑠
Sensitivity (%) = * 100
𝑇𝑟𝑢𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒𝑠+𝐹𝑎𝑙𝑠𝑒 𝑁𝑒𝑔𝑎𝑡𝑖𝑣𝑒𝑠
 𝑇𝑟𝑢𝑒 𝑁𝑒𝑔𝑎𝑡𝑖𝑣𝑒𝑠
Specifity (%) = * 100
𝑇𝑟𝑢𝑒 𝑁𝑒𝑔𝑎𝑡𝑖𝑣𝑒𝑠+𝐹𝑎𝑙𝑠𝑒 𝑁𝑒𝑔𝑎𝑡𝑖𝑣𝑒𝑠

𝑇𝑟𝑢𝑒 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒𝑠+𝑇𝑟𝑢𝑒 𝑁𝑒𝑔𝑎𝑡𝑖𝑣𝑒𝑠

Accuracy (%) = * 100
𝑇𝑜𝑡𝑎𝑙 𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑆𝑎𝑚𝑝𝑙𝑒𝑠

Where the True Positive (TP), True Negative (TN), False Positive (FP), and False Negative (FN) are explained in

Table 2.

Abbreviation Description
True Positive (TP) The number of people who diagnosed with
disease or disorders
 Target output = 1
 Network output = 1
False Positive (FP) The number of people who are healthy but
diagnosed with the disease
or disorders
 Target output = 0
 Network output = 1
True Negative (TN) The number of people who are healthy but
diagnosed as healthy
 Target output = 0
 Network output = 0
False Negative (FN) The number of people who have a disease but
are diagnosed as healthy
 Target output = 1
 Network output = 0

Conclusion

Medical images analysis is a key subject before diagnosing a disease. Due to less number of expert
radiologists, computer-aided technologies can fill this gap. In this regard, machine learning and deep
learning are progressing so rapidly to match the line. In this research, some core algorithms shall be used
and fine-tuned for better accuracy and precision. It is assumed that after this research some new directions
shall be explored in the field of artificial neural networks.
References

Abiwinanda, N., Hanif, M., Hesaputra, S. T., Handayani, A., & Mengko, T. R. (2019). Brain tumor
classification using a convolutional neural network. Paper presented at the World
congress on medical physics and biomedical engineering 2018.

Agrawal, P., & Trivedi, B. (2021). Machine learning classifiers for Android malware detection Data
Management, Analytics, and Innovation (pp. 311-322): Springer.

Akil, M., Elloumi, Y., & Kachouri, R. (2021). Detection of retinal abnormalities in fundus image
using CNN deep learning networks State of the Art in Neural Networks and their
Applications (pp. 19-61): Elsevier.

Alakwaa, W., Nassef, M., & Badr, A. (2017). Lung cancer detection and classification with 3D
convolutional neural network (3D-CNN). Lung Cancer, 8(8), 409.

Blanco-Justicia, A., & Domingo-Ferrer, J. (2019). Machine learning explainability through

comprehensible decision trees. Paper presented at the International Cross-Domain
Conference for Machine Learning and Knowledge Extraction.

Devikanniga, D., Ramu, A., & Haldorai, A. (2020). Efficient Diagnosis of Liver Disease using Support
Vector Machine Optimized with Crows Search Algorithm. EAI Endorsed Transactions on
Energy Web, 7.

Duong, C. N., Quach, K. G., Jalata, I., Le, N., & Luu, K. (2019). Mobiface: A lightweight deep learning
face recognition on mobile devices. Paper presented at the 2019 IEEE 10th International
Conference on Biometrics Theory, Applications and Systems (BTAS).

Gibson, E., Li, W., Sudre, C., Fidon, L., Shakir, D. I., Wang, G., . . . Hu, Y. (2018). NiftyNet: a deep-
learning platform for medical imaging. Computer methods and programs in biomedicine,
158, 113-122.

Gurney, K. (2018). An introduction to neural networks: CRC press.

Han, W., Zhang, Z., Zhang, Y., Yu, J., Chiu, C.-C., Qin, J., . . . Wu, Y. (2020). Contextnet: Improving
convolutional neural networks for automatic speech recognition with the global context.
arXiv preprint arXiv:2005.03191.
Islam, J., & Zhang, Y. (2017). An ensemble of deep convolutional neural networks for Alzheimer's
disease detection and classification. arXiv preprint arXiv:1712.01675.
Jiang, J., Chen, M., & Fan, J. A. (2020). Deep neural networks for the evaluation and design of
photonic devices. Nature Reviews Materials, 1-22.

Kalaiselvi, T., Padmapriya, S., Sriramakrishnan, P., & Somasundaram, K. (2020). Deriving tumor
detection models using convolutional neural networks from MRI of human brain scans.
International Journal of Information Technology, 1-6.

Lee, H.-C., Yoon, S. B., Yang, S.-M., Kim, W. H., Ryu, H.-G., Jung, C.-W., . . . Lee, K. H. (2018).
Prediction of acute kidney injury after liver transplantation: machine learning approaches
vs. logistic regression model. Journal of clinical medicine, 7(11), 428.

Luo, H., Xiong, C., Fang, W., Love, P. E., Zhang, B., & Ouyang, X. (2018). Convolutional neural
networks: Computer vision-based workforce activity assessment in construction.
Automation in Construction, 94, 282-289.

Mosavi, A., & Varkonyi-Koczy, A. R. (2017). Integration of machine learning and optimization for
robot learning Recent Global Research and Education: Technological Challenges (pp. 349-
355): Springer.

Nassif, A. B., Shahin, I., Attili, I., Azzeh, M., & Shaalan, K. (2019). Speech recognition using deep
neural networks: A systematic review. IEEE Access, 7, 19143-19165.

Panigrahi, P. K. (2012). A comparative study of supervised machine learning techniques for spam
e-mail filtering. Paper presented at the 2012 Fourth International Conference on
Computational Intelligence and Communication Networks.

Peng, T., Harris, I., & Sawa, Y. (2018). Detecting phishing attacks using natural language
processing and machine learning. Paper presented at the 2018 IEEE 12th international
conference on semantic computing (icsc).

Picardi, C., Hawkins, R., Paterson, C., & Habli, I. (2019). A pattern for arguing the assurance of
machine learning in medical diagnosis systems. Paper presented at the International
Conference on Computer Safety, Reliability, and Security.

Rahman, M. G., & Islam, M. Z. (2022). Adaptive Decision Forest: An incremental machine learning
framework. Pattern Recognition, 122, 108345.
Rajagopalan, N., Narasimhan, V., Vinjimoor, S. K., & Aiyer, J. (2020). Deep CNN framework for
retinal disease diagnosis using optical coherence tomography images. Journal of Ambient
Intelligence and Humanized Computing, 1-12.

Razzak, M. I., Naz, S., & Zaib, A. (2018). Deep learning for medical image processing: Overview,
challenges and the future. Classification in BioApps, 323-350.

Seng, K. P., Ang, L.-M., Schmidtke, L. M., & Rogiers, S. Y. (2018). Computer vision and machine
learning for viticulture technology. IEEE Access, 6, 67494-67510.

Tonin, F., Patrinos, P., & Suykens, J. A. (2021). Unsupervised learning of disentangled
representations in deeply restricted kernel machines with orthogonality constraints.
Neural Networks, 142, 661-679.

Tripathi, B. K. (2017). On the complex domain deep machine learning for face recognition. Applied
Intelligence, 47(2), 382-396.

Vu, T.-D., Ho, N.-H., Yang, H.-J., Kim, J., & Song, H.-C. (2018). Non-white matter tissue extraction
and deep convolutional neural network for Alzheimer’s disease detection. Soft
Computing, 22(20), 6825-6833.

Wang, X., Lu, Y., Wang, Y., & Chen, W.-B. (2018). Diabetic retinopathy stage classification using
convolutional neural networks. Paper presented at the 2018 IEEE International
Conference on Information Reuse and Integration (IRI).

Wang, Z., Li, M., Wang, H., Jiang, H., Yao, Y., Zhang, H., & Xin, J. (2019). Breast cancer detection
using extreme learning machine based on feature fusion with CNN deep features. IEEE
Access, 7, 105146-105158.

Zhang, W., Cui, X., Finkler, U., Kingsbury, B., Saon, G., Kung, D., & Picheny, M. (2019). Distributed
deep learning strategies for automatic speech recognition. Paper presented at the ICASSP
2019-2019 IEEE International Conference on Acoustics, Speech, and Signal Processing
(ICASSP).

Zhao, Z.-Q., Zheng, P., Xu, S.-t., & Wu, X. (2019). Object detection with deep learning: A review.
IEEE transactions on neural networks and learning systems, 30(11), 3212-3232.