2019 IEEE 7th International Conference on Computer Science and Network Technology (ICCSNT)
Experimental Disease Prediction Research on
Combining Natural Language Processing and
Machine Learning
Hong Qing Yu
School of Computer Science and
Technolgy
University of Bedfordshire
Luton, United Kingdom
hongqing.yu@beds.ac.uk
Abstract— Nowadays Artificial Intelligent (AI) technologies
are applied widely in many different areas to assist knowledge
gaining and decision-making tasks. Especially, health
information system can get most benefits from the AI
advantages. In particular, symptoms based disease prediction
research and production became increasingly popular in the
healthcare sector recently. Various researchers and
organizations have turned their interest in using modern
computational techniques to analyze and develop new
approaches that can efficiently predict diseases with reasonable
accuracy. In this paper, we propose a framework to evaluate
the efficiency of applying both Machine Learning (ML) and
Nature Language Processing (NLP) technologies for disease
prediction system. As an example, we scraped a disease-
symptom dataset with NLP features from one of the UK most
trustable National Health Service (NHS) website. In addition,
we will exam our data in depth having symptom frequency,
similarity and clustering analysis. As result, we can see that the
prediction can have a very positive efficient rate but still open
issues need to be addressed.
Keywords— Artificial Intelligent, Data analysis, Machine
Learning, health, Nature Language Processing
I. INTRODUCTION
There are more and more medical and healthcare related
products available on mobile and web applications
nowadays. Many machine learning research work start to use
the information they captured from online websites such as
social media information, forum communications and many
other resources to create AI supported healthcare
recommended applications. The research results were very
encouraging and these AI applications can provide helpful
tips or even pre-diagnose advices based on very simple
datasets e.g. condition and symptom relational datasets [1].
However, most of the work lacks of transparent framework
for managing the data and applying ML algorithms on the
trust and reliable data. Researches in the past like, predicting
cancer using machine learning and clustering, predicting
dermatological conditions using naïve Bayes classifier [2]
and predicting occurrence of swine flu using naïve Bayes
classification as well [3] have been done and these
approaches seem to produce reasonably good results.
However, most of these researches are narrowed down one
or a few diseases or medical conditions that are very crucial
in the healthcare field. Some investigations, however,
proposed new methods which can predict more disease based
on symptoms, but their reliability and accuracy are still being
a significant concern. Therefore, the research we do in this
paper is to develop an evaluable and transparent disease
prediction framework by applying combinative most
advanced technologies as experimental case study to exam
the quality and issues of the result in order to conclude a
978-1-7281-3299-0/19/$31.00 ©2019 IEEE 145
Authorized licensed use limited to: REVA UNIVERSITY. Downloaded on January 31,2024 at 05:38:11 UTC from IEEE Xplore. Restrictions apply.
future research path. The proposed approach can efficiently
predict various diseases with 90% accuracy. The paper is
organised as following:
Section II will discuss the related work. Section III
introduces the information extracting and NLP process.
Section IV will explore the dataset features and information
quality for ML. Section V will introduce the prediction ML
algorithms and experimental results. Section VI will discuss
the experimental results for further open issues. Finally, the
conclusion will be drawn at section VII.
II. RELATED WORK
Support Vector Machines [4] is a popular supervised
learning algorithm that can be used for both classification
and regression problems. The concept behind this algorithm
is to find extremities in the dataset by using a hyperplane
(decision surface) that can detect and separate different
features into different classes. Due to its higher accuracy
when compared to other machine learning algorithms, SVM
has become one of the popular machine learning algorithms
that are used in the medical diagnosis field.
Chen et al. conducted a study using three multivariate
models to find patients with Chronic Kidney Disease (CKD)
[5]. The study used three ML algorithms K-Nearest
Neighbour (KNN), Support Vector Machine (SVM) and Soft
Independent Modelling of Class Analogy (SIMCA) using the
clinical dataset from UCI Machine learning repository. It is
found out that SVM had 99% accuracy rate in predicting
CKD when compared with KNN, SIMCA with 98.2% and
85.9% accuracy rate respectively. Besides, it is also essential
to specify that the SVM had a quicker processing time
compared to the other two models. Almansor et al., on the
other side, suggest that when comparing ANN and SVM in
predicting CKD, ANN performed better with higher
accuracy, also taking longer runtime [6]. However, the study
finally concluded that there is no significant difference in the
accuracy rate.
Based on the previous researches, it is safer to conclude
that the SVM tends to produce higher accuracy rate while
comparing with other machine learning models. However,
SVM has few significant drawbacks; the higher accuracy rate
comes with the cost of a long training period. Also, SVM can
be only be applied for the two-class classification problem,
i.e. given the symptoms SVM can only be able to classify if
the given symptoms relate to the disease or not.
Naive Bayes [7] is also a popular ML algorithm primarily
used for text classification. It uses probabilistic classifiers
based on the principles of Bayes Theorem. It works based on
the probability distributions of variables in a dataset. While
Dalian, China•Oct 19-20, 2019
providing new attributes, the algorithm will predict the
probability of the response variable data belonging to a
particular class.
Pattekari & Parveen [8] conducted research to predict
heart disease using machine learning and data mining
techniques, and this research aims to develop an intelligent
support system for the doctors which can predict the heart
disease provided using Naïve Bayes Algorithm, based on the
various input attributes. The work was concluded by stating
that the proposed approach is most accurate solution to
predict patients with heart disease. However, the work didn’t
seem to use any real-world datasets or use cases to support
the statement and also, no evidence of results can be found.
Thus, the question around the work is the reliability.
An approach that can predict breast cancer using
Weighted Naï ve Bayes (WNB) algorithm was proposed [2],
along with the Wisconsin Breast cancer dataset. To reach a
higher accuracy rate, the author used a grid search algorithm
to find an optimum weight value for each attribute in the
dataset. While comparing with the traditional Naïve Bayes
approach, the proposed method resulted in performing better
with higher accuracy of 98.54%, sensitivity and specificity
rate although it is also to be noted that the proposed plan was
computationally expensive and application dependent.
Rashid et al. in 2016 proposed an Automatic disease
prediction system (ADPS) using Naïve Bayes Algorithm [9],
unlike other researches, focusing on one particular disease,
the proposed study concentrates on general disease
prediction. The approach uses Natural Language Processing
(NLP) to process the user input and tagging the keywords
based on their attribute type, i.e. 1.) Symptom 2.) Time 3.)
Intensity 4.) Organ 5.) Duration and stores them as Relevant
Attribute Array (RA). This helps the algorithm to compute
efficiently with higher accuracy and ranking the possible
diseases list. The research concluded with 14.35% higher
accuracy in comparison with other existing solutions. Also,
only four conditions were tested in the system, and the
dataset used in this approach is not available for other
researchers to evaluate.
However, there is one major drawback in Rashid et al.
approach. The author states that if any word or term in user
input cannot be determined during the NLP process, then that
particular word or phrase will be assumed to have no
apparent significance and will be avoided. Though it is
relatively reasonable to opt-in such an approach to prevent
the noisy inputs and improve the prediction accuracy,
however, chances are there that this approach might avoid
some critical factors due to the misspelling of input data. In
that case, this would affect the overall accuracy of the
prediction process.
K-Nearest Neighbour (KNN) [10] is a popular machine
learning algorithm used to solve both classification and
regression problems. The algorithm aims to find the nearest
neighbours of a given unclassified input for a given feature
space. This is achieved by measuring the distance between
the input data and the classified data which already exists
using various distance metric techniques like Euclidian,
Manhattan or Weighted etc. A critical factor in this algorithm
is determining the value of K. K is user-defined constant and
also known as the query point that determines the number of
neighbours ‘voting’ for new input data.
146
Authorized licensed use limited to: REVA UNIVERSITY. Downloaded on January 31,2024 at 05:38:11 UTC from IEEE Xplore. Restrictions apply.
Various researchers in recent times have turned their
interest in using the KNN algorithm for disease prediction
problems. Princy & Thomas in 2016 proposed a heart disease
prediction system using data mining techniques focusing
mainly on the KNN and Iterative Dichotomiser 3 (ID3)
algorithm [11]. The research suggests that KNN produces
good consistency result in disease prediction when compared
to other machine learning algorithms. The study, however,
didn’t provide any information about the dataset which was
used. It is concluded that this approach provided a prediction
accuracy of 80.6%.
Tayeb et al. in 2017 researched predicting chronic kidney
disease and heart disease using the KNN algorithm [12]. This
research used the datasets from UCL’s Irvine Machine
learning repository. The most crucial step of the approach is
to choose the value of K for KNN. It is suggested that the
value of K is critical in this type of approach as it made a
substantial change in the accuracy rate. The research
concluded that the approach could successfully predict CKD
and Heart disease with 90% accuracy rate, which is better
than previous studies which also uses the same dataset.
KNN, however, has some common drawbacks comparing
to other classification algorithms, Tayeb et al. suggest that
the algorithm will be computationally expensive as it needs
to calculate the distance of each test instance to all training
samples. Another drawback is that the accuracy rate will be
primarily affected when using datasets with low proximity
between different classes. Also, the accuracy rate is affected
when using multidimensional datasets with noisy data.
Our proposed framework will investigate, analyze
existing solutions and develop an efficient framework that
can be used to predict general diseases or medical conditions
based on the symptoms provided. The project uses NLP, K-
Means and Multinomial Naï ve Bayes Algorithm together to
predict 298 different medical conditions. The framework has
three major components:
1. Data extracting and NLP processing
2. Data evaluation and clustering
3. Prediction
We will discuss components details in next three sections.
III. INFORMATION EXTRACTING
For the data extracting and processing component, we
developed a data “web-crawling” engine first that can
extract disease information automatically and provide
symptom descriptions from UK NHS website (see Figure 1
shows Hay fever information as an example).

Fig. 1. Condition information example from NHS website
Figure 2 shows the process of our data extracting process,
which majorly include three steps: Website crawling to
generate raw data of conditions and their symptoms (see
Figure 3), data cleansing/merging and nature language
processing (NLP) for ML.
Fig. 2. Data extracting process
By studying the raw data, we found that there are a lots
of repeated condition names with different symptoms.
Therefore, we need to clean up the raw data to assist NLP
step efficiently. The clearn up tasks include delete duplicated
raw data and merging the symptoms grouped by condition
names.
The NLP tasks are achieved by coding with Python's
‘nltk’ library. The process involves four major pre-
processing methods 1.) Capitalisation 2.) Stop word removal
3.) Tokenisation and 4.) stemming. Figure 3 shows parts of
the final results.
Fig. 3. Cleaned data with NLP semantic tokens
147
Authorized licensed use limited to: REVA UNIVERSITY. Downloaded on January 31,2024 at 05:38:11 UTC from IEEE Xplore. Restrictions apply.
Capitalization is an important step in text pre-processing
as the text data often involves a mixture of capitalized words.
For instance, a computer program will consider “Nausea”
and “nausea” as two completely different words even though
they possess the same meaning. A simple approach to solve
this issue would be converting all the data into lowercase.
Stop word removal helps to get rid of the meaningless
words in dataset. For example, filler words like ‘of’ or ‘to’
hold no meaning and are often not important. In addition,
stop word removal will also reduce the dimensional space
and provide a good advantage when using bag of words
method e.g. term-frequency inverse document frequency [13].
Tokenisation is a very straightforward process which
helps in converting paragraphs into sentences or sentence
into words. In the dataset the symptoms for each condition
are stored in paragraphs. Using the NLTK tokenisation
method, the paragraphs are spilt into tokens of sentences
making it easier to use for the clustering and classification
algorithms.
Stemming, in simple words, is a process of removing the
possible suffix of a word and converting into its root stem.
This process helps normalise the data and improves the
quality of dictionary. In order to achieve this, an inbuilt
stemming algorithm named Snowball stemmer was used.
IV. INFORMATION EVALUATION
From data extracting process, we extracted 3691 records
of raw data and cleaned up 298 condition records with their
related symptoms and NLP tokens. We would like to have
two evaluations on our disease-symptom dataset: finding the
most frequent symptoms for conditions and clustering the
conditions based on the semantic similarities.
To demonstrate the first evaluation, we use WordCloud
figure (see Figure 4) to show the most frequent appearing
symptoms extracted from NLP word and sentences tokens.
Then we can see that “loss of appetite”, “tiredness”, “a high
temperature”, “feeling Sick or vomiting”, “diarrhoea”,
“dizziness” and “loss hair/weight” and “confusion” are
clearly showing as most common symptoms. However,
frequent analysis cannot explain the whole story and
symptoms are closed link to the conditions. In another word,
symptoms are scattered according to different types of
condition. Therefore, the clustering analysis is required.
Fig. 4. Symptom Wordcloud
Based on the number of conditions (298), we evaluated
the two different clustring algorithms that are k-mean
clustering and LDA (Latent Dirichlet Allocation) algorithms
[14]. The evaluation result shows that k-mean clustering
algorithm is more suitable for this clustering task. We
automatically tested different numbers of clusters to find the
best k-mean clustering result. The automation testing process
is presented in Figure 5 and the k-mean clustering results
represent in Figure 6 as plotpot graph with 15 clusters. The
largest size cluster contains 32 conditions and the smallest
two cluster contains 6 and 9 conditions. To make the
understanding more easily on the quality of the clusters, two
smallest clusters are discussed here and they are cluster 8 and
13 (see table 2) . Checking cluster 8 (C8), we can see that
most of the condition is homologous (e.g. Acute
lymphoblastic leukaemia) but for different age group or vas
versa. Cluster 13 (C13) is not too obvious, however we can
see that the conditions are related to each other’s (e.g.
Anxiety, Panic disorder, OCD and PTSD).
Fig. 5. Clustering process
Fig. 6. Clustering plot graph
148
Authorized licensed use limited to: REVA UNIVERSITY. Downloaded on January 31,2024 at 05:38:11 UTC from IEEE Xplore. Restrictions apply.
TABLE I. SMALLEST CLUSTERS
Example clusters Disease names in the cluster
Cluster 8 1. Acute lymphoblastic leukaemia: Children
2. Acute lymphoblastic leukaemia: Teenagers and
young adults
3. Acute myeloid leukaemia: Children
4. Acute myeloid leukaemia: Teenagers and young
adults
5. Non-Hodgkin lymphoma: Children
6. Spleen problems and spleen removal
Cluster13 1. Anxiety
2. Fibromyalgia
3. Hypoglycaemia (low blood sugar)
4. Migraine
5. Obsessive compulsive disorder (OCD)
6. Panic disorder
7. Post-traumatic stress disorder (PTSD)
8. Peripheral neuropathy
9. Menopause
V. THE PREDICTION METHODOLOGY
Fig. 7. Clustering plot graph
Figure 7 shows the disease prediction methodology by
giving a set of symptoms. We had discussed NLP and
Clustering parts in previous sections. This section focuses on
the last prediction component using Multinomial Naï ve
Bayes (MNB) probability classification method.
The first step is calculating the prior probability of each
class in the training dataset. This can be achieved by using
the formula below.
Explanation, in order to calculate the prior probability of
each class in the training dataset, we take total number of
times the class occurred divided by size of training set.
The second step is to find the likelihood of a symptom
occurring in a particular class this can be achieved by a
simple calculation.
Explanation, in order to find the likelihood of a symptom
occurring in a particular class, we take the total number of
times the symptom occurred in that class with Laplace
smoothing 1 and divide it by the sum of total number of
symptoms in that class and size of the vocabulary of that
class i.e. total number of distinct symptoms in that class.
After calculating the required parameters, it is now
possible to determine the class of new unseen data. This can
be achieved by the following technique.

Explanation, to find the probability score of the new
input belonging to a class, we multiply prior probability of
that class with the likelihood of the input within that class.
For example, in the project’s context, the formula for
calculating the probability of a user given symptom can be
calculated like:
The process will be repeated with each class and their
resulting scores will be used to determine prediction results.
VI. DISCUSSIONS
For the evaluating this project, three similar existing
online commercial products are chosen and they are
mayoclinic.org, medicinenet.com and heathdirect.gov.au.
Figure 8 shows the interface of our implemented C-Predict
system. A comparison on the overall functionalities of each
system was made aiming to evaluate the accuracy and
efficiency of proposed method. Testing cases of 20 different
conditions with specific symptoms was obtained from nhs.uk.
The final result shows all other systems have relative
accuracy rate around 80% and our proposed framework can
reach 93% as the highest. However, here we only talk about
relative accuracy and there are many issues around current
technologies.
Fig. 8. The C-Predict system interface
First is the undetermined prediction problem. If there is
only couple of symptoms provided, then the prediction result
will have multiple suggestions with same probability scores.
For example, if the inputs are fever, dizzy and cough, then
about 20 results will be displayed with very similar
prediction scores. However, many of the prediction results
have their specific context. For instance, the highest score is
Sinusitis, but its common major signs are thick nasal mucus,
a plugged nose and pain in the face. Fever, dizzy and cough
can be the side effects of the major signs. Thus, the
comprehensive knowledge modelling techniques are required
149
Authorized licensed use limited to: REVA UNIVERSITY. Downloaded on January 31,2024 at 05:38:11 UTC from IEEE Xplore. Restrictions apply.
to enable specifying the detail information about symptoms
and diseases rather than just calculating the similarities of
NLP token matching.
Second key issue is the black box prediction process,
which means the algorithm cannot explain the results in a
transparent way to tell why (the reason) one disease has
higher scores than others. This is also the general issues of
current ML algorithms [15], where you can only specify the
computation algorithm but difficult to have a traceability for
evaluating the reason behind it. In contrast, finding why is
the most import task in healthcare research. For example, if a
person has more than 7 days high temperature, understanding
what reasons caused this condition is critical, e.g. the reason
of long period high fever can be infections, however what
causes the infection.
Finally, the learning and prediction framework are static
and fixed to certain dataset. In another words, the prediction
model is hard to be updated unless the whole process is
redone.
Therefore, there is a long way to go in this research area.
As short-term goal, the future research should integrate
semantic technologies like semantic web data (or called
knowledge graph) with NLP tokens to structure machine
understandable graph data, e.g. to indicate position of the
symptom, side effects of using drugs and relations between
different diseases and so on. In addition, new logic models
like causal analysis logic should be considered to enhance
the causal reasoning capability. Transparent probability
distribution algorithm should also been injected into the
framework.
VII. CONCLUSION
In the paper, we presents an experimental evaluation
research framework for developing a symptom-based disease
prediction system. We extracted the 298 disease information
from UK most trust NHS website and applied advanced NLP
and ML algorithms in the framework. The relative accuracy
can achieve 93%. However, there are many remaining issues
around this research area, which are determination, black box
reasoning and difficulties of updating knowledge. Therefore,
we need to guide our research to directions of semantic
enhancement, knowledge graph, Open Linked Data with
traceable causal analysis logic frameworks.
REFERENCES
[1] Kaggle.com, “Heart Disease UCI”, 2018. [Online]. Available at:
https://www.kaggle.com/cdabakoglu/heart-disease-classifications-
machine-learning/data. [Accessed: 23- July- 2019].
[2] M. Karabatak and M. C. Ince, “An Expert System for Detection of
Breast Cancer Based on Association Rules and Neural Network,”
Expert Systems with Applications, 2009, 36(2), 2009, 3465-3469
[3] B. A. Thakkar, M. I. Hasan, M. A. Desai, “Healthcare decision
support system for swine flu prediction using naïve bayes classifier”,
In Proceedings of the 2010 International Conference on Advances in
Recent Technologies in Communication and Computing (ARTCOM
'10). IEEE Computer Society, Washington, DC, USA, 2010, pp. 101-
105.
[4] H. William, S. A. Teukolsky, A. Saul, T. W. Vetterling, B. P.
Flannery, "Support Vector Machines". Numerical Recipes: The Art of
Scientific Computing (3rd ed.). New York: Cambridge University
Press. ISBN 978-0-521-88068-8. Section 16.5, Archived from the
original on 2011-08-11.
[5] Chronic kidney disease: KDIGO CKD-MBD guideline update:
evolution in the face of uncertainty. Wei Chen, David A. Bushinsky,
Nat Rev Nephrol. 2017 Aug 21 Published online 2017 Aug 21. doi:
10.1038/nrneph.2017.118
[6] N. Almansour, H. Syed, N. Khayat, R. Altheeb, R. Juri, J. Alhiyafi, S.
Alrashed and S. Olatunji. “Neural network and support vector
machine for the prediction of chronic kidney disease: A comparative
study”, Computers in Biology and Medicine, 2019, 109, pp.101-111.
[7] I. Rish, “ An empirical study of the naive Bayes classifier”. IJCAI
Workshop on Empirical Methods in AI, 2001. [Online]. Available at:
https://www.cc.gatech.edu/~isbell/reading/papers/Rish.pdf [Accessed:
27- July- 2019].
[8] S. Pattekari and A. Parveen, (2019). “Prediction System for Heart
Disease Using Naïve Bayes”. International Journal of Advanced
Computer and Mathematical Sciences, [online] 3(3), pp.290-294.
Available at: https://pdfs.semanticscholar.org/ d32e/ e90a5de89093
a4fc95f43e0409cb91414726. pdf [Accessed 31 May 2019].
[9] Md. T. R. Laskar, Md. T. Hossain, A. R. M. Kamal and N. Rashid,
“Automated Disease Prediction System (ADPS): A User Input-based
Reliable Architecture for Disease Prediction”. International Journal
of Computer Applications 133(15):24-29, January 2016. Published by
Foundation of Computer Science (FCS), NY, USA
[10] D. Coomans, D. L. Massart. "Alternative k-nearest neighbour rules in
supervised pattern recognition: Part 1. k-Nearest neighbour
classification by using alternative voting rules". Analytica Chimica
Acta. 1982, 136: 15–27. doi:10.1016/S0003-2670(01)95359-0.
[11] J. Thomas and R. T. Princy, "Human heart disease prediction system
using data mining techniques," 2016 International Conference on
Circuit, Power and Computing Technologies (ICCPCT), Nagercoil,
2016, pp. 1-5. doi: 10.1109/ICCPCT.2016.7530265
[12] S. Tayeb, M. Pirouz, J. Sun, K. Hall, A. Chang, J. Li, C. Song, A.
Chauhan, M. Ferra, T. Sager, J. Zhan and S. Latifi, “Toward
predicting medical conditions using k-nearest neighbors”. 2017 IEEE
International Conference on Big Data (Big Data). [online] Available
at: https://ieeexplore.ieee.org/document/8258395 [Accessed 31 May
2019].
[13] A. Aizawa, "An information-theoretic perspective of tf–idf measures".
Information Processing and Management. 39 (1): 45–65. doi: 10.
1016/ S0306-4573(02)00021-3.
[14] Blei, David M.; Ng, Andrew Y.; Jordan, Michael I (January 2003).
Lafferty, John (ed.). "Latent Dirichlet Allocation". Journal of
Machine Learning Research. 3 (4–5): pp. 993–1022. doi: 10. 1162/
jmlr. 2003. 3.4-5.993.
[15] C. Zednik, “Solving the Black Box Problem: A Normative
Framework for Explainable Artificial Intelligence”, 2019. Available
at: https://arxiv.org/ftp/arxiv/papers/1903/1903.04361.pdf [Accessed
31 July 2019]

150
Authorized licensed use limited to: REVA UNIVERSITY. Downloaded on January 31,2024 at 05:38:11 UTC from IEEE Xplore. Restrictions apply.