Symptom Checker Application

Presented by
Pattranit Teerakoson st123050
Nutdanai Sritunya st123055
Nattabude Tanasansurapong st123131
Bao Chen st124448

Examination Committee
Prof. Chaklam Silpasuwanchai

Machine Learning Class

 2

1. Introduction (PHASE 1: PROPOSAL)

In Thailand, as more and more foreigners visit, there's a growing need for easy-to-use
health tools that cater to diverse languages. Many visitors want a quick way to understand
any health worries they might have while being away from home. Our app is designed for
this very purpose. It lets users check their symptoms in their preferred language and even
guides them to the nearest hospital, ensuring they feel safe and cared for during their stay.

2. Related work
Digital and web-based symptom checker tools are software applications that allow
users to enter their symptoms and personal information to generate a list of potential
diagnoses and medical guidance. The symptom checker industry is experiencing dynamic
growth, driven by advancements in AI, natural language processing, and data analytics.
Symptom checker applications can be classified into two main types based on their user
interaction methods:
Checklist/Question-based Symptom Checkers:
1. Employ predefined decision trees and rule-based algorithms.
2. Guide users through structured question-and-answer sessions.
3. Match responses to predefined patterns, ensuring systematic assessment.
In the checklist format, users match their symptoms to a list of scenarios, guiding them to
appropriate care options based on severity. This systematic approach educates users about
symptom severity and empowers them to learn self-care

Fig 1: checklist format in Symptomate application

 3

Chatbot-based Symptom Checkers:

1. Utilize NLP and AI for user input interpretation.
2. Process natural language, grasp user intent, and provide interactive responses.
3. Mimic human-like conversations for a more engaging experience.
Using a conversational chatbot, patients engage in a series of friendly, text-like
interactions, receiving guidance on their symptoms and appropriate healthcare steps.
Although the chat format aids in guiding users through their assessments, the lack of
transparency regarding the factors influencing the final outcome remains a drawback. Some
advanced bots leverage AI to enhance diagnostic accuracy, particularly for rare conditions,
yet prolonged questioning, and uncertainty about the process endpoint may lead to user
disengagement.
Companies like Ada Health, Buoy, and Babylon are hoping AI will offer a marked
improvement on the current status quo of Googling symptoms or using decision-tree based
symptom checkers like WebMD — not to replace doctors, but to provide a more robust first
step on the healthcare journey.

Fig 2: Chatbot-based Symptom Checkers

 4

3. Problems
1. Diagnostic Accuracy: Ensuring reliable and accurate diagnoses remains a significant
challenge for symptom checker apps. The diagnostic accuracy of the primary diagnosis
was low across included studies (range: 19 - 37.9%) and varied between individual
symptom checkers, despite consistent symptom data input. Triage accuracy (range:
48.8–90.1%) was typically higher than diagnostic accuracy. Overall, the diagnostic and
triage accuracy of symptom checkers are variable and of low accuracy.(Wallace,2022)
Physicians demonstrated significantly higher diagnostic accuracy compared to computer
algorithms, with an 84.3% versus 51.2% according to Semigran’s (2016) research
comparison on clinical vignettes.
2. User Misinterpretation and Over-Reliance: Users may misinterpret the provided
information or over-rely on the app's output without seeking professional medical advice.
This reliance can lead to delays in seeking appropriate medical care or unnecessary
anxiety based on potentially inaccurate self-diagnoses.
3. Data Privacy and Security Concerns: Symptom checker apps often handle sensitive
user health data. Ensuring robust data privacy and security measures to protect this
information from breaches or unauthorized access is a critical challenge for developers
and healthcare providers, requiring adherence to strict regulatory standards and
compliance measures.

Reference: Wallace, W., Chan, C., Chidambaram, S. et al. The diagnostic and triage accuracy of digital and
online symptom checker tools: a systematic review. npj Digit. Med. 5, 118 (2022).
https://doi.org/10.1038/s41746-022-00667-w

Semigran, Hannah L., et al. "Comparison of physician and computer diagnostic accuracy." JAMA internal
medicine 176.12 (2016): 1860-1861.
 5

4. Solutions
4.1 Rationale
In Thailand, the influx of foreign visitors highlights a pressing need for accessible
health tools that cater to diverse languages and cultural nuances. Our symptom checker
application addresses this gap, providing immediate healthcare insights in multiple
languages and guiding users to nearby medical facilities. This tool not only empowers
visitors to take charge of their health in unfamiliar territory but also enhances Thailand's
reputation as a tourist-friendly destination that prioritizes the well-being of its guests.
4.1.1 Value Proposition
● Healthcare Access (The Symptom Checker application offers an initial symptom
assessment and recommends possible conditions for the user to consider. In order not
to waste time contacting the hospital.)
● Find the location of nearby hospitals. (For some people who have an emergency
situation in an unfamiliar place)
● It is easily accessible for domestic people or foreigners who come to live in Thailand.
Because there is a language changing system
4.1.2 Target Audience
● For Everyone: The app is for anyone, helping them quickly check their symptoms and
get guidance on common illnesses and when to see a doctor.
4.1.3 Requirement Elicitation
1. High level requirements
● User-Friendly Interface
● Timely Responses
● Comprehensive Information
2. Measurement
● User Satisfaction Surveys
● Response Time Monitoring
● Content Database Check
4.1.4 Feasibility
● We can finish classification and Docker as planned. But the website might not look as
pretty as we talked about in our proposal. It's because we have limited time.
4.1.5 Risks
● Complex or Rare Conditions: Symptom checkers may not be as effective in
diagnosing complex or rare medical conditions that require in-depth medical
knowledge and diagnostic tests.
● Lack of Trust: Some individuals may be skeptical of the accuracy and reliability of
symptom checker applications, leading them to prefer consulting with healthcare
professionals they trust.
● Limited Scope: Symptom checkers may not cover all possible health conditions or
may not consider rare or newly emerging diseases.
 6

4.2 Mockup: using Figma Application

Fig 3: Main page Fig 4: Find Hospital page Fig 5: Description page

Fig 6: Main symptom page Fig 7: Questionnaire page Fig 8: Final page
 7

4.3 Software Architecture Diagram

Fig 10: Main symptom page

4.4 Team Assignment

For our team assignments, we use ASANA to delegate tasks. We hold meetings every
Monday and Friday. On Mondays, we plan our weekly objectives and assign tasks to each
member, documenting these in the "Sprint Planning" section. Fridays are reserved for
summarizing our accomplishments for the week. The "Meeting Archive" is used to record
meeting descriptions and summaries for future reference. Lastly, the "Done Task" section
archives tasks we've completed during the week.

Fig 11: Main symptom page

 8

4.5 Cooperation with hospitals

Cooperating with hospitals would definitively enhance our symptom checker app. Below are
some ideas for collaborating with hospitals to provide more value to users:
4.5.1. Integration with Hospital Directories:
- Collaborate with hospitals to integrate their directories into the app. This would allow
users to not only check their symptoms but also find nearby hospitals and clinics. Ensure that
the information is regularly updated to provide accurate and reliable data.
4.5.2 Appointment Scheduling:
- Partner with hospitals to implement an appointment scheduling feature. Users who
identify a need for medical attention through the symptom checker can seamlessly schedule
appointments with the recommended hospitals or clinics directly through the app.
4.5.3 Emergency Services Integration:
- Work with hospitals to integrate emergency services information into your app. In case of
urgent situations, users can quickly locate the nearest emergency room or contact emergency
services directly from the app.
4.5.4. Language Support:
- collaborate with hospitals to ensure that their services are also accessible in multiple
languages. Provide information about hospitals, clinics, and departments in languages
commonly spoken by tourists and residents.
4.5.5. Telehealth Services:
- Explore partnerships with hospitals that offer telehealth services. Users can receive virtual
consultations through the app, providing an additional layer of convenience, especially for
non-emergency cases.
4.5.6. Health Records Integration:
- Work on integrating health record systems with your app. This would enable users to store
and access their medical history, making it easier for healthcare providers to understand their
health background during visits.
4.5.7. Feedback and Ratings:
- Implement a feature that allows users to provide feedback and ratings for hospitals and
clinics they visit through your app. This can help other users make informed decisions and
improve the overall user experience.
4.5.8. Privacy and Data Security:
- Work closely with hospitals to ensure that all data exchanged between your app and the
hospitals' systems comply with privacy and data security regulations. Build trust by
transparently communicating how user data is handled.
 9

By incorporating these features and collaborations, they can evolve into a comprehensive
healthcare tool that not only assists users in identifying symptoms but also facilitates a
seamless journey towards appropriate medical care.

5. Evaluation
5.1 Performance
● Accuracy: Given the need for "Comprehensive Information", it's crucial that our
product provides correct and reliable results. Accuracy will determine how often our
product's outputs are correct compared to a gold standard.
● Inference Speed (Response Time): In line with the "Timely Responses" requirement,
we will measure the speed at which our product provides results after a user's input. A
faster inference speed ensures that users receive timely information.
5.2 Human
● Satisfaction: We will conduct user surveys to gauge satisfaction levels. This will give
insights into whether our "User-Friendly Interface" and "Comprehensive Information"
requirements are being met to the users' expectations.
● Ease of Use: Directly mapping to the "User-Friendly Interface" requirement, we'll
assess how easy and intuitive it is for users to navigate and use our product. This can
be evaluated through user feedback forms and usability tests.
● Preference: By comparing our product with similar tools in the market, we can
understand users' preferences. Do they prefer our interface, the comprehensiveness of
our information, and the timeliness of our responses over other tools?

6. Actual Solution (PHASE 2:PROGRESS REPORT)

The app can guide users from symptom checking to appointment scheduling and
online consultation, providing a seamless healthcare experience with cooperation with
hospitals.
Below is some user case description and relevant functions that need to be implemented.
● User Case Description:
- User has Cough Symptoms: User opens the app due to coughing symptoms.
- Symptom Checker Feature: App provides a symptom checker feature for
initial assessment.
- Recommendation for Nearby Hospital or Lung Department:Based on
symptoms, the app recommends nearby hospitals or lung departments.
- Online Consultation Option: App suggests the option for online consultation
with a physician for quick advice.
- Hospital Search Functionality: Users search for hospitals through the app.
- Department Filter: App allows filtering hospitals by relevant departments (e.g
Respiratory or Lung).
 10

- Physician Consultation Booking: Users book an online consultation with a

physician through the app.
- Appointment Scheduling: App integrates with hospital systems to schedule an
appointment.
- Department Selection: User selects the relevant department for the
appointment (e.gRespiratory).
- Real-Time Confirmation: User receives real-time confirmation of the
appointment.
- Online Consultation Messaging: App facilitates messaging between user and
physician for online consultation.
- In-App Hospital Navigation: App provides navigation features to guide the
user to the selected hospital.
- Review and Rating Feature: After the consultation, the app prompts the user to
provide a review and rating.

● Relevant Functions to Implement

- Symptom Checker Feature: Implement a symptom assessment algorithm.
- Recommendation Engine: Develop a recommendation engine for suggesting
nearby hospitals or departments.
- Online Consultation Integration: Integrate a platform for online consultation
with physicians.
- Hospital Search and Filter: Implement hospital search functionality with filters
for relevant departments.
- Appointment Scheduling API: Collaborate with hospitals to integrate
appointment scheduling APIs.
- Real-Time Confirmation Notifications: Implement push notifications for
real-time confirmation.
- In-App Messaging: Develop a secure messaging system for communication
between users and physicians.
- Hospital Navigation Integration: Integrate with mapping services to provide
in-app navigation to hospitals
- Review and Rating System: Implement a review and rating system for user
feedback.
 11

6.1 Screenshots of your product

The figure below is just a prototype of our product and does not yet include the Figma
design.

Fig 12: Webpage for our project

Fig 13: Survey Page

Fig 14: Result Page

 12

6.2 Obstacles/risks you face and the plan to mitigate them

Obstacles and Risks
- A survey with 131 questions might be too much for people to handle. It's really long
and could make users tired or even make them stop answering the questions halfway
through. Because there are so many questions, people might not take the time to think
carefully about each one. This means they might not give the right answers about their
symptoms or how they are feeling. This can happen because the survey is so big and
takes a lot of time and effort to complete.
- Medical terminology, like "polyuria," can be confusing for users without a medical
background. This can lead to misunderstandings or incorrect symptom reporting. But
with approximately 160 features across two models, manually creating user-friendly
descriptions for each symptom is an overwhelming task.
Mitigation Plan
- Reinforcement learning is a smart way to make surveys better. It's like teaching a
computer to find the most important questions by learning from what works best. This
can make a long questionnaire shorter and easier to handle. By using this method, we
can cut down the number of questions but still get the right answers we need for an
accurate diagnosis. The computer looks at which questions give the most useful
information and keeps those, while getting rid of the ones that aren't as helpful. This
way, people don't have to spend as much time answering questions, but doctors and
experts still get the information they need to understand what's going on.
- Implementing a large language model to generate user-friendly questions
corresponding to each medical symptom. This approach can efficiently convert
complex medical terminology into simple, understandable language with the prompt
like show in figure 14

Fig 14: User-Friendly Question with GPT-4

 13

Finally we will get the very nice pair of symptom name and question (Figure 15)

Fig 15: Example for Lung Cancer Classification Question

6.3 Deployment details - where you deploy, and how you deploy. This must match to
your "Software Architecture Diagram"

In our project, we're implementing the microservice approach, allowing multiple

containers to function collaboratively in a cluster. Refer to Figure 10 for our System
Architecture Design. Our strategy involves deploying the website within a single
container, named 'symptom-checker-web-service.'. This setup is currently operational on
port 800 at localhost(figure 16) because it is in the debug state. However, once we finish
all services, the website will be deployed on AWS ECS.

Fig 16: Symptom Checker Web Page deployment

For the lung cancer classification service, We have created a container named
'symptom-checker-lung-cancer-services' and written a Flask API. This API waits for users
to POST a JSON file as shown in Figure 17 to port 5000. Upon receiving the file, the
service will output a response in JSON format, like this: {'cancer': '80.50'}.

Fig 17: Example Input Data for Classification Service

After users complete the questionnaire, the web service will POST the user data as
shown in Figure 17 to the Classification service. Then, it will output the JSON and
display the percentage on the results page (figure 14).
 14

6.4 Contribution of the team member (EVERYONE)

1. Pattranit Teerakoson (st123050)
● Design website and Experiment
● Deployment using MLFlow and Docker
2. Nutdanai Sritunya (st123055)
● Deploy the website
● Lung cancer classification service
3. Nuttabude Tanasansurapong (st123131)
a. Disease prediction service
b. Disease prediction system
4. Chen Bao(st124448)
● Functions for cooperating with hospitals and clinics

7. Actual Evaluation (on progress)

7.1 Introductory
In the ever-changing world of technology, a product's ability to provide a dependable
and smooth user experience is critical to its success. Accuracy and inference speed are
two essential characteristics that characterize this experience. As we traverse the complex
terrain of offering "Comprehensive Information," the accuracy and reliability of our
product are the cornerstones of its effectiveness.
7.2 Export any performance / human evaluation metrics in tables
7.2.1 Disease prediction results

algorithms model accuracy

Multinomial naive bayes 100

logistic regression 100

support vector 100

decision tree 99.3

XGboost 100

random forest 100

 15

after train the datasets with many models it show that there no problems in accuracy

Fig 18: Cross validation of disease datasets

Fig 19: Cross validation of Multinomial Naive bayes

Fig 20: Classification report of XGboost model

 16

7.2.2 Reinforcement result

Fig 21: training results

7.3 Discuss them briefly whether your requirements are met

not done yet..