Base paper Title: Optimal Drug Dosage Control Strategy of Immune Systems Using

Reinforcement Learning

Modified Title: Reinforcement learning as an optimal drug dosage control strategy for
immune systems

Abstract

In this article, a reinforcement learning-based drug dosage control strategy is

developed for immune systems with input constraints and dynamic uncertainties to sustain
the number of tumor and immune cells in an acceptable level. First of all, the state of the
immune system and the desired number of tumor and immune cells are constructed into an
augmented state to derive an augmented immune system. By designing a discounted non-
quadratic performance index function, the robust tracking control problem of immune
systems with uncertainties is transformed into an optimal tracking control problem of
nominal immune systems and the drug dosage can be limited within the specified range.
Hereafter, a reinforcement learning algorithm and a critic-only structure are adopted to
acquire the approximate optimal drug dosage control strategy. Furthermore, theoretical proof
reveals that the proposed reinforcement learning-based drug dosage control strategy ensures
the number of tumor and immune cells reaches the preset level under limited drug dosages
and model uncertainties. Finally, simulation study verifies the availability of the developed
drug dosage control strategy in different growth models of tumor cell.

Existing System

Cancer is a leading cause of death worldwide in recent decades, accounting for nearly
10 million deaths in 2020. Its morbidity expects up to 29 million cases by 2040 [1]. Cancer
development is a multistep process. The risk factors of tumorigenesis are highly diverse,
including genetic alterations, poor diet, physical inactivity, chronic infections and so on [2],
[3]. Normal cells grow out of control when harmful changes interfere with orderly cellular
biological process, forming precancerous lesions. Further, precancerous lesions develop into
tumors. Cancer is characterized as malignant tumor. Traditional treatments of cancer mainly
include surgery, radiotherapy, chemotherapy. Treatment options depends on the type and
stage of cancer and the individual status of patients. Most types of cancer are separated by
tumor-node-metastasis classification system including stage I to stage IV [4]. Stage I cancer
is limited to primary location and can be removed through surgery. Stage II-III cancers have
spread deeply into nearby tissues and even lymph nodes. Stage IV cancer that has spread to
remote organs of the body is called advanced or metastatic cancer. Widespread metastases are
the leading causes of cancer death. Once the cancer is diagnosed at stage II-IV, it should be
treated with radiotherapy, chemotherapy or combined chemo-radiation therapy.

Drawback in Existing System

 High computational complexity: Reinforcement learning algorithms often require a

large number of iterations and computations to converge to an optimal solution. This
can be computationally intensive and time-consuming, especially when dealing with
complex immune system models.
 Lack of interpretability: Reinforcement learning algorithms often learn optimal
strategies by exploring and exploiting different actions. Consequently, the resulting
control policy may be difficult to interpret and explain, making it challenging to
understand the reasoning behind certain dosage decisions.
 Limited generalization capabilities: Reinforcement learning models tend to perform
well within the domains they have been trained on, but they may struggle to
generalize to new and unseen conditions. This can be a limitation when dealing with
the complex and dynamic nature of immune system dynamics, as the learned control
strategy may not adapt well to new disease or patient-specific conditions.
 Ethical considerations: When using reinforcement learning for drug dosage control,
there may be ethical considerations related to the exploration of different dosage
levels on patients. Experimenting with different dosages using reinforcement learning
can potentially lead to adverse effects or harm to patients.

Proposed System

 Proposed a novel VI algorithm to speed up the convergence rate of the iterative value
function and ensure the admissibility of the iterative control law.
 Proposed event-triggered integral RL scheme to cope with the constrained H∞
tracking control problem.
  Proposed RL-based methods to acquire Nash equilibrium solutions of zero-sum
games nonzero sum games, and Stackerberg games .
 The proposed RL-based drug dosage control strategy is available, that is, tumor cells
and immune cells can be maintained at desired levels by using limited drug dosages.

Algorithm

 Model Dynamics: Develop a model describing how the immune system responds to
different drug dosages over time. This model is crucial for the reinforcement learning
algorithm.
 Reinforcement Learning Algorithm: Choose an appropriate RL algorithm (e.g., Q-
learning, Deep Q Networks) to learn optimal drug dosage policies.
 Safety Measures: Implement safety constraints to prevent the algorithm from
suggesting dosages that may cause harm.

Advantages

 Adaptability: Reinforcement learning enables dynamic adaptation to changes in the

patient's immune response, allowing for personalized and responsive dosage
adjustments.
 Potential for Complex Regimens: RL algorithms can handle complex drug regimens
involving multiple agents or dosages, optimizing combinations for enhanced
therapeutic effects.
 Real-time Decision Making:The algorithm can provide real-time recommendations,
facilitating timely adjustments in response to changing patient conditions.
 Resource Optimization: Efficient drug dosing can lead to better resource allocation
in healthcare settings by minimizing unnecessary treatments and hospital visits.
Software Specification

 Processor : I3 core processor

 Ram : 4 GB
 Hard disk : 500 GB

Software Specification

 Operating System : Windows 10 /11

 Frond End : Python
 Back End : Mysql Server
 IDE Tools : Pycharm