882 IEEE/ACM TRANSACTIONS ON COMPUTATIONAL BIOLOGY AND BIOINFORMATICS, VOL. 18, NO.

3, MAY/JUNE 2021

A Unified Smart Chinese Medicine Framework

for Healthcare and Medical Services
Qingchen Zhang , Changchuan Bai, Laurence T. Yang , Zhikui Chen , Peng Li, and Hang Yu

Abstract—Smart Chinese medicine has emerged to contribute to the evolution of healthcare and medical services by applying machine
learning together with advanced computing techniques like cloud computing to computer-aided diagnosis and treatment in the health
engineering and informatics. Specifically, smart Chinese medicine is considered to have the potential to treat difficult and complicated
diseases such as diabetes and cancers. Unfortunately, smart Chinese medicine has made very limited progress in the past few years. In
this paper, we present a unified smart Chinese medicine framework based on the edge-cloud computing system. The objective of the
framework is to achieve computer-aided syndrome differentiation and prescription recommendation, and thus to provide pervasive,
personalized, and patient-centralized services in healthcare and medicine. To accomplish this objective, we integrate deep learning and
deep reinforcement learning into the traditional Chinese medicine. Furthermore, we propose a multi-modal deep computation model for
syndrome recognition that is a crucial part of syndrome differentiation. Finally, we conduct experiments to validate the proposed model by
comparing with the staked auto-encoder and multi-modal deep learning model for syndrome recognition of hypertension and cold.

Index Terms—Healthcare and medical services, smart chinese medicine, machine learning, cloud computing, syndrome differentiation and
prescription recommendation

1 INTRODUCTION

R ECENTLY,
cyber-physical-social systems have made a
great progress by the deep fusion of many cyber and
physical systems with some emerging technologies such as
Internet of Things and cloud computing [1], [2], [3]. Cyber-
physical-social systems integrate various enabling techni-
ques like smart space design, artificial intelligence, big
data analytics, and cloud computing that are needed for
building healthcare and medical systems [4], [5]. An
important objective of healthcare systems is to provide per-
sonalized, pervasive, and patient-centralized healthcare
and medical services. To accomplish this objective, some
smart medical systems based on cyber-physical-social sys-
tems have been developed for computer-aided diagnosis
and treatment [6]. As a component of medicine, the tradi-
tional Chinese medicine shows its promising therapeutic
effect for some chronic and complex diseases such as
hypertension, diabetes, and cancers [7], [8]. More recently,
smart Chinese medicine has been proposed, and conceptu-
ally it integrates various potential artificial intelligence

Q. Zhang and L. T. Yang are with the School of Information and Communication
Engineering, University of Electronic Science and Technology of China,
Chengdu 611731, China, and also with the Department of Computer Science,
St. Francis Xavier University, Antigonish, NS B2G 2W5, Canada.
E-mail: qzhang@stfx.ca, ltyang@ieee.org.

C. Bai is with the Dalian Hospital of Traditional Chinese Medicine,
Dalian, Liaoning 116013, China. E-mail: bcc_clinic@163.com.

Z. Chen and P. Li are with the School of Software Technology, Dalian
University of Technology, Dalian 116620, China.
E-mail: zkchen@dlut.edu.cn, lipeng2015@mail.dlut.edu.cn.

H. Yu is with the Department of Computer Science, St. Francis Xavier
University, Antigonish B2G 2W5, Canada. E-mail: x2018ujw@stfx.ca.
Manuscript received 30 Jan. 2019; revised 3 Apr. 2019; accepted 15 Apr. 2019.
Date of publication 2 May 2019; date of current version 3 June 2021.
(Corresponding author: Laurence T. Yang.)
Digital Object Identifier no. 10.1109/TCBB.2019.2914447
1545-5963 © 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See ht_tps://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Francis Xavier Engineering College. Downloaded on February 21,2024 at 10:32:22 UTC from IEEE Xplore. Restrictions apply.
techniques such as deep learning, clustering, and reinforce-
ment learning into the traditional Chinese medicine for
computer-aided diagnosis and prescription recommenda-
tion [7]. Specially, Zhang et al. [7] presented a potential
deep learning model for syndrome recognition in the treat-
ment of hypertension.
At present, smart Chinese medicine is considered to be
promising to treat the difficult and complicated diseases
such as hypertension, diabetes and cancers, and therefore it
is potential to provide pervasive and personalized health-
care and medical services. Unfortunately, smart Chinese
medicine has not make a remarkable progress since no uni-
fied architecture has been developed to combine various
enabling computing techniques with the traditional Chinese
medicine theory and practice. Moreover, a large volume of
traditional Chinese medicine experience that has been accu-
mulated for thousands of years has not been well summa-
rized with the modern artificial intelligence models. In this
paper, we present a unified smart Chinese medicine frame-
work based on edge-cloud computing system to integrate
advanced machine learning models including deep learning
and deep reinforcement learning into the traditional Chinese
model effectively for the increasing evolution of smart Chi-
nese medicine [9], [10], [11], [12]. By the fundamental theory
of the traditional Chinese medicine, therapy according to
syndrome is the core idea in clinical practice. In detail, syn-
drome differentiation and prescription selection are the most
important two steps of this idea. In particular, syndrome
differentiation points out one key difference between the
traditional Chinese medicine and the modern medicine. In
detail, drugs and prescriptions are selected according to the
patient’s disease in the modern medicine. For example, the
cold medicine like Advil and Tylenol will be selected when
people catch a cold. However, drugs and prescriptions are
ZHANG ET AL.: A UNIFIED SMART CHINESE MEDICINE FRAMEWORK FOR HEALTHCARE AND MEDICAL SERVICES
Fig. 1. Unified smart chinese medicine framework.
usually chosen according to the patient’s syndrome rather
than the patient’s disease in the traditional Chinese medi-
cine. For example, the traditional Chinese medicine classifies
the cold into several types of syndrome, the typical two of
which are cold pathogen of Taiyang and affection of Taiyang
by wind. Furthermore, the patient affected by cold pathogen
of Taiyang will be given the mahuang decoction while the
patient with affection of Taiyang by wind will be given the
guizhi decoction. Another representative example is hyper-
tension which is classified into five types of syndrome like
liver qi stagnation and abundant phlegm-dampness in
the traditional Chinese medicine. Xiaochaihu decoction is
the most appropriate prescription selected for treating the
hypertension of the syndrome of liver qi stagnation whereas
wendan decoction is selected to treat the hypertension of the
syndrome of abundant phlegm-dampness. Therefore, the
presented unified smart Chinese medicine framework aims
at computer-aided syndrome differentiation and prescrip-
tion recommendation. To achieve the aim of the framework,
we integrate the advanced machine learning models, espe-
cially deep learning and deep reinforcement learning, into
the traditional Chinese medicine. Moreover, most machine
learning models are inefficient for the tasks of recognition
and decision-making because they often have many parame-
ters to be trained [13], [14], [15]. However, real-time com-
puter-aided syndrome differentiation and prescription
recommendation is always crucial in clinic. An edge-cloud
computing system that combines cloud computing and edge
computing is designed to improve the efficiency of machine
learning models in the presented framework.
At present, inquiry and tongue inspection are the most
widely used ways for obtaining the patient’s symptoms data
which is heterogeneous. Specially, the symptoms data
obtained from inquiry is typically structural while tongue
inspection obtains a image of the patient’s tongue. In this
paper, we present a multi-modal deep computation model
to recognize the syndrome based on the heterogeneous
symptoms data obtained by inquiry and tongue inspection.
In the presented model, a stacked auto-encoder and a convo-
lutional neural network are built to learn the features of the
symptoms data from inquiry and the image of the patient’s
Authorized licensed use limited to: Francis Xavier Engineering College. Downloaded on February 21,2024 at 10:32:22 UTC from IEEE Xplore. Restrictions apply.
883
tongue, respectively. Afterwards, the vector outer product is
used to concatenate the learned features to form a feature
matrix which is taken as input of a deep computation model
for syndrome differentiation. Finally, we conduct experi-
ments to validate the potential of the presented model by
comparing with the stacked auto-encoder and the multi-
modal deep learning model on two Chinese medical datasets
for recognizing the syndrome of hypertension and cold in
terms of the classification accuracy.
There contributions of this paper can be summarized as
follows:

We present a unified smart Chinese medicine frame-
work for providing healthcare and medical services
based on the edge-cloud computing system to inte-
grate advanced machine learning models including
deep learning models and deep reinforcement learn-
ing models into the traditional Chinese medicine
effectively.

We present a multi-modal deep computation model
by combining two deep learning models including a
stack auto-encoder and convolutional neural net-
work with a deep computation model to recognize
the syndrome based on the symptoms obtained from
inquiry and tongue inspection.

We evaluate the potential of the presented model
regarding the classification accuracy by comparing
with the stacked auto-encoder and the multi-modal
deep learning model for the computer-aided diagno-
sis on the syndrome differentiation of hypertension
and cold.
The rest of this paper is organized as follows. The unified
smart Chinese medicine framework based on the edge-cloud
computing system is presented and described in Section 2.
Section 3 illustrates the presented multi-modal deep compu-
tation model and the corresponding training algorithms and
Section 4 reports the experimental results in terms of the
training error. Finally, the paper is concluded in Section 5.
2 UNIFIED SMART CHINESE MEDICINE
FRAMEWORK BASED ON EDGE-CLOUD
COMPUTING SYSTEM
In this section, we describe our presented unified smart Chi-
nese medicine framework which runs on an edge-cloud com-
puting system. Specially, the presented framework has two
layers, i.e., edge layer and cloud layer, as shown in Fig. 1.
The edge layer aims to provide the pervasive healthcare
services. In detail, people can use the functionalities such as
computer-aided diagnosis and prescription recommenda-
tion provided by the framework with any edge device like a
mobile phone or a personal computer anytime and any-
where. The task of the edge layer is to collect the patient’s
symptoms for computer-aided syndrome differentiation
which is accomplished in the cloud layer. Typically, the
patient’s clinical symptoms are obtained by the four ways,
i.e., inquiry, inspection, listening and smelling, and palpa-
tion and pulse taking, in the traditional Chinese medicine.
Inquiry is the most important way of collecting the patient’s
symptoms since most of the patient’s symptoms can be
obtained only by inquiry. The doctor inquiries the patient
884 IEEE/ACM TRANSACTIONS ON COMPUTATIONAL BIOLOGY AND BIOINFORMATICS, VOL. 18, NO. 3, MAY/JUNE 2021
TABLE 1
Groups of Questions in Inquiry
Group of questions Examples
chill and fever only chill without fever, alternate attacks of
chill and fever, afternoon tidal fever
sweating anhidrosis, spontaneous sweating, night
sweating, head sweating
headache migraine, parietal headache, forehead pain
sleep insomnia, dreaminess
body uncomfort heavy body, weakness, soreness and weakness
of waist and knees
ear and eye tinnitus, deafness, itching of eyes, photophobia
cough and dyspnea coughing of phlegm, dry cough, wheezing
diet anorexia, rapid digestion of food and
polyorexia, hunger without desire to eat
micturition and diarrhea, constipation, heat fecaloma with
defecation watery discharge
others spermatorrhea, asexuality
what uncomfortable symptoms he/she has. Furthermore,
the doctor must be able to identify the questions that are
closely related to the patient’s potential syndrome and
accordingly encourages the patient to tell the relevant symp-
toms. However, it is usually difficult for the young Chinese
doctors to identify the appropriate questions since they do
not have rich medical experience. Generally, all the questions
in inquiry are classified into ten groups, summarized in
Table 1.
Inspection is also an important way of collecting the
patient’s symptoms, including inspecting the patient’s face,
tongue, and the whole body. Specially, tongue inspection is
especially important for syndrome differentiation since the
appearance of the tongue can accurately reveal the patient’s
diseases. For example, teeth-printed tongue implies that the
patient suffers from dampness while the tongue with yellow
fur indicates that the patient may have syndrome of endoge-
nous heat due to yin deficiency. Typically, the tongue inspec-
tion mainly includes tongue quality inspection and fur
inspection. Table 2 summarizes some representative patho-
logic tongue manifestations.
Palpation and pulse taking is another important way of
collecting the patient’s symptoms. Pulse taking is the core of
this way since many diseases will be reflected in pulse mani-
festation. For example, the patient may catch a cold when
he/she presents floating pulse. When he/she suffers from
dampness, the patient has slippery pulse. At present, pulse
manifestation is usually taken as a corroboration of syn-
drome differentiation.
Listening and smelling is the fourth way of collecting the
patient’s symptoms and it is also helpful to recognize the
syndrome. For example, a patient in delirium may suffer
from the syndrome of heat disturbing heart-mind while a
patient with halitosis may suffer from the syndrome of defi-
ciency of stomach yin or the syndrome of exuberant stom-
ach fire. However, listening and smelling has seldom been
used in Chinese clinical practice in recent years.
In summary, inquiry and tongue inspection are the most
important ways of collecting the symptoms. Therefore,
the two main sub-tasks of the edge layer are to collect the
patient’s symptoms and tongue image. To accomplish the
two sub-tasks, an automatic question and answering system
and a tongue localization system should be designed and
Authorized licensed use limited to: Francis Xavier Engineering College. Downloaded on February 21,2024 at 10:32:22 UTC from IEEE Xplore. Restrictions apply.
TABLE 2
Representative Pathologic Tongue Manifestations
Group of tongue
Symptoms
manifestation
tongue color pale tongue, pink tongue, green tongue
tongue spirit lustrous tongue, withered tongue, tender tongue
tongue shape teeth-printed tongue, fissured tongue, dry tongue,
ecchymosis on tongue
tongue condition flaccid tongue, stiff tongue, protracted tongue,
frequent protrusion of tongue
fur character rootless fur, sticky greasy fur, dry and cracked fur
fur color white fur, yellow fur, black fur
implemented in the edge layer for collecting the patient’s
symptoms and tongue image, respectively.
Specially, the automatic question and answering system
should be able to automatically identify the questions that
are closely related to patient’s diseases to encourage the
patients to tell his/her main symptoms. Furthermore, the
system should be able to ask as few related questions as pos-
sible to identify the patient’s syndrome, which is very impor-
tant in Chinese medicine clinic practice. Too many questions
that are less related to the patient’s diseases usually lead the
patient to tell some secondary symptoms which will some-
times increase the difficulty of syndrome differentiation. In
general, the number of questions should not exceed 7 during
an inquiry case. However, it is usually hard to identify the
closely related questions for most difficult and complication
diseases. How to design an automatic question and answer-
ing system that could identify questions closely associated
with the patient’s diseases is a challenging issue. Li et al. [16]
presented a dialog generation system using the deep rein-
forcement learning model in 2016. An automatic question
and answering system can be viewed as a special dialog gen-
eration system. So, the deep reinforcement learning models
such as the deep Q-network and the deep Q-learning are
potentially used to design such an automatic question and
answering system for computer-aided syndrome differentia-
tion in smart Chinese medicine framework [17], [18].
When a patient uses the camera that may be an indepen-
dent one or inside any edge device to take his/her tongue
picture, the picture will include the background such as his/
her mouth or face. Such the background will increase data
traffic on the network, and more importantly it will disturb
the syndrome differentiation as the noise. Thus, a tongue
localization algorithm should be embedded in the edge layer
for exacting the tongue image or removing the background.
Recently, deep reinforcement learning has achieved state-
of-the-art performance for tackling the problem of object
localization. Furthermore, it has been used for landmark
detection in medical imaging data. For instance, Ghesu et al.
[19] applied the deep reinforcement learning model to ana-
tomical landmark detection on 2D magnetic resonance
images. Therefore, the deep reinforcement learning model is
the most promising model for the task of tongue localization
in the smart Chinese medicine framework [20].
The cloud layer is responsible for syndrome differentia-
tion and prescription recommendation. Syndrome differen-
tiation is achieved by two tasks, i.e., syndrome recognition
and the main accompanied symptoms recognition, based
on the symptoms data and tongue image uploaded from the
ZHANG ET AL.: A UNIFIED SMART CHINESE MEDICINE FRAMEWORK FOR HEALTHCARE AND MEDICAL SERVICES
Fig. 2. Example of the multi-modal deep learning model.
edge layer. Specially, syndrome recognition can be viewed
as a classification problem that takes the symptoms and the
tongue image as input and outputs the patient’s syndrome
as the class label. For example, a patient with the symptoms
of aversion to cold, rigidity of nape and headache, and
anhidrosis suffers from the syndrome of cold pathogen of
Taiyang. When a patient with hypertension feels vertigo
and headache, he/she suffers from the syndrome of liver qi
stagnation. In the past ten years, deep learning has shown
its promising results for the task of classification in image
recognition and natural language processing and thus it is
the most potential for syndrome recognition. However, the
symptoms data and the tongue image are heterogeneous. It
is difficult for the traditional deep learning models to clas-
sify the heterogeneous data since they are initially designed
for singe type of data feature learning [21], [22]. Recently,
multi-modal deep learning models have been proposed for
multi-modal data feature learning. A representative exam-
ple is multi-modal deep Boltzmann machine which was
presented by Srivastava and Salakhutdinov [23] to learn fea-
tures on bimodal data that is composed of text and images.
Ngiam et al. [24] proposed a bimodal deep auto-encoder for
feature learning on bimodal data with image and audio.
Specially, Fig. 2 shows an example of the multi-modal deep
learning model.
More recently, Zhang et al. [25] presented a deep compu-
tation model which generalizes the deep learning models
with tensor-based big data representation for heterogeneous
data feature learning, as shown in Fig. 3.
Multi-modal deep learning models and deep computa-
tion models have achieved state-of-the-art results for the
tasks of feature learning and classification for heterogeneous
data, and therefore they are promising for syndrome recog-
nition in smart Chinese medicine. Specially, multi-modal
deep learning models learn the features of each modality
and then construct a joint representation for each multi-
modal object by concatenating the learned features. How-
ever, multi-modal deep learning models do not achieve the
desirable results sometimes since they cannot capture the
correlations over the different modalities effectively by
concatenating the learned feature in a simply linear way. To
address this problem, we present a multi-modal deep com-
putation model for syndrome recognition by combining the
ideas of multi-modal deep learning models and deep com-
putation models in this paper. Specially, we use the outer
product to concatenate the learned features to construct a
Authorized licensed use limited to: Francis Xavier Engineering College. Downloaded on February 21,2024 at 10:32:22 UTC from IEEE Xplore. Restrictions apply.
885
Fig. 3. Example of the deep computation model.
feature tensor for each object, which will be described in next
section.
The ultimate goal of the cloud layer is prescription recom-
mendation. This goal can also be achieved by two tasks, i.e.,
prescription selection and prescription re-organization. To
select the most appropriate prescription, a deep learning
model that takes the patient’s syndrome and the main
accompanied symptoms as input and outputs the appropri-
ate prescription is required to train. Specially, the prescrip-
tion selection depends on both the patient’s syndrome and
the main accompanied symptoms. For example, mahuang
decoction should be selected to give the patient with the syn-
drome of cold pathogen of Taiyang. However, if the patient
is accompanied by the symptoms of dysphoria and floating
and rapid pulse, guizhi decoction should be selected.
Prescription re-organization aims to provide the personal-
ized and patient-centralized treatment plan by adding one or
more Chinese medicine into the selected prescription and/or
removing one or more Chinese medicine from the prescrip-
tion based on the patient’s primary symptoms accompanied
with the syndrome. For example, a hypertensive patient suf-
fers from the syndrome of liver qi stagnation, he/she should
take xiaochaihu decoction. However, when the patient has
the symptoms of dry mouth, red urine and red tongue, the
stir-baked cape jasmine fruit should added into xiaochaihu
decoction. Specially, prescription re-organization can be
viewed as a multi-step decision-making problem. In detail,
one Chinese medicine is decided to be added and/or
removed in each step. Deep reinforcement learning is espe-
cially good at solving the problem of continuous decision-
making and it has achieved remarkable results in recent
years [26]. For example, it has achieved considerable level in
some computer games as human professional. Specially, a
deep reinforcement learning model-based agent, AlphaGo,
beat Fan Hui, the European Go champion, five times out of
five in tournament conditions. Therefore, deep reinforce-
ment leaning is the most promising model for prescription
re-organization. However, how to define the appropriate
reward function that is the key in deep reinforcement learn-
ing for prescription re-organization poses a big challenge. It
requires the in-depth cooperation between medical scientists
and artificial intelligence scientists for this problem.
886 IEEE/ACM TRANSACTIONS ON COMPUTATIONAL BIOLOGY AND BIOINFORMATICS, VOL. 18, NO. 3, MAY/JUNE 2021

Fig. 5. Graph of hyperbolic tangent function.

To train the parameters, an error function Ju regarding m

training instances is defined as:
1X m  2 1 X X 2
Ju ¼ yðiÞ  xðiÞ þ W : (3)
m i¼1 2
Typically, the gradient descent method with the back-
propagation algorithm that is used to compute the partial
derivatives is employed to train the parameters for minimiz-
ing the error function Ju , described in Algorithm 1. Specially,
the forward propagation for calculating the output of each
unit in the network is on lines 3-8 while the back propagation
Fig. 4. Architecture of the multi-modal deep computation model. for calculating the partial derivatives is on lines 10-21.
Finally, the gradient descent method is on lines 22-23 for
3 MULTI-MODAL DEEP COMPUTATION MODEL FOR updating the parameters.
SYNDROME RECOGNITION
In this section, we present a multi-modal deep computation Algorithm 1. Training Parameters of Auto-Encoder
model for syndrome recognition based on the symptoms Input: fðX ðiÞ ; Y ðiÞ Þgm i¼1 , h, threshold
 
and the tongue image that are uploaded from the edge Output: u ¼ W ð1Þ ; bð1Þ ; W ð2Þ ; bð2Þ
layer. The architecture of the presented multi-model deep 1 for iteration ¼ 1; 2; . . . ; iteratermax do
computation model is shown in Fig. 4. 2 for example ¼ 1; 2; . . . ; N do
The multi-modal deep computation model is made of 3 for j ¼ 1; 2; . . . ; m do
ð2Þ P ð1Þ ð1Þ
three components, i.e., a stacked auto-encoder, a convolu- 4 aj ¼ fð ni¼1 Wji þ bi Þ;
ð2Þ P ð1Þ ð1Þ
tional neural network and a deep computation model. 5 Let zj ¼ ni¼1 Wji þ bi ;
The stacked auto-encoder that is made of an input layer 6 for i ¼ 1; 2;P . . . ; n do
and two hidden layers is used to learn features of the ð3Þ ð2Þ ð2Þ ð2Þ
7 ai ¼ fð m j¼1 Wij aj þ bi Þ;
patient’s symptoms. Specially, the stacked auto-encoder can ð3Þ P ð2Þ ð2Þ ð2Þ
8 Let zi ¼ m j¼1 Wij aj þ bi ;
be trained by two auto-encoders from bottom to up.
9 if JTAE ðuÞ > threshold then
In each auto-encoder, the hidden representation h is 10 for i ¼ 1; 2; . . . ; n do
obtained by the encoding function f [27]: 11
ð3Þ ð3Þ
s i ¼ ðai  xi Þ  f 0 ðzi Þ;
ð3Þ

12 for j ¼ 1; P 2; . . . ; m do
h ¼ fðW ð1Þ x þ bð1Þ Þ: (1) ð2Þ ð2Þ ð3Þ ð2Þ
13 s j ¼ ð ni¼1 Wij s i Þ  f 0 ðzj Þ;
Specially, the hyperbolic tangent function fðxÞ ¼ ðex  14 for i ¼ 1; 2; . . . ; n do
ð2Þ ð2Þ ð3Þ
e Þ=ðex þ ex Þ is used as the encoding function. The hyper-
x
15 Dbi ¼ Dbi þ s i ;
bolic tangent function has two important properties. First, 16 for j ¼ 1; 2; . . . ; m do
ð2Þ ð2Þ ð2Þ ð3Þ
its range is in (-1,1), as shown in Fig. 5. 17 Dwij ¼ Dwij þ aj  s i ;
Second, the derivative of the hyperbolic tangent function 18 for j ¼ 1; 2; . . . ; m do
ð1Þ ð1Þ ð2Þ
regarding its argument is fðxÞ ¼ 1  f 2 ðxÞ. 19 Dbj ¼ Dbj þ s j ;
Afterwards, the decoding function g is used to map the 20 for i ¼ 1; 2; . . . ; n do
ð1Þ ð1Þ ð2Þ
hidden representation to the output layer y: 21 Dwji ¼ Dwji þ xi  s j ;
22 // Update parameters;
y ¼ gðW ð2Þ h þ bð2Þ Þ; (2)
W ¼ W  h  ðN1 DwÞ;
where the hyperbolic tangent function is employed as the 23 b ¼ b  h  ðN1 DbÞ;
decoding function g as well in the presented model.
Authorized licensed use limited to: Francis Xavier Engineering College. Downloaded on February 21,2024 at 10:32:22 UTC from IEEE Xplore. Restrictions apply.
ZHANG ET AL.: A UNIFIED SMART CHINESE MEDICINE FRAMEWORK FOR HEALTHCARE AND MEDICAL SERVICES 887

Fig. 6. Example of max pooling.

The second component is a convolutional neural network Fig. 7. Architecture of tensor auto-encoder.
which is responsible for feature learning of the patient’s
tongue image. Specially, the convolutional neural network
" #
m 
consists of a convolutional layer, a pooling layer and a fully-
connected layer. 1X 1 T
JTAE ðuÞ ¼ ðhW;b ðxÞ  yÞ GðhW;b ðxÞ  yÞ
The convolutional layer aims to compress the parameters m i¼1 2
and to recognize the principal region of the tongue image XNX
J1 J X

I1 IN
 ð1Þ 2
by the weight sharing, defined as [28]: þ  Wpi1 in (8)
2 p¼1 i1 ¼1 iN ¼1
X  !
yj ¼ f K ij  x i þ bj ; (4) XNX
I1 I J1 X
JN
 ð2Þ 2
i þ  Wqj1 jn ;
q¼1 j1 ¼1 jN ¼1
where Kij and  denote the convolutional kernel and the
discrete convolutional operator, respectively. where G is the coefficient matrix used to compute the tensor
The pooling layer is used to reduce the number of param- distance [32]. Specially, the tensor distance between two
eters further by a max pooling operation. Fig. 6 shows an tensors X 2 RI1 I2 IN and Y 2 RI1 I2 IN is defined by:
example of a max pooling operation. rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
XI1 I2 IN
The fully-connected layer is used to store the features of dTD ¼ glm ðxl  yl Þðxm  ym Þ
the patient’s tongue image. Typically, the parameters of the l;m¼1
qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi (9)
convolutional neural network is trained by the gradient
¼ ðx  yÞT Gðx  yÞ;
descent method with the back-propagation algorithm.
After learning the features of the symptoms and the ( )
1 jjpl  pm jj22
tongue image, the vector outer product is used to concate- glm ¼ exp  ; (10)
nate the learned features for obtaining a joint represen- 2pd2 2d2
tation. Given two vectors a 2 Rm and b 2 Rn , their outer qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
product generates a matrix c 2 Rmn with cij ¼ ai  bj jjpl  pm jj2 ¼ ði1  i01 Þ2 þ    þ ðiN  i0N Þ2 : (11)
[29], [30], [31]. Eq. (5) shows an example of the vector outer
product . In the presented model, the high-order back-propagation
1 0 algorithm is used to train each tensor auto-encoder [25].
2 6
ð 2 3 1 Þ  ð 1 3 Þ ¼ @ 3 9 A: (5)
1 3 4 EXPERIMENT
In the experiment, we validate the presented model (MDCM)
Finally, a deep computation model with two hidden layers by comparing with the stacked auto-encoder (SAE) and the
is built for syndrome recognition on the joint representation. multi-modal deep learning model (MDLM) for syndrome
Specially, the deep computation model is stacked by two ten- recognition of hypertension and code in terms of classifica-
sor auto-encoders. Each tensor auto-encoder has the architec- tion accuracy on two Chinese medical datasets. Specially, the
ture shown in Fig. 7 [25]. experiments were conducted using MATLAB on the Think-
Specially, it is defined as the following forward pass: Server with 6-cores, 2.4GHz Intel E5-2620V3 CPU and 64GB
! DDR memory.
I1X
IN The first Chinese medical dataset consists of 400 training
ð1Þ ð1Þ
Hj1 ...jM ¼ f Wai1 iN Xi1 iN þ bj1 jM ; (6) instances and 100 test instances of patients with hyperten-
i1 iN
sion. Specially, hypertension is divided into five types of syn-
PM1 QM drome, i.e., liver qi stagnation, liver fire flaring up, upper
where a ¼ jM þ t¼1 ðjt  1Þ s¼tþ1 Js . hyperactivity of liver yang, deficiency of both yin and yang,
and abundant phlegm-dampness.
J1X
JM
!
In the experiments, each model was trained for five times
ð2Þ ð2Þ
Yi1 ...iN ¼ f Wbj1 jM Hj1 jM þ bi1 ...iN ; (7) and the parameters were initialized differently every time
j1 jM so that the robustness of the presented model can be evalu-
P QN ated. The classification results with each initialization are
where b ¼ iN þ N1
t¼1 ðit  1Þ s¼tþ1 Is . displayed in Fig. 8.
Correspondingly, the error function of a tensor auto- Table 3 lists the experimental results in terms of the
encoder regarding m training instances is defined as: worst accuracy, the best accuracy, and the mean accuracy.
Authorized licensed use limited to: Francis Xavier Engineering College. Downloaded on February 21,2024 at 10:32:22 UTC from IEEE Xplore. Restrictions apply.
888 IEEE/ACM TRANSACTIONS ON COMPUTATIONAL BIOLOGY AND BIOINFORMATICS, VOL. 18, NO. 3, MAY/JUNE 2021
Fig. 8. Classification accuracy on the hypertension dataset.
TABLE 3
Experimental Results on Hypertension Dataset
Model Worst Best Mean
MDCM 86% 91% 88.8%
MDLM 84% 89% 86.6%
SAE 81% 88% 83.4%
TABLE 4
Main Symptoms of Each Syndrome of Cold
Type of syndrome Primary symptoms
cold pathogen of Taiyang aversion to cold, fever, rigidity of nape and
headache, anhidrosis, floating and tight pulse
affection of Taiyang aversion to wind, fever, headache, sweating,
by wind floating and moderate pulse
By the results shown in Fig. 8 and Table 3, the presented
model significantly outperforms the other two models for
syndrome recognition of hypertension. The presented
model obtained the higher mean accuracy than the other
two models. Specially, the presented model obtained 2.2
and 5.4 percent higher mean accuracy than the multi-modal
deep learning model and the stacked auto-encoder model,
respectively. In most cases, the presented model produced
the higher classification accuracy than the multi-modal deep
learning model since the presented model uses the vector
outer product to fuse the learned features while the multi-
modal deep learning model concatenates the features
linearly. Such the experimental results indicate that the pre-
sented model can learn the joint representation more effec-
tively than the multi-modal deep learning model. The
stacked auto-encoder performed the worst since this model
did not take the features of the tongue image into account.
Such the results indicate that the tongue inspection is very
important to the syndrome differentiation. However, the
stacked auto-encoder could still obtain relatively good classi-
fication results (e.g., greater than 80 percent) according to the
inquiry data, implying that inquiry is the most important to
the syndrome differentiation for computer-aided diagnosis.
The experimental results demonstrated the potential of the
presented model for syndrome recognition of hypertension
in smart Chinese medicine.
Authorized licensed use limited to: Francis Xavier Engineering College. Downloaded on February 21,2024 at 10:32:22 UTC from IEEE Xplore. Restrictions apply.
Fig. 9. Classification accuracy on the cold dataset.
TABLE 5
Experimental Results on Cold Dataset
Model Worst Best Mean
MDCM 90% 94% 92%
MDLM 85% 93% 89.8%
SAE 86% 91% 89%
The second dataset consists of 400 training instances and
100 training instances of patients suffering from cold. Cold is
divided into two main types of syndrome, i.e., cold pathogen
of Taiyang and affection of Taiyang by wind. Table 4 summa-
rizes the primary symptoms of each type of syndrome of cold.
Each model was trained for five times and the results
with each initialization are displayed in Fig. 9.
Table 5 lists the experimental results in terms of the
worst accuracy, the best accuracy, and the mean accuracy.
Based on the results shown in Fig. 9 and Table 5, the pre-
sented model performs better than the other models in terms
of the worst accuracy, the best accuracy, and the mean accu-
racy. Specially, for each initialization, the presented model
could obtain the classification accuracy greater than 90 percent,
implying that the presented model is more robust than the
other two models. Moreover, the presented model produced
2.2 and 3 percent higher mean accuracy than the multi-modal
deep learning model and the stacked auto-encoder, respec-
tively. Such the results demonstrate the potential of the pre-
sented model for syndrome recognition of cold. However,
only distinguishing the cold pathogen of Taiyang and affection
of Taiyang by wind is not enough for prescription recommen-
dation in clinical practice. Generally, mahuang decoction
should be chosen to treat the cold pathogen of Taiyang while
guizhi decoction should be chosen to treat the affection of
Taiyang by wind. However, when a patient with the cold path-
ogen of Taiyang is suffering from the deficiency of healthy qi,
the mahuang decoction should not be used. In this case, the
guizhi decoction should be chosen. Therefore, the presented
model will be validated for detailed syndrome differentiation
in the future work.
Finally, we compare the training time and the inference
time of the three models on the two Chinese medicine clini-
cal datasets. To present the results more clearly, the training
and inference time is normalized regarding the highest
ZHANG ET AL.: A UNIFIED SMART CHINESE MEDICINE FRAMEWORK FOR HEALTHCARE AND MEDICAL SERVICES
TABLE 6
Average Training Time
Dataset/Model MDCM MDLM
Hypertension dataset 1 0.92
Cold dataset 1 0.94
TABLE 7
Average Inference Time
Dataset/Model MDCM MDLM
Hypertension dataset 1 0.98
Cold dataset 1 0.99
training and inference time. Tables 6 and 7 present the aver-
age training time and the average inference time of the three
models, respectively.
By the results in Table 6, it took much shorter to train the
stacked auto-encoder than the presented model and the
multi-modal deep learning model as the stacked auto-
encoder has the least number of parameters. Specially, the
stacked auto-encoder cost 25 and 17 percent less average
training time than the presented modal and the multi-modal
deep learning model, respectively, on the hypertension data-
set. The presented model cost slightly more average training
time than the multi-modal deep learning model as the pre-
sented model produces more parameters using the outer
product to fuse the learned features. However, the parame-
ters need to be trained only once offline. Furthermore, from
the results in Table 7, it took almost the same average time for
the three models in syndrome recognition. Such the results
demonstrate the potential of the presented model for efficient
syndrome differentiation in smart Chinese medicine.
5 CONCLUSION
Recently, some healthcare and medical systems have
emerged to provide personalized, pervasive, and patient-
centralized healthcare. Specially, smart Chinese medicine
has more recently been proposed to contribute the evolution
of healthcare 4.0. In this paper, we presented a unified smart
Chinese medicine framework based on an edge-cloud com-
puting system to integrate the advanced machine learning
models including deep learning models and deep reinforce-
ment learning models into the traditional Chinese medicine.
The objective of the framework is to provide computer-aided
syndrome differentiation and prescription recommendation
in Chinese medicine clinic practice. Specially, the advanced
machine learning models are used to provide the personal-
ized and patient-centralized healthcare to patients regarding
syndrome differentiation and prescription recommendation
while the edge-cloud computing system is designed to
improve the efficiency of machine learning models used in
the framework and to provide the pervasive healthcare serv-
ices to patients with edge computing. Furthermore, we dis-
cuss the potential models applicable to each challenge in the
presented framework. Finally, we presented a multi-modal
deep computation model for syndrome recognition in smart
Chinese medicine. The results clearly demonstrates the
promising of the presented model for syndrome recognition
of hypertension. In the future work, we will design and
Authorized licensed use limited to: Francis Xavier Engineering College. Downloaded on February 21,2024 at 10:32:22 UTC from IEEE Xplore. Restrictions apply.
889
implement the potential models used in the framework and
validate them using the real clinic datasets. In detail, the
SAE potential models include deep reinforcement learning mod-
els for inquiry questions identification, tongue extraction
0.75
0.81 and prescription re-organization, and deep learning models
for the main accompanied symptoms recognition.
ACKNOWLEDGMENTS
SAE Q. Zhang and C. Bai are the co-first authors.
0.95
0.98 REFERENCES
[1] R. K. Lenka, A. K. Rath, Z. Tan, S. Sharma, D. Puthal, N. Simha,
M. Prasad, R. Raja, and S. S. Tripathi, “Building scalable cyber-
physical-social networking infrastructure using IoT and low
power sensors,” IEEE Access, vol. 6, pp. 30162–30173, 2018.
[2] J. Li, M. Qiu, J. Niu, L. T. Yang, Y. Zhu, and Z. Ming, “Thermal-
aware task scheduling in 3D chip multiprocessor with real-time con-
strained workloads,” ACM Trans. Embedded Comput. Syst., vol. 12,
no. 2, pp. 24:1–24:22, 2013.
[3] T. Wang, M. Z. A. Bhuiyan, G. Wang, M. A. Rahman, J. Wu, and
J. Cao, “Big data reduction for smart city’s critical infrastructural
health monitoring,” IEEE Commun. Mag., vol. 56, no. 3, pp. 128–133,
Mar. 2018.
[4] M. Lin and L. T. Yang, “Hybrid genetic algorithms for scheduling
partially ordered tasks in a multi-processor environment,” in Proc.
Int. Conf. Real-Time Comput. Syst. and Appl., 1999, pp. 382–387.
[5] T. Wang, G. Zhang, A. Liu, M. Z. A. Bhuiyan, and Q. Jin, “A secure
IoT service architecture with an efficient balance dynamics based on
cloud and edge computing,” IEEE Internet Things J., vol. 6, no. 3,
pp. 4831–4843, Jun. 2019.
[6] Z. Liu, C. Yao, H. Yu, and T. Wu, “Deep reinforcement learning
with its application for lung cancer detection in medical internet of
things,” Future Generation Comput. Syst., vol. 97, pp. 1–9, Aug. 2019.
[7] Q. Zhang, C. Bai, Z. Chen, P. Li, S. Wang, and H. Gao, “Smart chi-
nese medicine for hypertension treatment with a deep learning
model,” J. Netw. Comput. Appl., vol. 129, pp. 1–8, 2019.
[8] Q. Zhang, C. Bai, Z. Chen, P. Li, H. Yu, S. Wang, and H. Gao,
“Deep learning models for diagnosing spleen and stomach
diseases in smart chinese medicine with cloud computing,”
Concurrency Comput.: Practice Experience, to be published. [Online].
Available: https://onlinelibrary.wiley.com/doi/abs/10.1002/
cpe.5252, doi: 10.1002/cpe.5252.
[9] X. Liu, R. H. Deng, Y. Yang, H. N. Tran, and S. Zhong, “Hybrid pri-
vacy-preserving clinical decision support system in FogCCloud
computing,” Future Generation Comput. Syst., vol. 78, pp. 825–837, 2018.
[10] Q. Zhang, L. T. Yang, Z. Chen, P. Li, and F. Bu, “An adaptive drop-
out deep computation model for industrial IoT big data learning
with crowdsourcing to cloud computing,” IEEE Trans. Ind. Inform.,
vol. 15, no. 4, pp. 2330–2337, Apr. 2019.
[11] L. Y. Zhang, Y. Zheng, J. Weng, C. Wang, Z. Shan, and K. Ren,
“You can access but you cannot leak: Defending against
illegal content redistribution in encrypted cloud media center,”
IEEE Trans. Depend. Secure Comput., to be published. [Online].
Available: https://ieeexplore.ieee.org/document/8432128,
doi: 10.1109/TDSC.2018.2864748.
[12] J. Dean, G. Corrado, R. Monga, K. Chen, M. Devin, M. Mao,
M. Ranzato, A. Senior, P. Tucker, K. Yang, Q. V. Le, and A. Y. Ng,
“Large scale distributed deep networks,” in Proc. Advances Neural
Inf. Process. Syst., 2012, pp. 1223–1231.
[13] H. Li, K. Ota, and M. Dong, “Learning IoT in edge: Deep learning
for the internet of things with edge computing,” IEEE Netw., vol. 32,
no. 1, pp. 96–101, Jan./Feb. 2018.
[14] S. Teerapittayanon, B. McDanel, and H. T. Kung, “Distributed
deep neural networks over the cloud, the edge and end
devices,” in Proc. IEEE Int. Conf. Distrib. Comput. Syst., 2017,
pp. 328–339.
[15] J. Li, W. Monroe, A. Ritter, M. Galley, J. Gao, and D. Jurafsky, “Deep
reinforcement learning for dialogue generation,” arXiv:1606.01541,
2016, https://arxiv.org/abs/1606.01541
[16] K. Narasimhan, T. Kulkarni, and R. Barzilay, “Language under-
standing for text-based games using deep reinforcement learning,”
arXiv:1506.08941, 2015, https://arxiv.org/abs/1506.08941
890 IEEE/ACM TRANSACTIONS ON COMPUTATIONAL BIOLOGY AND BIOINFORMATICS, VOL. 18, NO. 3, MAY/JUNE 2021

[17] K. Arulkumaran, M. P. Deisenroth, M. Brundage, and A. A. Bharath, Changchuan Bai is currently a chief physician at
“Deep reinforcement learning: A brief survey,” IEEE Signal Process. the Dalian Hospital of Traditional Chinese Medi-
Mag., vol. 34, no. 6, pp. 26–38, Nov. 2017. cine, China, and a guest professor at the Liaoning
[18] F. C. Ghesu, B. Georgescu, T. Mansi, D. Neurmann, J. Horegger, and University of Traditional Chinese Medicine, China.
D. Comaniciu, “An artificial agent for anatomical landmark detection He is one of the first famous Chinese medicine doc-
in medical images,” in Proc. Int. Conf. Med. Image Comput. Comput.- tors in China. His research interests include exoge-
Assisted Intervention, 2016, pp. 229–237. nous fever disease, classical prescriptions, and
[19] J. C. Caicedo and S. Lazebnik, “Active object localization with smart Chinese medicine.
deep reinforcement learning,” in Proc. IEEE Int. Conf. Comput.
Vis., 2015, pp. 2488–2496.
[20] Q. Zhang, L. T. Yang, Z. Yan, Z. Chen, and P. Li, “An efficient
deep learning model to predict cloud workload for industry
informatics,” IEEE Trans. Ind. Inform., vol. 14, no. 7, pp. 3170–3178, Laurence T. Yang received the BEng degree in
Jul. 2018. computer science and technology and the BSc
[21] M. Ghersari, G. Wang, and M. Z. A. Bhuiyan, “A survey on deep degree in applied physics from Tsinghua Univer-
learning in big data,” in Proc. IEEE Int. Conf. Comput. Sci. Eng. sity, China, and the PhD degree in computer sci-
IEEE Int. Conf. Embedded Ubiquitous Comput., 2017, pp. 173–180. ence from the University of Victoria, Canada. He is
[22] N. Srivastava and R. R. Salakhutdinov, “Multimodal learning with currently a professor and W.F. James Research
deep boltzmann machines,” in Proc. Advances Neural Inf. Process. chair at St. Francis Xavier University, Canada. His
Syst., 2012, pp. 2222–2230. research has been supported by the National Sci-
[23] J. Ngiam, A. Khosla, M. Kim, J. Nam, H. Lee, and A. Y. Ng, ences and Engineering Research Council, and the
“Multimodal deep learning,” in Proc. Int. Conf. Mach. Learn., 2011, Canada Foundation for Innovation. His research
pp. 689–696. interests include parallel and distributed comput-
[24] Q. Zhang, L. T. Yang, and Z. Chen, “Deep computation model for ing, embedded and ubiquitous/pervasive comput-
unsupervised feature learning on big data,” IEEE Trans. Serv. ing, big data, and cyber-physical-social systems.
Comput., vol. 9, no. 1, pp. 161–171, Jan./Feb. 2016.
[25] Y. Li, “Deep Reinforcement Learning: An overview,”
arXiv:1701.07274, 2017, https://arxiv.org/abs/1701.07274 Zhikui Chen received the BEng degree from
[26] Q. Zhang, M. Lin, L. T. Yang, Z. Chen, S. U. Khan, and P. Li, Chongqing Normal University, China, and the PhD
“A double deep Q-learning model for energy-efficient edge degree from Chongqing University, China. He is
scheduling,” IEEE Trans. Serv. Comput., to be published. currently a professor at the Dalian University of
[Online]. Available: https://ieeexplore.ieee.org/document/ Technology, China. His research interests include
8449105, doi: 10.1109/TSC.2018.2867482. internet of things and big data processing.
[27] P. Li, Z. Chen, L. T. Yang, J. Gao, Q. Zhang, and M. J. Deen, “An
incremental deep convolutional computation model for feature
learning on industrial big data,” IEEE Trans. Ind. Inform., vol. 15,
no. 3, pp. 1341–1349, Mar. 2019.
[28] A. Cichocki, “Era of big data processing: A new approach via ten-
sor networks and tensor decompositions,” arXiv:1403.2048, 2014,
https://arxiv.org/abs/1403.2048 Peng Li received the BEng degree from Dezhou
[29] P. Li, Z. Chen, L. T. Yang, L. Zhao, and Q. Zhang, “A privacy- University, China. He is working toward the PhD
preserving high-order neuro-fuzzy C-means algorithm with cloud degree at the Dalian University of Technology,
computing,” Neurocomputing, vol. 256, pp. 82–89, 2017. China. His research interests include deep learn-
[30] Q. Cheng, P. Pal, M. Tsuji, and Y. Hua, “An MDL algorithm for ing and big data.
detecting more sources than sensors using outer-products of array
output,” IEEE Trans. Signal Process., vol. 62, no. 24, pp. 6438–6453,
Dec. 2014.
[31] Y. Liu, Y. Liu, and K. C. C. Chan, “Tensor distance based multilinear
locality-preserved maximum information embedding,” IEEE Trans.
Neural Netw., vol. 21, no. 11, pp. 1848–1854, Nov. 2010.

Hang Yu is working towards the BSc honors

Qingchen Zhang received the BEng degree from degree in computer science at St. Francis Xavier
Southwest University, China, and the PhD degree University, Canada. His research interests include
from the Dalian University of Technology, China. deep learning and smart medicine.
He is currently an assistant professor at St. Francis
Xavier University. His research interests include
big data, deep learning, and smart medicine.

" For more information on this or any other computing topic,

please visit our Digital Library at www.computer.org/csdl.

Authorized licensed use limited to: Francis Xavier Engineering College. Downloaded on February 21,2024 at 10:32:22 UTC from IEEE Xplore. Restrictions apply.