Received 3 August 2023; accepted 8 September 2023. Date of publication 29 September 2023; date of current version 23 October 2023.

The review of this paper was arranged by Associate Editor Amitava Chatterjee.
Digital Object Identifier 10.1109/OJIM.2023.3320765

Conv-Random Forest-Based IoT: A Deep Learning

Model Based on CNN and Random Forest
for Classification and Analysis of Valvular
Heart Diseases
TANMAY SINHA ROY 1 , JOYANTA KUMAR ROY 2 (Senior Member, IEEE),
AND NIRUPAMA MANDAL 3 (Senior Member, IEEE)
1 Electrical Engineering Department, Haldia Institute of Technology, Haldia 721657, India

2 Electronics and Communication Engineering Department, Narula Institute of Technology, Kolkata 700109, India

3 Electronics Engineering Department, Indian Institute of Technology (Indian School of Mines) Dhanbad, Dhanbad 826004, India

CORRESPONDING AUTHOR: T. S. ROY (e-mail: tanmoysinha.roy@gmail.com)

ABSTRACT Cardiovascular diseases are growing rapidly in this world. Around 70% of the world’s
population is suffering from the same. The entire research work is grouped into the classification and
analysis of heart sound. We defined a new squeeze network-based deep learning model—convolutional
random forest (RF) for real-time valvular heart sound classification and analysis using industrial Raspberry
Pi 4B. The proposed electronic stethoscope is Internet enabled using ESP32, and Raspberry Pi. The said
Internet of Things (IoT)-based model is also low cost, portable, and can be reachable to distant remote
places where doctors are not available. As far as the classification part is concerned, the multiclass
classification is done for seven types of valvular heart sounds. The RF classifier scored a good accuracy
among other ensemble methods in small training set data. The CNN-based squeeze net model achieved
a decent accuracy of 98.65% after its hyperparameters were optimized for heart sound analysis. The
proposed IoT-based model overcomes the drawbacks faced individually in both squeeze network and
RF. CNN-based squeeze net model and RF classifier combined together improved the performance of
classification accuracy. The squeeze net model plays a pivotal part in the feature extraction of heart
sound, and an RF classifier acts as a classifier in the class prediction layer for predicting class labels.
Experimental results on several datasets like the Kaggle dataset, the Physio net challenge, and the Pascal
Challenge showed that the Conv-RF model works the best. The proposed IoT-based Conv-RF model is
also applied on the selected subjects with different age groups and genders having a history of heart
diseases. The Conv-RF method scored an accuracy of 99.37 ± 0.05% on the different test datasets with
a sensitivity of 99.5 ± 0.12% and specificity of 98.9 ± 0.03%. The proposed model is also examined
with the current state-of-the-art models in terms of accuracy.

INDEX TERMS Cardiovascular disorder, convolutional neural network, electronic stethoscope, ensemble
learning, PCG signal, random forest (RF), Raspberry Pi, squeeze network.

I. INTRODUCTION as well. Sparse networks find better applications in terms

D EEP Learning finds many applications in natural

language processing (NLP), speech recognition, pattern
recognition, image analysis, and medical image diagno-
of heart sound analysis, as it is most suited to learn-
ing local features in a heart sound compared to deep
neural networks. Deep networks generally have many
sis. In medical image diagnosis, heart sound diagnosis and added layers, which, in turn, makes their time complexity
early screening prove to be very effective and challenging higher.
© 2023 The Authors. This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/

VOLUME 2, 2023 2500717

SINHA ROY et al.: CONV-RF-BASED IoT: A DEEP LEARNING MODEL BASED ON CNN AND RF

FIGURE 1. Bagging ensemble method. FIGURE 2. Stacking ensemble method.

We defined our work based on the following two

hypotheses.
1) Squeeze network-based CNN model is used as a feature
learning part which scored very high accuracy in heart
sound analysis problems.
2) Random forest (RF) algorithm is used for the classifi-
cation part, which attained better accuracy in terms of
classification of heart sound disorder compared to a
single decision tree (DT) method and gradient boosting
(GB) methods.
In classification problems, ensemble methods are primar-
ily used and are more suitable than any other supervised
machine learning algorithms.
The objective of this research work is to develop an
Internet-enabled low-cost, portable, and accurate Raspberry FIGURE 3. Boosting ensemble method.
Pi-based electronic stethoscope system. The proposed stetho-
scope system can be applied to subjects even in remote places
for the measurement and analysis of valvular heart sounds. Ensemble machine learning methods like RF, DT, GB, and
The said stethoscope is also ear contactless since auscultation extreme GB (XGB) methods are found a special place in
can be done through a Bluetooth-connected microspeaker. classification problems as they achieve a reasonable amount
Mainly, the ensemble methods can be of three types. of accuracy for large training datasets as well as low training
1) Bagging. time.
2) Stacking. Supervised machine learning methods like Naïve Bayes
3) Boosting. (NV), support vector machine (SVM), and multilayer per-
ceptron (MLP) are also used in classification problems, but
In the bagging algorithm, the input training set is subdi- they are not suitable for large training data sets.
vided into different random samples, and each sample is fed
to the DT. The result of DTs is combined through voting, II. PAPER ORGANIZATION
and finally, the output is generated, as presented in Fig. 2. Section III provides a literature study of different deep learn-
In the stacking ensemble method, the input training set is ing methods used in heart sound analysis. Section IV briefs
fed to different models, and their predictions are combined to about the methods and materials used in this research paper.
produce the final predicted output from the ensemble model, Section V highlights the result analysis of the research
as shown in Fig. 2. In boosting the algorithm, the input train- work. Eventually, Section VI summarizes the conclusions
ing set is fed to different models. The effectiveness of the and future scope of the research.
next model is improved through the previous model output
prediction. Finally, all the predicted outputs from other mod- III. LITERATURE REVIEW
els are combined to produce the final predicted output, as In this section, a detailed discussion on heart sound anal-
given in Fig. 3. ysis is carried out for classification. As per the literature

2500717 VOLUME 2, 2023

survey, many research works are carried out in this area.
Dwivedi et al. [1] did a study on methods for automation
in heart sound analysis and classification. Many researchers
have worked on various techniques for the classification of
heart sounds. Mishra et al. [2] did research on identifying
different segments of heart sounds for the identification of
basic heart sound segments by applying the CNN method.
This study has limited scope on analysis of S3 heart sounds.
Mishra et al. [3] derived a novel method for the separation
of heart and lung sounds of the phonocardiogram signal
for the identification of the S3 heart sound. However, this
method contains a few limitations when working in real-
time applications. Muduli et al. [4] derived a novel algorithm
for the extraction of biomedical signals from noisy calcula-
tions by sparse recovery analysis, and Barma et al. [5] did
research on the study of S2 heart sounds, which engages
with calculating the time period and the energy of normal-
ized cardiac sounds, however, they could not distinguish
heart sounds. Dewangan et al. [6] and Mishra et al. [7] did
research work on heart sound analysis using the wavelet
transform algorithm. It has specific restrictions in online
PCG signal analysis. Othman and Khaleel [8] did work
on PCG signal analysis using Shannon energy envelop and
DWT features, but it could not distinguish the heart sounds
accurately. Lubaib and Muneer [9] did work on heart defect
analysis and classification using the pattern recognition tech-
nique, but the adopted technique is entirely based on echo
imaging. Singh and Cheema [10] did work on PCG sig-
nal analysis using classification by feature extraction, but
the method has limited scope working in real-time applica-
tions. Ahmad et al. [11] did work on heart sound analysis
using a soft computing-based fuzzy classifier model based
on Mamdani-type fuzzy computation. The authors worked
on a standard cardiac sound repository using an offline-based
method which was not validated with the human subjects.
Roy et al. [12] reviewed the papers on discrimination of car-
diac signals. Gupta et al. [13] researched various stages in
cardiac sound signal for heart sound signal analysis. There
is no important work on designing the real-time screening
device for valvular diseases. Apart from these works, a few
other current research works that are related to CNN-based
PCG signal analysis and classification are highlighted in
Table 1.
IV. METHODS AND MATERIALS
A. CARDIAC SOUND BANK DESCRIPTION
Heart sound [50] samples that are used for the valvular heart
sound analysis have been considered from four heart sound
repositories as in [50] and [51] by T. S. Roy et al. in 2023.
1) https://github.com/yaseen21khan/Classification-of-
Heart-Sound-Signal-Using-Multiple-Features [15],
[17], [50]. A brief explanation of the cardiac sound
bank is shown in Heart Sound Dataset1 in Table 2.
Five types of cardiac sound samples are considered,
namely, NS, MR, MS, MVP, and AS. Every cardiac
sound lasts for a time duration of 5–10 s, sampling
VOLUME 2, 2023
frequency of 44 000 Hz, and has a bandwidth of
65–500 Hz.
2) Kaggle Dataset [49], [50], [51] is also used for the
valvular heart sound analysis. Kaggle’s heart sound
repository contains a collection of NS and heart sound
data containing murmurs.
Cardiac Sound Dataset 2, as mentioned in Table 3, is
obtained from the Pascal Heart Sound Repository-Dataset
B [15], [36], [50]. The cardiac sound samples last for a time
duration of 3–9 s, have a sampling frequency of 45 000 Hz,
and have a bandwidth of 68–512 Hz as described in the
following.
Table 4 highlights a detailed description of the Physio
Net Challenge Training Set, [16], [37], [50] that contains
six training databases (A through F) comprising a collection
of 3128 cardiac sample data. The cardiac sound samples
last for a time duration of 4–8 s, a sampling frequency of
47 000 Hz, and contain a bandwidth of 72–492 Hz.
B. METHODOLOGY ADOPTED IN THE PCG SIGNAL
ANALYSIS
The methodology used in the cardiac sound analysis is
described as follows.
Fig. 4 describes the schematic diagram of heart sound sig-
nal classification [22], [30], [50]. The preprocessing block
comprises normalization and filtering operations. Normalized
heart sound goes to the filtering block, where a bandpass fil-
ter of bandwidth 70–450 Hz is used for background filtering
of unwanted noise. A time frame of 3 s is considered for
every heart sound. The various in-depth features are captured
from the preprocessed signal, and eventually, categorization
of the cardiac sound signal is done for validation of the
adopted model
s (t) = f (s(t)) (1)
where s (t) is the preprocessed cardiac sound data.
The heart sound samples [50] are split into train data
(85%) and test data (15%). Second, training samples are
again split into validation data (15%) and the remaining for
training the software model.
Fig. 5(a) highlights the block diagram of the display unit
and Fig. 5(b) is the experimental setup of the hardware
system where it uses a conventional stethoscope chest piece,
preamplifiers, filters, 7-inch touch screen led, and industrial
Raspberry Pi 4B.
Fig. 6 provides the schematic representation of the
proposed hardware model. The PCG signal is captured
through the PCG signal acquisition module that contains
a stethoscope chest piece, preamplifiers, filters, and buffer
amplifier circuits followed by the processing and displaying
unit as the industrial Raspberry Pi 4B.
Fig. 7 is the block diagram of the proposed electronic
stethoscope that contains input sensor unit and computational
unit. The real-time PCG signal is captured with a chest piece
and microphone. The captured PCG signal goes through the
2500717
SINHA ROY et al.: CONV-RF-BASED IoT: A DEEP LEARNING MODEL BASED ON CNN AND RF
TABLE 1. Literature study table of some recently done works.
TABLE 2. Cardiac sound dataset1 [50], [51].
TABLE 3. Cardiac sound dataset2 [50], [51].
preamplifier unit of gain 20, followed by a 50-Hz notch filter
to reject the electrical interferences. The processed signal is
fed to an analog tunable band-pass filter with a 32–472-Hz
2500717
TABLE 4. Cardiac sound dataset3 [50], [51].
spectrum. The PCG signal [51], [52] generally belongs to
32–472 Hz for normal and abnormal sounds.
The unity gain buffer amplifier is selected for impedance
matching. The signal-conditioned output goes to the node
MCU (ESP32) which is WiFi enabled and contains a 12-bit
ADC with a sampling frequency of 44.1 kHz. Finally, the
converted digital PCG signal [19], [33] goes to the Raspberry
Pi through its enabled WiFi for further signal processing and
analysis. The classified heart sound is displayed on a 7-inch
LCD screen attached with the Raspberry Pi and is heard in
a Bluetooth-enabled microspeaker.
VOLUME 2, 2023

FIGURE 4. Methodology used in heart sound classification.
FIGURE 5. (a) Raspberry Pi attached with 7-inch display unit. (b) Experimental setup
of the hardware system.
FIGURE 6. Schematic representation of the proposed electronic stethoscope
system.
Time domain Features: RMS, signal energy, signal power,
ZCR, THD, skewness, and kurtosis.
Frequency domain Features: DWT.
Classification Unit: The proposed CNN model built in
Python 3.9.2 ver. stored and implemented using Raspberry
Pi 4B.
The valvular cardiac samples used for the study of PCG
signal analysis are broadly categorized into the following.
1) Ns.
2) MS.
3) AS.
4) MR.
5) AR.
6) MVP.
7) EXT.
Acoustic stethoscopes based on sensors, such as
diaphragms and piezoelectric crystals, work on the princi-
ple of conversion of sound pressure into electrical energy.
They suffer from distortion in output electrical signal, so
electronic stethoscopes are developed that can incorporate
capacitive electret microphone sensors for better efficiency
and stability as given in Fig. 7.
Message queuing telemetry transport (MQTT) is a stan-
dard protocol used for sending the processed heart sound data
to the Raspberry Pi (MQTT Broker) from ESP32 (MQTT
Client) through a WiFi connection. MQTT broker and sub-
scriber are on the same device (Raspberry Pi). The input
sensor unit and computational unit are connected through
WiFi and they are Internet enabled as shown in Fig. 8. Thus,
they are in the Internet of Things (IoT).
Features [13] of the cardiac sound used for the overall
analysis are as follows.
1) Acoustic Features: MFCCs, Mel, chroma, contrast, and
Tonnetz.
2) Time-Domain Features: RMS, signal energy, signal
power, ZCR, THD, skewness, and kurtosis.
The deep learning method proposed to categorize the heart
sounds is as follows.
1) Proposed Conv-RF Method: The machine learning
methods proposed to classify the heart sounds are as
follows.
a) RF.
b) GB.
c) XGB.
All deep learning-based algorithms are written in Python
ver. 3.9.2 using Thonny Python editor (Linux). The brief
description of the proposed algorithm mentioned above is
explained under the software development of the proposed
deep learning model.

Preprocessing Unit: Preamplifier, notch filter, bandpass C. PROPOSED CONV-RANDOM FOREST LEARNING
filter, and unity gain buffer. ALGORITHM
Feature Extraction Unit: Let X = {(xj , yj ); 1 ≤ j ≤ T}, where T denotes the length
Acoustic Features: MFCCs, Mel, chroma, contrast, and of the training data set, xj = [x1√, x2√, . . . , xN ] is the set of
Tonnetz. N feature matrices in R9 or R 9x 9 , and yj denotes the

VOLUME 2, 2023 2500717

SINHA ROY et al.: CONV-RF-BASED IoT: A DEEP LEARNING MODEL BASED ON CNN AND RF

FIGURE 7. Schematic of the input sensor unit and computational unit used in the proposed electronic stethoscope.

the output, γ i (l), of the lth layer for the ith fea-
ture map, is considered from the output of the earlier
layer, γ i (l − 1). For every layer, l, in 1 . . . L: compute
the convolutions to produce the γi (l) for layer, l
f (l−1)

γi (l) = ∅ (Bi (l) + kij (l) ∗ γj (l − 1)
j=1

where ∅ denotes the ReLU activation function, Bi (l)

is a bias matrix, and kij (l) is the filter of size (2wk +
FIGURE 8. Data transmission via WIFI between ESP32 and Raspberry Pi using the 1 × 2hk + 1). √ √
Convert γi (l) to a vector of size ( N × N × Z (l) )
MQTT Protocol.
6)
− γi γi (l).
7) Start with a new training sample data for the class
prediction layer X new = γi γi (l), yj; 1 ≤ j ≤ T.
name of vector xj . The proposed method for the Conv-RF
8) In the reshape layer, random data are considered from
is explained as follows.
the given dataset.
1) Declare the training data set, X = {(xj , yj ); 1 ≤ j ≤ T}. 9) Next, this proposed method shall produce a DT for
2) If required, add zeros to the N items of every training each considered random data. Then, it shall get the
data element, xj , so that a new sample
√ element √ could predicted output from each formed DT.
be framed into a matrix of shape, √N ×√ N. 10) Voting is carried out for every predicted outcome.
3) Transform xj into required shape, ( N, N, Z (l) ). 11) Eventually, the maximum voted prediction outcome is
4) Fix the metrics of the conv as: considered as the eventual predicted outcome.
12) Computing the time complexity of the classifier:
a) total conv layers, L;
Train-Time Complexity of RF = O(d ∗ log(d) ∗ c ∗ f ),
b) output depth, Z;
where
c) in every layer, fix the filter sizes, K (l) , and;
(l) c = count of DTs.
d) filter strides, sk .
d = count of sound samples in the train set.
5) In every conv layer, denoted by l, a conv function, f = count of information in the sound samples.
and an additive bias are done to the input for a Time Complexity of RF = O (depth of tree * c) Depth
feature vector denoted by f ∈ {1, . . . , f (l)}. Thus, of tree = 5.

2500717 VOLUME 2, 2023


FIGURE 9. (a) Squeeze net block. (b) Squeeze filters and expand filters.
The time complexity of the RF classifier plays an impor-
tant part in determining the total floating point operations
per second (FLOPs) and number of trainable parameters in
the proposed model.
D. PROPOSED CNN-BASED SQUEEZE NETWORK
A CNN-based Squeeze Net [22], [31] contains many
intermediate layers and fire modules between the input and
output nodes. The said CNN model can work with any
real-world problem having a large number of data. Neural
networks [8], [17], [47] are very much helpful in providing
answers to the challenges that are witnessed in real life.
A deep network reacts to the inputs provided, does diffi-
cult computations on them, and eventually generates output.
Besides this, backpropagation is the working algorithm in
training these deep learning models [23], [30]. The skele-
ton structure of the convolutional neural network-based deep
learning network is provided in Fig. 10. The model sum-
mary is presented after training and validation of the dataset
through this proposed model.
Fig. 9(a) is the block diagram of a used fire block in
a squeeze network, where it uses a combination of both
squeeze filter and expand filter. It can be seen that outputs
from both the filters get concatenated. Fig. 9(b) provides
the skeleton of the used squeeze filter and expand filter.
VOLUME 2, 2023
FIGURE 10. Proposed CNN-based squeeze net system description.
The squeeze filter contains three 1×1 convolutions, whereas
the expand filter comprises of four 1×1 convolutions and
four 3×3 convolutions. Basically, it is a sparsely connected
network having a maxpool and multiple convolutions of
kernel sizes 1, 3, and 5 at the same layer, followed by an
application of concatenation operation from all filter outputs.
Table 5 provides the proposed Squeeze Net architecture
using five fire blocks, two convolutional layers with a
ReLU activation function, two maxpool layers, an input
layer, and an output layer with a softmax activation
function.
Fig. 10 provides the entire architecture of the proposed
Squeeze Net model used for the valvular heart sound analy-
sis. In this model, two convolutional layers are considered,
three maxpool layers are used, five fire blocks have been
implemented, followed by one global average pool layer,
and an output layer having a softmax activation function is
used.
The third and fourth hidden layers have fire block mod-
ules, followed by the fifth layer having a maxpool layer. The
sixth and seventh hidden layers contain fire block modules,
again followed by the maxpool layer and fire block module.
The second conv layer is followed by the global average
pool layer. The output layer comprises of five nodes using
the softmax activation function to categorize five various
classes of cardiac sounds [24], [29].
Characteristic plots are taken up for the proposed system
implemented with normal and abnormal cardiac sound
samples, and these are provided in Figs. 11 and 12.
Fig. 11 presents the cross-entropy loss of the adopted
squeeze network during the training and validation stage
applied on dataset1. The curve of cross-entropy loss with the
number of epochs (i.e., 100) predicts that loss reduces as the
2500717
SINHA ROY et al.: CONV-RF-BASED IoT: A DEEP LEARNING MODEL BASED ON CNN AND RF
TABLE 5. Proposed squeeze NET layers.
FIGURE 11. Characteristics curve of loss versus epoch during training and
validation in CNN-Based squeeze net.
TABLE 6. Description of squeeze network MODEL effectiveness.
number of epochs grows during the training and validation
stage for the proposed squeeze network.
Fig. 12 is the accuracy of the improved squeeze network
during the training and validation stage applied on dataset1.
The curve of accuracy versus the number of epochs (i.e., 100)
predicts that accuracy rises close to 1 as the number of
epochs grows during the training and validation stage for
the proposed CNN system.
2500717
FIGURE 12. Characteristic curve of average accuracy versus epoch.
FIGURE 13. Flow diagram of RF.
Table 6 represents the proposed CNN-based squeeze
network that achieved an accuracy of 98.65%.
E. RANDOM FOREST ALGORITHM
RF is an established and popular supervised ensem-
ble machine learning method used mainly for real-time
classification-based fields [22], [25], [50]. This method is
also used to challenge its effectiveness in classifying normal
and abnormal cardiac samples.
In Fig. 13, a cardiac sound repository is split into the train-
ing and test data. Training data is further decomposed into
800 data samples called estimators for producing predictions.
The eventual predicted result for valvular cardiac sound [50]
is generated via the mean of all the predictions obtained
through the estimators.
Fig. 14 provides [50] the accuracy of the modified RF
system during the training and validation stage. The curve
of accuracy versus training sample size depicts that accu-
racy during the training and validation stage goes down to
one as the training sample data grows for the improved RF
model.
The effectiveness of the RF system is evaluated, and
various parameters are selected for the computation of its
efficiency as provided in Table 7. In Section V, by studying
VOLUME 2, 2023

FIGURE 14. Characteristics curve of accuracy versus epoch in RF.
TABLE 7. Description of RF model effectiveness.
the results of Figs. 20 and 21, the model shall categorize the
cardiac samples more effectively with better accuracy than
the other classifiers.
F. ARCHITECTURE OF THE ADOPTED CONV-RANDOM
FOREST
The proposed model of Conv-RF is given in Fig. 15. The
proposed model contains six layers: 1) input layer; 2) data
preprocessing layer; 3) conv layers; 4) reshape layer; 5) class
prediction layer; and 6) output layer.
All layers are subdecomposed into two sections: one for
feature extraction and another for the prediction of the class
labels. Each layer has a different function.
Feature Extraction Section: This section extracts the
important information from the data samples undergoing
training. It contains three layers: 1) the input; 2) data pre-
processing; and 3) conv layers. The prediction accuracy of
the proposed system relies on efficient feature learning.
Details of each layer are explained as follows.
1) Input Layer: The input layer takes input from the stan-
dard heart sound sample in the proposed model. It is
considered that a training data set, X, contains a set of
tuples √(xj , yj ),√where j denotes the index of the data set.
xj is a N × N feature vectors, and yj stands for class
label considered for vector, xj . In case the training set
is in the mentioned shape, it shall be directly allowed
to the conv layers of the feature extraction stage. Else,
it shall be transformed in the data preprocessing layer.
2) Data Preprocessing Layer: In the data preprocessing
layer, a square matrix shape is considered for the ten-
sors in the convolution block. If√the input √ data is not
in the matrix shape, with size N × N, it will be
VOLUME 2, 2023
converted, by adding zeros as required. Various data
types are also transformed in this layer.
3) Convolutional Layer: The convolutional layers play
a significant role in this proposed model. They are
responsible for learning the features of the input by
applying a convolution method known as Squeeze
network.
√ √ The data is basically a tensor, with shape,
N, N, z(l) , where z(l) is the total filters in the l
layer
⎛ ⎞
f (l−1)

γi (l) = ∅ ⎝Bi (l) + kij (l) ∗ γj (l − 1)⎠.
j=1
The pooling layer changes the convolution layer outcome.
The said model downsamples and restricts the overfitting of
the proposed model. It modifies the outcome with the high-
est or mean value within a rectangular window of square
matrix form. For instance, if (γil )m,n is an output of the ear-
lier layer with ∅ an activation function, then P(.) is a pooling
function that acts on (γil ) by allowing it through a pool-
ing method with stride, Sp , and wp × hp pooling window.
Typically, pooling works by placing windows at nonover-
lapping places in every feature map and taking one item
per window so that the feature maps are subsampled. Two
types of poolings are mainly used: 1) average pooling and
2) max pooling. In max pooling, the highest value of every
window is considered. Hence, the output of a max-pooling
function is
P(γ li )m,n = max(γ li )m,n
where the max function is fed to the max-pooling window
of the mentioned shape.
Predicting the Class Labels: RF is an ensemble algo-
rithm that is better than a single DT because it decreases
over-fitting by taking the mean of the outcome. It works
very fine with a large number of data elements than a
single DT.
Reshape Layer: This layer converts the convolutional layer
tensor output into the required vector form.
Class Prediction Layer: The major application in this layer
is to predict the class using RF.
Output Layer: The output layer shall receive the predicted
class result, and depending on that accuracy of the model
can be computed.
Fig. 15 is the architectural block diagram of the
convolutional-RF model. The proposed Conv-RF model
comprised of feature learning part and predicting the class
label part. Table 8 provides a summary of volunteers having
different age groups and genders with past medical history.
Table 9 highlights PCG signal analysis made on selected
volunteers in different postures, such as sitting, standing,
and supine with different locations like upper right sternal
border (URSB), upper left sternal border (ULSB) position,
and lower left sternal border (LLSB).
Table 10 is the analysis of PCG recordings done with
the developed stethoscope on the selected volunteers with
2500717
SINHA ROY et al.: CONV-RF-BASED IoT: A DEEP LEARNING MODEL BASED ON CNN AND RF

FIGURE 15. Architectural block diagram of the proposed Conv-RF model.

TABLE 8. Description of the volunteers [50], [51].

FIGURE 16. Operational process flow diagram in the proposed Conv-RF model.

past medical history. The comparison of the past medical Fig. 16 shows the operational process flow diagram in
history of the volunteers with the result obtained from the the proposed Conv-RF model, which highlights the different
said stethoscope is carried out. For evaluation purposes, a tensor transformations in the input convolutional layer and
score value is obtained in the range of 1 to 5. reshape layer. Table 11 is the architecture of the proposed

2500717 VOLUME 2, 2023

TABLE 9. Proposed conv-RF model applied on test volunteers [50], [51].
Conv-RF model adopted in the valvular cardiac sound
analysis.
Fig. 17 presents the accuracy of the said Conv-RF system
during the training and validation stage applied on dataset2.
The curve of accuracy with the number of epochs shows
that accuracy rises as the number of epochs grows for the
adopted model.
Fig. 18 presents the cross-entropy loss of the adopted
Conv-RF model during the training and validation stage
applied on dataset2. The curve of cross-entropy loss with
the number of epochs predicts that cross-entropy loss
reduces as the number of epochs grows for the used CNN
model.
VOLUME 2, 2023
V. RESULT ANALYSIS
Experimental Methods and Results: In this experiment, var-
ious standard datasets have been considered, and fivefold
cross-validation is used for the training and test data samples.
The results from each testing set are recorded, and the aver-
age value is computed, as shown in the figure below. The
ensemble RF classifier result is compared with other ensem-
ble methods, such as single DT, GB, and XGB method.
The result obtained with the proposed Convolutional-RF
algorithm is compared with different CNN-based models
like LeNet-5, Alex Net, VGG16, VGG19, DenseNet121,
Inception Net, Residual Net, Xception Net, and ConvXGB.
It is found that the Conv-RF method gives the best result
2500717
SINHA ROY et al.: CONV-RF-BASED IoT: A DEEP LEARNING MODEL BASED ON CNN AND RF

TABLE 10. Analysis of PCG recordings produced using the proposed stethoscope with past medical history [50], [51].

TABLE 11. Proposed conv-RF architecture.

in terms of valvular heart sound analysis. Figs. 19 and 20 Fig. 21 is the comparison of the developed Conv-RF
highlight the comparison of the RF algorithm with other model with other CNN-based models for different datasets.
ensemble algorithms and other models for different datasets The electronic stethoscopes available in the market cost
used in this article. around €300–€399. The total expenditure incurred for the

2500717 VOLUME 2, 2023


FIGURE 17. Characteristics curve of accuracy versus epoch in conv-RF.
FIGURE 18. Characteristics curve of loss versus epoch during training and
validation.
FIGURE 19. RF model versus other ensemble classifier models for different
datasets.
development of the proposed system for predicting heart dis-
eases is only around €220. Since auscultation is done through
a Bluetooth-enabled speaker, it is safe to use for patients
as well as for health professionals. The said stethoscope is
also very easy to use as it is AI enabled. Table 12 is the
VOLUME 2, 2023
FIGURE 20. RF model versus other models for different datasets.
TABLE 12. Comparative study of the developed stethoscope with other
stethoscopes.
comparison study of the developed stethoscope with other
stethoscopes based on various factors like use, price, safety
and protection, digital storage, etc.
Table 13 provides the comparison study of runtime (in
seconds) of the Conv-RF model with the RF method and
Squeeze Net-based CNN model for different datasets used
in the heart sound analysis. The runtime environment of
the proposed model is the Raspbian operating system with
Thonny ide in Raspberry Pi 4B using Python version 3.9.2.
Table 14 provides the comparison study of runtime (in
sec) of the Conv-RF model with other ensemble learning
methods like DT, GB method, and XGB method.
The proposed model is also compared with SVM and
MLPs for different datasets used in the valvular heart sound
analysis.
VI. CONCLUSION AND FUTURE SCOPE
Experimental data showed that the Conv-RF model provides
decent outputs in terms of accuracy, sensitivity, recall, and
f 1-score. This is also observed that the runtime (in sec-
onds) of the Conv-RF method is the lowest among all other
methods for different datasets used. The lowest runtime is
very helpful in having a fast early screening of any kind
of valvular cardiac disorder. The proposed modified squeeze
network is highly compatible with the ensemble RF method
in terms of their respective architectural breakdown. A few
limitations of the proposed model are the ambient noise
that cannot be removed to a full extent. The cable of the
microphone chest piece movement generates noise during
2500717
SINHA ROY et al.: CONV-RF-BASED IoT: A DEEP LEARNING MODEL BASED ON CNN AND RF
FIGURE 21. Conv-RF model versus other models for different datasets.
TABLE 13. Runtime of the proposed Conv-RF model compared with other models.
TABLE 14. Runtime of the proposed Conv-RF model compared with other ensemble methods.
auscultation. Some of the future scopes of this research are
auscultation time which is closely around 2 min and requires
further minimization. More number of volunteers with clin-
ical assessment are needed for statistical validation of the
developed model.
VII. DISCUSSION
In the hardware development part, this work deals with the
design of an Internet-enabled electronic stethoscope. The
2500717
proposed electronic stethoscope is based on the combination
of Raspberry Pi and ESP32.
The Signal conditioned output goes to the node MCU
(ESP32) that is WiFi enabled and contains a 12-bit ADC with
a sampling frequency of 44.1 kHz. Finally, the converted
digital PCG signal goes to the Raspberry Pi through its
enabled WiFi for further signal processing and analysis. The
classified heart sound is displayed on a 7-inch LCD screen
attached with the Raspberry Pi and is heard in a Bluetooth-
enabled microspeaker.
VOLUME 2, 2023
 In the software development part, a novel CNN-based
convolutional-RF algorithm is developed. The squeeze
network is used as a CNN model for feature extraction and
the RF method is used as a classifier in the output part. The
combination of both of them proved to be a decent valvular
heart sound classification algorithm in this field.
DECLARATIONS
The authors have worked with the same type of heart sounds
that have been used in [50] and [51] by T. S. Roy et al. in
2023. The same set of online available heart sound reposito-
ries are used in this paper for training the proposed Conv-RF:
A CNN-based deep learning model which is used in [50]
and [51].
Analysis of captured heart sound recordings is done on the
same set of volunteers using the same positions and postures
that have been done in [50] and [51] by T. S. Roy et al. in
2023. Postures considered are sitting, standing, and supine.
Positions considered are upper left sternal border (ULSB),
upper right sternal border (URSB), and lower left sternal
border (LLSB).
AUTHOR CONTRIBUTIONS
All authors contributed to the algorithm conception and
design. Experimental evaluation was also performed by
all authors. Eventually, all authors have gone through and
confirmed the final manuscript.
DECLARATION OF COMPETING INTEREST
The authors confirm no conflict of interest.
FUNDING
Not applicable, as the entire funding for this research work
was provided by the authors themselves.
DATA AVAILABILITY STATEMENT
All heart sound data samples used for training, validation,
and testing of the proposed model are available in standard
heart sound repositories like:
1) Yaseen et al. [15] https://github.com/yaseen21khan/
Classification-of-Heart-Sound-Signal-Using-Multiple-
Features-.
2) 2016 Physio Net heart sound recording challenge. [21]
https://physionet.org/content/challenge-2016/1.0.0/
3) 2012 pascal heart sound recording challenge. [32]
http://www.peterjbentley.com/heartchallenge/.
4) Kaggle heartbeat sounds [49]
https://www.kaggle.com/datasets/kinguistics/heartbeat-
sounds
REFERENCES
[1] A. K. Dwivedi, S. A. Imtiaz, and E. Rodriguez-Villegas, “Algorithms
for automatic analysis and classification of heart sounds—A systematic
review,” IEEE Access, vol. 7, pp. 8316–8345, 2019.
[2] M. Mishra, H. Menon, and A. Mukherjee, “Characterization of S1 and
S2 heart sounds using stacked autoencoder and convolutional neural
network,” IEEE Trans. Instrum. Meas., vol. 68, no. 9, pp. 3211–3220,
Sep. 2019.
VOLUME 2, 2023
[3] M. Mishra, S. Banerjee, D. C. Thomas, S. Dutta, and A. Mukherjee,
“Detection of third heart sound using variational mode decomposi-
tion,” IEEE Trans. Instrum. Meas., vol. 67, no. 7, pp. 1713–1721,
Jul. 2018. [Online]. Available: https://ieeexplore.ieee.org/document/
8306138
[4] P. R. Muduli, A. K. Mandal, and A. Mukherjee, “An anti-noise-folding
algorithm for the recovery of biomedical signals from noisy measure-
ments,” IEEE Trans. Instrum. Meas. vol. 66, no. 11, pp. 2909–2916,
Nov. 2017, doi: 10.1109/TIM.2017.2734018.
[5] S. Barma, B.-W. Chen, W. Ji, F. Jiang, and J.-F. Wang, “Measurement
of duration, energy of instantaneous-frequencies, and splits of sub-
components of the second heart sound,” IEEE Trans. Instrum. Meas.,
vol. 64, no. 7, pp. 1958–1967, Jul. 2015.
[6] N. K. Dewangan, S. P. Shukla, and K. Dewangan, “PCG signal analysis
using discrete wavelet transform,” Int. J. Adv. Manage. Technol. Eng.
Sci., vol. 8, no. 3, pp. 412–417, 2018.
[7] G. Mishra, K. Biswal, and A. K. Mishra, “Denoising of heart sound
signal using wavelet transform,” Int. J. Res. Eng. Technol., vol. 2,
no. 4, pp. 719–723, 2013.
[8] M. Z. Othman and A. N. Khaleel, “Phonocardiogram signal anal-
ysis for murmur diagnosing using Shannon energy envelop and
sequenced DWT decomposition,” J. Eng. Sci. Technol., vol. 12, no. 9,
pp. 2393–2402, 2017.
[9] P. Lubaib and K. V. A. Muneer, “The heart defect analysis based on
PCG signals using pattern recognition techniques,” in Proc. Int. Conf.
Emerg. Trends Eng., Sci. Technol., vol. 24, 2016, pp. 1024–1031.
[10] M. Singh and A. Cheema, “Heart sounds classification using feature
extraction of phonocardiography signal,” Int. J. Comput. Appl., vol. 77,
no. 4, pp. 13–17, 2013.
[11] T. J. Ahmad, H. Ali, and S. A. Khan, “Classification of phono-
cardiogram using an adaptive fuzzy inference system,” in Proc. Int.
Conf. Image Process., Compute. Vis., Pattern Recognit., vol. 2, 2009,
pp. 1–6.
[12] A. K. Roy, A. Misal, and G. R. Sinha, “Classification of PCG signals:
A survey,” Int. J. Comput. Appl., Recent Adv. Inf. Technol., vol. 83,
no. 2, pp. 22–26, Jan. 2014.
[13] C. N. Gupta, R. Palaniappan, S. Rajan, S. Swaminathan, and
S. M. Krishnan, “Segmentation and classification of heart sounds,” in
Proc. Int. Conf. Can. Conf. Electr. Comput. Eng., Jun. 2005,
pp. 1674–1677, doi: 10.1109/CCECE.2005.1557305.
[14] A. Mdhaffar, I. B. Rodriguez, K. Charfi, L. Abid, and
B. Freisleben, “CEP4HFP: Complex event processing for heart fail-
ure prediction,” IEEE Trans. NanoBiosci., vol. 16, no. 8, pp. 708–717,
Dec. 2017, doi: 10.1109/TNB.2017.2769671.
[15] Yaseen, G.-Y. Son, and S. Kwon, “Classification of heart sound signal
using multiple features,” Appl. Sci., vol. 8, no. 12, p. 2344, 2018,
doi: 10.3390/app8122344.
[16] “Your first deep learning project in Python with Keras step.” 2022.
[Online]. Available: https://machinelearningmastery.com/tutorial-first-
neural-network-python-keras/
[17] B. Xiao, Y. Xu, X. Bi, J. Zhang, and X. Ma, “Heart sounds
classification using a novel 1-D convolutional neural network with
extremely low parameter consumption,” Neurocomputing, vol. 392,
pp. 153–159, Jun. 2020, doi: 10.1016/j.neucom.2018.09.101.
[18] G. V. H. Prasad and P. R. Kumar, “Analysis of various DWT methods
for feature extracted PCG signals,” Int. J. Eng. Res. Technol., vol. 4,
no. 4, pp. 1279–1290, 2015.
[19] Anju and S. Kumar, “Detection of cardiac murmur,” Int. J. Comput.
Sci. Mobile Comput., vol. 3, no. 7, pp. 76–80, 2014.
[20] J. K. Roy and T. S. Roy, “A simple technique for heart sound detection
and real-time analysis,” in Proc. Int. Conf. Sens. Technol. (ICST), 2017,
pp. 1–7, 10.1109/ICSensT.2017.8304502.
[21] E. F. Gomes, P. J. Bentley, M. Coimbra, E. Pereira, and Y. Deng,
“Classifying heart sounds: Approaches to the PASCAL challenge,” in
Proc. Int. Conf. Health Informat., Barcelona, Spain, Feb. 2013,
pp. 337–340.
[22] B. Bozkurt, I. Germanakis, and Y. Stylianou, “A study of time–
frequency features for CNN based automatic heart sound classification
for pathology detection,” Comput. Biol. Med., vol. 100, pp. 132–143,
Sep. 2018, doi: 10.1016/j.compbiomed.2018.06.026.
[23] J. Muruganantham, R. Amarnath, K. V. Jawahar, and
C. Kalyanasundaram, “Methods for classification of phonocar-
diogram,” in Proc. Conf. Convergent Technol. Asia–Pacific Region,
vol. 4. Bengaluru, India, 2003, pp. 1514–1515.
2500717
SINHA ROY et al.: CONV-RF-BASED IoT: A DEEP LEARNING MODEL BASED ON CNN AND RF
[24] T. H. Chowdhury, K. N. Poudel, and Y. Hu, “Time-frequency
analysis, denoising, compression, segmentation, and classifica-
tion of PCG signals,” IEEE Access, vol. 8, pp. 160882–160890,
2020.
[25] J. K. Roy, T. S. Roy, and S. C. Mukhopadhyay, “Heart
sound: Detection and analytical approach towards diseases,” in
Modern Sensing Technologies, S. Mukhopadhyay, K. Jayasundera,
O. Postolache, Eds. Cham, Switzerland: Springer, 2019, pp. 103–145.
[Online]. Available: https://doi.org/10.1007/978-3-319-99540-3_7
[26] F. Li, H. Tang, S. Shang, K. Mathiak, and F. Cong, “Classification of
heart sounds using convolutional neural network,” Appl. Sci., vol. 10,
no. 11, p. 3956, 2020, doi:10.3390/app10113956.
[27] J. K. Roy, T. S. Roy, N. Mandal, and O. A. Postolache, “A simple
technique for heart sound detection and identification using Kalman
filter in real time analysis,” in Proc. Int. Symp. Sens. Instrum. IoT Era
(ISSI), 2018, pp. 1–8.
[28] D. B. Springer, L. Tarassenko, and G. D. Clifford, “Support vector
machine hidden semi Markov model-based heart sound segmenta-
tion,” in Proc. Comput. Cardiol., 2014, pp. 625–628.
[29] A. Cheema and M. Singh, “Steps involved in heart sound analysis—A
review of existing trends,” Int. J. Eng. Trends Technol., vol. 4, no. 7,
pp. 2921–2925, 2013.
[30] J. B. Wu, S. Zhou, Z. Wu, and X. M. Wu, “Research on the method
of characteristic extraction and classification of phonocardiogram,” in
Proc. Int. Conf. Syst. Informat. (ICSAI), 2012, pp. 1732–1735.
[31] C. D. Papadaniil and L. J. Hadjileontiadis, “Efficient heart sound
segmentation and extraction using ensemble empirical mode decompo-
sition and kurtosis features,” IEEE J. Biomed. Health Inform., vol. 18,
no. 4, pp. 1138–1152, Jul. 2014.
[32] C. Liu et al., “An open-access database for the evaluation of heart
sound algorithms,” Physiol. Meas., vol. 37, no. 12, pp. 2181–2213,
2016. [Online]. Available: https://iopscience.iop.org/article/10.1088/
0967-3334/37/12/2181
[33] M. Deng, T. Meng, J. Cao, S. Wang, J. Zhang, and H. Fan, “Heart
sound classification based on improved MFCC features and convolu-
tional recurrent neural networks,” Neural Netw., vol. 130, pp. 22–32,
Oct. 2020.
[34] Z. Abduh, E. A. Nehary, M. A. Wahed, and Y. M. Kadah,
“Classification of heart sounds using fractional fourier transform based
mel-frequency spectral coefficients and traditional classifiers,” Biomed.
Signal Process. Control, vol. 57, Mar. 2020, Art. no. 101788.
[35] T. Alafif, M. Boulares, A. Barnawi, T. Alafif, H. Althobaiti, and
A. Alferaidi, “Normal and abnormal heart rates recognition using
transfer learning,” in Proc. 12th Int. Conf. Knowl. Syst. Eng. (KSE),
2020, pp. 275–280.
[36] F. Demir, A. Sengür, V. Bajaj, and K. Polat, “Towards the classification
of heart sounds based on convolutional deep neural network,” Health
Inf. Sci. Syst., vol. 7, p. 16, Aug. 2019.
[37] B. Xiao et al., “Follow the sound of children’s heart:
A deep-learning-based computer-aided pediatric CHDs diagnosis
system,” IEEE Internet Things J., vol. 7, no. 3, pp. 1994–2004,
Mar. 2020.
[38] F. A. Khan, A. Abid, and M. S. Khan, “Automatic heart sound classi-
fication from segmented/unsegmented phonocardiogram signals using
time and frequency features,” Physiol. Meas., vol. 41, no. 5, 2020,
Art. no. 55006.
[39] A. Raza, A. Mehmood, S. Ullah, M. Ahmad, G. S. Choi, and B.-W. On,
“Heartbeat sound signal classification using deep learning,” Sensors,
vol. 19, no. 21, p. 4819, 2019.
[40] H. Ryu, J. Park, and H. Shin, “Classification of heart sound recordings
using convolution neural network,” in Proc. Comput. Cardiol. Conf.
(CinC), Vancouver, BC, Canada, Sep. 2016, pp. 1153–1156.
[41] J. Rubin, R. Abreu, A. Ganguli, S. Nelaturi, I. Matei, and K. Sricharan,
“Classifying heart sound recordings using deep convolutional neu-
ral networks and mel-frequency cepstral coefficients,” in Proc.
Comput. Cardiol. Conf. (CinC), Vancouver, BC, Canada, 2016,
pp. 813–816.
[42] V. Maknickas and A. Maknickas, “Recognition of normal-abnormal
phonocardiographic signals using deep convolutional neural networks
and mel-frequency spectral coefficients,” Physiol. Meas., vol. 38, no. 8,
pp. 1671–1684, 2017.
2500717
[43] F. Li et al., “Feature extraction and classification of heart sound using
1D convolutional neural networks,” EURASIP J. Adv. Signal Process.,
vol. 10, p. 59, Dec. 2019.
[44] J. M.-T. Wu et al., “Applying an ensemble convolutional neural
network with Savitzky–Golay filter to construct a phonocardio-
gram prediction model,” Appl. Soft Comput., vol. 78, May 2019,
pp. 29–40.
[45] T. C. Yang and H. Hsieh, “Classification of acoustic physiologi-
cal signals based on deep learning neural networks with augmented
features,” in Proc. Comput. Cardiol. Conf. (CinC), Vancouver, BC,
Canada, Sep. 2016, pp. 569–572.
[46] Q. Suo et al., “Deep patient similarity learning for personalized
healthcare,” IEEE Trans. NanoBiosci., vol. 17, no. 3, pp. 219–227,
Jul. 2018, doi: 10.1109/TNB.2018.2837622.
[47] A. Mario et al., “Cardiac conduction model for generating 12
lead ECG signals with realistic heart rate dynamics,” IEEE Trans.
NanoBiosci., vol. 17, no. 4, pp. 525–532, Oct. 2018.
[48] D. Li, M. Huang, X. Li, Y. Ruan, and L. Yao, “MfeCNN:
Mixture feature embedding convolutional neural network for data
mapping,” IEEE Trans. NanoBiosci., vol. 17, no. 3, pp. 165–171,
Jul. 2018, doi: 10.1109/TNB.2018.2841053.
[49] “Kaggle heartbeat sounds.” kaggle.com. Accessed: Feb. 28, 2021.
[Online]. Available: https://www.kaggle.com/datasets/kinguistics/
heartbeat-sounds.
[50] T. S. Roy, J. K. Roy, and N. Mandal, “Classifier identification using
deep learning and machine learning algorithms for the detection
of valvular heart diseases,” Biomed. Eng. Adv., vol. 3, Jun. 2022,
Art. no. 100035.
[51] T. S. Roy, J. K. Roy, and N. Mandal, “Design of ear-contactless
stethoscope and improvement in the performance of deep learning
based on CNN to classify the heart sound,” Med. Biol. Eng. Comput.,
vol. 61, pp. 2417–2439, Apr. 2023. [Online]. Available: https://doi.
org/10.1007/s11517-023-02827-w
[52] T. S. Roy, J. K. Roy, and N. Mandal, “Early screening of valvular
heart disease prediction using CNN-based mobile network,” in Proc.
Int. Conf. Comput., Electr. Commun. Eng. (ICCECE), Kolkata, India,
2023, pp. 1–8, doi: 10.1109/ICCECE51049.2023.10085513.
TANMAY SINHA ROY was born in India, in 1988.
He received the B.Tech. degree in instrumen-
tation and control engineering and the M.Tech.
degree in applied electronics and instrumentation
engineering from the West Bengal University of
Technology, Kolkata, India, in 2009 and 2011,
respectively. He is currently pursuing the Ph.D.
degree with the Indian Institute of Technology
(Indian School of Mines) Dhanbad, Dhanbad,
India.
He is currently an Assistant Professor with the
Electrical Engineering Department, Haldia Institute of Technology, West
Bengal University of Technology. His research interests include PCG signal
analysis, developing systems for heart sound acquisition, instrumentation,
and control, and designing low-cost acoustic stethoscopes for diseased
patients.
VOLUME 2, 2023
 JOYANTA KUMAR ROY (Senior Member, IEEE)
received the Ph.D. degree from Calcutta
University, Kolkata, India, in 2005.
He has been an Electronics and Automation
Engineer for the last 40 years as a Company
Director, Consulting, Engineer, Developer,
Researcher, and Educationist. He is a Visiting
Professor with the Narula Institute of Technology,
Kolkata, a Company Director with System
Advance Technologies, Kolkata, and a Freelance
Consultant with a number of industries to give
design support toward intelligent technology in the water sector. He
published a significant number of scientific and technical publications in
the form of books, book chapters, design documents, and research papers.
His research interests include the development of smart measurement and
control systems, multifunction sensors, IoT-based health and technology-
assisted living, and smart homes and cities.
Dr. Roy is an Associate Editor of the International Journal on Smart
Sensing and Intelligent Systems. He is a Senior Member of IET, a Fellow
Member of IETE and IWWA, and a regular reviewer of IEEE and Springer
journals.
VOLUME 2, 2023
NIRUPAMA MANDAL (Senior Member, IEEE)
received the Ph.D. degree from Calcutta
University, Kolkata, India, in 2012.
She is currently an Associate Professor with
the Department of Electronics Engineering,
Indian Institute of Technology (Indian School
of Mines) Dhanbad, Dhanbad, India. She was
the Head of the Department of Electronics and
Instrumentation Engineering, Asansol Engineering
College, Asansol, India, in 2013. Her research
interests include transducer development, con-
troller design, process plant instrumentation, process modeling, smart
sensing system, and smart instrumentation.
Dr. Mandal received the National Scholarship Award from the
Government of India in 2001.
2500717