Computer application

A computer application was created with the aim to display rehabilitation exercises which must be
done by the patient, and also to see previous sensor data stored in a database, in order to check
patient’s rehabilitation progress.

1.1.1. Architecture design

Sensor data is sent from Arduino to a MySQL database via WiFi. This data is stored in a database in
different tables and columns. Then a graphical user interface (GUI) is created using Python, with the
Tkinter library which has many resources for GUI design.

Figure 1. Computer application architecture

1.1.2. Database design

To store sensor values, seven different tables are created. The training table contains a column named
running, which will have a value of 0 if the no patient is doing the rehabilitation exercices and 1 if some
is. The training id is updated whenever a new training begins.

The remaining 6 tables are used to store sensor data, with sensor values columns, a timestamp column
usual for previous data visualization, a num column which is the primary key and allows data
representation, and a training id column which is used to select all data from a specific training.

Figure 2. Tables from the project's database

1.1.3. Arduino and MySQL communication

Arduino data is inserted into a specific database via WiFi connection. First of all, a MySQL user must be
created in the server and it must have permission to manipulate the database (insert or select data for
example). An IP adress must be assigned to this user, and in this case it must be the board’s IP adress
when it is connected to the WiFi network, so data can be sent from the board to the server.

MySQL connector library for Arduino was used to insert data to the server and to select data from the
database [59].
 Figure 3. MySQL server User creation, using the IP adress of the ESP32 board.

Figure 4. Setting all Administrative Roles to the user so it can interact with the server.

1.1.4. Computer Application design

Tkinter library from Python was used to design the GUI. It allows to create different frames, add
buttons, text labels, insert images, plots and many other tools [60].

SQLAlchemy is the database tool for Python. This library was used to connect to the database server
and to make queries, such as select, insert or manipulate data [61].

Pandas library is a fast, powerful, flexible and easy to use open source data analysis and manipulation
tool, built on top of the Python programming language. This was used to store selected data in data
frames to be plotted later on the application [62].
Finally, Matplotlib library was used to plot resulting data, which is also a widely used library in Python
for data visualization. Matplotlib is a comprehensive library for creating static, animated, and
interactive visualizations in Python [63]
1.1.5. Computer Application results

Figure 5. Diagram of the computer application strucuture.

In the start page of the application, the user can choose to start the training exercises, to see previous
training results, or to see last results.

Figure 6. Start page of the computer application.

In the training page the user can start the training video and do the proposed hand rehabilitation
exercises. When the Start video button is pressed, the ESP32 starts reading sensor values and sending
them sequentially to the database server.

Figure 7. Training page of the computer application.

The exercises showed in the training video are shown on the figure below. The first two exercises are
meant to measure strength with the FSR sensors, the third exercise is used to measure finger
bending, the exercises on the fourth figure are used to calculate the wrist range of movement (roll,
pitch and yaw angles) and the last figure which is fist clenching is used to record the EMG signal.

Figure 8. Grip strengthening [64]. Figure 9. Flat pinch [64].

Figure 10. Fingers bending exercises [69].

.
Figure 11. Wrist range of motion exercises [70].

Figure 12. Fist clenching [65].

These exercises were chosen since they allow to measure fingers and wrist range of motion, pressure
applied when gripping or pinching objects, and muscle activation thanks to the EMG sensor. All these
variables can be measured using the sensors of this prototype Smart Wearable glove. Different
exercises and routines can be designed according to the therapist’s criteria.

If clicking the previous results button dates from previous trainings are displayed. The user can select
the day and time of the trainings available and choose what sensor data he wants to see.

Figure 13. Previous date trainings selection.

Figure 14. Sensor data selection from a previous training on a specific date.
The results corresponding to the force sensitive resistors are given in grams as the corresponding
weight applied over the sensor, but it couls also be specified in Newtons or in mmHg by doing a simple
calculus.

Figure 15. Pinky finger FSR results from a previous training.

In the figures below the results from the wrist range of motion are shown. They are expressed in
degrees and values are between -90 and 90 degrees.

Figure 16. Wrist rotation and flexion results from a previous training.
 Figure 17. Wrist rotation results from a previous training.

Figure 18. Wrist adduction and abduction results from a previous training.

In the Last Results page, the user can select the data from each of the sensors and plot the results for
the last training.

Figure 19. Last results page of the computer application.

The following figures correspond to the results of the same sensors as earlier but from a new training.

Figure 20. Pinky finger FSR results.

Figure 21. Wrist flexion and extension results.

 Figure 22. Wrist rotation results.

Figure 23. Wrist adduction and abduction results.

Also the toolbox at the left bottom of the plot, allows to zoom in the graph, change its size and location,
and to save it as a picture on the computer.

When moving the mouse over the plot, we can see the exact x and y values at the right bottom of the
figure.

Figure 24. X and Y value at the bottom right of the figure.