DESIGN AND IMPLEMENTTATION OF A BIONIC ARM

INTERFACE
BY

MAKAFU JOSEPH JAMES

19/U/BIO/178/GV

A PROJECT REPORT SUBMITTED TO KYAMBOGO

UNIVERSITY IN PARTIAL FULFILLMENT OF THE
REQUIREMENTS FOR THE AWARD OF A BACHELORS
DEGREE IN MECHATRONICS AND BIOMEDICAL
ENGINEERING

1
DECLARATION
I MAKAFU JOSEPH JAMES, declare to the best of my knowledge and understanding that
this report was originally done by me with the exception of references quoted and has not
been published or submitted for any academic award to any university or institution before.

Signature Date

……………………………. ……………………………

2
APPROVAL
This is to certify that this report has been prepared by MAKAFU JOSEPH JAMES to the best
of his knowledge and experience. His report clearly reflects the work he did during the
project development and implementation.

Mr. OBETI FRANCIS

Signature…………………………………… Date……………………………………

3
DEDICATION
I dedicate this report to my beloved family for the tremendous work and selfless services
done towards my academic work. Their encouragement, advice, financial support and
devotion to my endeavors have kept me striving for greater heights.

May the Almighty God bless you in all your endeavors.

4
ACKNOWLEDGEMENT
First and foremost I thank the Almighty God who has kept me alive and healthy throughout
this period and completion of this work.

I also would like to express my gratitude to Mr. Obeti Francis for his enthusiastic
encouragements and precious instructions during project development period. I would also
like to thank my groupmates for the participation and timely response whenever called upon
which helped me develop both academically, professionally and socially.

Last but not least, I wish to express my deepest thanks to my family for their unconditional
love and support. Their sincere caring and support had helped me successfully completing my
project.

5
ABSTRACT
The purpose of this report is to provide an overview of the design and development of a
bionic arm interface using Arduino, MPU6050 sensor, and Unity 3D software. The design is
inspired by the biological structure and mechanism of the human hand. The project aimed to
create a functional interface that allows users to control a bionic arm using motion data
captured by the MPU6050 sensor and input from potentiometers. The interface was
developed by integrating hardware and software components to provide a seamless user
experience.

6
KEYWORDS
Data Glove: glove that contains various sensors and captures finger articulation, hand motion
and orientation

Accelerometer: circuit element that gives gravitational acceleration as an object is moved

Gyroscope: circuit element that gives rotational orientation of an object

Arduino ESP32 microcontroller: microcontroller board to control physical or virtual objects

Hardware/Software Interface: bridge that links hardware stimulation to virtual action

SCC (serial Communication channel): communication channel that links microcontroller to

PC and allows information passage between them

7
Table of Contents
DECLARATION........................................................................................................................................2
APPROVAL.............................................................................................................................................3
DEDICATION..........................................................................................................................................4
ACKNOWLEDGEMENT...........................................................................................................................5
ABSTRACT..............................................................................................................................................6
KEYWORDS............................................................................................................................................7
LIST OF FIGURES..................................................................................................................................10
CHAPTER ONE: INTRODUCTION...........................................................................................................11
1.1 Background................................................................................................................................11
1.2 Problem Statement....................................................................................................................11
1.3 Project objectives......................................................................................................................12
1.3.1 General objective................................................................................................................12
1.3.2 Specific Objectives..............................................................................................................12
1.4 Justification................................................................................................................................12
1.5 Significance................................................................................................................................12
1.6 Scope.........................................................................................................................................13
1.7 Methodology and tools..............................................................................................................14
1.7.1 Sensor Selection..................................................................................................................15
1.7.2 Hardware/Software Interface Design.................................................................................15
CHAPTER TWO: LITERATURE REVIEW..................................................................................................17
2.1 THEORETICAL DESIGN PRINCIPLES.............................................................................................17
2.2 Identification of Hand Motion...................................................................................................18
2.3 Contact technologies.................................................................................................................19
2.4 Research Paper [1].....................................................................................................................19
2.5 Commercial glove contact devices.............................................................................................21
2.5.1 Optical (VPL Data glove)......................................................................................................21
2.5.2 Resistor-based (Virtex Cyber glove)........................................................................................21
2.6 Negative aspects of the data gloves..........................................................................................21
2.7 Exoskeletons..............................................................................................................................22
2.8 Non-contact technologies..........................................................................................................23
CHAPTER THREE: PROJECT DEVELOPMENT.........................................................................................24
3.1 Equipment Design......................................................................................................................24
3.1.1 3D Parts..............................................................................................................................24

8
 3.1.2 Potentiometer....................................................................................................................25
3.1.3 ESP32 Microcontroller Unit................................................................................................26
3.1.4 MPU6050 Sensor (accelerometer + gyroscope)..................................................................27
3.1.5 Integrated Development Environment...............................................................................27
3.1.6 Virtual Reality Software......................................................................................................28
3.2 Cost Management.....................................................................................................................28
3.3 Theoretical Design of Signal Conversion Circuit Schematic.......................................................29
CHAPTER FOUR: RESULTS AND DISCUSSIONS......................................................................................30
4.1 3D PRINTED PARTS....................................................................................................................30
4.2 Acquisition by Arduino IDE and Processing IDE.........................................................................33
4.2 Unity 3D Simulation and Hardware Integration.........................................................................33
4.3 Unity 3D Simulation Environment Setup...................................................................................34
4.3.1 Virtual Arm Model..............................................................................................................34
4.3.2 Physics Properties...............................................................................................................34
4.3.3 User Interface Elements.....................................................................................................34
4.4 Integration of Hardware Components with Unity 3D................................................................34
4.4.1 Serial Communication.........................................................................................................34
4.4.2 Data Reception and Processing...........................................................................................35
4.4.3 Arm Control and Visualization............................................................................................35
4.4.4 Visual Feedback and Interaction.........................................................................................35
4.4.4 Testing and Calibration.......................................................................................................35
CHAPTER FIVE: CONCLUSION AND RECOMMENDATIONS...................................................................38
5.1 Conclusion.................................................................................................................................38
5.2 Recommendation......................................................................................................................38

9
LIST OF FIGURES

Figure 1: Overall process diagram (earlier plan for the project execution).............................14
Figure 2: Hardware/Software Interface of this capstone project.............................................16
Figure 3: Anatomical details of the hand exoskeleton.............................................................17
Figure 4: Classification of hand reconstruction techniques.....................................................18
Figure 5: Prototype Data Glove. Flex sensors mounted on the glove and white colored force
sensor taped around thumb.......................................................................................................20
Figure 6: System design of Data Glove with force sensors as presented in the research paper
..................................................................................................................................................20
Figure 7: Signal Conversion circuit (converts resistance to voltage)......................................20
Figure 8: Optical (VPL Data glove).......................................................................................21
Figure 9: Resistor-based (Virtex Cyber glove).....................................................................21
Figure 10: Exoskeleton used as a master to control artificial hand..........................................22
Figure 11: Final Assembly of All the parts..............................................................................32
Figure 12: Output from serial monitor in Arduino IDE...........................................................33

10
CHAPTER ONE: INTRODUCTION
This chapter gives a brief explanation of the project background, problem statement,
justification, objectives and summary of the methodology.
1.1 Background
Over the past decade of the new millennium, the world has seen the ever increasing
population around the globe. However, along with this increasing population trend, came
forth a whole new domain of technological innovations and improvements for the betterment
and sustenance of this population. Specifically, the innovations in the Computer & Gaming
industry and Health Care industry is so overwhelming that human life has become a lot easier
and appreciating. In medicine, for instance, the integration of medical science and
engineering has enabled almost anything from simple and efficient electronic record keeping
to complicated tele robotic surgery.
An interface is a combination of hardware and software which allows a user to interact with
computer systems. Computer interfaces refer to the various ways in which users interact with
computer systems. The goal of computer interfaces is to make it easy and intuitive for users
to interact with complex computer systems, allowing them to perform tasks and access
information quickly and efficiently.
Common types of computer interfaces include: the Graphical User Interface (GUI),
Command Line Interface (CLI), Natural Language Interface (NLI), Touch Screen Interface
and Virtual Reality (VR) Interface.
With the development of new technologies like natural language processing and virtual
reality, a number of novel interfaces that employ gesture control and motion capture
technologies have been developed to meet more demanding tasks of the modern times.

1.2 Problem Statement

Traditional control interfaces for bionic arms often involve complex input devices or limited
control options, which can be challenging for users to operate effectively. There is a need for
a more intuitive and seamless control system that can capture natural hand gestures and
translate them into precise movements of the bionic arm.

11
1.3 Project objectives
1.3.1 General objective
To design and implement a Bionic Arm-Interface.

1.3.2 Specific Objectives

 To research about the existing interfaces available in the market

 To identify key features regarding electronics, structure, and usability.

 To design the device's structural, electrical, and user interface elements.

 To test the functionality of the implemented hardware and software against different
performance metrics and safety standards

1.4 Justification
The interface provides precise control over movement, allowing the users to execute fine
tasks with greater accuracy

In Contrast to the Keyboard as a primary input device, the wearable piece of equipment is
more ergonomic and possesses a more natural form of interaction with computer systems.

The simplicity and reliability of the interface makes it a cost effective choice, ensuring a
wider accessibility for the user in need

1.5 Significance
The Bionic Arm-Interface is designed to be cheap and affordable for any user as the system is
immediately adaptable to different hand sizes and geometries. Once the system is working
properly, it can be easily adapted to capture the motion of any other body part.

The real time feedback provided by the potentiometers allows users to have a more intuitive
interaction with the bionic arm. They can adjust their grip strength and movement speed
seamlessly, making the device feel more natural and responsive.

12
The potentiometers allow for fine-tuned control over the bionic arms movement, enabling
users to perform delicate tasks with greater accuracy. This precision is crucial for tasks that
require careful manipulation.

The non-intrusiveness allows for more natural motion of the hand, and also allows for the
addition of new sensors without altering the motion sensing system.

1.6 Scope
The scope of this project included the design and development of the bionic arm interface
using the ESP32 microcontroller, MPU6050 sensor, potentiometers and Unity 3D software.

The focus was to process the data extracted from finger movements in a manner that lets us
extract signal information of interest.

Once the information was extracted we then used it for different purposes such as controlling
and manipulating an entity in a virtual environment.

13
1.7 Methodology and tools
The project consists of two major parts i.e. Data Glove implementation and simulation of
virtual Robot Manipulator design. These two parts are further subdivided into two smaller
parts:

 Implementation of motion capturing Data Glove fingers and real time articulation of
stick figure representation of virtual robot manipulator grippers
 Hand motion and orientation tracker and interfacing it with real-time virtual stick
figure model

To execute the project step by step, a process diagram has been developed. It is shown below

Analog signal Real time virtual finger

conditioning motion simulation

Potentiometers detect
the angular change and
outputs the resistance Real time command for
change virtual fingers

Actions: finger curling, ADC

flexion, extension, hand Programming to process
Analog to digital
movements digital data
conversion

MPU 6050
Real time action
Accelerometer +
command hand motion
gyroscope detects
change in relative
motion, position

Real time virtual hand

Accelerometer signal motion simulation
output

Figure 1: Overall process diagram (earlier plan for the project execution)

14
1.7.1 Sensor Selection
The motion capture of finger articulation and hand movement cannot be tracked by a single
sensor.

For capturing finger articulation, an appropriate method would be to use potentiometers that
produce real-time change in its parametric value when adjusted.

Usually potentiometers are resistive sensors that change their resistance value when adjusted.
Therefore, it must be converted to voltage for further signal analysis or modifications.

On the other hand, tracking the movement and orientation of hand in space can be
accomplished by an accelerator and/or gyroscope (MPU 6050).

Since a traditional analog output or response is a voltage waveform, it is necessary to obtain

real-time voltage outputs from the potentiometers and the accelerometer.

1.7.2 Hardware/Software Interface Design

Controlling any virtual or real object using an external controller requires a communication
channel between the controller and the controllable object. This channel is the interface that
links the two separate systems. There are several sophisticated and efficient methods
available through MATLAB-SIMULINK software systems that allow a programmer to both
establish a real-time virtual interactive system and link in to an external controller.

However, execution of these methods are rather complicated, time consuming and expansive.
Therefore, for this project, a simplistic approach has been adapted. The hardware/software
interface can be developed using the microcontroller Arduino, Arduino IDE and UNITY 3D.
The idea of the method is as following:

 The analog output from the potentiometers (and accelerometer) are supplied to the
microcontroller.
 The microcontroller is connected to the PC via a USB port for serial communication.
 The Arduino IDE (a C based programming environment) is used to program the
microcontroller.
 Arduino IDE has a serial port monitor that is used to monitor the real-time response of
the hardware in numerical values.
 Virtual hand is implemented in UNITY 3D. Using serial communication command in
Processing, a serial communication channel (SCC) can be established between virtual
hand, developed in processing, and the microcontroller connected to the sensors.

15
 The generalized data path for Serial Communication between PC and microcontroller is
shown below. As described above, following diagram is the overview of the method used to
implement the hardware software interface.

Serial Port Real time

Monitor simulation

Arduino Ide

Processing IDE
Potentiometer Arduino
Analog Inputs Microcontroller
PC

Figure 2: Hardware/Software Interface of this project

16
CHAPTER TWO: LITERATURE REVIEW
This chapter entails an overview of the different theoretical elements of the project and
includes the various technologies included in the system’
2.1 THEORETICAL DESIGN PRINCIPLES
The hand is part of the extremity of human located at the distal segment of an arm or fore
limb. It is composed of a palm with five fingers with the wrist connecting it to the forearm.
Human hands are considered as one of the most intricate mechanical structures of the body
with 27 bones and 24 degrees of freedom, as shown

Figure 3: Anatomical details of the hand exoskeleton

In hand function rehabilitation, quantitative analysis and measurements of human hand

movements play a vital role in designing hand robotic exoskeletons. Kinematic parameters
need to be evaluated in the development of human-machine interfaces. In general, kinematics
deal with the motions without assessing the forces that caused those movements. The study of
kinematics of the human hand involves measurements of hand and finger positions during
gestures, computation of velocities and accelerations, orientations, angular velocities and
accelerations.

17
2.2 Identification of Hand Motion
There is a great amount of work done in identifying the motion of the human body and, in
particular, of the human hand. The fields of interest are also diverse; much of the work has
been done in the area of computer graphics, in order to create realistic virtual motion for
avatar animation, for automatic hand language identification, for automatic sketching, etc.;
also in the areas of humanoid robotics, in order to program human-like motion in the robots,
and in the area of biomechanics.

The extraction of motion and posture information has been categorized by Varga et al.
according to three aspects, that we reproduce here as a good framework to classify the
previous and actual research. The extraction of motion information can be categorized
according to the sensing device used as contact (in which the device is mounted on, or
touches the hand) or non-contact (in which the information is extracted at a distance).
Regarding whether the whole hand or only some characteristic points are tracked, the
information is classified as complete or incomplete. This information can be transferred
directly to some geometric modeling system (direct transfer ) or the information can be fed to
an intermediate hand model that adjusts the raw information before sending it to the
geometric modeler (indirect transfer, also called model-based).

Figure 4: Classification of hand reconstruction techniques

18
2.3 Contact technologies
Contact devices can be classified as data gloves (with different sensing technologies),
electromagnetic emitter/receiver tracking systems, and exoskeletons.

Data Glove is one of the most exciting research topic at this current time. With accelerated
growth in the electronic industry, consumers demand to interact with virtual objects or remote
machinery in a realistic fashion. Examples of which include virtual gaming, touch panel
computers, human machine interaction, tele robotic surgery, virtual surgical training and
many more. Through literature research for this project, it has been discovered that many
research based on Data Glove, related to each of the developments mentioned above, has
been done. Some of those research paper, as they were reviewed for this project, include:
Data Glove with a force Sensor, 3D hand motion tracking and gesture recognition using a
Data Glove, Motion Sensitive Glove based Korean finger-spelling tutor, Teleoperation of
robot based on Data Glove and virtual reality, Hand-in-Glove Human-Machine Interface and
Interactive Control and Development of a functional neuromuscular stimulation system using
Data Glove.

Among these research papers, Data Glove with a force Sensor was the most valuable toward
the development of this project.

2.4 Data Glove with a Force Sensor

(Kostas N. Tarchanidis, Member, IEEE, and John N. Lygouras, Member, IEEE)

The objective of this paper is to demonstrate the development stages of a data glove equipped
with force sensors and bend sensors. This paper demonstrates that using flex sensors mounted
on the glove, the finger articulation can be captured. In addition, using force sensors mounted
on the glove, the force produced due to gripping mechanism of the hand can be obtained and
transferred through serial communication port and replicated within a remote mechanical
gripper. As a result, wearing the Glove, if an object is held with a certain force input (i.e. 4N),
a remote mechanical robotic arm should also produce the same grasp force. As mentioned in
the paper, this development can be used in robotics, telecheric, biomechanics, rehabilitation
and virtual reality applications. The following is the prototypical design of the Data Glove
used in the project.

19
Figure 5: Prototype Data Glove. Flex sensors mounted on the glove and
white colored force sensor taped around thumb.
The simplistic system design of the project presented in this research paper is as following:

Figure 6: System design of Data Glove with force sensors as presented in the research paper

The circuit is composed of three subdivided op-amps circuits known as the buffer, the adder
and the amplifier. The buffer circuit simply act as a voltage follower to output the voltage
produced at the voltage divider circuit. The adder circuit adds positive voltage value to the
buffer circuit so that the low negative voltage is offset by the comparably higher voltage of
the adder output. Finally, the small voltage is amplified through the amplifier circuit and
gives an amplified voltage that falls between desired voltage ranges.

Figure 7: Signal Conversion circuit (converts resistance to voltage)

20
2.5 Commercial glove contact devices
2.5.1 Optical (VPL Data glove)
Sensitive to hand size. Neoprene-fabric glove with two fiber
optic loops on each finger; each loop goes up to one knuckle.
That is the problem with the hand size. When hand is bent, light
escapes through small cuts of the fiber. Needs recalibration for
each user. DOF: 5 metacarpo-phalangeal joints, interphalangeal
joint of the thumb, 4 proximal interphalangeal joints of the
other fingers. Resolution: ¡ 1 degree (in good conditions), rate: Figure 8: Optical (VPL Data
up to 160 Hz. Price $ 11000 approx. glove)

2.5.2 Resistor-based (Virtex Cyber glove).

Versions of 18 or 22 sensors( 3 per finger 4 abduction
sensors, palm arch sensor, and sensors for flexion and
abduction, resolution 0.5 degrees, data rate 90 records/s.
Wireless version. Price from $ 9800 approx.

Figure 9: Resistor-based
(Virtex Cyber glove)

2.6 Negative aspects of the data gloves

If hand positions need to be detected, the system needs to assume or measure the distances
between finger joints. If that is not done, using the same glove for individuals with different
hand dimensions leads to a significant increase in the error of the measurements.

In addition, the implicit model of the hand considers the joints as parallel and perpendicular;
in any case, no information can be extracted about the real directions of the joints of the
human hand. In many applications, a different sensing device needs to be used in order to
identify the motion of the wrist and the location and orientation of the hand in space.

21
2.7 Exoskeletons
Exoskeletons are rigid structures designed to follow, up to certain extent, the motion of the
hand. They are in fact less used than gloves for applications in hand tracking. One of the
reasons is that it is difficult to create an exoskeleton that can adapt to the hand deformation
and still keep the accuracy in measuring certain joint motions; they are mostly used to
identify and track the motion of a few degrees of freedom. Nakagawara et al. presents a hand
exoskeleton used as a master to control a slave hand. In this paper, some of the common
problems of exoskeletons are also identified. Kim et al. presents a wearable device called
SCURRY which has characteristics of both exoskeletons and data gloves.

Figure 10: Exoskeleton used as a master to control artificial hand

Magnetic tracking follows the position and orientation of an emitter attached to a point of the
hand. It seems to yield a very good resolution and it is a promising technology to be applied
to hand tracking; for application of magnetic tracking systems to characterize human motion.

One of the disadvantages of electromagnetic tracking systems is interference: metallic objects

near the transmitter or receiver may affect the performance; this may be a problem if other
sensors need to be used to characterize a human task, for instance force sensors. Magnetic
trackers are also incomplete tracking systems; for a more rich motion identifications, more
sensors need to be placed in different points of the hand.

22
2.8 Non-contact technologies
Non-contact technologies refer almost exclusively to vision systems. Vision-based systems
can use a single camera, stereo cameras or orthogonally-placed cameras. Advantages of
vision-based systems are that they are less intrusive than any of the contact technologies;
however, small motions, certain views and occlusions may present problems in order to
identify the motion. This is partially overcome with the recovery of 3D position and
orientation information, but this in turn is difficult and computationally expensive.

2.8.1 Negative aspects of the Non-contact technologies

One of the main problems of the vision-based systems is that of the automatic identification
of the hand and the hand posture from the image. In order to distinguish the interesting
features in the hand, strategies such as using color, motion or edge information have been
implemented. Among other strategies, there are the following: skin-color detection (in which
the subject must wear long-sleeved clothes of a different color), color cues, motion cues
(when the hand is the only moving object), or shape detection.

23
CHAPTER THREE: PROJECT DEVELOPMENT
3.1 Equipment Design
3.1.1 3D Parts
In order to capture finger motion, it is necessary to design a glove equipped with
potentiometers. Observing the Data Glove designs, 3D parts were printed as shown below.

Table 1: Project parts for printing

Tensioner:
A tensioner is a device that applies a force
to create or maintain tension.

Spool
A spool is a cylindrical device on which a
string is wound

Holder
This is the part that keeps the potentiometer
in one position

24
 Guide Node/Guide Ring:
Guide the string that is attached to the end
of the finger in order to produce the force on
the tensioner.

End cap
Attached to the end of the finger and also
the end of the string

Cover
attached to the tensioner to keep the spool in
position

3.1.2 Potentiometer
The potentiometer is a resistance element with three terminals and the resistance value can be
adjusted according to a certain change law, which is equivalent to a variable resistor. Because
its role in the circuit is to obtain a certain relationship with the input voltage (external
voltage) to output Voltage, so -called potentiometer. Potentiometers can be divided into
rotary potentiometers, push-pull potentiometers, straight slide potentiometers, etc. according
to the adjustment method.
This project uses a rotary potentiometer. Its three pins are shown below:

25
The rotary potentiometer is an adjustable resistance element. It is composed of a resistor and
a rotating system. When a voltage is applied between the two fixed contacts of the resistive
body, the position of the contact on the resistive body is changed by the rotating system, and
a voltage that has a certain relationship with the position of the moving contact can be
achieved between the moving contact and the fixed contact. Potentiometers can be used to
adjust the voltage and current. Our course uses a rotary potentiometer. Its structure is as
shown in the figure below. By rotating the knob, the position of pin 2 is changed, thereby
changing the resistance value from pin 2 to both ends. In the experiment. Connect pin 1 and
pin 3 to the GND and 5V of the development board respectively. And then read the voltage
divided by the pin 2 of the potentiometer through the analog input pin A0. The range is

Figure 11: Potentiometer Pins

Figure 12: Potentiometer
between 0V and 5V. The analog input function of Arduino has 10-bit precision, that is, it can
convert the voltage signal of 0 to 5V into an integer form of 0 to 1024.

3.1.3 ESP32 Microcontroller Unit

Microcontroller is the bridge between hardware and PC in this project. As mentioned earlier,
ESP32 microcontroller board has been selected for this project. It comes with its own inbuilt
boot-loader and driver for the USB port. This board has 6 analog I/O pins and 14 digital I/O
pins. Therefore, two analog input from the potentiometer circuit and four 23 analog input
from the accelerometer module will fully occupy all the useable pins.
Features
- Ultra-Low-Power
- Wi-Fi-and-Bluetooth integration
- 32-bit dual core MPU @ 40MHz

26
- Up to 18 12bit Analog Pins

Figure 13:ESP32 Microcontroller

3.1.4 MPU6050 Sensor (accelerometer + gyroscope)

An Inertial Measurement Unit (IMU) that measures acceleration and rotation along three
axes. It captures motion data used to control the bionic arm.

Figure 14: MPU 6050 Accelerometer + Gyroscope

3.1.5 Integrated Development Environment

Arduino IDE is the C based open source development environment to program any
microcontroller from Arduino microcontroller series. It has a serial monitor that can be used
to monitor real-time response of the sensors. However, appropriate programs must be written
and uploaded to Arduino microcontroller to setup a serial port communication channel
between the microcontroller and the signal conversion circuit. Arduino IDE has been
obtained from the official Arduino microcontroller website. Furthermore, Processing IDE is
also a C based development environment to develop animation, simulation, real-time

27
interaction with Arduino microcontroller etc. However, appropriate programs must be written
to achieve desired functionality. Development in Processing IDE is relatively easy and
therefore it has been selected for the development of simulated real-time responsive stick
figure virtual robotic arm.

3.1.6 Virtual Reality Software

We used UnityTM Software to simulate control of an entity in a virtual environment.
UnityTM is a3D/2D, VR & AR engine used to create video games, apps and simulations for
computers and consoles

3.2 Cost Management

Cost management of the project is a very important aspect of the overall project development.
One of the project goal is to develop the design concept significantly under the market price.
The advantage of this goal is twofold: keeping the cost low while making the best functional
outcome; make it affordable and a marketable product.
Item Quantity Amount
Potentiometers 5 5,000
ESP32 Microcontroller 1 45,000

28
 MPU6050 1 15,000
Glove 1 5,000
Cables 5,000
TOTAL 80,000

In addition to the above purchases, the 3D parts were printed at the university biomedical
laboratory.
3.3 Theoretical Design of Signal Conversion Circuit Schematic
Schematic Representation

Visual Representation

29
CHAPTER FOUR: RESULTS AND DISCUSSIONS
4.1 3D PRINTED PARTS

Figure 15: 3D printing in progress

We were able to successfully print out some of the parts that were required for the successful
assembly of the arm interface but due to some circumstances some of the parts were not
properly printed.

30
Some of the printed parts included:

Table 2: Final output of the 3D printed parts

Cover

Tensioner

Spool

Guide Ring

Guide nodes

31
 Final assembly of the printed
parts with a potentiometer

Figure 16: Final Assembly of All the parts

32
4.2 Acquisition by Arduino IDE and Processing IDE
Using the Arduino IDE, analog and digital data is obtained. Arduino IDE provides examples
that can be slightly modified to obtain the real-time serial port output. To directly display the
real time response of the potentiometer the following program was uploaded to the
microcontroller through the USB cable.
/*Analog serial reader
Program description: Reads 2 analog inputs and sends their values out.
*/
void setup() {
// serial port initialization
Serial.begin(9600);
}
void loop() {
int thumb = analogRead(4); //define thumb input
int index = analogRead(5); //define index input
// print its value out as an ASCII numeric string
Serial.print(thumb, DEC);
// if this isn't the last sensor to read,
// then print a comma after it
Serial.print(",");
Serial.print(index, DEC);
Serial.println();
}
Figure 17: Output from serial monitor in
Arduino IDE

4.2 Unity 3D Simulation and Hardware Integration

This chapter focuses on the setup of the Unity 3D simulation environment for the bionic arm
interface. It includes the configuration of the virtual arm model, physics properties, and user
interface elements. Additionally, the integration of the hardware components with Unity 3D
is discussed, outlining how the captured sensor data and control signals are utilized to control
the virtual arm.

33
4.3 Unity 3D Simulation Environment Setup
The setup of the Unity 3D simulation environment involves the following components:

4.3.1 Virtual Arm Model

A realistic virtual arm model was created in Unity 3D to visually represent the movements of
the bionic arm interface. The virtual arm model consists of various components such as
joints, segments, and end effectors. These components are designed and rigged to mimic the
structure and motion of a human arm. The model should be anatomically accurate and
visually appealing to enhance the user experience.

4.3.2 Physics Properties

To simulate realistic arm movements, physics properties are assigned to the virtual arm
model. Unity's physics engine enables the emulation of natural movements and interactions.
Properties such as mass, gravity, and joint constraints are configured to provide realistic
dynamics and simulate the arm's behavior accurately.

4.3.3 User Interface Elements

User interface elements are incorporated into the Unity 3D environment to provide visual
feedback, control options, and status indicators. These elements may include buttons, sliders,
or textual information displayed on the screen. The user interface design should be intuitive
and user-friendly, allowing users to interact with the virtual arm and access relevant
information easily.

4.4 Integration of Hardware Components with Unity 3D

The integration of hardware components with Unity 3D involves the following steps:

4.4.1 Serial Communication

Unity 3D establishes serial communication with the ESP32 microcontroller to receive the
sensor data and control signals. This is achieved using the Serial Port class or a suitable

34
communication library in Unity. The serial port is configured with the appropriate settings,
such as baud rate and data format, to match the settings in the ESP32 microcontroller.

4.4.2 Data Reception and Processing

Unity 3D scripts receive the data stream from the ESP32 microcontroller through the serial
port. The scripts parse the incoming data, extracting relevant sensor data or control signals
required to control the virtual arm. This data processing step ensures that the captured arm
motion and hand gestures are accurately reflected in the movements of the virtual arm.

4.4.3 Arm Control and Visualization

Based on the received sensor data or control signals, Unity 3D scripts control the movements
of the virtual arm model. The scripts apply the appropriate transformations, such as rotating
joints or changing arm positions, to mimic the user's gestures and actions. These
transformations are synchronized with the captured arm motion, allowing the virtual arm to
respond in real time.

4.4.4 Visual Feedback and Interaction

Unity 3D provides visual feedback and interaction between the user and the virtual arm. This
includes updating the position, orientation, and animations of the virtual arm in real time
based on the sensor data. Additionally, visual cues such as highlighting or color changes can
be incorporated to provide feedback on the arm's state or the success of gesture recognition.

4.4.4 Testing and Calibration

After integrating the hardware components with Unity 3D, it is important to conduct
thorough testing and calibration. This involves verifying the accuracy and responsiveness of
the virtual arm's movements, ensuring that it accurately reflects the captured arm motion and
hand gestures. Calibrations may be performed to fine-tune mapping functions, thresholds, or
gesture recognition algorithms to achieve optimal performance.

The images below represent the outcome of the simulation of the glove.

35
36
37
CHAPTER FIVE: CONCLUSION AND RECOMMENDATIONS
5.1 Conclusion
In conclusion, the designed bionic arm interface successfully combines hardware and
software components to create a functional system for controlling a bionic arm. The
integration of the Arduino board, MPU6050 sensor, and Unity 3D software enables real-time
motion tracking, potentiometer simulation, and interactive visualization. The interface
provides an intuitive control mechanism for users, allowing them to control the bionic arm
with their arm motions. This project demonstrates the potential of using Arduino, sensors,
and 3D visualization software to create innovative and interactive human-machine interfaces.

5.2 Recommendation
In this project, the data glove was also connected to a mouse circuit and interfaced with the
Arduino microcontroller. The end result was that by bending the fingers, the mouse buttons
can be controlled. This is just one of the many dynamic uses of this device. With more time
and proper management, this project can be taken much further and an example would be
remotely controlling the computer without even touch the monitor mouse or keyboard. It can
only be imagined, what good it can do for the paralyzed people who cannot walk. This device
would then be able to give these individuals ability to use computers with simple hand and
finger movements.
Moreover, Data glove can put promising results in the field of medicine. It can be used for
monitoring hand function for rehabilitation purposes, it can be used in telerobotic surgery and
not to mention high quality virtual interactive gaming.
Therefore, it is a highly recommended research topic for the obtainment of many well and
advanced outcomes.

38
REFERENCES

[1] LucidVR, “LucidVR/Lucidgloves Github Repository,” GitHub,

https://github.com/LucidVR/lucidgloves/blob/main/hardware/Prototype4/
RigidMount_rightHand.STL (accessed Jun. 6, 2023).

[2] Data Glove With a Force Sensor Kostas N. Tarchanidis, Member, IEEE, and John N.
Lygouras, Member, IEEE

[3] Hand-in-Glove Human-Machine Interface and Interactive Control: Task Process

Modeling Using; Dual Petri Nets Stephen Mascaro and H. Harry Asada d’ Arbeloff
Laboratory for Information Systems and Technology Department of Mechanical Engineering
Massachusetts Institute of Technology Cambridge, MA 02139, e-mail: asada@mit.edu ,
smascaro@mit.edu

[4] Teleoperation of Robot Based on Virtual Reality Xiaoling Lv , Minglu Zhang , Feng Cui ,
Xiaoli Zhang （School of Mechanical Engineering Hebei University of Technology Tianjin
300130, China）

[5] Automated Control System of Irrigation for SMEs in Real Time. Aquije, P.; Huaman, D.;
Boada, J.; Tentaya, M.; Chauca, M.; Electrical, Communications, and Computers, 2009.
CONIELECOMP 2009. International Conference on 26-28 Feb. 2009 Page(s):35 – 40

[6] Mohammed, A.A. and Sunar, M., 2015, May. Kinematics modeling of a 4-DOF robotic
arm. In Control, Automation and Robotics (ICCAR), 2015 International Conference on (pp.
87-91). IEEE.

[7] F. C. Chen, A. Favetto, M. Mousavi, E. P. Ambrosio, S. Appendino, A. Battezzato, D.

Manfredi, F. Pescarmona, B. Bona, “Human Hand: Kinematics, Statics, and Dynamics,” in
41st International Conference on Environmental Systems, Conference Proceedings (Portland,
Oregon, 2011).

[8] Qassem, M.A., Abuhadrous, I. and Elaydi, H., 2010, March. Modeling and Simulation of
5 DOF educational robot arm. In Advanced Computer Control (ICACC), 2010 2nd
International Conference on (Vol. 5, pp. 569-574). IEEE

39
[9] Robotic hand simulation with kinematics and dynamic analysis. Ramasamy, S.; Arshad,
M.R.; TENCON 2000. Proceedings. Volume 3, 24-27 Sept. 2000 Page(s):178 - 183 vol.3

[10] Taha Beyrouthy, Samer Al. Kork, J. Korbane, M. Abouelela, “EEG Mind Controlled
Smart Prosthetic Arm A Comprehensive Study”, Advances in Science, Technology and
Engineering Systems Journal, vol. 2, no. 3, pp. 891-899 (2017). google scholar

[11] Ahmed Roshdy, Samer Al Kork, Sherif Said, Taha Beyrouthy, “A Wearable
Exoskeleton Rehabilitation Device for ParalysisA Comprehensive Study,” Advances in
Science, Technology and Engineering Systems , Vols. Journal Vol. 4, , no. No. 1, , pp. 17-
26 , 2019.

[12] A. Ganiev, H.-S. Shin and K.-H. Lee, “Study on Virtual Control of a Robotic Arm via a
Myo Armband for the Self Manipulation of a Hand Amputee,”,” International Journal of
Applied Engineering Research, vol. vol. 11, pp. pp. 775-782, 2016.

[13] Janne M. Hahne, Meike A. Schweisfurth, Mario Koppe, and Dario Farina,
“Simultaneous control of multiple functions of bionic hand prostheses: Performance and
robustness in end users,” Science Robotics, vol. 3, no. 19, 2018.

[14] L. LucidVR, “LucidVR/Lucidgloves: Arduino/ESP32 based DIY VR haptic gloves.

compatible with Steamvr via OpenGloves.,” GitHub,
https://github.com/LucidVR/lucidgloves (accessed Oct. 18, 2022).

40