RoboTechArm: Leading the Future, Guiding Growth

Final Year Project – Mid Report

Session 2020-2024

by

Hadiqa Ismail – 2020-UKT-006906

Muhammad Qutab – 2020-UKT-006942

Department of Artificial Intelligence

Faculty of Computing and Engineering
University of Kotli Azad Jammu and Kashmir

22 April 2024
 Project Detail

Type (Nature of project) [ ] Development [ ] Research [] R&D

Area of specialization Robotics, Computer Vision

Project Group Members

Sr.
Reg. # Student Name Email ID *Signature
#

2020-
(i) UKT- Muhammad Qutab m.qutab03@gmail.com
006942

2020-
(ii) UKT- Hadiqa Ismail hadiqam23@gmail.com
006906

(iii)
*The candidates confirm that the work submitted is their own and appropriate credit has
been given where reference has been made to work of others

Plagiarism Free Certificate

This is to certify that, I am ________________________ S/D/o _______________________, group leader
of FYP under registration no CIIT/_________________/KOTLI at Artificial Intelligence Department,
University of Kotli, Main Campus. I declare that my FYP report is checked by my supervisor and the
similarity index is ________% that is less than 20%, an acceptable limit by HEC. Report is attached
herewith as Appendix A.

Date: ____________ Name of Group Leader: ________________________ Signature: _____________

Name of Supervisor: _____________________ Co-Supervisor (if any):____________________

Designation: _____________________ Designation: _____________________
Signature: _____________________ Signature: _____________________

ii
Abstract
This proposal presents the improvement of a mechanical arm outfitted with man-made
intelligence empowered control for upgraded adaptability and proficiency in different
applications. The joining of man-made reasoning empowers the automated arm to adjust
to dynamic conditions and perform complex assignments with accuracy. The proposed
strategy includes a far reaching writing survey, trailed by the plan and execution of the
man-made intelligence empowered control framework.
The extent of the venture envelops applications in assembling, medical services, and
different enterprises where independent and smart automated control is required. Through
their high level capacities, automated arms can accomplish remarkable degrees of
accuracy, empowering errands like getting a handle on, control, and get together with
excellent exactness. Also, the coordination of PC vision empowers automated arms to
perceive and follow objects continuously, working with consistent association with
dynamic conditions and supporting independent activity. This independence stretches out
to human-robot coordinated effort, as the arm can distinguish and answer human
presence and signals, encouraging protected and proficient collaboration in shared work
areas. With ongoing criticism components and ceaseless learning capacities, man-made
intelligence empowered mechanical arms consistently work on their presentation,
guaranteeing versatility and effectiveness in a large number of uses, from assembling and
operations to medical services and horticulture. Through this exploration, we expect to
add to progressions in mechanical innovation and its applications in different fields.

iii
Table of Contents
Project Detail....................................................................................................................................................ii
Plagiarism Free Certificate...............................................................................................................................ii
Abstract...........................................................................................................................................................iii
1 Introduction.............................................................................................................................................1
2 Objectives................................................................................................................................................2
3 Problem Statement...................................................................................................................................2
3.1 Object Acknowledgment............................................................................................................2
3.2 Grasping Technique....................................................................................................................2
3.3 Positioning Precision..................................................................................................................3
3.4 Adaptability and Adaptability.....................................................................................................3
3.5 Real-Time Execution..................................................................................................................3
4 Assumptions & constraints......................................................................................................................3
4.1 Assumptions................................................................................................................................3
4.2 Constraints..................................................................................................................................3
5 Project scope............................................................................................................................................4
6 Literature review......................................................................................................................................5
7 Functional requirements..........................................................................................................................6
8 Non-functional requirements...................................................................................................................7
8.1 Performance Requirements.........................................................................................................7
8.2 Safety Requirements...................................................................................................................7
8.3 Security Requirements................................................................................................................7
8.4 Software Quality Attributes........................................................................................................7
9 Requirements traceability matrix.............................................................................................................8
10 Gantt Chart..............................................................................................................................................8
11 Proposed Architecture.............................................................................................................................9
12 References.............................................................................................................................................10

iv
List of Tables
Table 1 List of Research Journals....................................................................................................................6
Table 2 List of Requirements Traceabilities....................................................................................................8

v
List of Figures
Figure 1 Gantt Chart.........................................................................................................................................8
Figure 2 Proposed Architecture........................................................................................................................9

vi
1 Introduction
Mechanical arm is a gift for present day improvement in this time. Many gathering
processes which add to the altruistic things incorporate usage of mechanical arm.
Robotized arm has rotational development and direct development to play out the
endeavour. It has wide applications in industry like welding, turning, squeezing things
on auto mechanical creation framework. The working of mechanized arm is imperative
as it incorporates many joints and advancement of mechanical arm is dependent upon
the improvement of individual joint. The solitary advancement of mechanical arm joint
is depicted by invert kinematics. In mechanical innovation, in reverse kinematics uses
the kinematics conditions to choose the joint limits that give a very smart arrangement
to all of the robot's end-effectors. To get to know the working of mechanical arm,
inverse kinematics should be considered [1] and a short time later the computerized arm
should be changed by the application. To address the limits of mechanics and gadgets,
the reconciliation of man-made consciousness (simulated intelligence) has arisen as a
promising way to deal with improve the usefulness of automated arms. Hiba Sekkat et
al. [2] proposed the arrangement of mechanical arm which utilizes support figuring out
how to put objects by going with a choice. We will be training our model to response
according to the object detection algorithm which can lead us to the development of a
model for the specific tasks.

Object tracking is the most common way of finding a moving item (or different items)
after some time utilizing a camera. Object following is valuable in the applications where
some activity should be performed in the wake of checking the edge from camera. We are
carrying out object identification calculations too. Object recognition is a PC innovation
which is connected with PC vision and picture handling. Object following has its base in
object discovery. It has applications like face discovery, pedestrian location, and
identification of various components in video outline. For modern applications where car
sequential construction system has various sorts of parts, the utilization of item
identification permits legitimate isolation of those parts.

1 22 April 2024
2 Objectives
i. Conducting a total composing overview to perceive critical assessment and
methodology in man-made reasoning based mechanical control.
ii. Creating computer vision assessments for bearing and control of the robotized
arm.
iii. Assessing the presentation of the man-made information controlled automated
arm through diversion and certifiable assessments.
iv. Documenting the arrangement communication, execution nuances, and
exploratory results for future reference and dissipating.
v. Collaborate with industry accessories to recognize unequivocal use cases and
necessities for computerized reasoning enabled robotized arms in different
regions.

3 Problem Statement
In present day current settings, the capable treatment of things in consecutive
development frameworks or circulation focuses expects a significant part in further
developing proficiency and decreasing utilitarian costs. Standard mechanized arms have
been by and large used for pick-and-spot endeavours. In any case, they oftentimes require
precise programming and fight with alterability in object shapes, sizes, and positions. To
address these difficulties, there is a making need for man-made cognizance empowered
mechanical frameworks outfitted with PC vision capacities to openly perceive, handle,
and precisely place objects in current conditions. Key difficulties include:

3.1 Object Acknowledgment

The framework should conclusively perceive and bundle different things paying little
mind to what their course, shape, size, or surface. This requires solid PC vision
calculations furnished for overseeing complex visual information perpetually.

3.2 Grasping Technique

Orchestrating major areas of strength for a feeling of system is vital for guaranteeing
solid thing control. The electronic arm should be really great for picking authentic handle
sparkles and changing hold strength thinking about the thing's qualities.

2 22 April 2024
3.3 Positioning Precision
Accomplishing precise orchestrating is fundamental for unsurprising coordination into
existing creation lines and limiting stumbles in the get-together cycle. The construction
should utilize progressed control calculations to precisely explore and change the
mechanical arm to target objects for pick-and-spot works out.

3.4 Adaptability and Adaptability

Present day conditions frequently highlight dynamic circumstances with moderate
changes in object types, positions, and creation necessities. The PC based understanding
empowered mechanical arm framework ought to be at this point adaptable and flexible to
oversee such combinations without sweeping repeating or manual intervention.

3.5 Real-Time Execution

To satisfy the necessities of quick creation lines, the framework should show rapid
reaction times and consistent strong limits. This requires competent assessments for
picture taking care of, object affirmation, and improvement intending to confine process
ranges and broaden throughput.

4 Assumptions & constraints

4.1 Assumptions
i. The structure anticipates that that the things should be picked and set have fairly
unsurprising shapes, sizes, and surface surfaces inside the utilitarian environment.
ii. The system acknowledges acceptable and consistent lighting conditions in the
workspace to ensure trustworthy article disclosure and affirmation by the
computer vision structure.
iii. Objects to be picked and set are accepted to be fixed or insignificant moving
during the activity, permitting the robotized arm to find and control them
precisely.

3 22 April 2024
4.2 Constraints
i. The plan and execution of the mechanical arm structure are obliged by the
accessibility of stuff parts like actuators, sensors, cameras, and dealing with units,
which could force limits on the framework's abilities and execution.
ii. The computational resources open for picture taking care of, object affirmation,
and development organizing are confined by the dealing with power of the
prepared or related handling contraptions, which could impact the structure's
speed and responsiveness.
iii. The structure ought to conform to prosperity rules and standards directing present
day progressed mechanics, including measures to prevent crashes, protect human
directors, and assurance safe movement inside the workspace.

5 Project scope
Our project expects to cultivate a best in class mechanical arm system that utilization
reproduced knowledge engaged control and significant level computer vision strategies.
Our fundamental community is to make areas of strength for a strong circumstance fit for
careful control and participation with its ongoing situation. To achieve this, we will
carefully pick fitting re-enacted insight computations and computer vision progresses,
ensuring they line up with our errand targets.
Testing and endorsement will be driven completely to ensure the display and value of the
robotized arm system in more favourable conditions. Security considerations will be
crucial all through the assignment, with extreme measures executed to ease bets related
with autonomous action and human-robot correspondence.
Flexibility will be a key thought, taking into account future upgrades and improvements
to oblige propelling requirements and developments. Broad documentation will be stayed
aware of to work with getting a handle on, support, and further headway of the system.
Fruitful correspondence with accomplices will be stayed aware of all through the
endeavour, ensuring plan with presumptions and watching out for any concerns or
analysis right away.

What to Consider

4 22 April 2024
 i. We will ensure the system operates reliably in diverse environments, effectively
handling uncertainties and variations.
ii. Continuous optimization of AI algorithms and computer vision systems will be
pursued to achieve efficient and responsive operation of the robotic arm.
iii. Intuitive UIs will be made to deal with the necessities of both specific and non-
particular clients, working with arrangement, noticing, and control of the system.
iv. Plans for ongoing maintenance and support will be made to ensure long-term
reliability and usability of the system.

What Not to Consider

i. We will avoid unnecessary complexity in the design and implementation,

focusing on meeting functional requirements efficiently.
ii. We will resist incorporating excessive features that do not directly contribute to
achieving our project objectives, prioritizing simplicity and clarity.
iii. Realistic expectations for the capabilities and limitations of the system will be set,
considering technological constraints and resource limitations.
iv. We will guard against scope creep by carefully evaluating proposed changes to
ensure alignment with project goals and priorities.

6 Literature review
The application grants clients to conclude their ideal thing, which will be gotten and
placed by a mechanical arm into the objective region. The application grants clients to
direct articles considering assortment, shape, and size. The arm then, goes through a lot
of preset circumstances to get the thing, place the article, and subsequently return the arm
to the fundamental position. The joint headings are shipped off a microcontroller that sets
the arm's joint point at each position. [3]

Research Journals Drawbacks Differences

Intisar et. al. Computer Vision (i) There is no criticism (i) Our system will feedback the
Based Robotic Arm Controlled component for the framework system about the objects not
Using Interactive GUI. separated from being looked found.
after and revealed by the client

5 22 April 2024
 (ii) The automated arm goes (ii) Our system will stop there
through the expected movements where it drops the object and
regardless of whether the article will try to pick it up again
got is dropped

Shaikat et. al. Computer vision
based industrial robotic arm for
sorting objects by color and
height.
(i) It is limited to specific colors.
(ii) It uses traditional .Net
framework and OpenCV
(i) Our system will be applied to
various colors.
(ii) Since, it uses traditional
sorting algorithms, we are going
to use advance object tracking
and sorting algorithms.

Md. Abdullah-Al-Noman et. al. The system encountered Our system will be resulting in
Computer vision-based robotic challenges in accurately maximizing the accuracy
arm for object color, shape, and measuring the distance of
size detection. objects, resulting in a 80%
success rate due to failures in
detecting object colors.

Hiba Sekkat et. al. Vision-Based Agent's memory imbalance We will be using updated
Robotic Arm Control Algorithm prompting challenges in version of YOLO to improve
Using Deep Reinforcement dynamic between sitting tight for agent’s decision-making
Learning for Autonomous extra information or grouping abilities.
Objects Grasping. deficient arrangements

Table 1 List of Research Journals

7 Functional requirements
By using computer vision strategies, the framework tries to distinguish and classify
things to further develop dealing with effectiveness. Also, it finds out the specific
position and direction of things recognized according to the work area of the mechanical
arm. For the mechanical arm to securely get a handle on objects of various sizes, shapes,
and directions, it requirements to utilize areas of strength for a technique. The
characteristics of the goal article can be used to change the gripper part. Development
organizing makes ideal bearings for accurate and capable thing approach, handle, and
plan. The structure should achieve raised levels of precision and exactness in picking and
setting objects inside resolved strengths. It should work ceaselessly with fast response
times for object revelation, affirmation, and development orchestrating. It should
consolidate reliably with existing present day computerization structures for created

6 22 April 2024
movement. The system should give a simple to utilize association highlight executives to
screen and control mechanical arm undertakings.

8 Non-functional requirements
8.1 Performance Requirements
i. The framework ought to have the option to recognize and limit objects inside the
work area.
ii. The framework's reaction time to client orders ought to associate with 500
milliseconds to guarantee continuous communication and control.
8.2 Safety Requirements
i. The automated arm's developments ought to be restricted inside predefined safe
zones to forestall impacts with snags or faculty.
ii. Emergency stop buttons ought to be effectively available and equipped for ending
all framework tasks quickly if there should arise an occurrence of a crisis.
iii. The framework ought to incorporate safeguard systems to forestall accidental
developments or activities in case of sensor disappointments or correspondence
mistakes.
8.3 Security Requirements
i. Access to framework controls and arrangement settings ought to be confined to
approve work force just, with client verification and job based admittance control
components executed.
ii. Data transmission between parts of the framework ought to be encoded to
safeguard against unapproved interference or altering.
8.4 Software Quality Attributes
i. The framework ought to stick to coding guidelines and best practices to guarantee
code viability, coherence, and particularity.
ii. It ought to be planned with a strong blunder taking care of component to deal with
unforeseen mistakes and special cases without compromising framework
steadiness nimbly.

7 22 April 2024
 iii. The framework ought to go through careful testing, including unit testing,
incorporation testing, and framework testing, to confirm usefulness, execution,
and dependability.
Object Object Object Grasping Gripper Motion Accuracy and
Detection Recognition Localization Strategy Control Planning Precision Simulation
X X
X X X
X X X X
X X X
X X X
X X
X X
X X
X X
X X
X
X
X
X

9 Requirements traceability matrix

Table 2 List of Requirements Trace abilities

10 Gantt Chart

RoboTechArm: Leading the Future, Guiding Growth

4 4 4 4
-2
3 -2 -2 -2 -2
ec an ar pr un
D -J -M -A -J
2- 21 11 30 19

Data Choosing & Selection

Literature Review: Studies related to Robotic Arms
Hardware Integration
Data Preprocessing: Python Libraries
Models: Training & Testing
Evaluating Model: Performance Metrices
Results & Discussion
Conclusion of Project

Start Date Days Needed

Figure 1 Gantt Chart

8 22 April 2024
11 Proposed Architecture

Figure 2 Proposed Architecture

9 22 April 2024
12 References

[1] Sekkat, H., Tigani, S., Saadane, R., & Chehri, A., “Vision-based robotic arm control
algorithm using deep reinforcement learning for autonomous objects grasping,” Applied
Sciences, vol. 17, no. 11, p. 7917, 2021.

[2] Kofinas, N., Orfanoudakis, E., & Lagoudakis, M. G., “Complete analytical forward and
inverse kinematics for the NAO humanoid robot,” Journal of Intelligent & Robotic Systems,
vol. 77, pp. 251-264, 2015.

[3] M. Abdullah-Al-Noman, A. N. Eva, T. B. Yeahyea and R. Khan, “Computer vision-based

robotic arm for object color, shape, and size detection,” Journal of Robotics and Control
(JRC), vol. 2, no. 3, pp. 180-186, 2022.

[4] M. Intisar, M. M. Khan, M. R. Islam and M. Masud, “Computer Vision Based Robotic Arm
Controlled Using Interactive GUI,” Intelligent Automation & Soft Computing, vol. 2, no. 27,
p. 18, 2021.

[5] A. S. Shaikat, S. Akter and U. Salma, “Computer vision based industrial robotic arm for
sorting objects by color and height,” Journal of Engineering Advancements, vol. 4, no. 1, pp.
116-122, 2020.

10 22 April 2024