Use of Mixed Reality and “Smart” robotic surgical systems

for Remote-surgery

There has always been a never-ending dearth of efficient surgeons all over the world. A complex
and critical medical procedure like surgery can involve unnecessary logistics like movement of
patient and care-takers, that too between countries if the facility is unavailable in home-country.
All this ultimately translates to increased financial burden on patient.
Initially a Robot in surgery was used to assist the surgeons with a simple task - maneuvering the
endoscope with the help of voice commands inside the patient’s body. AESOP (Automated
Endoscopic System for Optimal Positioning) was first of the 3 robotic arms of the up-coming Zeus
Robotic Surgical System (ZRSS) which performed that function. The other two arms of ZRSS
mimicked the surgeon’s movements and were responsible for precise incisions and extractions.
Today with the help of Telemanipulators and consoles controlling high-precision robots, surgeons
perform surgeries with minimal blood loss, smaller incisions and greater healing time. Robotic
arms are used to improve accuracy and to provide freedom of movement which a human hand is
unavailable to achieve. Most of the use of Robotic Surgical Systems today is “on-site” only, with
the robots assisting the surgeon visually and mechanically. Though Remote-surgery has already
been attempted successfully with examples like Operation Lindbergh, but it still isn’t a common
practice and alternative to regular surgery. Cost is an indispensable reason for this but the visual
restriction of 2-D feed from traditional camera setup and limited immersion of the surgeon is one
of the most important limiting factors. The other reason often listed is the unforeseen cases of
emergency where robots would be incapable to carry out emergency procedures on patients on
their own.
We have found that this is a problem area and a gap that can be filled with the help of
technologies like Mixed Reality and Artificial Intelligence.
We intend to use Mixed Reality with the help of devices like HoloLens to provide a more
interactive and ready interface to surgeon in place of existing console with a plain visual feed. The
projections in HoloLens would entail scanned and 3-D imagery of patient’s systems which would
help to view the insides and exploded aspect better. To feel feedback and sense-of-touch, surgeons
will require to wear Haptic receivers since this is one of the most important part of surgery. It is
essential for them to feel the thickness and tension of tissues while making incisions and other
critical moves. Surgeons can even practice and increase familiarity with the subject to be operated
on. Motions of surgeons can be captured better with the help of motion-sensing tech supplemented
with the MR’s intuitive gesture control.
The response of surgeon in the form of digital signals will be propagated using ATM
(Asynchronous Transfer Mode) Technology which is already used for remote-surgery. Signals use
fibre-optic cables for a fixed-lag. The control box at patient’s end will receive these signals and
they will be interpreted. The “smart” robots will perform surgery as per the surgeon’s instruction.
 Command Signals from source (doctor)

 Mixed Reality System Motion-sensing, Gestures  Robotic Arms make

with Surgeon sense of the signals
(Operating on a and operate
Hologram Model)

Feedback from Remote location

(Real-time Hologram, Haptic feedback)

Prepared By – Group 06
Anirudh Bansal (PGP-18-008)
Gauri Gupta (PGP-18-024)
Harshit Rajput (PGP-18-028)
Kshitij Gaur (PGP-18-035)
Saumya Agarwal (PGP-18-063)