3D Haptics and Robotics

Krasimir Kolarov
Interval Research Corporation

HCC Seminar, UC Berkeley, 12/3/98.

What is this talk about?

Introduction to Haptics and Haptic Interfaces
Commercial and University Haptics
Force Feedback Devices
3D Haptics at Interval Research Corp.

Haptic Interfaces
Haptic - an information processing perceptual system
that uses inputs from the receptors embedded in the
skin, as well as in muscles, tendons and joints (Loomis
and Lederman, 1986)
hap.tic (haptik) adj. of or having to do with the sense of
touch; tactile (Websters New World Dictionary)
haptic interfaces - devices that measure the motion of,
and stimulate the sensory capabilities within, our hands
(as used in human interface technology )

Unique Characteristics of Haptics

Haptics relies on action to stimulate perception.
The haptic system can sense and act on the environment
while vision and audition have purely sensory nature.
Being able to touch, feel, and manipulate objects in an
environment, in addition to seeing (and hearing) them,
provides a sense of immersion in the environment that is
otherwise not possible (Srinivasan, 1995)

Other Topics
Actuators (electrical, hydraulic, pneumatic)
Sensors
Tactile Feedback Interfaces (sensors, texture, slip,
surface temperature)
Control of Haptic Interfaces (distributed computation,
quality)
Physical Modeling (collision detection, surface
deformation, mechanical compliance, smoothness,
physical construction)

Applications
Enhancement of GUIs (graphical user interfaces) enable users to feel where the buttons on their programs
are.
Computer Games - engaging touch interactions, costsensitive market.
Simulation for training humans - to perform tasks that
require sensorimotor skills (surgery, training for naval
personnel).
Interaction with computer-generated 3D data - users of
CAD/CAM, data visualization and other engineering and
scientific applications.
Medicine, Entertainment, Telerobotics.

Human Haptics System

Tactile Sensory System - distinguish vibrations up to 1
KHz; detection threshold on smooth glass plate - 2
high single dot, 0.06 high grating
Kinesthetic Sensory System - bandwidth 20-30 Hz; JND
(just noticeable distance) - 2.5o for finger joint, 2o for
wrist and elbow, 0.8o for the shoulder
Motor System - human bandwidth for limb motions is
between 1-10 Hz as a function of the mode of operation
Active Touch with all three systems - stiffness > 25N/mm
is needed for an object to be perceived as rigid

Functions of Haptic Interfaces

to measure the position and contact forces (and time
derivatives) of the users hand (or other body parts).
to display contact forces and positions (or their spatial
and temporal distributions) to the user.
Alerting function - vibrations.
Premise: The sense of touching simple shapes could be
evoked by programming computers to control
electromechanical master devices. We can build devices
that give us a sense of feel when controlling remote
actions with a high degree of dexterity.

Categories of Haptics Interfaces

1. Free motion, in which no physical contact is made with
objects in the environment
2. Contact involving unbalanced resultant forces (like
pressing an object with a finger pad)
3. Contact involving self-equilibrating forces (like
squeezing an object in a pinch grasp)
Additional consideration - we can touch, feel and
manipulate the objects directly or with a tool

Currently Available Haptic

Interfaces
Ground-Based Devices
joysticks/ hand controllers

Body-Based Devices
exoskeletal devices
flexible (gloves and suits worn by users)
rigid links (jointed linkages affixed to users)

Tactile Displays
shape changers
shape memory actuators
pneumatic actuators
microelectromechanical actuators
vibrotactile
electrotactile

History of Haptic Interfaces

Haptics Research in Universities

MIT AI Lab
Salisbury (design of high performance mechanisms and sensors)

MIT Human-Machine Systems Lab

Srinivasan (understand human haptics, enhance human-machine interaction)
Sheridan (tactile and auditory substitution of force feedback for teleoperation)

MIT Media Lab

Margaret Minsky (tactile feedback from a graphics simulation, home haptics)
Plesniak (haptics and holographic systems)

Harvard University
Rob Howe (tactile display of shape and vibrations)

UNC
Taylor, Fred Brooks (nanomanipulator force feedback, medical research)

CMU
Baraff, Vedula (force feedback in interactive dynamic simulation)

University Research (cont.)

Stanford University
ME Dept., Cutkosky (force feedback grasping, multi-finger manipulation)
CS Dept., Khatib, Ruspini (haptics library, force control, dynamics)
CCRMA, OModrian (grand piano simulation, haptics for the blind)

UC Berkeley
Canny (optimum stability of grasp, dynamic simulations)

Northwestern University
Ed Colgate (dynamically effects like mechanical impedance)

Japan
Iwata (6 dof stewart platform joystick Haptic Master, mechanical design)

University of New Mexico, University of Virginia, University of

Colorado, Rutgers University, Georgia Tech, McGill University,
Naval Postgraduate School, University of Washington, Simon
Fraser University, ...

Industrial Research and

Development
SensAble Devices (PhanTom)
Immersion Corp. (Impulse Engines, Joysticks)
Cybernet Systems Corp. (CyberImpact Joystick,
Steering Wheel, Flight Yoke)
Microsoft (formerly - EXOS Inc. Power Stick, Surgical
Simulator, SAFiRE)
Boston Dynamics (Tangible Reality, Interactive Humans)
VTT, Finland (virtual prototyping)
Interval Research Corp., MERL, GE Corporate R&D, High
Techsplanations Inc., Army, Navy, ...

Immersion Corporation

Cybernet Systems Corp.

EXOS Inc.

SensAble Technologies Inc.

Video on Applications (SensAble)

before that
b&w slides from Web Page
color slides from Hassers report
after that color slides from VTT

Research Interests
Develop a cooperative graphic and haptic interface
that allows to manipulate and sculpture 3D objects
more effectively

Goals and Assumptions

Provide a high level interface to haptic devices that:
Complements existing interactive graphic systems
Works robustly in multi-surface environments
Provide common framework to allow stable and safe haptic
control.
Ability to perform tasks that are not possible with the current
technology
The combined graphical and haptic interface to 3D objects will
allow us much richer and powerful interaction.

Existing Graphic Systems

Capable of displaying a large number of simple
polygons at interactive rates (>20,000 polygons at
30Hz)
Intersecting polygons & gaps common
Topology seldom available (Polygon Soup)
Gourand/Phong Shading & Texture

Questions
How can you support a powerful and general set of
modeling 3D primitives.
Allow the haptic server to operate with a great amount
of autonomy from the host computer and simulate a wide
range of virtual environments
Explore issues like latency in manipulating large 3D
data sets

Basis for Research

Test distance/collision calculation and dynamic
simulation.
Building a library to support arbitrary complex rigid
objects.
Allow the developer to specify constraints between the
objects in the environment and control the motion of
objects in the virtual world.
Model the contact forces caused by contact and
collisions between the objects in the environment

Stanford Students Projects

The Virtual Xylophone.
The Haptic Roaches.
Haptic Exploration of rigid 3D objects.

Video on Roaches
before that - color slides on xylophone, roaches
after that
color slide on staircase
slides from the HL talk
goals
bounding sphere covering
bounding sphere hierarchy
simulating smooth surfaces
results

Issues
How to display (and compute) elemental sensations
such as impact, friction, softness, motion and
constraint.
How to involve more complicated interactions (the
Phantom concentrates on forces at the fingertip or
tool tip). Those include pressure distribution,
temperature and high-frequency vibration.

Solutions

Penalty Based Haptic Systems

Bounding Sphere Hierarchy
Virtual Proxy Model
Surface Properties
Force Shading/Texturing
HL Library with Application Programmers
Interface similar to GL

Bounding Sphere Hierarchy

The Virtual Proxy

Virtual Proxy Description

A representative

object that is constrained by

obstacles in the environment
Proxy is reduced to a point (C-space). User
definable size of proxy.
Constraint planes locally describe the range of
potential proxy motion.
Proxy moves to locally minimize the distance to
users position
Haptic device physically moves user to proxys
position

Force Shading

Haptic System Implementation

HL library, syntax similar to GL
graphic Client/Haptic Server Model
Bounding Sphere Hierarchy
- O(log n) growth
> 24,000 polygonal primitives on 200 MHz
Pentium - Stiffness 1800 Newtons/meter
- > 1000 Hz servo rate

Demonstrations

Sample References
Force and Touch Feedback for Virtual Reality, Grigore
Burdea, Rutgers Univ., 1996, John Wiley & Sons.
The PHANToM Users Group Workshop Proceeding,
MIT September 1996, 1997, 1998 (published as MIT AI
Lab Tech Reports).
Haptics Home Page at Northwestern Univ.:
http://haptic.mech.nwu.edu/
Ruspini, D., Kolarov, K. and Khatib, O. "The Haptic
Display of Complex Graphical Environments", Computer
Graphics Proceedings, Annual Conference Series,
SIGGRAPH'97, Los Angeles, California, September 1997