Input Devices:

Trackers, Navigation and Gesture Interfaces

Input Devices

What is Virtual Reality?

“A high-end user interface that involves real-time
simulation and interaction through multiple sensorial
channels.” (vision, sound, touch, smell, taste)
Input Devices

Virtual objects have 6 degrees

of freedom (D.O.Fs):
-three translations;
-three rotations.

3-D System of coordinates of a VR object

Input Devices

Trackers measure the motion of “objects” such as user’s

wrist or his head vs. a fixed system of coordinates.

Technologies to perform this task:

Magnetic trackers (used to be prevalent but less used today);
Ultrasonic trackers (least used);
Mechanical trackers (special cases);
Inertial/ultrasonic trackers (new). (volunteers?)
Vision-based trackers (new and geared to be favorite)
GPS
Input Devices
Input Devices

Tracker characteristics:
 Measurement rate – Readings/sec;
 Sensing latency;
 Sensor noise and drift;
 Measurement accuracy (errors);
 Measurement repeatability;
 Tethered or wireless;
 Work envelope;
 Sensing degradation
Price.
Input Devices

Tracker characteristics:
Accuracy

Resolution

Real object position

Tracker position
measurements
Input Devices

Tracker characteristics:

Tracker data

Real object fixed

position

Signal noise

Time
Input Devices

Tracker characteristics:

Tracker data

Real object fixed Sensor drift

position

Time
Input Devices

Tracker characteristics:
Tracker data

Real object
position

Sensor latency

Time
Input Devices

Tracker characteristics:
Tracker Update Rate
Input Devices

Mechanical Trackers
Definition: A mechanical tracker consists of a serial or
parallel kinematic structure composed of links interconnected
by sensorized joints.
Input Devices

Mechanical Trackers
Pros
 Use sensors imbedded in exoskeletons to measure
position;
 Have extremely low latencies;
 Are immune to interference from magnetic fields and
large metal objects;

 Cons
 But limit the user’s freedom of motion;
 Can be heavy is worn on the body
Input Devices

Push 1280 (Fakespace Inc)

Input Devices

Exoskeleton
structure

Interface
With
computer
Input Devices

Magnetic Trackers
Definition: A magnetic tracker is a non-contact position
measurement device that uses a magnetic field produced
by a usually stationary TRANSMITTER to determine the real-time
position of a usually moving RECEIVER element
Input Devices

Magnetic Trackers
Use low-frequency magnetic fields to measure position;
Fields are produced by a fixed source;
Size of source grows with the tracker work envelope;
Usually the receiver is attached to the tracked object and
has three perpendicular antennas;
Distance is inferred from the voltages induced in the
antennas – needs calibration…
Input Devices

Magnetic tracker with the old Data Glove

Input Devices
Fastrack magnetic tracker system
Electronic interface

Source

Stylus Receiver
Input Devices

Long Ranger source for the tracker system

Source
 Receivers

Source

Input Devices
Fastrack magnetic tracker electronics
 Metal in lab double ceiling

Polhemus Long Ranger

tracking errors (Rutgers)
 Magnitude of Error Vector / Moving Tripod

35.0
Magnitude of Error Vector (inches)

30.0

25.0
Err (54)
20.0 Err (60)
Err (66)
15.0 Err (72)

10.0 Err (80)

5.0

0.0
0 10 20 30 40 50 60 70 80
Transmitter-Receiver Distance (inches)

Tracking error as a function of tripod height

Liberty LATUS – A Polhemus Wireless
Tracker
Uses wireless sources, now called “markers”, each with
a different frequency;
Each source position is measured by a “receptor” within
8 ft (50 sq. ft). 1 receiver can track 4 markers.
The system can have up to 12 markers and up to 16
receptors
Sampling rate is 188 Hz up to 8 markers and
drops to 94 Hz from 9 to 12 markers
Markers are battery powered up to 2.5 hours and weigh
2 ounces each.
 For one marker and one receptor accuracy is 0.04 mm
and 0.0012 degree at 1 ft range and drops afterwards.
Communication from base unit is RS232 or USB
Cost – UP TO $64,000
 source (marker)

sensor
controller
Input Devices

Motion Star wireless tracker (courtesy of Ascension Technology)

Input Devices

DC Magnetic Tracker Block Diagram

Input Devices Wireless suit (Ascension Technology)
Sensors: 20/suit

100 updates/sec
3 meters range
from base unit
Resolution<2 mm Electronic unit
and <.2 degrees (2 hours battery life)
1.7 kg (3.8 lb)
Input Devices

• Two opposite ERTs 3.63.6 m

• Two side-by-side ERTs 1.8 4.2 m
• Readings weighted to the closer ERTs
• Several Base Stations for several users

Motion Star block diagram

Input Devices

Magnetic Tracker Calibration

Use mechanical measurements to reduce errors;
Sensor noise – variation in measurement with no
real object motion – solved by over-sampling;
Size of errors grow from source outwards;
Errors both in position and orientation.
Input Devices

Magnetic Tracker Errors

 due to ambient noise:

e ambient = Kn (d transmitter-receiver)4

 due to metal:

Kr (d transmitter-receiver)4
e metal = ---------------------------------
(d transmitter-metal)3  (d metal-receiver)3
Input Devices

Magnetic tracker accuracy degradation

Input Devices

Comparison of AC and DC magnetic trackers

 DC trackers are immune to non-ferromagnetic metals
(brass, aluminum and stainless steel)
Both DC and AC trackers are affected by the presence of
Ferromagnetic metals (mild steel and ferrite).
Both are affected by copper;
AC trackers have better resolution and accuracy.
AC trackers have slightly shorter range
Input Devices
Input Devices

Ultrasonic Trackers
Definition: A non-contact position measurement
device that uses an ultrasonic signal produced by a
stationary transmitter to determine the real-time
position/orientation of a moving receiver.
Input Devices

Ultrasonic Trackers
Use low-frequency ultrasound to measure position;
Sound produced by a fixed triangular source (speakers);
Number of sources grows with the tracker work envelope;
The receiver is triangular and attached to the tracked
object and has three microphones;
Distance is inferred from the sound time of flight;
Sensitive to air temperature and other noise sources;
Requires “direct line of sight”;
Slower than magnetic trackers (max 50 updates/sec).
Input Devices
Ultrasonic tracker (Logitech)
Input Devices

Large-volume ultrasonic tracker (Logitech)

Input Devices

Optical Trackers
Definition: A non-contact position measurement
device that uses optical sensing to determine the
real-time position/orientation of an object
Input Devices

Optical trackers: a) outside-looking-in; b) inside-looking-out

Input Devices
Outside-looking-in
LaserBIRD optical tracker
Input Devices
Inside-looking-out
LaserBIRD optical tracker
 The two beams are offset
 The three vertices are
computed based on known
offset, geometry and angular
velocity of the beams
The average of the three
vertices is computed and sent
to the host at 240 sets/sec
2 ms latency
Range is smaller than that
of magnetic trackers
Outside-looking in: Vicon MX
Uses 4 Mpixel cameras with
own 120 LED array (infrared, or
visible red). Accuracy 0.02 of a
pixel,
 Camera has real-time onboard
image processing (masking and
thresholding)
 Resolution 2352x1728 @ 160
fps
 8 cameras are connected to a
MX net unit which then
communicates with the PC
Input Devices

Outside-looking in: Vicon MX

 MX Link connects several MX

Net units for more cameras
 To interface with other devices
like a force plate, sensing glove or
eye tracker – use a MX control
unit.
User wears reflective markers
(small spheres).
http://www.youtube.com/watch?
v=xq_nRfoBSm0&feature=related
Input Devices

Outside-looking in: The

Rutgers Arm

 Used in rehabilitation
 Arm and shoulder
movements are tracked with
active markers and an
infrared camera
Low cost
Input Devices
Inside-out optical tracker advantages

The best accuracy is close to the

work envelope.
Very large tracking surface and
resistance to visual occlusions
(line of sight).
 HiBall 3100 wide area tracker
The sensor advantages are:
 High sampling rate (2000 Hz, 1000Hz for 2 sensors, 500Hz for 4);
 High accuracy (0.4 mm, 0.02°) and high resolution (0.2 mm, 0.03°)
 Impervious to metallic or ultrasonic interference;
 Very large tracking area (up to 40 ft x 40 ft), small weight (6 oz).
HiBall Optical Sensor HiBall Optical Sensor interior

6 photodiodes

6 optical lenses

Signal conditioning
electronics

(courtesy of 3rdTech Inc.)

HiBall 3000 tracker
on an HMD

Lateral effect
photo diodes
Types of VR Applications

Beacon array modules

(6 strips with 8 LED/strip)
 Vision-based tracking for the xBox
 The xBox “Project Natal” eliminates the remote.
 Natural interaction, and nothing to wear or hold
 Uses vision tracking and real-time scene recognition, based in part
software from 3DV Systems, an Israeli company.
 Is based on the Time-Of-Flight (TOF) principle, which generates
distance (depth) information for each pixel or object captured by a
camera. Depth data is determined by emitting pulses of infra-red
light to all objects in the scene and sensing the reflected light from
the surface of each object.

xboxprojectnatal's Channel.flv
Inertial Trackers

Christos Giannopoulos
 Introduction
 Tracking technologies may be grouped into three
categories: active, passive, and inertial.
 While active and passive systems rely on the
occurrence of either produced or naturally occurring
signals, inertial systems are completely self contained,
sensing physical phenomena created by linear
acceleration and angular motion.
 How it Works
 Inertial sensors determine orientation and position through
Newton's laws of motion. The two devices that are key to
perform inertial tracking are gyroscopes and
accelerometers.
 Accelerometers measure linear acceleration vectors with
respect to the inertial reference frame
 Gyroscopes measure the rotation rate relative to changes in
orientation.
 Properties
Advantages:
• Very low latency
• Prediction based on directly sensed motion derivatives
• High resolution
• Negligible jitter over entire range
• Unrestricted range
• No line-of-sight problems
• Portability: no source to set up

Disadvantages:
• Susceptible to noise & calibration
errors due to gravity field
• Position & orientation drift can
occur
 Applications
Due to the versatility of inertial tracking systems, a
multitude of applications for its technology have been
developed including:
• Subterranean Navigation
 Inertial navigation technology provides a passive self-contained tracking and
location solution for GPS-denied environments.

• Military Simulation & Training

• Entertainment Applications (“Mo-Cap”)
 Utilizes inertial tracking for “motion capturing,” which many 3D animation and
videogame studios use today (http://www.youtube.com/watch?v=JeGflcAW_-g)

• Medical Settings
• Oil & Gas Discovery
 Sun SPOT System
 SPOT - Small Programmable Object Technology
 Small wireless battery powered experimental platform
programmed almost entirely in Java, allowing regular
programmers to create projects that used to require
specialized embedded system development skills.
 Future Development
 Aside from the constant attempt to make these
devices smaller, faster and more precise, an interest in
improving inertial tracker relies on the concept of
Hybrid Tracking.
 References
1) http://www.ensco.com/Inertial-Navigation-and-Tracking
2) http://www.sti.nasa.gov/tto/Spinoff2005/ch_10.html
3) http://www.xsens.com
4) Foxlin, Eric. “Miniature 6-DOF inertial system for tracking HMDs”
5) You, Suya. “Hybrid Inertial and Vision Tracking for Augmented Reality
Registration”
6) Lang, Peter. “Inertial Tracking for Mobile Augmented Reality”
7) Wormell, D. “Advanced Inertial-Optical Tracking System for Wide Area
Mixed and Augmented Reality Systems”
8) Bandala, Manuel. “Wireless inertial sensor for tumour motion tracking”
9) http://www.sunspotworld.com
Input Devices

Hybrid Ultrasonic/Inertial Trackers

No interference from metallic objects;

No interference from magnetic fields;
Large-volume tracking;
“Source-less” orientation tracking;
Full-room tracking;
A newer technology.
Input Devices

But…

Accelerometer errors a lead to decreased

accuracy since x= a t2
2
Errors grow geometrically in time!
Gyroscope errors compound position errors;
Needs independent position estimation to reduce

“drift”;
Input Devices

IS 900 block diagram

VC 2.1- book CD
IS 900 software block diagram
 Base unit

Sonic Strips

I-cube

Tracker components (InterSense Co.)

Degrees of freedom: 6
Resolution: 1.5 mm RMS
Angular: 0.05o RMS
Update rate: 180 sets/s max
– one station
Down to 90 updates/sec
- for four stations.
Latency 4–10 ms
Max tracking area: 900 meters2
(300 strips, 24 hubs)

Tracker components (courtesy of Intersense Co.)

I-Cube
Accel./gyro

Ultrasonic
emitter

InterSense Stereo Glasses tracker (courtesy of Intersense Co.)

GPS Trackers and Virtual Reality
Global Navigation Satellite System (GNSS)
- Developed by the US DoD
- Only fully functional GNSS in the world
- Uses 24-32 medium Earth orbit satellites
- Transmits data via microwave signals
Basic Concept of Tracking

- Four or more satellites are used to determine x, y, z,

and t
- GPS satellites broadcast their navigation signals at 50
bit/s
• ~ 65 – 85 milliseconds for the signal to travel
from the satellite to the receiver
- The signal transmits:
• Pseudo-random code
• Ephemeris
• Almanic
- GPS receiver calculates it position by precisely timing
the signals from satellites
Basic Concept of Tracking (continued)
Basic Concept of Tracking (continued)
Basic Concept of Tracking (continued)
Basic Concept of Tracking (continued)
Basic Concept of Tracking (continued)
Typical GPS Receiver Characteristics:

- Refresh Rate:
• Automotive Needs: < 5Hz
• Military Needs: 5Hz to greater than 100Hz
- Accuracy: 5 meters
- Wireless receivers
- Work Envelope: Worldwide, restricted to view of
satellites
- Price: 50 USD - 5000 USD
GPS Error Diagram
1Hz vs. 5 Hz GPS
Receivers
Sony PSP GPS Receiver
Reception Frequency: 1575.42Mhz
Refresh time: Approx. 1 second
Precision: 5 meters
Uses:
• Navigation System
• Stargazing
• Several GPS enabled games
Tourality

- GPS multiplayer game for your mobile phone

- Object is to race to a certain spot before others in the
game
JOYity

- GPS enabled games for the

Android
- YouCatch – GPS enabled
version of ‘Manhunt’
- Roads of San Francisco –
Scavenger Hunt
- City Race Munich – Tourist
trivia game
 The GPS Revolution
- iNap
- Shop Savvy
- Google Earth
- Locale
- Goskywatch
- Safety Net
- Sit or Squat
- Twitter
 Military operations in Urban
Therein (MOUT)
- Developed at U. New
Orleans
-Uses a web browser with
CosmoPlayer
-Up to 4 team leaders are
rendered in the scene
-Lander et al 2000
Wireless InertiaCube3
 up to 4 sensors per receiver
 Orientation measurement
 180 Hz update rate;
 Receiver interfaces via USB or RS232 with
host computer

Wireless receiver Wireless sensor

Hybrid Vision/Inertial Tracker
 6 DOF, accuracy 3 mm, 0.1 degree, IS 1200
 Latency 2 ms; InertiaCAM
Fixed-focus lens, on-board DSP
 Requires a minimum of 1 fiducial Weight 35 grams
Tracking range 10 m per 10 cm
for every 2 m2 at 2 m from camera;
Fiducials
of fiducial diameter

 Update rate 120 Hz;

Accelerometer

Ultrasonic
emitter

InterSense Stereo stylus tracker (courtesy of Intersense Co.)

Input Devices
Input Devices

Navigation and Gesture Input Devices

 Navigation interfaces allow relative

position control of virtual objects
(including a virtual camera);
 Gesture interfaces allow dextrous

control of virtual objects and interaction

through gesture recognition.
Input Devices

Navigation Input Devices

 Are the Cubic Mouse, the trackball and the

3-D probe;
 Perform relative position/velocity

control of virtual objects;

 Allow “fly-by” application by controlling

a virtual camera.
Input Devices

VC 2.2 book CD
The Cubic Mouse
Input Devices

Trackballs
Input Devices

The MicroScribe (Immersion Co.)

Input Devices

Gesture Input Devices

 Are sensing gloves such as:
- Fakespace “Pinch Glove”
- 5DT Data Glove;
- The DidjiGlove
- Immersion “CyberGlove”
 Have larger work envelope than trackballs/3-D probes;
 Need calibration for user’s hand.
Input Devices

Finger Degrees of Freedom

Input Devices

Hand work envelope vs. interface type

The Pinch Glove (Fakespace Co.)
- no joint measures, but contact detection
The Pinch Glove (Fakespace Co.)
 P5 Glove
Resistive sensor one per finger

Wrist Tracking

Finger bending resolution 0.5 degrees

0.1 inch wrist tracking resolution at 3 ft range
60 sets/sec $20 to $89
http://www.vrealities.com/demo_hi.mov Movie Link
 One optical fiber/finger 5DT Data Glove

A) 5DT Data Glove Ultra

Roll/pitch sensing
100 datasets/sec, 12 bit A/D flexion resolution,
wireless version transmits data at 30 m, needs calibration

The glove interface: a) five-sensor version; b) newer design

 5DT Data Glove - continued

Two sensors/finger plus

Two gloves use one Bluetooth 2.4
abduction sensors
GHz transmitter on the user’s belt

The glove interface: a) 14-sensor version; b) wireless kit

5DT Data Glove
Glove has less sensors 5DT Data Glove
than hand joints …
Needs to infer distal
joint flexion angle

The coupling of intermediate and distal finger joints

Input Devices 5DT Data Glove
Input Devices 5DT Data Glove

Linear calibration method

 The DG5
glove

 Uses one flex sensor/finger;

 Angles are obtained by measuring
voltages
 100 gestures/sec
 Sensor resolution 10 bit (1024 points),
 cannot measure individual joints
Needs calibration
 less expensive (about $495)
 The new DG5
VHand glove 2.0

 Uses one bend flex sensor/finger+ accelerometers for position/orientation;

 Angles are obtained by measuring voltages
 25 gestures/sec
 Sensor resolution 10 bit (1024 points),
 USB or wireless connection
Incorporates roll, pitch, tracking, 0.5 degree resolution
Needs calibration
 less expensive ($585 wired $785 wireless) www.dg-tech.it
 The CyberGlove

 Uses 18-22 linear sensors – electrical strain gauges;

 Angles are obtained by measuring voltages on
a Wheatstone bridge;
 112 gestures/sec “filtered”.
 Sensor resolution 0.5 degrees, but errors accumulate
to the fingertip (open kinematic chain);
 Sensor repeatability 1 degree
 Needs calibration when put on the hand;
 Is expensive (about $10,000)
 The CyberGlove
(Vertex Co.)

VC 2.3 on book CD
Specifications Pinch P5 Glove 5DT Data Glove DG5 Glove CyberGlove
Glove DG5 VHand
Number of 7/glove 5 5 or 14/glove 5 /glove 18 or
sensors (2 gloves) sensors/glove (1 glove) (1 gloves) 22/glove
Sensor type Electrical (1 glove) Fiber-optic ?(ink film) (1 glove)
Record/sec NA Ink film 100(5DT 5W), 100 Strain gauge
200(5DT 5) 25 (DG 5 VHand) 150(unfiltere
Sensor 1bit 60 8bit 10 bit d)
resolution (2 Points) records/sec (256 Points) (1024 points) 112(filtered)
0.5°
Communicatio Wired Wireless(9.600k Wired (19.2kb)
n Weird b)
(19.2kb) Wireless (DG5
rate None USB 1.0, 2.0 Wired(19.2kb) VHand) Wired
Wrist sensors LED trackers Pitch Accelerometers (115kb)
(5DT 5 model) (DG5 Vhand) Pitch and
yaw