The SixthSense prototype contains a number of

SixthSense is a wearable gestural interface  demonstrations applications.

device developed by Pranav Mistry, a PhD
The map application lets the user navigate a
student in the Fluid Interfaces Group at the MIT
Media Lab. It is similar to Telepointer, a map displayed on a nearby surface using hand
neckworn projector/camera system developed gestures to zoom and pan
by Media Lab student Steve Mann (which Mann
The drawing application lets the user draw on
originally referred to as "Synthetic Synesthesia
of the Sixth Sense") any surface by tracking the fingertip movements
. of the user’s index finger.
Construction and workings SixthSense also implements Augmented reality;
The SixthSense prototype comprises a projecting information onto objects the user
pocket projector, a mirror and a camera interacts with.
contained in a pendant like, wearable device.
The system recognizes a user's freehand
Both the projector and the camera are
gestures as well as icons/symbols drawn in the
connected to a mobile computing device in the
air with the index finger, for example:
user’s pocket. The projector projects visual
A 'framing' gesture takes a picture of the scene.
information enabling surfaces, walls and
The user can stop by any surface or wall and
physical objects around us to be used as
flick through the photos he/she has taken.
interfaces; while the camera recognizes and
tracks user's hand gestures and physical objects Drawing a magnifying glass symbol takes the

using computer-vision based techniques.[3] The user to the map application while an ‘@’ symbol

software program processes the video stream lets the user check his mail.

data captured by the camera and tracks the
locations of the colored markers (visual tracking
fiducials) at the tip of the user’s fingers. The
movements and arrangements of these fiducials
are interpreted into gestures that act as
interaction instructions for the projected
application interfaces. SixthSense supports
multi-touch and multi-user interaction.
The gesture of drawing a circle on the user’s
wrist projects an analog watch.
we can play motion games via piece of paper
and digital information about a product can be
know instantaneous
Cost and license
SixthSense prototypes cost approximately $350
to build (not including the computer)[4][5][6], the
main cost being the micro-projector. The source
code will be released under Open Source[7]
Gesture recognition is a topic in computer science and language technology with the goal of interpreting
human gestures via mathematical algorithms. Gestures can originate from any bodily motion or state but
commonly originate from the face or hand. Current focuses in the field include emotion recognition from
the face and hand gesture recognition. Many approaches have been made using cameras and computer
vision algorithms to interpret sign language. However, the identification and recognition of posture,
gait, proxemics, and human behaviors is also the subject of gesture recognition techniques. [1]

Gesture recognition can be seen as a way for computers to begin to understand human body language,
thus building a richer bridge between machines and humans than primitive text user interfaces or
even GUIs (graphical user interfaces), which still limit the majority of input to keyboard and mouse.

Gesture recognition enables humans to interface with the machine (HMI) and interact naturally without
any mechanical devices. Using the concept of gesture recognition, it is possible to point a finger at
the computer screen so that the cursor will move accordingly. This could potentially make
conventional input devices such as mouse, keyboards and even touch-screens redundant.

Gesture recognition can be conducted with techniques from computer vision and image processing.

The literature includes ongoing work in the computer vision field on capturing gestures or more general
human pose and movements by cameras connected to a computer. [2][3][4][5]

Gesture recognition and pen computing:

 In some literature[examples needed], the term gesture recognition has been used to refer more narrowly
to non-text-input handwriting symbols, such as inking on a graphics tablet, multi-touch gestures,
and mouse gesture recognition. This is computer interaction through the drawing of symbols with a
pointing device cursor (see discussion at Pen computing).
Uses
Gesture recognition is useful for processing information from humans which is not conveyed through
speech or type. As well, there are various types of gestures which can be identified by computers.
 Sign language recognition. Just as speech recognition can transcribe speech to text, certain
types of gesture recognition software can transcribe the symbols represented through sign
language into text.[6]
 For socially assistive robotics. By using proper sensors (accelerometers and gyros) worn on
the body of a patient and by reading the values from those sensors, robots can assist in patient
rehabilitation. The best example can be stroke rehabilitation.
 Directional indication through pointing. Pointing has a very specific purpose in our[clarification
needed]
 society, to reference an object or location based on its position relative to ourselves. The use of
gesture recognition to determine where a person is pointing is useful for identifying the context of
statements or instructions. This application is of particular interest in the field of robotics.[7]
 Control through facial gestures. Controlling a computer through facial gestures is a useful
application of gesture recognition for users who may not physically be able to use a mouse or
keyboard. Eye tracking in particular may be of use for controlling cursor motion or focusing on
elements of a display.
 Alternative computer interfaces. Foregoing the traditional keyboard and mouse setup to
interact with a computer, strong gesture recognition could allow users to accomplish frequent or
common tasks using hand or face gestures to a camera.[8][9][10][11][12]
 Immersive game technology. Gestures can be used to control interactions within video games
to try and make the game player's experience more interactive or immersive.
 Virtual controllers. For systems where the act of finding or acquiring a physical controller could
require too much time, gestures can be used as an alternative control mechanism. Controlling
secondary devices in a car, or controlling a television set are examples of such usage. [13]
 Affective computing. In affective computing, gesture recognition is used in the process of
identifying emotional expression through computer systems.
 Remote control. Through the use of gesture recognition, "remote control with the wave of a
hand" of various devices is possible. The signal must not only indicate the desired response, but also
which device to be controlled.[14][15][16]
[edit]Input devices
The ability to track a person's movements and determine what gestures they may be performing can be
achieved through various tools. Although there is a large amount of research done in image/video based
gesture recognition, there is some variation within the tools and environments used between
implementations.

 Depth-aware cameras. Using specialized cameras such as time-of-flight cameras, one can
generate a depth map of what is being seen through the camera at a short range, and use this data to
 approximate a 3d representation of what is being seen. These can be effective for detection of hand
gestures due to their short range capabilities.[17]
 Stereo cameras. Using two cameras whose relations to one another are known, a 3d
representation can be approximated by the output of the cameras. To get the cameras' relations, one
can use a positioning reference such as a lexian-stripe or infrared emitters.[18]
 Controller-based gestures. These controllers act as an extension of the body so that when
gestures are performed, some of their motion can be conveniently captured by software. Mouse
gestures are one such example, where the motion of the mouse is correlated to a symbol being
drawn by a person's hand, as is the Wii Remote, which can study changes in acceleration over time
to represent gestures.[19][20][21]
 Single camera. A normal camera can be used for gesture recognition where the
resources/environment would not be convenient for other forms of image-based recognition. Although
not necessarily as effective as stereo or depth aware cameras, using a single camera allows a
greater possibility of accessibility to a wider audience. [22]
[edit]Challenges

There are many challenges associated with the accuracy and usefulness of gesture recognition software.
For image-based gesture recognition there are limitations on the equipment used and image noise.
Images or video may not be under consistent lighting, or in the same location. Items in the background or
distinct features of the users may make recognition more difficult.

The variety of implementations for image-based gesture recognition may also cause issue for viability of
the technology to general usage. For example, an algorithm calibrated for one camera may not work for a
different camera. The amount of background noise also causes tracking and recognition difficulties,
especially when occlusions (partial and full) occur. Furthermore, the distance from the camera, and the
camera's resolution and quality, also cause variations in recognition accuracy.

In order to capture human gestures by visual sensors, robust computer vision methods are also required,
for example for hand tracking and hand posture recognition [23][24][25][26][27][28][29][30][31] or for capturing
movements of the head, facial expressions or gaze direction.

[edit]"Gorilla arm"
"Gorilla arm" was a side-effect that destroyed vertically-oriented touch-screens as a mainstream input
technology despite a promising start in the early 1980s. [32]

Designers of touch-menu systems failed to notice that humans aren't designed to hold their arms in front
of their faces making small motions. After more than a very few selections, the arm begins to feel sore,
cramped, and oversized—the operator looks like a gorilla while using the touch screen and feels like one
afterwards. This is now considered a classic cautionary tale to human-factors designers; "Remember the
gorilla arm!" is shorthand for "How is this going to fly in real use?".

Gorilla arm is not a problem for specialist short-term-use uses, since they only involve brief interactions
which do not last long enough to cause gorilla arm.

Mobile computing is "taking a computer and all necessary files and software out into the field." [1]

Many types of mobile computers have been introduced since the 1990s, including the:

 Wearable computer
 Personal digital assistant/Enterprise digital assistant
 Smartphone
 Carputer
 Ultra-Mobile PC

[edit]Technical and other limitations of mobile computing

 Insufficient bandwidth

Mobile Internet access is generally slower than direct cable connections, using technologies such
asGPRS and EDGE, and more recently HSDPA and HSUPA 3G networks. These networks are usually
available within range of commercial cell phone towers. Higher speed wireless LANs are inexpensive, but
have very limited range.

 Security standards

When working mobile one is dependent on public networks, requiring careful use of VPNs.

 Power consumption
When a power outlet or portable generator is not available, mobile computers must rely entirely on battery
power. Combined with the compact size of many mobile devices, this often means unusually expensive
batteries must be used to obtain the necessary battery life.

 Transmission interferences

Weather, terrain, and the range from the nearest signal point can all interfere with signal reception.
Reception in tunnels, some buildings, and rural areas is often poor.

 Potential health hazards

More car accidents are related to drivers who were talking through a mobile device. Cell phones may
interfere with sensitive medical devices. There are allegations that cell phone signals may cause health
problems.[citation needed]

 Human interface with device

Screens and keyboards tend to be small, which may make them harder to use. Alternate input methods
such as speech or handwriting recognition require training.

[edit]In-vehicle computing and fleet computing

Many commercial and government field forces deploy a ruggedized portable computer such as the
Panasonic Toughbook or larger rack-mountedcomputers with their fleet of vehicles. This requires the
units to be anchored to the vehicle for driver safety, device security, and userergonomics. Ruggedized
computers are rated for severe vibration associated with large service vehicles and off-road driving, and
the harsh environmental conditions of constant professional use such as in Emergency medical services,
fire and public safety.

The Compaq Portable

Other elements that enables the unit to function in vehicle:

 Operating temperature: A vehicle cabin can often experience temperature swings from -20F to
+140F. Computers typically must be able to withstand these temperatures while operating. Typical
fan based cooling has stated limits of 95F-100F of ambient temperature, and temperature below
freezing require localized heaters to bring components up to operating temperature(based on
independent studies by the SRI Group and by Panasonic R&D).
 Vibration: Vehicles typically have considerable vibration that can decrease life expectancy of
computer components, notably rotational storage such as HDDs.
 Daylight, or sunlight readability: Visibility of standard screens becomes an issue in bright sunlight.
 Touchscreens: These enable users to easily interact with the units in the field without removing
gloves.
 High-Temperature Battery Settings:. Lithium Ion batteries are sensitive to high temperature
conditions for charging. A computer designed for the mobile environment should be designed with a
high-temperature charging function that limits the charge to 85% or less of capacity.
 External wireless Connections, and External GPS Antenna Connections: Necessary to contend
with the typical metal cabins of vehicles and their impact on wireless reception, and to take advantage
of much more capable external tranception equipment.

Several specialized manufacturers such as First Mobile Technologies, National Products Inc (Ram
Mounts), Gamber Johnson and LedCo build mounts for vehicle mounting of computer equipment for a
wide range of vehicles. The mounts are built to withstand the harsh conditions and maintain ergonomics.

Specialized installation companies specialize in designing the mount design, assembling the proper parts,
and installing them in a safe and consistent manner away from airbags, vehicle HVAC controls, and driver
controls. Frequently installations will include a WWAN modem, power conditioning equipment, and
WWAN/WLAN/GPS/etc… transceiver antennæ mounted external to the vehicle.Mobile internet access is
generally slower than direct cable connections, using technologies such as GPRS(General packet radio
service) and EDGE( Enhanced Data for GSM Evolution), and more recently 3G networks. These
networks are usually available within range of commercial cell phone towers. Higher speed wireless LANs
are inexpensive, but have very limited range

[edit]Portable computing devices

This section may require cleanup to meet Wikipedia's quality
standards. Please improve this section if you can. (February 2009)

There are several categories of portable computing devices that can run on batteries but are not usually
classified as laptops: portable computers, keyboardless tablet PCs, Internet tablets, PDAs, Ultra Mobile
PCs (UMPCs) and smartphones.
A Nokia N800 Internet tablet

A Portable computer is a general-purpose computer that can be easily moved from place to place, but
cannot be used while in transit, usually because it requires some "setting-up" and an AC power source.
The most famous example is the Osborne 1. Portable computers are also called a "transportable" or a
"luggable" PC.

A Tablet PC that lacks a keyboard (also known as a non-convertible Tablet PC) is shaped like slate or a
paper notebook, features a touchscreen with a stylus and handwriting recognition software. Tablets may
not be best suited for applications requiring a physical keyboard for typing, but are otherwise capable of
carrying out most tasks that an ordinary laptop would be able to perform.

An Internet tablet is an Internet appliance in tablet form. Unlike a Tablet PC, an Internet tablet does not
have much computing power and its applications suite is limited, and it can not replace a general purpose
computer. Internet tablets typically feature an MP3 and video player, a web browser, a chat application
and a picture viewer.

A Personal digital assistant (PDA) is a small, usually pocket-sized, computer with limited functionality. It is
intended to supplement and to synchronize with a desktop computer, giving access to contacts, address
book, notes, e-mail and other features.

A Palm TX PDA

An Ultra Mobile PC is a full-featured, PDA-sized computer running a general-purpose operating system.
A Smart phone is a PDA with an integrated cellphone functionality. Current smartphones have a wide
range of features and installable applications.

A Carputer is a computing device installed in an automobile. It operates as a wireless computer, sound

system, GPS, and DVD player. Also contains word processing software and it's bluetooth compatible. [2]

A Fly Fusion Pentop computer is a computing device the size and shape of a pen. It functions as a writing
utensil, MP3 player, language translator, digital storage device, and calculator. [3]

Boundaries that separate these categories are blurry at times. For example, the OQO UMPC is also a
PDA-sized tablet PC; the Apple eMate had the clamshell form factor of a laptop, but ran PDA software.
The HP Omnibook line of laptops included some devices small enough to be called Ultra Mobile PCs. The
hardware of the Nokia 770 internet tablet is essentially the same as that of a PDA such as
the Zaurus 6000; the only reason it's not called a PDA is that it doesn't have PIM software. On the other
hand, both the 770 and the Zaurus can run some desktop Linux software, usually with modifications.

Augmented reality (AR) is a term for a live direct or indirect view of a physical real-world environment
whose elements are augmented by virtual computer-generated imagery. It is related to a more general
concept called mediated reality in which a view of reality is modified (possibly even diminished rather than
augmented) by a computer. As a result, the technology functions by enhancing one’s current perception
of reality.

In the case of Augmented Reality, the augmentation is conventionally in real-time and in semantic context
with environmental elements, such as sports scores on TV during a match. With the help of advanced AR
technology (e.g. adding computer vision and object recognition) the information about the surrounding
real world of the user becomes interactive and digitally usable. Artificial information about the environment
and the objects in it can be stored and retrieved as an information layer on top of the real world view. The
term augmented reality is believed to have been coined in 1990 by Thomas Caudell, an employee of
Boeing at the time[1].
Augmented reality research explores the application of computer-generated imagery in live-video streams
as a way to expand the real-world. Advanced research includes use of head-mounted displays and virtual
retinal displays for visualization purposes, and construction of controlled environments containing any
number of sensors and actuators.