MULTI TOUCH

A SEMINAR REPORT

Submitted by

ALIKUTTY K A

BACHELOR OF TECHNOLOGY

in

COMPUTER SCIENCE & ENGINEERING

SCHOOL OF ENGINEERING

COCHIN UNIVERSITY OF SCIENCE & TECHNOLOGY,

KOCHI-682022

NOVEMBER 2008
ABSTRACT

Multi-touch technology is an advanced human-computer interaction technique that recogonises

multiple touch points and also includes the hardware devices that implement it, which allow
users to compute without conventional input devices . Multi-touch consists of a touch screen
(screen, table, wall, etc.) or touchpad, as well as a software that recognizes multiple simultaneous
touch points, as opposed to the standard touchscreen which recognizes only one touch point at a
time. Multi touch using Frustrated Total Internal Reflection is a simple, inexpensive, and
scalable technique for enabling high-resolution multi- touch sensing on rear-projected interactive
surfaces. Different applications for multi-touch interfaces both exist and are being proposed.
Some uses are individualistic eg iPhone, iPod touch, MacBook Pro, MacBook Air. The use of
multi-touch technology is expected to rapidly become common place.
TABLE OF CONTENTS

1. INTRODUCTION

1.1 TOUCH SCREEN

1.1.1 TECHNOLOGIES

1.1.2 DEVELOPMENT

2. MULTI TOUCH

2.1 HISTORY

2.2 RECENT DEVELOPMENTS

2.3 FUTURE

3. FTIR

3.1 PRINCIPLES

3.2 DESIGN

3.3.1 REQUIREMENTS

3.3.1.1 HARDWARE

3.3.1.2 SOFTWARE

3.4 WORKING

3.5 ADVANTAGES

3.6 APPLICATIONS

4.CONCLUSION

5.APPENDICES

5.1 APPENDIX I

6.REFERENCES
1. INTRODUCTION

1.1 TOUCH SCREEN

A touch screen is a display which can detect the presence and location of a touch within the
display area. The term generally refers to touch or contact to the display of the device by a
finger or hand. Touch screens can also sense other passive objects, such as a stylus. However, if
the object sensed is active, as with a light pen, the term touch screen is generally not applicable.
The thumb rule is: if you can interact with the display using your finger, it is likely a touch
screen - even if you are using a stylus or some other object. Up until recently, most touch
screens could only sense one point of contact at a time, and few have had the capability to sense
how hard one is touching. This is starting to change with the emergence of multi-touch
technology - a technology that was first seen in the early 1980s, but which is now appearing in
commercially available systems. The touch screen has two main attributes. First, it enables you
to interact with what is displayed directly on the screen, where it is displayed, rather than
indirectly with a mouse or a touchpad. Secondly, it lets one do so without requiring any
intermediate device, again, such as a stylus that needs to be held in the hand. Such displays can
be attached to computers or, as terminals, to networks. They also play a prominent role in the
design of digital appliances such as the personal digital assistant , satellite navigation devices
and mobile phone

1.1.1 TECHNOLOGIES

There are a number of types of touch screen technology

Resistive

A resistive touch screen panel is composed of several layers. The most important are two thin
metallic electrically conductive and resistive layers separated by thin space. When some object
touches this kind of touch panel, the layers are connected at certain point; the panel then
electrically acts similar to two voltage dividers with connected outputs. This causes a change in
the electrical current which is registered as a touch event and sent to the controller for
processing. Surface acoustic wave SAW technology uses ultrasonic waves that pass over the
touchscreen panel. When the panel is touched, a portion of the wave is absorbed. This change in
the ultrasonic waves registers the position of the touch event and sends this information to the
controller for processing. Surface wave touch screen panels can be damaged by outside
elements. Contaminants on the surface can also interfere with the functionality of the touch
screen.

Capacitive

A capacitive touch screen panel is coated with a material, typically indium tin oxide that
conducts a continuous electrical current across the sensor. The sensor therefore exhibits a
precisely controlled field of stored electrons in both the horizontal and vertical axes - it achieves
capacitance. The human body is also an electrical device which has stored electrons and
therefore also exhibits capacitance. When the sensor's 'normal' capacitance field (its reference
state) is altered by another capacitance field, i.e., someone's finger, electronic circuits located at
each corner of the panel measure the resultant 'distortion' in the sine wave characteristics of the
reference field and send the information about the event to the controller for mathematical
processing. Capacitive sensors can either be touched with a bare finger or with a conductive
device being held by a bare hand. Capacitive touchscreens are not affected by outside elements
and have high clarity. The Apple iPhone is an example of a product that uses capacitance
touchscreen technology.

Infrared

An IR touchscreen panel employs one of two very different methods. One method uses thermal
induced changes of the surface resistance. This method is sometimes slow and requires warm
hands. Another method is an array of vertical and horizontal IR sensors that detect the
interruption of a modulated light beam near the surface of the screen.

Optical imaging

A relatively-modern development in touchscreen technology, two or more image sensors are

placed around the edges (mostly the corners) of the screen. Infrared backlights are placed in the
camera's field of view on the other sides of the screen. A touch shows up as a shadow and each
pair of cameras can then be triangulated to locate the touch. This technology is growing in
popularity, due to its scalability, versatility, and affordability, especially for larger units.

Dispersive signal technology

Introduced in 2002, this system uses sensors to detect the mechanical energy in the glass that
occur due to a touch. Complex algorithms then interpret this information and provide the actual
location of the touch. The technology claims to be unaffected by dust and other outside
elements, including scratches. Since there is no need for additional elements on screen, it also
claims to provide excellent optical clarity. Also, since mechanical vibrations are used to detect a
touch event, any object can be used to generate these events, including fingers and stylus. A
downside is that after the initial touch the system cannot detect a motionless finger. 1.1.2
DEVELOPMENT

Virtually all of the significant touchscreen technology patents were filed during the 1970s and
1980s and have expired. Touchscreen component manufacturing and product design are no
longer encumbered by royalties or legalities with regard to patents and the manufacturing of
touchscreen-enabled displays on all kinds of devices is widespread.
The development of multipoint touchscreens facilitated the tracking of more than one finger on
the screen, thus operations that require more than one finger are possible. These devices also
allow multiple users to interact with the touchscreen simultaneously. With the growing
acceptance of many kinds of products with an integral touchscreen interface the marginal cost of
touchscreen technology is routinely absorbed into the products that incorporate it and is
effectively eliminated. As typically occurs with any technology, touchscreen hardware and
software has sufficiently matured and been perfected over more than three decades to the point
where its reliability is unassailable. As such, touchscreen displays are found today in airplanes,
automobiles, gaming consoles, machine control systems, appliances and handheld display
devices of every kind. The ability to accurately point on the screen itself is taking yet another
step with the emerging graphics tablet/screen hybrids.

2.MULTI TOUCH

Multi-touch is a human-computer interaction technique and the hardware devices that

implement it, which allow users to compute without conventional input devices e.g., mouse,
keyboard. Multi-touch consists of a touch screen like screen, table, wall or touchpad, as well as
software that recognizes multiple simultaneous touch points, as opposed to the standard
touchscreen i.e. computer touchpad, ATM, which recognizes only one touch point. This effect is
achieved through a variety of means, including but not limited to: heat, finger pressure, high
capture rate cameras, infrared light, optic capture, tuned electromagnetic induction and shadow
capture.

2.1 HISTORY

Multi-touch technology dates back to 1982, when the University of Toronto developed the first
finger pressure multi-touch display. The same year, Bell Labs at Murray Hill published what is
believed to be the first paper discussing touch-screen based interfaces.

Bell Labs

In 1984 Bell Labs engineered a multi-touch screen that could manipulate images with more than
one hand. The group at the University of Toronto stopped working on hardware and moved on
to software and interfaces, expecting that they would have access to the Bell Labs work.

A breakthrough occurred in 1991, when Pierre Wellner published a paper on his multi-touch
Digital Desk, which supported multi-finger and pinching motions.

Finger works

In 1998, FW, a Newark-based company run by University of Delaware academics John Elias
and Wayne Westerman, produced a line of multi-touch products including the iGesture Pad and
the TouchStream keyboard. Westerman published a dissertation in 1999 on the subject. In 2005,
after years of maintaining a niche line of keyboards and touch pads, Finger works was acquired
by Apple Computer.

2.2 RECENT DEVELOPMENTS

Various companies expanded upon these discoveries in the beginning of the twenty- first
century. Mainstream exposure to multi-touch technology occurred in the year 2007, when Apple
unveiled the iPhone and Microsoft debuted surface computing. The iPhone in particular has
spawned a wave of interest in multi-touch computing, since it permits greatly increased user
interaction on a small scale. More robust and customizable multi-touch and gesture-based
solutions are beginning to become available, among them TrueTouch, created by Cypress
Semiconductor. The following is a compilation of notable uses of multi-touch technology in
recent years. Microsoft Surface In 2001 Steve Bathiche and Andy Wilson of Microsoft began
work on an idea for an interactive table that mixes both physical and virtual worlds. Research
and Development expanded rapidly in 2004, once the idea caught the attention of Microsoft
Chairman Bill Gates. In 2007 Microsoft introduced MS, a functional multi- touch table-top
computer based on a standard PC platform including an Intel Core 2 Duo processor, Windows
Vista, and 2 GB of RAM

Perceptive Pixel

PP is a company founded by New York University consulting research scientist Jefferson Y.

Han that creates wall displays and tables that can accommodate up to 20 fingers. Han introduced
the FTIR technique to multi touch screens. The displays use light emitting diodes along with
infrared light to determine the point of contact. Han envisions large collaborative spaces that
will allow multiple users to work and interact. PPâ„¢s technology is currently being utilized, in
the form of the Multi-Touch Collaboration Wall, by CNN and an unspecified government
contractor everyday programs with ease and most importantly more than one user can operate
the system at any given time.

Apple iPhone, iPod touch, MacBook Air, and MacBook Pro In 2005, Apple acquired
Fingerworks. In 2007 they introduced the iPhone, marking the first time multi-touch technology
was used on a phone. The iPhone includes such components as a web browser, music player,
video player, and a cell phone without the use of a hard keypad or stylus. Following the release
of the iPhone, Apple also expanded its use of multi-touch computing with the new iPod Touch,
as well as the new MacBook Air. Multi-touch was later added to the 2008 MacBook Pro line in
the form of a trackpad. Apple is currently in the process of trying to patent its Multi-touch
technology and to trademark the term "multi-touch".

2.3 FUTURE

The use of multi-touch technology is expected to rapidly become common place. For example,
touch screen telephones are expected to increase from 200,000 shipped in 2006, to 21 million in
2012. Developers of the technology have suggested a variety of ways that multi-touch can be
used including:

 Enhanced dining experience

 Concierge service
 Governmental use
 Concept mapping
 Collaboration and instruction on Interactive Whiteboards

3.FRUSTRATED TOTAL INTERNAL REFLECTION

3.1 PRINCIPLES

Total internal reflection is an optical phenomenon that occurs when a ray of light strikes a
medium boundary at an angle larger than the critical angle with respect to the normal to the
surface. If the refractive index is lower on the other side of the boundary no light can pass
through, so effectively all of the light is reflected. The critical angle is the angle of incidence
above which the total internal reflection occurs. When light crosses a boundary between
materials with different refractive indices, the light beam will be partially refracted at the
boundary surface, and partially reflected. However, if the angle of incidence is greater (i.e. the
ray is closer to being parallel to the boundary) than the critical angle ” the angle of incidence
at which light is refracted such that it travels along the boundary ” then the light will stop
crossing the boundary altogether and instead be totally reflected back internally. This can only
occur where light travels from a medium with a higher refractive index to one with a lower
refractive index. For example, it will occur when passing from glass to air, but not when passing
from air to glass.

The critical angle is the angle of incidence above which total internal reflection occurs. The
angle of incidence is measured with respect to the normal at the refractive boundary. The critical
angle is given by: where n

is the refractive index of the less dense medium, and n

is the refractive index of the denser medium

An important side effect of total internal reflection is the propagation of an evanescent wave
across the boundary surface. Essentially, even though the entire incident wave is reflected back
into the originating medium, there is some penetration into the second medium at the boundary.
Additionally, the evanescent wave appears to travel along the boundary between the two
materials. This wave can lead to a phenomenon known as frustrated total internal reflection.

Under "ordinary conditions" it is true that the creation of an evanescent wave does not affect the
conservation of energy, i.e. the evanescent wave transmits zero net energy. However, if a third
medium with a higher refractive index than the second medium is placed within less than several
wavelengths distance from the interface between the first medium and the second medium, the
evanescent wave will be different from the one under "ordinary conditions" and it will pass
energy across the second into the third medium.

3.2 DESIGN

3.2.1 REQUIREMENTS

Multi touch is designed using the FTIR technology. FTIR describes the internal reflection of
light .It is force-sensitive, and provides unprecedented resolution and scalability .Large enough
to accommodate both hands and multiple users.This phenomenon is also used in fingerprint and
robot sensors.Allows us to create sophisticated multi-point widgets for applications 3.2.1.1
HARDWARE

The basic design has a hardware and software part. Hardware requires basically IR
ledâ„¢s,acrylic,camera,projector and a computer. Infra red light has a higher wavelength than
that of visible light .Therefore it has more intensity and will be felt everywhere inside the denser
medium. Acrylic is the denser medium .It is a synthetic fiber having half the density of glass.An
infra red camera or a webcam is used to catch IR light . A IR block filter of the camera has to be
removed since it blocks IR light. It also consists of a projector and a computer. The object has to
be projected on top of the acrylic from a computer . Thus acrylic is a virtual display. The IR
LEDâ„¢s about 10-20 are arranged on both sides of the acrylic along its edges .The camera
below the acrylic and the projector located behind to acrylic.

3.2.1.2 SOFTWARE

Computer applications are necessary to communicate between a multi touch display and a
computer.These applications can be developed within several languages / programming
environments.For example: Processing (P5), Flash, C, C++, Java and others. Because of the
usability of certain APIâ„¢s and the relatively simple visualisation possibilities, P5 or Flash 9, in
combination with Actionscript 3.0 will make a great combination
3.4 WORKING

FTIR describes the internal reflection of light, inside a certain material. In our case, it will be
infrared light, that internally reflects inside is a piece of acrylic, also known as plexiglas.This
way, infrared light is beamed inside the acrylic and reflects internally.In a simple way, you can
say that, IR-light bounces inside the acrylic, from one side to another.As soon as a finger
touches the acrylic surface, the internal reflection of the IR-light, is interrupted.The infrared
light scatters on the finger tips.Infrared light is invisible to the human eye, but by placing an
infrared camera behind the acrylic your fingertips will be visible on the infrared camera.The
images that are generated by the camera, contain white blobs (caused by the fingertips). These
blobs will be analyzed by software. Every blob corresponds to certain coordinates. Software can
by analyzing these coordinates perform certain tasks, for example move, resize or rotate objects.
Multiple points are obtained on the camera .Each point is a pixel position. Either a single pixel
or a group of pixel . Each point locations are identified and all operations are performed
.Suppose if we want to zoom a picture we use 2 fingers and move it in or out to perform zoom in
and zoom out respectievely . Two coordinates will be located on the camere .The difference is
found which is put as the offset and it is either added or subtracted with the locations to zoom
out and zoom in respectievely.

3.5 ADVANTAGES

 Multi touch based on FTIR is a simple and inexensive technique .It constructs a multi
touch display with the available and less costly materials .
 Scalable technique that enables high-resolution graphics .It provides support to any
resolution possible as all multiple points could be generated on a camera

 It acquires true touch image information at high spatial and temporal resolutions.The
actual finger print of the touch is obtained .This could be used to determine the force
sensitivity on displays , either too hard or soft touches can be analysed.
 It is scalable to large installations.Any kind of applications can be made to suit multi
touch using FTIR . Allows us to create sophisticated multi-point widgets for applications
 Larger shared-display systems ie it is well suited for use with rear- projection like wall
screens,table tops .All this lead to high resolution graphics.
3.6 APPLICATION

A myriad different applications for multi-touch interfaces both exist and are being proposed.
Some uses are individualistic e.g., iPhone, iPod touch, MacBook Pro, MacBook Air, HTC
Diamond . However, multi-touch technology is mainly used to incorporate collaboration into the
computing experience .

A multi touch display can be used in

 Personal computers,Laptops,Tabletops,Graphics Tablets .

 It supports both LCD and CRT monitors .
 Telephones ,Watches ,PDAs, Mobile phones.
 Advanced multi touch Gaming with high graphics support
 Governmental,office and business purposes
 An enhanced multimedia experience including audio,video and photo sharing
 Enhanced dining experience

Applications for a multi touch display are never ending . We can even convert a computer to a
mere piece of display attached to a wall or a
4.CONCLUSION

Touch screens are the interface for the 21 st century. Touch screens address the conflicting
demands for smaller portable electronics with larger displays, by eliminating traditional buttons
without sacrificing screen size. The recent release of the iPhone has created a buzz around touch
screen interfaces and its multi-touch acrobatics have caught the eye of many industry leaders.
There are many ways to make a multi-touch screen. Some of the early designs measured the
change in electrical resistance or capacitance on a surface when fingers touched it. But these
devices have limited resolution, are relatively complex, and don't easily and inexpensively scale
up to large dimensions. Multi-touch technologies have a long history. This technique using
FTIR is simple and easy to implement . It provides any resolution displays supported with high
graphics .The applications being both made and proposed are plenty in number . A drawback of
the approach is that, being camera-based, it requires a significant amount of space behind the
interaction surface, though we primarily expect application scenarios where rear-projection
would have been employed anyway (e.g. interactive walls, tables). Also, as an optical system, it
remains susceptible to harsh lighting environments.
6.REFERENCES

 Low-Cost Multi-Touch Sensing through FTIR by Jefferson Y. Han

 Buxton, W., Hill, R., and Rowley, P. 1985. Issues and Techniques in
 Touch-Sensitive Tablet Input
 Donald Hearn ,M Pauline Baker, Computer Graphics C version, 2/E
 Pearson Education ,2003
 Buxton, Bill. 2008. Multi-Touch Systems that I Have Known and Loved.
 http://www.billbuxton.com/multitouchOverview.html
 How to build a multi touch by Harry Vaan Der
 Opensource,MultitouchDisplay,http://www.technologyreview.com/Infotec