Surface Computing

ABSTRACT
The name Surface comes from " surface computing ," and Microsoft envisions the
coffee-table machine as the first of many such devices. Surface computing uses a blend of
wireless protocols, special machine-readable tags and shape recognition to seamlessly merge
the real and the virtual world — an idea the Milan team refers to as "blended reality." The
table can be built with a variety of wireless transceivers, including Bluetooth, Wi-Fi and
(eventually) radio frequency identification (RFID) and is designed to sync instantly with any
device that touches its surface.

It supports multiple touch points – Microsoft says "dozens and dozens" -- as well as
multiple users simultaneously, so more than one person could be using it at once, or one
person could be doing multiple tasks.

The term "surface" describes how it's used. There is no keyboard or mouse. All
interactions with the computer are done via touching the surface of the computer's screen
with hands or brushes, or via wireless interaction with devices such as smartphones, digital
cameras or Microsoft's Zune music player. Because of the cameras, the device can also
recognize physical objects; for instance credit cards or hotel "loyalty" cards.

For instance, a user could set a digital camera down on the tabletop and wirelessly
transfer pictures into folders on Surface's hard drive. Or setting a music player down would
let a user drag songs from his or her home music collection directly into the player, or
between two players, using a finger – or transfer mapping information for the location of a
restaurant where you just made reservations through a Surface tabletop over to a smartphone
just before you walk out the door.

Dept. of ECE 2008-

09
Surface Computing

CONTENTS
CHAPTERS PAGE. No .

1. INTRODUCTION……………………………………………………...………..01

2. WHAT IS SURFACE
COMPUTING?.................................................................02
3. HISTORY OF SURFACE COMPUTING………...………………………….…03

3.1 Hardware Design…………………………………………………………….05

3.2 From Prototype to Product…………………………………………………..06

4. KEY ATTRIBUTES OF SURFACE COMPUTING……………………………07

5. TECHNOLOGY BEHIND SURFACE COMPUTING……………………..….08

6. MICROSOFT SURFACE OVERVIEW………………………………………..09

6.1 The Hardware………………………………………………………………...09

6.2 System Software………...………………………………..………………….12

7. FEATURES………………………………………………………………..…….13

8. PERCEPTIVE PIXEL…………………………………………………………..15

9. APPLICATIONS OF SURFACE COMPUTING………………………………17

9.1 Water…………………………………………………………………………17

9.2 Video Puzzle…………………………………………………………………18

9.3 Paint………………………………………………………………………….19

9.4 Music…………………………………………………………………………20

9.5 Photos………………………………………………………………………...21

9.6 Casino………………………………………………………………………..22

9.7 Dining………………………………………………………………………..22

10. SURFACE COMPUTING – IN FUTURE.……………………………………..23

11. CONCLUSION…………………………………………………………..…..….24

12. BIBLIOGRAPHY……………………………………………………………….25

Dept. of ECE 2008-

09
Surface Computing 1

1. Introduction:

Over the past couple of years, a new class of interactive device has begun to
emerge, what can best be described as “surface computing”. Two examples are illustrated
in this report. They are-
Surface Table top
Perceptive Pixel
The Surface table top typically incorporates a rear-projection display coupled
with an optical system to capture touch points by detecting shadows from below.
Different approaches to doing the detection have been used, but most employ some form
of IR illumination coupled with IR cameras. With today’s camera and signal-processing
capability, reliable responsive and accurate multi-touch capabilities can be achieved.

The multitouch pioneer and his company, Perceptive , have devoted the
Pixel
better part of two years to building an entirely new multitouch framework from the
ground up. Instead of simply mapping multitouch technology to familiar interfaces and
devices, Han's goal is far more sweeping: To use the technology as a foundation for an
entirely new operating system.

Because they are new to most, the tendency in seeing these systems is to assume
that they are all more-or-less alike. Well, in a way that is true. But on the other hand, that
is perhaps no more so than to say that all ICs are more-or-less alike, since they are black
plastic things with feet like centipedes which contain a bunch of transistors and other
stuff. In short, the more that you know, the more you can differentiate. But even looking
at the two systems in the photo, there is evidence of really significant difference.

The really significant difference is that one is vertical and the other is horizontal.
Why is this significant? Well, this is one of those questions perhaps best answered by a
child in kindergarten. They will tell you that if you put a glass of water on the vertical
one, it will fall to the floor, leading to a bout of sitting in the corner. On the other hand, it
is perfectly safe to put things on a table. They will stay there.

Dept. of ECE
2008-09
Surface Computing 2

2. What is surface computing?

Surface computing is a new way of working with computers that moves beyond
the traditional mouse-and-keyboard experience. It is a natural user interface that allows
people to interact with digital content the same way they have interacted with everyday
items such as photos, paintbrushes and music their entire life: with their hands, with
gestures and by putting real-world objects on the surface. Surface computing opens up a
whole new category of products for users to interact with.

Surface computing is a completely intuitive and liberating way to interact with

digital content. It blurs the lines between the physical and virtual worlds. By using your
hands or placing other unique everyday objects on the surface – such as an item you’re
going to purchase at a retail store or a paint brush – you can interact with, share and
collaborate like you’ve never done before. Imagine you’re out at a restaurant with friends
and you each place your beverage on the table – and all kinds of information appears by
your glass, such as wine pairings with a restaurant’s menu. Then, with the flick of your
finger, you order dessert and split the bill. We really see this as broadening content
opportunities and delivery systems.

Surface computing is a powerful movement. In fact, it’s as significant as the move

from DOS [Disk Operating System] to GUI [Graphic User Interface]. Our research shows
that many people are intimidated and isolated by today’s technology. Many features
available in mobile phones, PCs and other electronic devices like digital cameras aren’t
even used because the technology is intimidating. Surface computing breaks down those
traditional barriers to technology so that people can interact with all kinds of digital
content in a more intuitive, engaging and efficient manner. It’s about technology adapting
to the user, rather than the user adapting to the technology. Bringing this kind of natural
user interface innovation to the computing space is what Surface Computing is all about.

Dept. of ECE
2008-09
Surface Computing 3

3. History of Surface Computing:

Surface computing is a major advancement that moves beyond the traditional user
interface to a more natural way of interacting with digital content. Microsoft Surface™,
Microsoft Corp.’s first commercially available surface computer, breaks down the
traditional barriers between people and technology to provide effortless interaction with
all forms of digital content through natural gestures, touch and physical objects instead of
a mouse and keyboard. The people will be able to interact with Surface in select
restaurants, hotels, retail establishments and public entertainment.

In 2001, Stevie Bathiche of Microsoft Hardware and Andy Wilson of Microsoft

Research began working together on various projects that took advantage of their
complementary expertise in the areas of hardware and software. In one of their regular
brainstorm sessions, they started talking about an idea for an interactive table that could
understand the manipulation of physical pieces. Although there were related efforts
happening in academia, Bathiche and Wilson saw the need for a product where the
interaction was richer and more intuitive, and at the same time practical for everyone to
use. This conversation was the beginning of an idea that would later result in the
development of Surface, and over the course of the following year, various people at
Microsoft involved in developing new product concepts, including the gaming-specific
PlayTable, continued to think through the possibilities and feasibility of the project. Then

Dept. of ECE
2008-09
Surface Computing 4

in October 2001 a virtual team was formed to fully pursue bringing the idea to the next
stage of development; Bathiche and Wilson were key members of the team.

In early 2003, the team presented the idea to Bill Gates, Microsoft chairman, in a
group review. Gates instantly liked the idea and encouraged the team to continue to
develop their thinking. The virtual team expanded, and within a month, through constant
discussion and brainstorming, the first humble prototype was born and nicknamed T1.
The model was based on an IKEA table with a hole cut in the top and a sheet of architect
vellum used as a diffuser. The evolution of Surface had begun. A variety of early
applications were also built, including pinball, a photo browser and a video puzzle. As
more applications were developed, the team saw the value of the surface computer
beyond simply gaming and began to favor those applications that took advantage of the
unique ability of Surface to recognize physical objects placed on the table. The team was
also beginning to realize that surface computing could be applied to a number of different
embodiments and form factors. Over the next year, the team grew significantly, including
the addition of Nigel Keam, initially software development lead and later architect for
Surface, who was part of the development team eventually tasked with taking the product
from prototype to a shipping product. Surface prototypes, functionality and applications
were continually refined. More than 85 early prototypes were built for use by software
developers, hardware developers and user researchers.

One of the key attributes of Surface is object recognition and the ability of objects
placed on the surface to trigger different types of digital responses, including the transfer
of digital content. This feature went through numerous rounds of testing and refining.
The team explored various tag formats of all shapes and sizes before landing on the
domino tag (used today) which is an 8-bit, three-quarter-inch-square tag that is optimal
thanks to its small size. At the same time, the original plan of using a single camera in the
vision system was proving to be unreliable. After exploring a variety of options,
including camera placement and different camera lens sizes, it was decided that Surface
would use five cameras that would more accurately detect natural movements and
gestures from the surface.

Dept. of ECE
2008-09
Surface Computing 5

3.1 Hardware Design:

The “Tub” model

By late 2004, the software development platform of Surface was well-established
and attention turned to the form factor. A number of different experimental prototypes
were built including “the tub” model, which was encased in a rounded plastic shell, a
desk-height model with a square top and cloth-covered sides, and even a bar-height
model that could be used while standing. After extensive testing and user research, the
final hardware design (seen today) was finalized in 2005. Also in 2005, Wilson and
Bathiche introduced the concept of surface computing in a paper for Gates’ twice-yearly
“Think Week,” a time Gates takes to evaluate new ideas and technologies for the
company.

Dept. of ECE
2008-09
Surface Computing 6

3.2 From Prototype to

Product:

“T1
Prototype”
The next phase of the development of Surface focused on continuing the journey
from concept to product. Although much of what would later ship as Surface was
determined, there was significant work to be done to develop a market-ready product that
could be scaled to mass production.

In early 2006, Pete Thompson joined the group as general manager, tasked with
driving end-to-end business and growing development and marketing. Under his
leadership, the group has grown to more than 100 employees. Today Surface has become
the market-ready product once only envisioned by the group, a 30-inch display in a table-
like form factor that’s easy for individuals or small groups to use collaboratively. The
sleek, translucent surface lets people engage with Surface using touch, natural hand
gestures and physical objects placed on the surface. Years in the making, Microsoft
Surface is now poised to transform the way people shop, dine, entertain and live. This is a
radically different user-interface experience than anything and it’s really a testament to
the innovation that comes from marrying brilliance and creativity.

Dept. of ECE
2008-09
Surface Computing 7

4. Key attributes of Surface Computing:

Surface computing features four key attributes:

• Direct interaction. Users can actually “grab” digital information with their
hands and interact with content through touch and gesture, without the use of a
mouse or keyboard.

• Multi-touch contact. Surface computing recognizes many points of contact

simultaneously, not just from one finger as with a typical touch screen, but up to
dozens and dozens of items at once.

• Multi-user experience. The horizontal form factor makes it easy for several
people to gather around surface computers together, providing a collaborative,
face- to- face computing experience.

• Object recognition. Users can place physical objects on the surface to trigger
different types of digital responses, including the transfer of digital content.

Dept. of ECE
2008-09
Surface Computing 8

5. Technology behind Surface Computing:

Microsoft Surface uses cameras to sense objects, hand gestures and touch. This
user input is then processed and displayed using rear projection. Specifically:

Microsoft Surface uses a rear projection system which displays an image onto the
underside of a thin diffuser.
Objects such as fingers are visible through the diffuser by series of infrared–sensitive
cameras, positioned underneath the display.
An image processing system processes the camera images to detect fingers, custom
tags and other objects such as paint brushes when touching the display.
The objects recognized with this system are reported to applications running in the
computer so that they can react to object shapes, 2D tags, movement and touch.

One of the key components of surface computing is a "multitouch" screen. It is an

idea that has been floating around the research community since the 1980s and is swiftly
becoming a hip new product interface — Apple's new iPhone has multitouch scrolling
and picture manipulation. Multitouch devices accept input from multiple fingers and
multiple users simultaneously, allowing for complex gestures, including grabbing,
stretching, swiveling and sliding virtual objects across the table. And the Surface has the
added advantage of a horizontal screen, so several people can gather around and use it
together. Its interface is the exact opposite of the personal computer: cooperative, hands-
on, and designed for public spaces.

Dept. of ECE
2008-09
Surface Computing 9

6. Microsoft Surface Overview:

Microsoft Surface turns an ordinary tabletop into a vibrant, interactive computing

experience. The product provides effortless interaction with digital content through
natural gestures, touch and physical objects. In Essence, it’s a surface that comes to life
for exploring, learning, sharing, creating, buying and much more. Currently available in
select in restaurants, hotels, retail establishments and public entertainment venues, this
experience will transform the way people shop, dine, entertain and live.
Surface is a 30 -inch display in a table -like form factor that’s easy for individuals
or small groups to interact with in a way that feels familiar, just like in the real world.
Surface can simultaneously recognize dozens and dozens of movements such as touch,
gestures and actual unique objects that have identification tags similar to bar codes.
Surface computing breaks down traditional barriers between people and technology,
changing the way people interact with all kinds of everyday content, from photos to maps
to menus. The intuitive user interface works without a traditional mouse or keyboard,
allowing people to interact with content and information by using their hands and natural
movements. Users are able to access information either on their own or collaboratively
with their friends and families, unlike any experience available today.

6.1 The Hardwa re:

Essentially, Microsoft Surface is a computer embedded in a medium-sized table,

with a large, flat display on top that is touch-sensitive. The software reacts to the touch of
any object, including human fingers, and can track the presence and movement of many
different objects at the same time. In addition to sensing touch, the Microsoft Surface unit
can detect objects that are labeled with small "domino" stickers, and in the future, it will
identify devices via radio-frequency identification (RFID) tags.

The demonstration unit I used was housed in an attractive glass table about three
feet high, with a solid base that hides a fairly standard computer equipped with an Intel
Core 2 Duo processor, an AMI BIOS, 2 GB of RAM, and Windows Vista. The team lead

Dept. of ECE
2008-09
Surface Computing 10

would not divulge which graphics card was inside, but they said that it was a moderately-
powerful graphics card from either AMD/ATI or NVIDIA.

(1) Screen: A diffuser turns the Surface's acrylic tabletop into a large horizontal
"multitouch" screen, capable of processing multiple inputs from multiple users. The
Surface can also recognize objects by their shapes or by reading coded "domino" tags.
(2) Infrared: Surface's "machine vision" operates in the near-infrared spectrum, using an
850-nanometer-wavelength LED light source aimed at the screen. When objects touch
the tabletop, the light reflects back and is picked up by multiple infrared cameras with
a net resolution of 1280 x 960.
(3) CPU: Surface uses many of the same components found in everyday desktop
computers — a Core 2 Duo processor, 2GB of RAM and a 256MB graphics card.
Wireless communication with devices on the surface is handled using WiFi and
Bluetooth antennas (future versions may incorporate RFID or Near Field
Co mmunications). The underlying operating system is a modified version of Microsoft
Vista.
(4) Projector: Microsoft's Surface uses the same DLP light engine found in many rear-
projection HDTVs. The footprint of the visible light screen, at 1024 x 768 pixels, is
actually smaller than the invisible overlapping infrared projection to allow for better
recognition at the edges of the screen.

Dept. of ECE
2008-09
Surface Computing 11

The display screen is a 4:3 rear-projected DLP display measuring 30 inches

diagonally. The screen resolution is a relatively modest 1024x768, but the touch detection
system had an effective resolution of 1280x960. Unlike the screen resolution, which for
the time being is constant, the touch resolution varies according to the size of the screen
used—it is designed to work at a resolution of 48 dots per inch. The top layer also works
as a diffuser, making the display clearly visible at any angle.

Unlike most touch screens, Surface does not use heat or pressure sensors to
indicate when someone has touched the screen. Instead, five tiny cameras take snapshots
of the surface many times a second, similar to how an optical mouse works, but on a larger
scale. This allows Surface to capture many simultaneous touches and makes it easier to
track movement, although the disadvantage is that the system cannot (at the moment)
sense pressure.

Five cameras mounted beneath the table read objects and touches on the acrylic
surface above, which is flooded with near-infrared light to make such touches easier to
pick out. The cameras can read a nearly infinite number of simultaneous touches and are
limited only by processing power. Right now, Surface is optimized for 52 touches, or
enough for four people to use all 10 fingers at once and still have 12 objects sitting on the
table.

The unit is rugged and designed to take all kinds of abuse. Senior director of
marketing Mark Bolger demonstrated this quite dramatically by slamming his hand onto
the top of the screen as hard as he could—it made a loud thump, but the unit itself didn't
move. The screen is also water resistant. At an earlier demonstration, a skeptical reporter
tested this by pouring his drink all over the device. Microsoft has designed the unit to put
up with this kind of punishment because it envisions Surface being used in environments
such as restaurants where hard impacts and spills are always on the menu.

The choice of 4:3 screen was, according to Nigel Keam, mostly a function of the
availability of light engines (projectors) when the project began. Testing and user

Dept. of ECE
2008-09
Surface Computing 12

feedback have shown that the 4:3 ratio works well, and the addition of a slight amount of
extra acrylic on each side leaves the table looking like it has normal dimensions.

Built-in wireless and Bluetooth round out the hardware capabilities of Surface. A
Bluetooth keyboard with a built-in trackpad is available to diagnose problems with the
unit, although for regular use it is not required.

6.2 System software:

Microsoft Surface works much like another Microsoft product, Media Center, in
that the main application runs on top of Windows and takes over the whole screen. Like
Media Center, it is designed to be difficult to exit the application without using a mouse
or keyboard. I asked if the Surface team considered allowing the user to drop into
Windows mode while retaining the touch functionality, but they felt that the product
worked better if it stayed in this mode.

The various demonstration programs are accessed from a main menu, which
scrolls left and right in an endless loop. The user moves the selection by swiping back
and forth and selects an application with a single tap. This works reasonably well and
feels quite natural. When an application is selected, a swirly purple ring appears in the
center of the screen to indicate that the program is loading.

There were eight different programs available: Water, Video Puzzle, Paint, Music,
Photos, Casino, a T-Mobile demonstration app, and Dining. Much of the software was
written using Microsoft's WPF (Windows Presentation Foundation), though the XNA
development toolkit, a framework originally created for writing PC and Xbox 360 games,
is also supported. XNA allows programmers to use managed code written in C# to
manipulate various DirectX features; managed code frees the programmer from worrying
about handling memory, allocating and discarding memory automatically. This approach
has allowed Microsoft and its partners to write impressive-looking demonstration
programs for Surface more quickly than would otherwise be possible.

Dept. of ECE
2008-09
Surface Computing 13

7. Features:

Multi-touch display. The Microsoft Surface display is capable of multi-touch

interaction, recognizing dozens and dozens of touches simultaneously, including
fingers, hands, gestures and objects.

Perceptive Pixel’s touch screens work via frustrated total internal reflection
Technology. The acrylic surface has infrared LEDs on the edges. When undisturbed, the
light passes along predictable paths, a process known as total internal reflection. When
one or more fingers touch the surface, the light diffuses at the contact points, changing
the internal-reflection pathways. A camera below the surface captures the diffusion and
sends the information to image-processing software, which translates it into a command.

Multitouch technology has been around since early research at the University of
Toronto in 1982. With multitouch devices, one or more users activate advanced functions
by touching a screen in more than one place at the same time. For example, a person
could expand or shrink images by pinching the edges of the display window with the
thumb and forefinger of one hand, explained Microsoft principal researcher Bill Buxton.
Users could also, while in contact with a point on a map, touch other controls to make the
system display information, such as nearby restaurants, about the area surrounding the

Dept. of ECE
2008-09
Surface Computing 14

indicated location. This is accomplished much as it has been in PCs for years. For
example, desktop users can press the Alt and Tab keys at the same time to toggle between
open windows. The OS translates the simultaneous keystrokes into a single command.
Industry observers say tabletop computers are likely to become a popular multitouch-
screen implementation. Because multiple users at different positions will work with
tabletop systems, the computers must be able to display material in different parts of the
screen and move controls around to keep them from blocking reoriented content. The
systems can determine users’ locations based on the positions from which they input
commands or data. The computers then orient their displays toward the tabletop edge
nearest to the user. Vendors are beginning to release commercial multitouch systems. For
example, Mitsubishi Electric Research Laboratories’ Diamond Touch table, which
includes a developer’s kit, can be used for small-group collaboration.

Horizontal orientation. The 30-inch display in a table-sized form factor allows

users to share, explore and create experiences together, enabling a truly
collaborative computing experience.

Dimensions. Microsoft Surface is 22 inches high, 21 inches deep and 42 inches

wide.

Materials.The Microsoft Surface tabletop is acrylic, and its interior frame is

powder-coated steel.

Dept. of ECE
2008-09
Surface Computing 15

8. Perceptive
Pixel:

Jeff Han demos one of Perceptive Pixel’s multitouch devices by calling up various Minority Report photos
from the Web on his virtual keyboard. With a flick of his finger, he severs Tom Cruise's head from his body
and sets it frenetically bobbing across the screen.

Computer scientists see technologies such as surface computing and multitouch as

the key to a new era of ubiquitous computing, where processing power is embedded in
almost every object and everything is interactive. Last year, New York University
professor Jeff Han launched a company called Perceptive Pixel, which builds six-figure-
plus custom multitouch drafting tables and enormous interactive wall displays for large
corporations and military situation rooms. "I firmly believe that in the near future, we
will have wallpaper displays in every hallway, in every desk. Every surface will be a
point of interaction with a computer," Han says, "and for that to happen, we really need
interfaces like this."

The display’s surface is a six-millimeter-thick piece of clear acrylic, with infrared

LEDs on the edges. Left undisturbed, the light passes along predictable paths within the
acrylic, a process known as total internal reflection. When objects such as fingers touch
the surface, the light diffuses at the contact point, causing the acrylic’s internal-reflection

Dept. of ECE
2008-09
Surface Computing 16

pathways to change. A camera below the surface captures the diffusion and sends the
information to image-processing software, which can read multiple touches
simultaneously and translate them into a command. The system sends information about
screen touches to applications via the lightweight Open Sound Control protocol, utilized
for network-based communication between computers and multimedia devices, and User
Datagram Protocol data transport technology. The applications then take the appropriate
actions. Perceptive Pixel, which has built a prototype that measures 36 _ 27 inches, is still
working on applications for its displays, Han noted. They could be used for collaborative
work on design-related and other projects, perhaps in place of interactive whiteboards, he
said.

Short-term success for a technology can be measured by how much attention a

product gathers when it is new. Long-term success is measured by how effectively that
product disappears into the everyday routine of life. Surface computing has enormous
potential to do both — it is a splashy new computer interface, surrounded by hype, but it
is also, quite literally, furniture. It is a technology in its infancy, where even the engineers
behind it can't predict its full impact; but the possibilities are everywhere, underhand and
underfoot — on every surface imaginable.

Dept. of ECE
2008-09
Surface Computing 17

9. Applications of Surface Computing:

9.1 Water

Water is used as an "attract mode" for the Surface desktop, and it is certainly
attractive. The default background picture is an image of smooth pebbles that appear
to sit beneath a thin layer of rippling water. By itself, the water moves as if it were being
disturbed by a light breeze, but it is when you touch the screen that it becomes more
interesting than just another screensaver.

Tapping anywhere on the surface causes larger ripples to spread out from the
point of contact. Many people can tap at the same time, making an effect similar to a
rainstorm. But by far the most fun is when you sweep your whole hand across and cause
waves to bounce back and forth. The physics of the water simulation is not perfect: the
ripples never get above a certain intensity, and there is no way to simulate diffraction.
However, the overall effect is strangely compelling and is certainly a good way to
introduce people to Surface.

Dept. of ECE
2008-09
Surface Computing 18

One interesting feature of Water is that if you take any object (the team used a
regular stove dial) and stick an identification sticker on the bottom, the program will
switch background pictures whenever you turn the dial.

9.2 Video Puzzle

Video Puzzle showcases the power of the little identification tags mentioned above.
The tags consist of a pattern of variously-sized dots; Keam mentioned that the dots
currently represent an 8-bit code (256 permutations) but that 128-bit tags were in the
works. The neat thing about the tags is that they can be very nearly transparent and the
system will still pick them up. Not only can the tags transmit numerical information, but
the geometrical arrangement of the dots means that Surface can also tell, to a high degree
of accuracy, how much the tag (and therefore the object) has rotated.

In Video Puzzle, these virtually invisible tags are placed upon small squares of
glass. When the pieces of glass are put on the table, the screen starts playing video clips
underneath each one. Because the video moves whenever you move the squares, it
creates the illusion that the glass itself is displaying the video, which looks very futuristic.
As you move the squares around, you quickly realize that the video clips are all pieces of
a larger video. Flipping the glass squares over inverts the video playing underneath,
making completing the puzzle even more of a challenge.

When you complete the puzzle correctly, the system senses the achievement,
congratulates you, and shows you the time taken to finish. According to Mark Bolger, the
current record for finishing when the pieces are fully randomized is 1 minute and 53
seconds. On my first attempt, I finished in just over 2 minutes, but the squares were all
right side up to begin with (Microsoft is nice to journalists, it seems).

Dept. of ECE
2008-09
Surface Computing 19

9.3 Pain
t

Paint programs have been a natural demonstration application for new platforms
ever since MacPaint graced the first Macintosh back in 1984. Surface’s paint program is
even lighter on features than MacPaint was, but the natural user interface makes up for
this deficiency.

There are three draw modes that can be toggled by touching an icon on the bottom
of the toolbar: brush, paint, and reveal, the last of which is kind of a negative brush that
shows a background bitmap underneath. The brush mode is a bit spotty and tends to skip,
but the paint mode is smooth and fun. You can draw using one finger, all your fingers at
once (good for drawing hair), the palm of your hand, or using any natural object such as a
regular paintbrush. Using the program is like having a flashback to finger painting back
in kindergarten (minus the mess), and certainly children will have tons of fun with this
kind of application.

That said, having this great touch interface absolutely cries out for a more full-
featured program, something that can mix colors (like Microsoft's own paint program that
comes with the Tablet PC version of Windows) and play around with textures and natural
materials. I immediately thought of Fractal Design Painter and how much fun it would be
with this interface. Of course, real digital artists have been using advanced pressure-

Dept. of ECE
2008-09
Surface Computing 20

sensitive graphics tablets for years, and Surface is not aimed at replacing this kind of
workflow. Still, a more full-featured Paint program would be nice to have, and Keam
mentioned that the team is still deciding whether or not to add features to Paint or
instead take an existing paint program and rework it for Surface.

9.4 Music

The Music application works like a virtual jukebox, displaying music arranged by
album and allowing the user to flip over albums, select songs, and drag them to the "Now
Playing" section. The album browser works a bit like Apple’s Cover Flow, although
many albums are visible at once without scrolling.

In addition to playing music that is already stored on the unit's hard drive, Music
can also transfer songs from portable music players. Mark Bolger demonstrated this by
placing two Zunes on top of the Surface and using the wireless connection to drag and
drop songs between the units, the song list, and the Now Playing section. I mentioned to
the team that this was the first time I had ever seen even one Zune "in the wild," and they
joked that Microsoft headquarters didn’t really count as being in the wild. Bolger noted
that sharing songs in this manner would be "subject to DRM restrictions, of course."

Dept. of ECE
2008-09
Surface Computing 21

9.5 Photos

Sharing photos is a much more unrestricted activity, thanks to the fact that the
consumer is also the creator of the content, and the photo album application reflected this
freedom. By simply placing a Bluetooth-equipped digital camera on the tabletop, Surface
was able to import the photos and place them in a pile on the screen, which Bolger
verified by taking a picture of Cindy, my Microsoft PR contact who was sitting in the
next chair. Most of the other photos were pictures of Microsoft employees' children;
Bolger joked that only the cutest kids were allowed to be put in the demonstration.

Photos are arranged into albums that look like piles. Tapping the pile once spreads
it around the screen and from there you can drag, rotate, and resize the images to your
heart’s content. Since Surface can detect many touches at the same time, multiple people
can sort and resize pictures, which could potentially turn a tedious job into a fun family
affair. The program can also apparently sort photos into stacks by using metadata tags,
although I did not see this feature demonstrated.

Not only pictures but full-motion videos can be viewed in this way; tapping the
video once starts the playback, and it can be smoothly resized and rotated while it plays.

Dept. of ECE
2008-09
Surface Computing 22

9.6 Casino

The Casino application was developed in cooperation with Harrah's of Las Vegas
and is a good example of how Surface can be used in a hospitality environment. The
background image is a giant map of the hotel and casino, with all the attractions marked
for further inspection. Hotel customers can place their card anywhere on the screen and
reserve tickets to any of these shows. The background map can be easily scrolled with a
brush of the hand, and zoomed in and out by performing the two-finger pinch.

9.7 Dining

The application allows diners to preview the entire menu by choosing a category
(drinks, appetizers, main courses, and so forth) and then scrolling left and right through
the available options. Items can be dragged into a central "ordering area" and when
everyone is satisfied with their choices, a single tap on the Order button sends the list out
to the waiter. This could potentially save service people huge chunks of time and would
be very useful for busy restaurants. The software can display the daily specials, and for
regular customers with their own identification cards, it could display a list of "favorites"
to make ordering even easier. Combine this with entertainment activities for the kids
(perhaps Paint?) and you can see how many restaurants could view this as a compelling
application.

Dept. of ECE
2008-09
Surface Computing 23

10. Surface Computing - In Future:

Although surface computing is a new experience for consumers, over time

Microsoft believes there will be a whole range of surface computing devices and the
technology will become pervasive in people’s lives in a variety of environments.

As form factors continue to evolve, surface computing will be in any number of

environments— schools, businesses, homes — and in any number of form factors — part
of the countertop, the wall or the refrigerator.

Dept. of ECE
2008-09
Surface Computing 24

11. Conclusions:

Some people will look at Surface and claim that it does nothing that hasn't been
tried before: computers with touch screens have been around for years and have already
found niches in ATMs, ticket ordering machines, and restaurant point-of-sale devices.

This view largely misses the point of the product. Like most projects, Surface
takes existing technology and presents it in a new way. It isn't simply a touch screen, but
more of a touch-grab-move-slide-resize-and-place-objects-on-top-of-screen, and this
opens up new possibilities that weren't there before.

Playing with the unit felt a bit like being in the movie Minority Report (in a good
way), but it also felt like a more natural and enjoyable method of doing certain computing
tasks. Sharing and looking at family photos, for example, is more fun on Surface than on
any other device. The retail applications, particularly the dining application, show how
businesses could use the technology to really stand out from competitors, though one
wonders how diners will react when their table locks up and needs a reboot.

Many people who viewed the early Xerox PARC demonstrations of the GUI came
out of that experience knowing that every computer would work that way someday, and
they were right. Playing with Surface, one gets the sense that although not every
computer will work like this someday, many of them will. More importantly, computers
running Surface-like software will end up in places that never had computers before, and
the potential applications are exciting. Imagine a multiplayer real-time strategy game
where you and another human opponent can move units around as quickly as you can
point to them. Or perhaps an educational environment, where university students could
assemble and disassemble anything from molecules to skyscrapers quickly and easily.

Dept. of ECE
2008-09
Surface Computing 25

12. BIBLIOGRAPHY:

1. Apple iPhone Multi-touch. http://www.apple.com/iphone/

2. www.scribd.com
3. www.whereisdoc.com
4. www.docjax.com

5. Microsoft Surface, http://www.surface.com

6. Perceptive Pixel, http://www.perceptivepixel.com

Dept. of ECE
2008-09