The Remediation of Paper

Creating Affinities between Paper and

Cross-Media Information Spaces
Beat Signer
Department of Computer Science
Vrije Universiteit Brussel

Cross-Media Information Spaces, April 22, 2009

The Paperless Office

Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

The Myth of the Paperless Office

Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

Bridging the Paper-Digital Divide

Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

EdFest Project

Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

Digital Pen and Paper
▪ Anoto offers
▪ camera technology
▪ pattern license
(virtual paper space
of 60 million km2)
▪ Pen manufacturers
▪ Nokia
▪ Maxell
▪ Adapx
▪ Livescribe
Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be
EdFest Demo

Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

PaperPoint Presentation Tool
▪ Mobile presentation tool
▪ Non-linear presentations
▪ Real-time annotations
▪ Digital whiteboard
▪ Multi-pen support
▪ brainstorming

Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

Applications
▪ Enhanced Reading ▪ Enhanced Writing
▪ interactive paper maps ▪ photo album
▪ interactive festival brochures ▪ researcher's notebook
▪ educational materials ▪ mammography
▪ auction brochures ▪ query by sketching
▪ scientific publications ▪ …
▪ …
▪ Artists' Installations
▪ Paper-based Interfaces ▪ Lost Cosmonaut
▪ PaperPoint ▪ Generosa Enterprise
▪ document proof-editing
▪ digital libraries ▪ Interactive Tabletops
▪ video analysis tool ▪ photo browser
▪ educational games ▪ collaborative sketch tool

Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

Research
Link Metamodel (RSL) Tools

Cross-Media Server / Publishing Applications

Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

Cross-Media Server (iServer)

Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

Active Page Areas and Layers

image

video clip

Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

RSL Metamodel and iServer

Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

 Structure

Links

Users

Layers
Plug-ins Active
Content

Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

iServer Resource Plug-ins
Medium Resource Selector

paper document page shape

web page XHTML document XPointer

movie mpeg file, avi file etc. time span

movie mpeg file, avi file etc. shape

sound mp3 file, wav file etc. time span

image gif file, jpeg file etc. shape

physical object RFID space RFID tag

... ... ...

Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

Active Components
▪ Execute code (services) on the client and server side

AC Manager
Active Component
ACS Manager A

Active Component Active Component

Stub B B
Request Sender
Client Device

Active Component
AC Names HTTP
C

iServer AC Names

Client Server

Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

Active Components …
Button Image

Slider Sound

Slider2D Movie

DateSelector SQLClient

Browser XCMRequest

MapSelector ClientCtrlRequest

Rating PaperPointControl

Capture PaperPointAnnot

CaptureAndSpeak ...

Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

Cross-Media Authoring and Publishing
▪ Manual link authoring
▪ Content-driven publishing
▪ e.g. EdFest brochure

▪ Automatic link authoring

▪ e.g. Print-n-Link

▪ Collaborative authoring

Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

Manual Authoring Tool

Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

 Cross-Media Publishing Framework
iServer
cross-media link server

XML link
XCM iPublish definition
content publishing publishing framework
data

publish • paper link data

Structure Style • active areas on paper
• layers
• digital resources
import • user model
PDF

XML data XSL CSS

of festival
venues
and events Information-centric approach
Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be
EdFest Architecture Overview
Context XCM
Engine content publishing
platform

Context DB Metadata DB Appln DB

Text-to-Speech
Engine ICR Software
Client handwriting
Controller recognition

Active
iPaper Client iServer/iPaper
cross-media
Components
link server

Link DB

Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

Print-n-Link
▪ Citations build the basis
for a scientific web of
publications

▪ Use digital services to

search / retrieve cited
publications

▪ Automatic authoring

Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

Collaborative Authoring
web pages
consumers /
publishers
publisher

videos
books

presentation slides

Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

Paper-Digital Proof-Editing (PaperProof)

Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

iGesture Framework
▪ Gesture recognition tool
▪ create/test gesture sets
▪ create/test algorithms

▪ Device independent
▪ digital pen, tablet PC,
mouse, Wii remote, …

▪ Open Source
▪ initial release in Feb 2007
http://www.igesture.org
▪ >4500 downloads

Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

Rough Guide

Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

Interactive Table (iTable)

Pen-Based Google Earth Browser Pen-Based Drawing Tool

Photo Browser Photo Browser (Slider)

Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

Interactive Table (iTable) …

Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

Pulse Smartpen
▪ Record and replay
synchronised ink & audio
▪ 1 GB memory
▪ 16'000 pages
▪ 200 hours voice recording
▪ 96x18 OLED Display
▪ Open application platform

Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

Ongoing and Future Work
▪ Cross-Media Information Spaces and
Architectures (CISA) research team
▪ cross-media lab space ("meeting room of the future")
▪ General and extensible representations for
cross-media information spaces
▪ interlinked structural components rather than
document-centric representation of information
▪ data no longer "owned" by applications
▪ replacement of traditional hierarchical file systems

Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

CISA Research
▪ New forms of cross-media publishing
methodologies and tools
▪ global publishing infrastructures and architectures
▪ personal cross-media information management
▪ information sharing and collaborative authoring
▪ cross-media annotations
▪ Innovative multimodal user interfaces (digital
and physical) to cross-media information spaces
▪ includes further research on interactive paper
Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be
Conclusions
▪ Interactive paper framework (iPaper)
▪ information-centric approach supports different
technologies, functionalities and applications
▪ rapid prototyping (reusable active components)
▪ separation of authoring and programming

▪ Extensible platform for cross-media information

management (iServer) based on RSL metamodel
▪ Cross-media authoring and publishing tools
▪ manual, content-driven, automatic and collaborative
Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be
Acknowledgements
▪ GlobIS group @ ETH Zurich
▪ in particular Prof. Moira C. Norrie, Nadir Weibel and
Adriana Ispas
▪ Paper++ and PaperWorks members
▪ ... and many other collaborators in individual
projects

Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be

Further Information
▪ http://vub.academia.edu/BeatSigner
▪ Fundamental Concepts for Interactive Paper
and Cross-Media Information Spaces, Beat
Signer, ISBN 978-3-8370-2713-6, May 2008

Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be