Augmented Reality

Günter Alce, Ph.D. candidate

: gunterAlce

1
 Günter Alce

• Industrial Ph.D. - AR Visor interaction

• Sony Mobile Communications
• Department of Design sciences

• What should the AR Visor offer the user in different

contexts?

• How to combine different interaction techniques, such

as gestures, gaze and speech?

2
 Overview
• Introduction of AR
• Evolution of AR
• Some applications areas
• AR technologies
• My research
• Summary
• HITLabNZ

3
 What is augmented reality?

4
5
6
7
 Definition

Augmented reality is a technology which:

1. Combines real and virtual

2. Interactive in real time
3. Registered in 3D

Azuma (1997)

8
 The evolution of
Augmented Reality hardware

9
 Steve Mann

10
 The evolution of AR hardware

• Generation “Kit bag”

11
 The evolution of AR hardware

• Generation “Hand bag”

12
13
 Keyhole AR

• Limited view
• Awkward interaction
• Relying on a camera sensor
• Limited use
• Ergonomic issues

14
 The evolution of AR hardware

• Generation “No bag”

15
“…uses a holographic waveguide…
we have developed in partnership with Nokia”
“They are scheduled to be delivered in 2013… between 350 to 600 dollars”
“We want these to be a consumer product”
“It connects to any mobile device through a HDMI cable”
Vuzix at CES 2012
17
 Upcoming glasses
•Olympus MEG4.0
•Project Glass
•Vuzix M100

18
 Project glass (2012)

• Video clip

19
 What could it be used for?

20
 Some application areas
• Marketing

• Entertainment

• Education

• Assistance

• Medical Applications

21
 Marketing

22
 Marketing

23
 Marketing (Kinect fitting room)

24
 Entertainment

ARDefender
Nerdherder

Danger copter
AR Blitz
25
 Entertainment

26
 Entertainment (Experience for tourists)

27
 Entertainment (Experience for tourists)

HistoriskaSkyltar
Platsbaserad historia i ögonblicket

28
 Education

Iulian Radu

29
 Assistance

30
 Assistance

ionRoad

31
 Medicine

M. Billinghurst
32
 Medicine

33
 Diminished Reality

Diminished reality

34
 AR Technologies

35
 AR Technologies

• AR Devices

• AR Interaction

36
 AR Devices

• Displays

• Input devices

• Tracking

37
 AR Devices

Displays

38
 Displays

Video see-through Optical see-through

Azuma 1997

39
 Video see-through

• Better consistency between real and virtual views

• Better handling of occlusion problems

40
 Optical see-through

• Natural, instantaneous view

• Zero parallax

41
 Optical see-through

42
 Projection-based displays

43
 OmniTouch

44
 AR Devices

• Displays

• Input devices

• Tracking

45
 Input devices
• Gloves

• Wristband (ReachMedia)

• Phone as pointing device

• Gestures

• Speech

• Gaze interaction

46
 AR Devices

• Displays

• Input devices

• Tracking

47
 Tracking
• The act of aligning the virtual content with the real
world view

48
 Tracking
• Sensor-based

• Vision-based

• Hybrid

49
 Sensor-based tracking
Technology Range (m) Precision (m)
• Accelerometers 1 0,1
• Inertial 1 0,01
• Magnetic 1 0,01
• WiFi 100 1
• RFID: active 20-100 0,5
• RFID: passive 0,05-5 0,5
• GPS ∞ 5

Table from Handbook of AR - An Overview

50
 Micro Electro-Mechanical Systems (MEMS)

51
 Tracking
• Sensor-based

• Vision-based

• Hybrid

52
 Vision-based tracking

• Marker-based

• Markerless
• Feature-based
• SLAM
• PTAM

53
 Marker-based tracking

54
 Vision-based tracking

• Marker-based

• Markerless
• Feature-based
• SLAM
• PTAM

55
 Feature-based tracking
Detecting feature points

56
 SLAM and PTAM
Simultaneous Localization and Mapping (SLAM)

Parallel Tracking and Mapping (PTAM)

Video clip

57
 Tracking
• Sensor-based

• Vision-based

• Hybrid

58
 Hybrid

Camera
Accelerometer, gyroscope & compass

59
 AR Technologies

• AR Devices

• AR Interaction

60
 Interaction

ISMAR 2011 papers and posters

User interfaces, interaction, human

factors, ergonomics, perception:
≈14%

ISMAR 2011
61
 Interaction styles

• Tangible AR

• Collaborative AR

• Hybrid AR

• Multimodal AR

62
 Tangible AR
• The user can manipulate the virtual information
through physical objects

63
 Collaborative AR

64
 Hybrid AR

65
 Multimodal AR

• Gestures

• Speech

• Touch

• Gaze

66
 Where are we today?

67
 ” Perceptually correct augmentation
remains a crucial challenge.

Ernst,, Kruijff, Swan, J. E., & Steve,, Feiner (2010). Perceptual Issues in
Augmented Reality Revisited. IEEE International Symposium on Mixed and
Augmented Reality (ISMAR 2010). Seoul, Korea.

68
 Vergence-accomodation conflict
Eyes focusing at a distance that differs from the
point of visual convergence

69
 Definition

Augmented reality is a technology which:

1. Combines real and virtual

2. Interactive in real time
3. Registered in 3D

Azuma (1997)

70
 Research

• What should the AR Visor offer the user in

different contexts?

• How to combine different interaction techniques,

such as gestures, gaze and speech?

71
 Attention resources

72
 What about SONY?

73
 AR Visor concept (2011)

• Video clip

74
 Wizard of Oz

75
 Simulate navigation

Navigation

76
 Simulate face recognition

Face recognition

77
 VENTURI – a three year EU project about AR

• A consolidated prototype at the end of

each year

• Evaluated through use cases

• Gaming
• Personal assistant
• Tourism

78
 Tools - AR SDK

1. Metaio SDK, http://www.metaio.com/

2. Vuforia, https://developer.qualcomm.com/mobile-
development/mobiletechnologies/augmented-reality
3. HITLabNZ, http://hitlabnz.org/index.php/products/mobile-ar-framework
4. AndAR - Android Augmented Reality, http://code.google.com/p/andar/

79
 Summary of AR

• Mixing real and virtual

• Interaction

• Display

• Tracking

80
 Master thesis

HITLabNZ, http://hitlabnz.org/

81
 Thanks for listening!

Questions?

82