Professional Documents
Culture Documents
223 Expanding The Sensory Experience With Core Haptics
223 Expanding The Sensory Experience With Core Haptics
223 Expanding The Sensory Experience With Core Haptics
© 2019 Apple Inc. All rights reserved. Redistribution or public display not permitted without written permission from Apple.
#WWDC19
© 2019 Apple Inc. All rights reserved. Redistribution or public display not permitted without written permission from Apple.
#WWDC19
© 2019 Apple Inc. All rights reserved. Redistribution or public display not permitted without written permission from Apple.
•
Techniques
•
Techniques
Can you feel it?
Can you feel it?
Core Haptics
Haptic Design 101
Haptic Design 101
Haptic Design 101
Haptic Design 101
Continuous
Haptic Design 101
Haptic Design 101
Haptic Design 101
Transient
Haptic Design 101
Transient
Haptic Design 101
Transient
Haptic Design 101
Transient
Haptic Design 101
Transient
Haptic Design 101
Round, Soft
Transient
Haptic Design 101
Transient
Haptic Design 101
Haptic Intensity
Haptic Design 101
Haptic Intensity
Haptic Sharpness
Haptic Design 101
•
Techniques
Three Guiding Principles
Three Guiding Principles
Causality
Three Guiding Principles
Causality
Harmony
Three Guiding Principles
Causality
Harmony
Utility
•
Causality
•
Causality
Cause
•
Effect
•
•
•
Cause •
Effect
•
•
•
Cause •
Effect •
Sound of impact
Feel of impact
•
•
Cause and Effect
Cause and Effect
Environment
Needs movie of
Apple Pay confirmation
animation + sound
Needs movie of
Apple Pay confirmation
animation + sound
Sound
Haptic
Sound
Haptic
Sound
Haptic
•
Harmony
•
Harmony
Real World
•
Real World
•
Visual
•
Audio
•
Haptic
•
•
Real World •
Digital World
•
Visual
•
Audio
•
Haptic
•
•
Real World •
Digital World
•
Visual
•
Audio
•
Haptic
•
•
Real World •
Digital World
•
Visual •
Visual
•
Audio
•
Haptic
•
•
Real World •
Digital World
•
Visual •
Visual
•
Audio
•
Haptic
•
•
Real World •
Digital World
•
Visual •
Visual
•
Audio Audio
•
Haptic
•
•
Real World •
Digital World
•
Visual •
Visual
•
Audio Audio
•
Haptic
•
•
Real World •
Digital World
•
Visual Visual
•
Audio Audio
•
Haptic •
Haptic
•
•
Real World •
Digital World
•
Visual Visual
•
Audio Audio
•
Haptic •
Haptic
•
•
Real World •
Digital World
•
Visual Visual •
Audio Audio •
Haptic Haptic
•
Synchronization
•
•
•
Real World •
Digital World
•
Visual Visual •
Audio Audio •
Haptic Haptic
•
Synchronization
•
•
•
Harmony
Interactions, Visuals, Audio and Haptics
•
•
Harmony
Interactions, Visuals, Audio and Haptics
•
•
Harmony
Interactions, Visuals, Audio and Haptics
•
•
Harmony
Interactions, Visuals, Audio and Haptics
•
•
Harmony
Interactions, Visuals, Audio and Haptics
•
•
Harmony
Interactions, Visuals, Audio and Haptics
•
•
Harmony
Interactions, Visuals, Audio and Haptics
•
•
Harmony
Interactions, Visuals, Audio and Haptics
•
•
Harmony
Interactions, Visuals, Audio and Haptics
•
•
Harmony
Interactions, Visuals, Audio and Haptics
•
•
Harmony
Interactions, Visuals, Audio and Haptics
•
•
Harmony
Interactions, Visuals, Audio and Haptics
•
Digital Crown Haptic
Digital Crown Haptic
Check + update video
Utility
•
Utility
“Add audio and haptics that provide clear value to your app experience”
•
Utility
Moderation
[Hugo]
Utility
Moderation
Focus
[Hugo]
Utility
Moderation
Focus
Causality
Guiding Principles
Causality
Harmony
Guiding Principles
Causality
Harmony
Utility
•
Techniques
Core Haptics Primitives
Core Haptics Primitives
Transient
Core Haptics Primitives
Transient Continuous
Core Haptics Transient
Core Haptics Transient
Core Haptics Transient
1.0
Haptic Intensity
0.0
Round, Soft
1.0
Haptic Intensity
0.0
1.0
Haptic Intensity
0.0
1.0
Haptic Intensity
0.0
1.0
Haptic Intensity
0.0
Haptic
Sound
Haptic
Sound
Haptic
Sound
Haptic
Anticipation
Sound
Haptic
Anticipation
Sound
Haptic
Number of Events
4 Events
Number of Events
4 Events
Haptic Priming
Sound
Haptic
Haptic Priming
Sound
Haptic
Haptic Priming
Sound
Haptic
Haptic Priming
Sound
Haptic
Contrast—Left
Sound
Haptic
Contrast—Left
Sound
Haptic
Contrast—Left
Sound
Haptic
Contrast—Left
Sound
Haptic
Contrast—Right
Sound
Haptic
Contrast—Right
Sound
Haptic
Contrast—Right
Sound
Haptic
Contrast—Right
Sound
Haptic
Contrast—Right
Sound
Haptic
Contrast—Right
Sound
Haptic
A Few More Thoughts
Collaborate
Experience It
Experiment
More Information
developer.apple.com/wwdc19/223
© 2019 Apple Inc. All rights reserved. Redistribution or public display not permitted without written permission from Apple.
•
Expressing content
•
Dynamic parameters
•
Expressing content
•
Dynamic parameters
•
Expressing content
•
Dynamic parameters
•
Expressing content
•
Dynamic parameters
•
Expressing content
•
Dynamic parameters
•
App
Core Haptics
App
AVFoundation UIKit
UIFeedbackGenerator
AVAudioSession
AVAudioPlayer
AVAudioEngine
CoreMIDI
Core Haptics
AudioToolbox
SystemSound
AudioUnits
App
AVFoundation UIKit
UIFeedbackGenerator
AVAudioSession
AVAudioPlayer
AVAudioEngine
CoreMIDI
Core Haptics
AudioToolbox
SystemSound
AudioUnits
UIFeedbackGenerator
Impact, Selection, Notification
Visceral feeling
Gaming Applications
Visceral feeling
Increase immersion
Core Haptics and Augmented Reality
Increase immersion
Increase immersion
Expressing Content
Classes for Content
CHHapticPattern
CHHapticEvent
CHHapticParameter
Classes for Content Classes for Playback
CHHapticPattern CHHapticEngine
CHHapticEvent
CHHapticParameter owns and vends
CHHapticPatternPlayer(s)
CHHapticAdvancedPatternPlayer(s)
Events, Parameters, and Patterns
CHHapticEvent
Time
Type
[EventParameters]
0.0 Time
Events, Parameters, and Patterns
CHHapticEvent
Time
Type
[EventParameters]
CHHapticEvent CHHapticEvent
Time Time
Type Type
[EventParameters] [EventParameters]
0.0 Time
Events, Parameters, and Patterns
CHHapticPattern CHHapticEvent
Time
Type
[EventParameters]
0.0 Time
Types of Events
HapticTransient
Think “striking”;
momentary;
instantaneous
Types of Events
Think “bowing”
Think “striking”;
momentary;
Can be background texture
instantaneous
Richer set of knobs
Types of Events
Think “bowing”
Think “striking”;
Developer-provided
momentary;
Can be background texture waveform
instantaneous
Richer set of knobs
Our First EventParameter
HapticIntensity / AudioVolume
0.0 1.0
Our First EventParameter
HapticIntensity / AudioVolume
0.0 1.0
HapticSharpness
An abstraction layer for physical dimensions
0.0 1.0
HapticSharpness
An abstraction layer for physical dimensions
Round, organic
0.0 1.0
HapticSharpness
An abstraction layer for physical dimensions
0.0 1.0
HapticSharpness
An abstraction layer for physical dimensions
0.0 1.0
HapticSharpness
An abstraction layer for physical dimensions
0.0 1.0
Flashlight button
HapticSharpness
An abstraction layer for physical dimensions
0.0 1.0
CHHapticPattern NSDictionary
Playing a Haptic Pattern: Recommended Flow
1. Create haptic content
CHHapticEngine
Playing a Haptic Pattern: Recommended Flow
1. Create haptic content
CHHapticPatternPlayer
CHHapticEngine
Playing a Haptic Pattern: Recommended Flow
1. Create haptic content
CHHapticEngine (Running)
Playing a Haptic Pattern: Recommended Flow
1. Create haptic content
CHHapticEngine (Running)
Playing a Haptic Pattern: Recommended Flow
1. Create haptic content
Done?
Playing a Haptic Pattern: Recommended Flow
1. Create haptic content
Demo
Using the Core Haptics API
// Using the API
import UIKit
import CoreHaptics
import CoreMotion
// Using the API
import UIKit
import CoreHaptics
import CoreMotion
// Using the API
import UIKit
import CoreHaptics
import CoreMotion
import UIKit
import CoreHaptics
import CoreMotion
Dynamic Parameters
Dynamic Parameters
{
"Version": 1.0
}
// An Example of a Simple AHAP File
{
"Version": 1.0
}
// An Example of a Simple AHAP File
{
"Version": 1.0,
“Pattern": [ ]
}
// An Example of a Simple AHAP File
{
"Version": 1.0,
“Pattern": [ ]
}
// Adding our first Event
{
"Version": 1.0,
“Pattern": [
{
“Event": {
"Time": 0.0,
"EventType": “HapticTransient"
}
},
]
}
// Adding our first Event
{
"Version": 1.0,
“Pattern": [
{
“Event": {
"Time": 0.0,
"EventType": “HapticTransient"
}
},
]
}
// Adding an optional EventParameter array
{
"Version": 1.0,
“Pattern": [
{
“Event": {
"Time": 0.0,
"EventType": “HapticTransient",
“EventParameters": []
}
},
]
}
// Adding an optional EventParameter array
{
"Version": 1.0,
“Pattern": [
{
“Event": {
"Time": 0.0,
"EventType": “HapticTransient",
“EventParameters": []
}
},
]
}
// Adding EventParameters for Intensity and Sharpness
{
"Version": 1.0,
“Pattern": [
{
“Event": {
"Time": 0.0,
"EventType": “HapticTransient",
“EventParameters": [
{ "ParameterID": "HapticIntensity", "ParameterValue": 0.8 },
{ "ParameterID": "HapticSharpness", "ParameterValue": 0.4 }
]
}
}
]
}
// Adding EventParameters for Intensity and Sharpness
{
"Version": 1.0,
“Pattern": [
{
“Event": {
"Time": 0.0,
"EventType": “HapticTransient",
“EventParameters": [
{ "ParameterID": "HapticIntensity", "ParameterValue": 0.8 },
{ "ParameterID": "HapticSharpness", "ParameterValue": 0.4 }
]
}
}
]
}
// A second event
“Pattern": [
{
“Event": { ... }
},
{
“Event": {
"Time": 0.5,
"EventType": “HapticContinuous”,
"EventDuration": 0.25,
“EventParameters": [
{ "ParameterID": "HapticIntensity", "ParameterValue": 0.8 },
{ "ParameterID": "HapticSharpness", "ParameterValue": 0.4 }
]
}
}
]
// A second event
“Pattern": [
{
“Event": { ... }
},
{
“Event": {
"Time": 0.5,
"EventType": “HapticContinuous”,
"EventDuration": 0.25,
“EventParameters": [
{ "ParameterID": "HapticIntensity", "ParameterValue": 0.8 },
{ "ParameterID": "HapticSharpness", "ParameterValue": 0.4 }
]
}
}
]
// EventDuration key is required for AudioContinuous, HapticContinuous
“Pattern": [
{
“Event": { ... }
},
{
“Event": {
"Time": 0.5,
"EventType": “HapticContinuous”,
"EventDuration": 0.25,
“EventParameters": [
{ "ParameterID": "HapticIntensity", "ParameterValue": 0.8 },
{ "ParameterID": "HapticSharpness", "ParameterValue": 0.4 }
]
}
}
]
// EventDuration key is required for AudioContinuous, HapticContinuous
“Pattern": [
{
“Event": { ... }
},
{
“Event": {
"Time": 0.5,
"EventType": “HapticContinuous”,
"EventDuration": 0.25,
“EventParameters": [
{ "ParameterID": "HapticIntensity", "ParameterValue": 0.8 },
{ "ParameterID": "HapticSharpness", "ParameterValue": 0.4 }
]
}
}
]
Building an AHAP File
AHAP Structure Summary
“Version”
1.0
“Pattern”
0.0 3.0
“EventDuration”
“1.0” “ParameterCurveControlPoints” “ParameterCurveControlPoints”
0.5
“EventParameters”
“Time”
0
“ParameterID”
“HapticIntensity” “ParameterValue”
1.0
“ParameterValue”
0.8
HapticSampler
Sample code
•
More to Discover
•Custom audio + synthesized audio •Use Parameter Curves to achieve the same effect
Audio Events and •Pitch; Pan; Brightness as a sequence of DynamicParameters
Modulation
Parameters •Attack, Decay, Release •Statically defined as part of the Pattern, or created
•Haptics-only mode during playback
•Routing and interruption behavior follows •Error Handling for Pattern Parsing
AudioSession AudioSession Category Error Handling •Error Handling and Recovery during Playback
Integration
•New API to allow haptics during recording •Query HW Capabilities
Performant API
Summary
Performant API
Performant API