Professional Documents
Culture Documents
407 Editing Media With Av Foundation
407 Editing Media With Av Foundation
407 Editing Media With Av Foundation
Eric Lee
iPhone Engineering
http://developer.apple.com/wwdc/attendee/
Technology Framework
MediaPlayer
UIKit
AVFoundation
CoreAudio
CoreMedia
CoreAnimation
Trim UI Thumbnails
Create an image for a time Trim a movie to a time range Cutting together multiple clips Audio mixing Video transitions Incorporating Core Animation in movies
Fundamentals
MediaPlayer
UIKit
AVFoundation
CoreAudio
CoreMedia
CoreAnimation
CMTime
Struct type for rational time
CMTime t = CMTimeMake( time value, time scale ); kCMTimeZero, kCMTimeInvalid x = CMTimeAdd( y, z ); if( CMTIME_COMPARE_INLINE( t1, <=, t2 ) ) { ... } CMTimeRange r = CMTimeRangeMake( start time, duration ); CMTimeMapping m = CMTimeMappingMake( source range, target range );
Moving Up
MediaPlayer
UIKit
AVFoundation
CoreAudio
CoreMedia
CoreAnimation
10
AVAssetTracks
11
Create an image for a time Trim a movie to a time range Cutting together multiple clips Audio mixing Video transitions Incorporating Core Animation in movies
12
Demo
Custom playback UI
AVPlayerDemo
13
AVAssetImageGenerator *imageGenerator = [AVAssetImageGenerator assetImageGeneratorWithAsset:myAsset]; [imageGenerator generateCGImagesAsynchronouslyForTimes:timeArray completionHandler:handlerBlock]; // need to retain imageGenerator until you get the images
14
15
Create an image for a time Trim a movie to a time range Cutting together multiple clips Audio mixing Video transitions Incorporating Core Animation in movies
16
Presets for different sizes, bitrates, etc. Optionally set timeRange to trim Optionally add metadata
AVAssetExportSession *exportSession = [[AVAssetExportSession alloc] initWithAsset:asset presetName:AVAssetExportPresetMediumQuality]; exportSession.outputURL = ...; exportSession.outputFileType = AVFileTypeQuickTimeMovie; exportSession.timeRange = CMTimeRangeMake(startTime, duration); exportSession.metadata = ...; [exportSession exportAsynchronouslyWithCompletionHandler:handlerBlock];
17
18
AVAssetExportSession will not overwrite files AVAssetExportSession will not write files outside of your sandbox
19
Other apps that start playback will interrupt a background export Even in the foreground, an incoming phone call will interrupt export
20
Create an image for a time Trim a movie to a time range Cutting together multiple clips Audio mixing Video transitions Incorporating Core Animation in movies
21
22
AVAssetImageGenerator
AVAssetExportSession
23
AVComposition
AVPlayerItem
AVAssetImageGenerator
24
Create an image for a time Trim a movie to a time range Cutting together multiple clips Audio mixing Video transitions Incorporating Core Animation in movies
25
Demo
Cutting together movie clips
Sample code: AVEditDemo (see SimpleEditor.m)
26
Composing a Timeline
27
AVComposition
AVCompositionTrack (audio)
AVCompositionTrackSegment seconds 36 of audio track of cat.mov AVCompositionTrackSegment seconds 12 of audio track of beach.mov AVCompositionTrackSegment seconds 510 of audio track of flowers.mov
28
AVMutableComposition
You can edit across all tracks of a composition:
[compositionTrack setSegments:...];
29
AVEditDemo
See buildSequenceComposition in SimpleEditor.m
AVMutableComposition *composition = [AVMutableComposition composition]; AVMutableCompositionTrack *compositionVideoTrack = [composition addMutableTrackWithMediaType:AVMediaTypeVideo preferredTrackID:...]; [compositionVideoTrack insertTimeRange:... ofTrack:clipVideoTrack atTime:... error:...];
30
Make a copy for these tasks; then its safe to modify the original:
AVPlayerItem *playerItem = [AVPlayerItem playerItemWithAsset: [[mutableComposition copy] autorelease]];
31
Create an image for a time Trim a movie to a time range Cutting together multiple clips Audio mixing Video transitions Incorporating Core Animation in movies
32
Demo
Mixing in additional audio
Sample code: AVEditDemo (see SimpleEditor.m)
33
34
AVAudioMix
35
AVAudioMix
Tool for adding volume adjustments
36
AVMutableAudioMix
AVMutableAudioMixInputParameters *trackMix = [AVMutableAudioMixInputParameters audioMixInputParametersWithTrack:mainAudioTrack]; [trackMix setVolume:1.0 atTime:kCMTimeZero]; [trackMix setVolumeRampFromStartVolume:1.0 toEndVolume:0.2 timeRange:CMTimeRangeMake(x,y-x)]; ...
1.0 0.2 x y
37
Using AVAudioMix
To apply AVAudioMix for playback:
playerItem.audioMix = audioMix;
exportSession.audioMix = audioMix;
38
Create an image for a time Trim a movie to a time range Cutting together multiple clips Audio mixing Video transitions Incorporating Core Animation in movies
39
Demo
Video transitions
Sample code: AVEditDemo (see SimpleEditor.m)
40
AVVideoComposition
A
A/B
B/A
41
AVVideoComposition
AVVideoComposition A
A B B A
Each instruction describes the output video in terms of input layers (AVVideoCompositionLayerInstruction)
Each layer has an opacity and an affine transform Opacity can be tweened e.g., for a cross-fade Affine transform can be tweened e.g., for a push transition
42
AVVideoCompositionInstruction
AVVideoComposition A
A B
B A
AVMutableVideoCompositionInstruction *transition = [AVMutableVideoCompositionInstruction videoCompositionInstruction]; transition.timeRange = ...; AVMutableVideoCompositionLayerInstruction *fromLayer = [AVMutableVideoCompositionLayerInstruction videoCompositionLayerInstructionWithAssetTrack:trackA]; // Fade out trackA by setting a ramp from 1.0 to 0.0. [fromLayer setOpacityRampFromStartOpacity:1.0 toEndOpacity:0.0 timeRange:...]; AVMutableVideoCompositionLayerInstruction *toLayer = ... transition = [NSArray arrayWithObjects:fromLayer, toLayer, nil];
43
AVVideoComposition
AVMutableVideoComposition *videoComposition = [AVMutableVideoComposition videoComposition]; videoComposition.instructions = [NSArray arrayWithObject:transition]; videoComposition.frameDuration = CMTimeMake(1, 30); videoComposition.renderSize = CGSizeMake(1280, 720); videoComposition.renderScale = 0.5; // for playback only
44
Using AVVideoComposition
For playback:
playerItem.videoComposition = videoComposition;
assetImageGenerator.videoComposition = videoComposition;
For export:
assetExportSession.videoComposition = videoComposition;
45
Pitfalls
AVVideoCompositionInstructions must not overlap or contain gaps
AVComposition AVVideoComposition
46
47
I P P P P I P P P P I P P P P I P P P P ...
AVComposition
48
preferredTransform set on the video track The transform will be ignored if the asset is placed in an AVComposition Use AVVideoComposition to reinstate the rotation
49
Create an image for a time Trim a movie to a time range Cutting together multiple clips Audio mixing Video transitions Incorporating Core Animation in movies
50
Demo
Core Animation in Movies
Sample code: AVEditDemo (see SimpleEditor.m)
51
CAAnimations
fade out after 10 sec
Magic!
52
titleLayer
ringOfStarsLayer
CABasicAnimation Spin
Magic!
Attend Sessions 424 and 425 for more information on Core Animation
53
AVPlayerLayer
real time seconds since boot 5000 5001 5002 5003 5004 5
AVSynchronizedLayer
Video
Magic!
54
55
videoLayer
animationTitleLayer
Magic!
Multitasking
Core Animation use in the background will cause the export to fail
57
Zero beginTime is automatically translated to CACurrentMediaTime() Use a small nonzero number: e.g., 1e-100 or -1e-100
(AVCoreAnimationBeginTimeZero)
58
59
Stretch It Out
Core Animation contributions may continue past the end of an
AVComposition
AVComposition Core Animation Animations a
60
Summary
Create an image for a time Outputting a movie Combining multiple clips Audio volume adjustment Video transitions Incorporating Core Animation
AVAssetImageGenerator AVAssetExportSession AVComposition AVAudioMix AVVideoComposition AVSynchronizedLayer and AVVideoCompositionCoreAnimationTool
61
More Information
Eryk Vershen
Media Technologies Evangelist evershen@apple.com
Documentation
AV Foundation Framework Reference http://developer.apple.com/iphone
62
Related Sessions
Discovering AV Foundation (Repeat) Using the Camera with AV Foundation Core Animation in Practice, Part 1 Core Animation in Practice, Part 2 Introducing Blocks and Grand Central Dispatch on iPhone
Nob Hill Thursday 4:30PM Presidio Tuesday 4:30PM Nob Hill Thursday 11:30AM Nob Hill Thursday 2:00PM Russian Hill Wednesday 11:30AM
63
Labs
AV Foundation Lab #1 AV Foundation Lab #2 Core Animation Lab
Graphics & Media Lab C Wednesday 9:00AM Graphics & Media Lab B Friday 11:30AM Graphics & Media Lab D Thursday 3:15PM
64
65
66
67