Reverberation algorithms

Augusto Sarti

Summary
n n n

The Reverb Problem Reverb Perception Acoustic impulse response:

Formation mechanisms Parameters

n n n

Early Reflections Late Reverb Numerical reverberation algorithms

Schroeder Reverbs Feedback Delay Network (FDN) Reverberators Waveguide Reverberators

Geometrical reverberation algorithms

Impulse response
n n

The sounds we perceive heavily depend on the surrounding environment Environment-related sound changes are of convolutive origin (filtering)
Well-modeled

by a space-varying impulse response

Impulse response

Amplitude

Time Direct signal Early reflections Reverberations

Reverberation tf Function
Three sources, one listener (two ears) n Filters should include pinnae filtering n Filters change if anything in the room changes (exact model)
n

Global descriptors
n

Energy decay curve (EDC)

Introduced by Schroeder to define reverberation time It measures the total signal energy remaining in the reverberators impulse response at time t It decays more smoothly than the impulse response, therefore it works better than the amplitudes envelope for defining the reverberation time In reverberant environments a large amount of the total energy is contained in the last portion of the impulse response

Reverberation time

T 60 = {t : EDC (t ) =EDC (0) 60dB}

Global descriptors

EDR of a violin body

Global descriptors
n

In the rooms transfer function we can single out resonant modes The spacing between two resonant modes is given by

which is valid above the threshold frequency

Global descriptors
n

Number of echoes in the impulse response before time t

Derivative of Nt:

Clarity index: ratio btw early reflections energy and late reverberation energy

Implementation

Moorer reverberator
n

accounts for late reverberations by placing an IIR filter after the FIR filter (tapped delay line)

Binaural impulse response

n Our

sound perception is affected by our own body

Head

Related Transfer Function (HRTF)

Acoustic paths can be grouped together to reduce cost

Comb filter

Allpass filter

Steady-state tones (sinusoids) really do see the same gain at every frequency in an allpass, while a comb filter has widely varying gains

Comb filters and reverberation time

n

The decay between successive samples in comb and allpass filters is described by the gain coefficient gi In order for the comb filters decay to correspond to a given reverberation time, we must have

Combination of comb filters

n n

Single comb filters do not provide sufficient echo density In order to improve the echo density, we need to combine multiple comb filters
Cascading comb filters corresponds to multiplying their transfer functions Frequency peaks not shared by all comb filters are cancelled by multiplication

Combination of comb filters

n Better

to place comb filters in parallel

Example

Parallel comb filters

n

The poles of comb filters are given by

The poles have the same magnitudes

The modal density (No. of modes per Hz) is

Parallel comb filters

n n n

Modal density turns out to be the same at all frequencies, unlike real rooms Above a threshold frequency, the modal density is constant The modal density of the comb filters is then set to the modal density above the threshold frequency

Parallel comb filters

n

The echo density of the comb filters is approximatively given by

Relating echo density and modal density provides:

Combination of allpass filters

n

Unlike comb filters, allpass filters must be cascaded

Multiplying

freq. responses corresponds to adding phase responses

Schroeders reverberator (1)

Schroeders reverberator
n

Delays of the comb and allpass filters are chosen so that the ratio of the largest and smallest delay is 1.5 (typically 30 and 45 ms) The gains gi of the comb filters are chosen to provide a desired reverberation time Tr according to

Allpass filters delays are set to 5 and 1.7 ms

Feedback Delay Networks at a glance

n

Unitary matrix: definition

A matrix is unitary if :

|| M u ||=|| u ||

We can also write that a matrix is unitary if

|| MMT ||=|| MT M ||= 1

FDN
a11 a12 a 21 a22 A= a31 a32 a41 a42 a13 a23 a33 a43 a14 a24 a34 a44

Stability of the feedback loop is guaranteed if A = gM where M is an unitary matrix and |g|<1 Outputs will be mutually incoherent: we can use the FDN to render the diffuse soundfield with a 4 loudspeaker system The early reverbeartions can be simulated by appropriately injecting the input signal into the delay lines

Jots reverberator

c =[

a11 a12 a1N a a22 a2 N 21 A = a N 1 a42 a4 N

b1 b = bN

c1 c = c N

Jots reverberator
The input-output relation of Jots reverberator is given by

with

and

Jots reverberator
n System

transfer function:

n Zeros:

n Poles:

Jots reverberator
n

Moorer noted that convolving exponentially decaying white noise with source signals produces a very natural sounding As a consequence, by introducing absorptive losses into a lossless prototype, we should obtain a natural sounding reverberator This is accomplished by associating a gain with each delay:

Jots reverberator
n

The logarithm of the gain is proportional to the length of the delay:

n n

The above modification has the effect of replacing z with z/ in the transfer function The lossless prototype response h[n] will be multiplied by an exponential envelope n

Modeling the Environment

Modeling the environment

n Simulate

reverberations due to environment

Motivations
Acoustical environment provides ... n Sense of presence n Comprehension of space n Localization of auditory cues n Selectivity of audio signals (cocktail party effect)

Geometric acoustic modeling

n

Spatialize sound by computing reverberation paths from source to receiver

Similarities to Graphics
n

Both model wave propagatation

Differences from Graphics I

n

Sound has longer wavelengths than light

Diffractions

are significant Specular reflections dominate diffuse reflections Occlusions by small objects have little effect

Differences from Graphics II

n

Sound waves are coherent

Modeling

phase is important

Differences from Graphics III

n

Sound travels more slowly than light

Reverberations

are perceived over time

Overview of approaches
n n n n n

Finite element methods Boundary element methods Image source methods Ray tracing Beam tracing

Finite element methods

n Solve

wave equation over grid-aligned

mesh

Boundary element methods

n Solve

wave equation over discretized surfaces

Boundary Element Trade-offs

n Advantages
Works

well for low frequencies Simple formulation

Boundary Element Trade-offs

n Disadvantages
Complex

function stored with each element Form factors must model diffractions & specularities Elements must be much smaller than wavelength

Image source methods

n Consider

direct paths from virtual

sources

Image source trade-offs

n Advantages
Simple

for rectangular rooms

Image source trade-offs

n Disadvantages
O(nr)

visibility checks in arbitrary environments Specular reflections only

Path tracing
n Trace

paths between source and receiver

Path Tracing Trade-offs

n Advantages
Models

all types of surfaces and scattering Simple to implement

Sampled reverberation s Incoming ray

Path Tracing Disadvantages

n Disadvantages
Subject

to sampling errors (aliasing) Depends on receiver position

Beam Tracing
n Trace

beams (bundles of rays) from source

Beam Tracing Trade-offs

n Advantages
Takes

advantage of spatial coherence Predetermines visible virtual sources

Beam Tracing Disadvantages

n Disadvantages
Difficult

for curved surfaces or refractions Requires efficient polygon sorting and intersection

BSPs

Cell adjacency graphs

Complex 3D Environments
n Precompute

beam tree for stationary

source

Interactive Performance
n Lookup

beams containing moving receiver

Summary
n FEM/BEM
best best

for low frequencies

n Image n Path

source methods

for rectangular rooms (very common) for high-order reflections (very common)

tracing tracing
for precomputation

best

n Beam
best

Current research in interactive audio spatialization

Back to the problem

n

Path/ray tracing according to the laws of geometric optics Applications to

Simulation

of acoustic reverberations in complex environments Prediction of EM propagation for wireless systems (multipath fading)

81

Beam tracing
n n

Construction of the beam tree through space subdivision Construction of paths through beam tree lookup

82

Using space subdivision

83

What is missing?
n Traditional
Every

beam tracing assumes that the source be fixed

time the source moves, the BT needs to be rebuilt from scratch (lengthy process based on space subdivision)

n Is

it possible to avoid space subdivision? n Is it possible to settle all visibility issues in advance (irrespective of the source location)? n Is it ultimately possible to build the BT through a simple lookup process?
84

Reformulating the problem

n Define

environments visibility independently from the sources location n Compute the environments visibility n Build the beam tree using
Visibility

info Sources location

n Build

the paths using

Beam

tree 85 Receivers location

Environments characterization
n n

Sources and Receivers

Assumed

to be point-like

Reflectors
Oriented
n n n

surface of a reflecting wall

A reflecting wall defines two reflectors Assumed as flat Identified by an index

Byproducts:
Beams
n n

Compact bundle of rays originated by the same source Identified by a source (real or virtual) and the illuminated portion of a reflector

Active
n n

reflectors

That portion of a reflector illuminated by a beam Identified by a beam and a reflector

86

Visibility
n

Visibility function
Function

that associates the index of the visible reflector to a viewpoint and a viewing direction Piece-wise constant function that takes on values in the parameter space that characterizes viewpoint and viewing direction
n

Visibility function from a reflector

Visibility

function where viewpoints are constrained on the pts of the reflector of the visibility functions associated to all the environments reflectors
n

Environments visibility description

Set

M reflecting walls => 2M visibility functions

87

Defining the parameter space

n

Parameter space: viewpoint and viewing direction

If

the point lies on a reflector

n n

4D parameter space in the 3D case 2D parameter space in the 2D case

Reflectors
n

normalization

affine transformation (rigid motion + scaling) of the geometric space that remaps the reflector onto the segment that goes from (0,-1) to (0,1), with reflecting surface facing x0

This

way viewpoint and viewing direction can be described by the eq. y = a x + b, where -1b1 describes the point on the reflector and a the viewing direction
n

Parameter space: (a,b)

88

Visibility region
n The

visibility region of a given reflector w.r.t. a reference reflector is the region of the parameter space (a,b) that corresponds to viewpoints on the reference reflector from which the given one results as visible
Due

to occlusions, this region can be empty or made of a set of convex polygons

n The

visibility region of reflector i is the region where the visibility function is equal to i
89

Visibility region
n A

generic reflector can be described by

(e+f,g+h) x=et+f y=gt+h 0t1 Substituting in y=ax+b we obtain (f,h) g t + h = a (e t + f) + b, 0t1 -1b1 Visibility region: Intersection btw a bundle of rays (a beam in parameter space) and the strip -1b1
90

Examples

91

Visibility region
n n n

Potential visibility region: visibility region with no other reflectors Potential visibility regions may overlap Actual visibility region is contained within the potential one Overlaps must be resolved considering occlusions Approach for evaluating visibility function Compute potential visibility regions Resolve overlaps and identify actual visibility regions Label actual visibility regions

92

Resolving overlaps
n When

two potential visibility regions overlap, the corresponding reflectors exhibit a partial occlusion w.r.t. the reference reflector n Who occludes who decides which region eats which on the overlap n This can be done by tracing a sample ray within the overlapping region

93

Parameter space (dual space)

94

Reflectors in the dual space

95

Normalized dual space

96

Building a beam tree from visibility

n Evaluating

the global visibility of the environment corresponds to building one visibility function per reflector
This

corresponds to constructing and labeling all the actual visibility regions for each reflector

n All

this ignores the location of the source n Given source location and visibility, how do we build the beam tree?
97

Sources in parameter space

n n

A source in parameter space is a line (dual of a pt) Source and active portion of a reflector define a beam
The

branching of a beam is defined by the intersection btw the line and the actual visibility regions
98

Beam tracing
n Given

a beam reflected by the i-th reflector, use visibility to to determine its branching in sub-beams (one per visible reflector)
Determine

virtual source location in the warped space corresponding to i-th reflector Determine illuminated portion of reflector and the corresponding narrowed reference strip Scan actual visibility regions over the line corresponding to the source in parameter space Update beam tree 99

Beam tracing

100

Computational efficiency

101

Computational efficiency

102

103

Modeling diffraction
n Use

geometric theory of diffraction n Diffraction modeled by placing sources (and the relative beam trees) at diffracting wedges n Beam trees computed in advance jointly with visibility information

104

A comparison btw imp. responses

Sim.

Meas.

Comparison
n

Parameters:
EDC Early Decay Time (EDT): time that imp. resp. takes to dim down of 10 dB. Center Time (CT): centroid of squared impulse response

Imp. Resp. envelope:

EDC
recorded Simulated refl. only

Sim. of refl. + diffr.

Complete simulation

EDT
recorded Simulated refl. only

Sim. of refl. + diffr.

Complete simulation\

Centre Time
recorded Simulated refl. only

Sim. of refl. + diffr.

Complete simulation

Auralization

110

Active Beamshaping

Rendering beams

Physical approach: WFS

Geometric appraoch: Beam Tracing

Huygens principle data-based (needs wavefield acquisition) Works on wavefronts

Implements general solution according to geometric propagation principles Boundary conditions become components of the implementation

112

Goal

Reconstruction of an arbitrary source (arbitrary radiation function) in an arbitrary location using an array of speaker in close range

113

Signal transfer
=

NxM

Mx1 Nx1

Gh = b
Matrix form
114

Example (1) central beam

n M

= 16; n f = 700Hz n y = not uniform n Gaussian mask

115

Example 2 skewed beam

nM

= 64; nf = 700Hz ny = 10 cm nMaschera Gauss

116

Rendered beam

Wideband extension
Instead

of setting constraints at a single frequency, we apply them to multiple frequencies (wideband minimization) 4 parameters:
n F,

No. of frequencies where we minimize n M, No. of speakers n N, No. of angles n T, No. of taps of the filter

118

Example 3 skewed beam

119

Interface

120

Testing in a dry room

Multiple audio cards in daisychain configuration 8-16-24 synchronized outputs

15-speaker non-uniform array

121

Results
n Expected
Low

contrasting needs

frequencies require extensive arrays High frequencies require closely-spaced speakers Cost constrains limit the No. of speakers
n With

15-16 speakers we do not go beyond 17-18 db of attenuation btw main lobe and side lobes with a limited frequency range (300Hz-6kHz)
122

Conclusions
n Results

are comparable to those achieved with WFS but we control them in a geometric fashion. Therefore we can
Reconstruct

an arbitrary source in an arbitrary

location Combine multiple beams through superposition principle, therefore it can be used as a geometric engine for synthesizing the response of the environment as well (early reverberation for spatial impression)
123