FEATURE ARTICLE

Ambisonics comes of age

Francis Rumsey
Consultant Technical Writer

Ambisonic representation provides a relatively versatile and compact way of storing

or transmitting spatial audio scenes. Signals in this format may exist in mixed
orders, requiring special decoders to be constructed that optimize spatial rendering.
When it comes to decoding ambisonics for different speaker layouts, there may be
advantages to active or hybrid decoders. Irregular layouts are harder to decode for
than regular ones. Binaural rendering is a popular means of reproducing ambisonic
content, and there is some evidence that individualized HRTF processing is useful
in this context. Ambisonic information can also be data compressed, and there may
be advantages to doing this after some form of signal decomposition has been done,
which takes advantage of interchannel redundancy.

A
mbisonics is a means by which
spatial audio sound fields can be
represented in the form of a hier-
archy of directional components. It has
become increasingly important in recent
years, resulting in a number of papers and
e-Briefs at AES conventions, and a num-
ber of these are summarized in this arti-
cle. First developed primarily in the 1970s,
ambisonic principles were manifested in
a relatively small number of systems and
products for many years—principally the
Soundfield microphone. Like a number of
other inventions that have emerged over
the years, it was perhaps “before its time,”
and remained mainly of interest in aca-
demic and research circles. Fascinatingly,
though, its time as a mainstream tech-
nology seems now to have come, brought
about by the need for an intermediate
method of representation for 3D audio that
can be rendered to a variety of different
speaker or headphone configurations. This
is perhaps not so surprising, as 3D audio
formats have proliferated in the recent past,
and the growing interest in interactive or
immersive media has generated a need
for things like rotatable sound fields and
a compact structure for capturing, stor-
ing, and transferring 3D audio alongside
360 video and virtual reality content. The
MPEG-H audio coding standard, for exam-
ple, incorporates a mode known as scene-
based audio (SBA) for transmitting spatial
audio as ambisonic components.
The first manifestations of ambisonics
mainly used what is known as first-order
representation (B-format), whereby the
sound field is represented as a pressure
(omnidirectional) component, plus three
orthogonal velocity components (like
figure-eight microphone patterns point-
ing front-back, left-right, and up-down).
This gives relatively low spatial resolution,
and later manifestations have increased the
order of representation by including further
“spatial harmonics” that have tighter direc-
tional patterns. These formats are typically
known under the generic hearing “high-
er-order ambisonics” or HOA. Increasing
the order leads to a larger number of chan-
nels needed for storage or transmission, and
recent ambisonic streaming applications
typically use 2nd or 3rd order representa-
tion, needing 9 or 16 channels respectively.
There can be diminishing returns, percep-
tually, as one increases the order above this,
with the suggestion that there is limited
value in using orders above 5th for typical
applications.
Encoded content can be rendered for
reproduction using decoding algorithms
designed for almost any loudspeaker
layout, as long as the locations of the
loudspeakers are known, although irreg-
ular layouts are more challenging than
regular ones. It can also be decoded in the
form of two-channel binaural signals suit-
able for headphone listening. Ambisonic
representation lends itself particularly
well to interactive contexts where the
listener’s head may be rotated or tilted, as
the decoding (rendering) algorithms can
easily be adapted.
OPTIMIZED DECODERS FOR
MIXED-ORDER AMBISONICS
In Paper 10507 (“Optimized Decoders
for Mixed-Order Ambisonics”) from the
150th Convention, Aaron Heller, Eric Ben-
jamin, and Fernando Lopez Lezcano dis-
cuss decoding in systems where the ver-
tical order is lower than the horizontal
order. They use a form of shorthand from

696 J. Audio Eng. Soc., Vol. 69, No. 9, 2021 September


Fig. 1. “The Stage” at CCRMA (Center for Computer Research in Music and Acoustics)—a 56-loudspeaker array for spatial audio rendering.
(Courtesy of Heller, Benjamin, and Lopez-Lezcano)
Chris  Travis to describe mixed-order sys-
tems, giving the example that 3H2V means
third-order horizontal, second-order ver-
tical. Such mixed-order systems will have
different spatial resolution in some direc-
tions to others. The authors point out that
mixed-order microphones are available, and
also that in domestic settings it’s easier to
deploy, say, eight loudspeakers in a horizon-
tal array than it is to have a lot of elevated
loudspeakers. Most real-world applications
involve arrays that are either irregular or
incomplete, it’s suggested. Approximately
hemispherical arrays are one example, the
missing bottom half of a full sphere leading
to increasing localization errors at or below
the horizon. In arrays that have more hor-
izontal than elevated speakers the spatial
accuracy will inevitably be better in this
plane than it is for height cues.
A useful explanation of the task of an
ambisonic decoder, given here, is “to
create the best perceptual impression
possible that the sound field is being
reproduced accurately, given the available
resources.” Historically, various criteria
have needed to be optimized in ambi-
sonic decoding, involving amplitude and
energy gain in different directions, veloc-
ity, and energy-based localization vectors
at low and high frequencies respectively.
Some of these are said to be more import-
ant than others, and it is important to
avoid conflicting perceptual cues. It’s
pointed out that a number of rendering
systems deal with mixed-order signals by
using full-order decoders and just leav-
ing out the missing channels, but decod-
ers designed specifically for mixed-order
signals seem always to outperform this
rather crude approach in terms of spatial
accuracy in the rendered directions.
In this paper the emphasis is on a new
implementation of the Ambisonic Decoder
J. Audio Eng. Soc., Vol. 69, No. 9, 2021 September
Toolbox (ADT), previously introduced by the
first author, and available online at https://
bitbucket.org/ambidecodertoolbox/adt.
git. This new version is said to include a
fast and robust nonlinear optimizer and a
new design procedure for dual-band decod-
ers. The high-frequency performance is
optimized first, then the low-frequency
performance is optimized to match it. The
solution described here involves designing
a mixed-order decoder for a spherical 240
loudspeaker array. Apparently the choice
of a spherical array design creates a well-
behaved optimization problem and an opti-
mal decoder matrix. The high frequency
energy vector (rE, which “points” in the
direction that a high frequency source
would be perceived) is then computed over
5200 virtual speaker points on a sphere, and
values at corresponding locations to those
points are used as target values for real
loudspeaker positions. The low frequency
velocity vector (rv, which “points” in the
direction that a low frequency source would
be perceived) is then optimized such that it
points the same way as rE at every spherical
point, and has a magnitude of 1.
One of the interesting outcomes of the
work is a visualization tool that helps
designers decide which of an available array
of loudspeakers should be used for horizon-
tal and height layers if parameters are to be
optimized. The authors applied the tool to
both a 56-loudspeaker professional installa-
tion (Fig. 1) and a 13-loudspeaker domestic
“dome” with good results, at least from
a modelling point of view, and an initial
one-listener listening test.
BINAURAL AMBISONIC RENDERING
To give a bit of background, back in 2017
Joseph Tylka and Edgar Choueiri pre-
sented “A Toolkit for Customizing the
ambiX Ambisonics-to-Binaural Renderer”
FEATURE ARTICLE
in e-Brief 403. The toolkit described by
this paper’s authors was based around a set
of ambisonic decoding plug-ins developed
by Matthias Kronlachner known as ambiX
plug-ins (http://www.matthiaskronlachner.
com/?p=2015). ambiX is a channel order-
ing convention used for ambisonic record-
ing exchange, and the ambiX plug-ins are a
set of ambisonic decoding plug-ins, which
includes a binaural renderer. Tylka and
Choueri’s contribution was essentially an
open source collection of MATLAB func-
tions known as the SABRE toolkit (https://
github.com/PrincetonUniversity/3D3A-SA-
BRE-Toolkit), which facilitated the cre-
ation of customized or personalized ambi-
sonic-to-binaural decoding presets, using
SOFA-formatted head-related transfer func-
tions (HRTFs).
HRTFs represent the unique effects of
one’s head, torso, and outer ear on the
acoustic cues processed by the auditory
system and are the principal means by
which that system decodes the directions
of sound sources. Every direction has a
different effect on the frequency response
and time delays of binaural signals. HRTFs
are often represented in their time-
domain equivalent form, the head-related
impulse response, or HRIR. SOFA is an
increasingly popular standard format for
distributing individual HRTFs, standing for
“spatially-oriented format for acoustics.”
Tylka and Choueri explained that there are
more comprehensive tools than theirs for
creating ambisonic decoders, such as the
ADT mentioned in the previous section. For
this reason they said that one application
of their toolkit was simply to add custom
HRTFs to existing ambiX decoding presets.
This needed the user to specify loudspeaker
locations because the information about
those is apparently not explicitly included in
ambiX decoding presets.
697
FEATURE ARTICLE
HRTFs included in SOFA files can be the measurement of them, partly because
represented as HRIRs gathered on a grid of measuring people’s individual HRTFs is
measuring locations. It is possible that these difficult. A novel feature is the extension of
may need to be interpolated if measured individual HRTFs generated in just a few
values are not available at desired grid posi- directions to rendering in any direction.
tions, so the toolbox had various means As shown in Fig. 2, a typical binau-
of doing this, including nearest neighbor, ral ambisonic renderer takes ambisonic
time domain, and frequency domain meth- components (spherical harmonics),
ods. There was an option not to interpolate decodes them to loudspeaker feeds for a
when a grid point in the SOFA file was close particular layout, and then uses HRTF
enough to a desired location, according to a processing to filter the loudspeaker signals
specified threshold value. Means were also from different directions, generating
provided in the toolkit to apply headphone left and right ear signals. As the authors
equalization if needed, in order to compen- explain, using non-individual HRTFs can
sate for specific headphone responses that give rise to perception errors such as
might otherwise interfere with optimal in-head localization, front-back confusion
binaural reproduction. For example, some or poor discrimination of elevation cues.
headphones are equalized to the diffuse- Individual HRTF modelling is therefore
field standard developed some years back, done according to the scheme shown in
whereas others may employ free-field equal- Fig. 3. Here HRTFs from a database based
ization or indeed none at all. on the KEMAR head, known as CIPIC,
are decomposed using a process known as
INDIVIDUALIZED BINAURAL SPCA (spatial principal component analy-
RENDERER FOR HOA sis). Eight different anthropometric param-
More recently at the 150th Convention, eters, including head dimensions, shoulder
Mengfan Zhang and colleagues from China width, and various outer ear measurements
published a paper on an “Individualized are chosen to represent the main physi-
HRTF-Based Binaural Renderer for High- cal factors that affect HRTFs in practice.
er-Order Ambisonics” (Paper 10454). These are captured from photographs of
They described a system whereby HOA sig- individuals, and the parameters used to
nals could be rendered for headphones with estimate new SPC weights for these indi-
better results using an individual binaural viduals (new people, whose HRTFs were
renderer than using a generic one. This was not contained in the original database).
based on their previous work which empha- Deep neural networks (DNNs) are trained
sised the modelling of HRTFs, rather than to predict new values of these in arbitrary
spatial directions,
and converted
back into HRTF
magnitude spectra.
Related interaural
time differences
(ITDs) are also
Fig. 2. A typical binaural ambisonic renderer takes ambisonic computed using a
components, decodes them to loudspeaker feeds for a particular layout, similar approach,
and then uses HRTF processing to filter the loudspeaker signals from
different directions, generating left and right ear signals. (Figs 2–4 also possible in
courtesy of Zhang et al.) arbitrary spatial
directions.
Binaural HRIRs are
then reconstructed
from minimum
phase monophonic
magnitude spectra,
based on modelled
ITDs.
In order to eval-
Fig. 3. Framework for modelling of individual HRTFs using spatial uate the approach,
principal components. the authors
698
Fig. 4. 20 vertices of a regular dodecahedron
used as virtual loudspeaker locations for
ambisonic decoding.
conducted a listening experiment. 3rd-
order HOA signals of source material were
decoded to 20 virtual loudspeakers at loca-
tions corresponding to the vertices of a
regular dodecahedron, as shown in Fig. 4.
The virtual loudspeaker signals were then
each convolved with the HRIR of the near-
est spatial direction in the CIPIC database,
for the “generic” renderer, or the individ-
ual HRIR created by the modelling process
described above for the individual renderers.
Source material for the tests was simply a
train of 250 ms bursts of windowed noise
panned ambisonically so as to appear to
come from different directions. As far as
can be determined from the paper, head
tracking was not employed in the render-
ing process, so the rendered signals would
not have been affected by listener head
movements. Listeners were expected to
use a graphical interface to indicate the
spatial locations of the noise bursts. Results
suggested that front-back confusion was
significantly lower for the individualized
rendering than for generic rendering.
AMBISONIC DECODER TEST
METHODOLOGY
Virtual loudspeakers and binaural ren-
dering were also used in work described
by Enda Bates, William David, and Daniel
Dempsey, in Paper 10457 (“Ambisonic
Decoder Test Methodologies Based on
Binaural Reproduction”). The authors
point out that there is an increasingly
large number of ways to decode ambi-
sonic source material, including passive
approaches, parametric methods based
on time-frequency analysis, and hybrid
approaches. It’s difficult, they say, to
J. Audio Eng. Soc., Vol. 69, No. 9, 2021 September

Fig. 5. Process for creating test and reference stimuli to compare ambisonic decoders.
(a) Object-based tests; (b) recording-based tests. (Courtesy of Bates, David and Dempsey)
compare ambisonic decoders, as it is to
compare spatial recording techniques,
without a clear “ground truth” reference
against which to compare the results.
Mostly people have assumed either the
expert subjects have a good internal
reference (memorized), against which
to compare what they hear, or ratings
have been made on specific spatial attri-
butes such as source width. For the tests
described in this paper, it seems, the
authors chose to use eight different attri-
butes for evaluation of three ambisonic
decoding strategies. A web-based multi-
ple comparison paradigm was employed
enabling stimuli to be compared against
a reference and high-low anchor signals.
Two different loudspeaker layouts were
used, one a cube and the other a typical
multichannel “cinematic” 11.1 (7.1.4)
layout. A cube layout is potentially easier to
J. Audio Eng. Soc., Vol. 69, No. 9, 2021 September
decode for, owing to its symmetry, but has
relatively few loudspeaker locations in the
horizontal plane. The 7.1.4 layout has more
loudspeakers in the horizontal plane but
is irregular in its layout. Source material
consisted of both recorded scenes (using
first order ambisonic microphones) and
object-based signals with sources panned
using an ambisonic panner. The ambisonic
source material was rendered binaurally by
convolving the loudspeaker feeds, decoded
using three different decoding algorithms
to be compared, with HRIRs relating to the
loudspeaker locations taken from a database
known as SADIE II. Reference scenes were
created using direct binaural recordings, or
direct binaural rendering of object sources,
rather than using ambisonic pickup/
panning and decoding as an intermediate
step. A form of equalization was used to
compensate for timbral differences between
FEATURE ARTICLE
binaural responses used in the processes
involved. Fig. 5 shows the process involved
for both types of scene capture.
The results suggested, among other
things, that hybrid and active decoding
methods could outperform traditional
passive ambisonic decoding for some source
location-related attributes, but only when
the loudspeaker layout was well matched to
the decoding method. No significant differ-
ences could be detected between the results
for the three decoders for more general
attributes such as timbre, spatial impression
or naturalness/presence. The cube loud-
speaker layout, though, was rated higher in
most cases for these than the 7.1.4 layout,
and overall it seemed there was a trend
towards better results with the cube layout.
It was noted that improvements to the
testing approach could include headphone
equalization, head tracking and personal-
ized HRIRs.
DATA COMPRESSION
OF HOA SIGNALS
Xu et al. explain that HOA signals need
a lot of channels to deliver high spatial
resolution, so it is worth considering effi-
cient data compression for efficient stor-
age and transmission. (“Higher Order
Ambisonics Compression Method Based
on Independent Component Analysis,”
Paper 10456). Of course independent
data compression could be employed on
each mono channel, but this would not
take advantage of redundancy between
channels. An alternative approach may
use a form of principal component anal-
ysis to extract predominant components
from a group of channels, encoding these
with conventional mono encoders, then
the residuals (background components)
are encoded as ambisonic signals at a
lower bit rate. This is the basis of the
so-called single value decomposition
(SVD) method given as an example in the
MPEG-H standard. It’s pointed out that
the predominant components extracted
using such a method may not correspond
directly to the sound sources, and the
components are calculated on a frame-by-
frame basis, which can result in abrupt
transitions. Consequently, the authors
propose an alternative based on indepen-
dent component analysis (ICA), a method
of signal decomposition widely used in
blind source separation applications. In
699
FEATURE ARTICLE

ate eight predomi- owing to its potential to be rendered for a

nant components, variety of different reproduction formats,
each encoded by an and for easy spatial manipulation of sound
EVS mono encoder scenes.
at 24 or 48 kbit/s, Signals in this format may exist in mixed
plus 3 kbit/s for orders, requiring special decoders to be
side information. constructed that optimize spatial render-
The third method ing. When it comes to decoding ambisonics
was the new for different speaker layouts, it seems that
method proposed there may be advantages to certain active
i n t h i s p a p e r, or hybrid decoders, but only if the decoder
again resulting in type is well matched to the speaker layout
eight components in question. Irregular layouts are harder
and encoded the to decode for than regular ones. On the
same way as the other hand, binaural rendering is a popular
second method. In means of reproducing ambisonic content,
both of the cases often by virtualizing decoded loudspeaker
involving signal feeds using HRTF processing. There is some
decomposition the evidence that individualized HRTF process-
background infor- ing is useful in this context.
mation was omit- Finally, ambisonic information can
Fig. 6. (a) Encoder and (b) decoder block diagram for low bit-rate ted. Compressed be data compressed, and there may be
ambisonic encoder based on independent component analysis. signals were advantages to doing this after some form
(Courtesy of Xu et al.)
decoded back to of signal decomposition has been done,
HOA signals and which takes advantage of interchannel
such a case, they say, the primary com- rendered for two-channel headphone redundancy. The jury is still out, to some
ponents extracted are directly related to listening using a binaural renderer. extent, on the best method of decom-
sound sources, while the side information The source material used for these tests posing ambisonic components into fore-
contains background spatial information. was either simulated from mono sources ground and background elements, but
The ICA algorithm thus employed works panned in different directions, or recorded preliminary evidence suggests some bene-
to “unmix” independent sources from directly, and consisted of a variety of fits from a method based on independent
an HOA signal matrix. These foreground program types such as singing, speaking, component analysis.
components form one element of the and instrumental music. There was also a
output, while the spatial characteristics, helicopter recording, some moving instru-
Editor’s note: All the papers referred to
in the form of a mixing matrix, form the ments, and a noisy room. Based on a multi- in this article can be found in the AES
side information. Residuals left after this ple stimulus paradigm, measuring overall E-Library at http://www.aes.org/e-lib/.
separation are regarded as Gaussian (noise- sound quality and spatial characteristics AES members get free access to the
E-Library.
like), and treated as a background stream (presumably combined into a single value),
that can be transmitted at a lower bit rate, employing a hidden reference and low
or truncated to a lower order. Foreground anchors (low-pass filtered and mono), 12
and background streams are then encoded listeners compared the results of the differ-
conventionally using an entropy encoder. ent coding approaches and bit rates. The
The overall structure of encoding and new approach was rated significantly higher
decoding using such an approach is shown than the other two, at both bit rates, provid-
in Fig. 6. ing some initial support for the effectiveness
In order to evaluate the success of this of the new approach. The authors acknowl-
approach Xu and his colleagues compared edged, nonetheless, that they should still
three different methods of data compres- evaluate the performance on more critical
sion on 4th order ambisonic source mate- test material such as very tonal or tran-
rial (25 channels), aiming for broadly sient signals, or those that do not contain
similar total bit rates around 200 and predominant sounds.
400 kbit/s for all three methods. The first
compression method encoded each of the SUMMARY
25 ambisonic channels separately, at either Ambisonic representation provides a rel-
8 or 16 kbit/s per channel, using an EVS atively versatile and compact way of stor-
mono encoder. The second method used ing or transmitting spatial audio scenes. It
SVD (the MPEG-H example), to gener- has become more popular in recent years,

700 J. Audio Eng. Soc., Vol. 69, No. 9, 2021 September