[[PageOutline]]

[=#audio]
= Audio =
This page will attempt to give examples of audio-based GStreamer pipelines using
GStreamer-1.0, with 'enhanced' encoders and decoders from the gstreamer-imx set of
plugins. Note that 'enhanced' just means that the software algorithms are
especially tuned for the imx6 family.

To see GStreamer-0.10 (deprecated) examples, please refer to

[http://trac.gateworks.com/wiki/Yocto/gstreamer/audio?version=2 this older revision
page].

For a description of Audio devices see:

* [wiki:ventana/audio Ventana Audio devices]

[=#gstreamer-alsa]
== GStreamer alsasrc and alsasink ==
The GStreamer {{{alsasrc}}} and {{{alsasink}}} provide audio capture and playback
to Advanced Linux Sound Architecture (ALSA) devices. ALSA is the modern device-
driver API for the Linux kernel.

The Gstreamer alsasrc and alsasink can be passed a {{{device}}} property which can
represent either the device name (reported from {{{aplay -L}}} for playback, or
{{{arecord -L}}} for record) (ie {{{sysdefault:CARD=sgtl5000audio}}}) or the hw:x,y
notation (ie {{{hw:0,0}}} for first device, first subdevice)

If not specified via the {{{device}}} property, the device used for alsasrc and
alsasink will depend on {{{/etc/asound.conf}}} and/or {{{~/.asoundrc}}}.

Playback and Capture also depends on your ALSA mixer ({{{amixer}}}) settings. See
[wiki:ventana/audio#Mixermuxgainvolume mixer settings] for more info.

[=#audio-output]
== Audio Output ==
Generally, a 'sink' plugin is one that will take an audio stream and send it to an
audio device. Please refer to [http://trac.gateworks.com/wiki/ventana/audio this
audio] page for more details on audio devices on the ventana platform.

A list of output sinks on the imx6:

* alsasink (recommended)
* autoaudiosink
* fakesink
* filesink

=== alsasink ===

This audio sink is our recommended audio sink. It can accept the following formats
via {{{audio/x-raw}}}: S8, U8, S16LE, S16BE, U16LE, U16BE, S24_32LE, S24_32BE,
U24_32LE, U24_32BE, S32LE, S32BE, U32LE, U32BE, S24LE, S24BE, U24LE, U24BE, S20LE,
S20BE, U20LE, U20BE, S18LE, S18BE, U18LE, U18BE, F32LE, F32BE, F64LE, F64BE

=== fakesink ===

This is a very useful debugging sink. It takes whatever data is given to it and
drops them. This might help debugging pipelines if problems ever arise.
=== filesink ===
This sink is very useful when a user wants to record and save audio to a file.

=== autoaudiosink ===

This GStreamer sink is not really an 'audio' sink in the traditional sense. Similar
to {{{playbin}}} and {{{decodebin}}}, this element selects what it thinks is the
best available audio sink and uses it. Generally not recommended.

=== Examples ===

The GStreamer {{{alsasink}}} element will playback a 1KHz tone to any Advanced
Linux Sound Architecture (ALSA) device driver:
{{{#!bash
gst-launch-1.0 audiotestsrc ! alsasink device="hw:0,0"
}}}
* see [wiki:ventana/audio#devices here] for details on specifying audio input and
output devices

[=#audio-input]
== Audio Input ==
An input source is anything coming from a capture device on the SBC, e.g. HDMI
audio in/analog audio in. Please refer to
[http://trac.gateworks.com/wiki/ventana/audio this audio] page for more details on
audio devices on the ventana platform.

A list of input sources on the imx6:

* alsasrc (recommended)
* autoaudiosrc
* audiotestsrc
* filesrc

=== alsasrc ===

This is the recommended audio capture source when capturing audio onto the SBC.
This plugin can output the following source types in {{{audio/x-raw}}}: S8, U8,
S16LE, S16BE, U16LE, U16BE, S24_32LE, S24_32BE, U24_32LE, U24_32BE, S32LE, S32BE,
U32LE, U32BE, S24LE, S24BE, U24LE, U24BE, S20LE, S20BE, U20LE, U20BE, S18LE, S18BE,
U18LE, U18BE, F32LE, F32BE, F64LE, F64BE

=== autoaudiosrc ===

This GStreamer source is not really an 'audio' source in the traditional sense.
Similar to {{{playbin}}} and {{{decodebin}}}, this element selects what it thinks
is the best available audio source and uses it. Generally not recommended.

=== audiotestsrc ===

This is a very useful plugin for testing. It can output 16-64bit {{{audio/x-raw}}}:
S16LE, S32LE, F32LE, F64LE

This can output audio signals from {{{sine}}} to {{{violet-noise}}}.

=== filesrc ===

This source is very useful when a user wants to playback audio from a file.
=== Examples ===
The GStreamer {{{alsasrc}}} element will capture audio from any Advanced Linux
Sound Architecture (ALSA) device driver:
{{{#!bash
gst-launch-1.0 alsasrc device="hw:0,0" ! "audio/x-
raw,rate=32000,channels=2,depth=16" ! \
audioconvert ! avimux ! filesink location=./audio.avi
}}}
* see [wiki:ventana/audio#devices here] for details on specifying audio input and
output devices

[=#audio-encoding]
== Encoding ==
Encoding is the term used to capture audio and encode it to a new format type.

A list of audio encoders on the imx6 platform:

* {{{gstreamer-imx}}} specific
* imxmp3audioenc
* Other GStreamer encoders
* vorbisenc
* mulawenc
* wavenc
* alawenc
* flacenc
* lamemp3enc

There are many more. You can search for your specified one by running a similar
search: {{{gst-inspect-1.0 | grep enc}}}.

=== imxmp3audioenc ===

This encoder can encode {{{audio/x-raw}}} S16LE to {{{audio/mpeg}}}.

=== vorbisenc ===

This encoder can encode {{{audio/x-raw}}} F32LE to {{{audio/x-vorbis}}}.

=== mulawenc ===

This encoder can encode {{{audio/x-raw}}} S16LE to {{{audio/x-mulaw}}}.

=== wavenc ===

This encoder can encode {{{audio/x-raw}}} S32LE, S24LE, S16LE, U8, F32LE, F64LE,
{{{audio/x-alaw}}}, and {{{audio/x-mulaw}}} to {{{audio/x-wav}}}.

=== alawenc ===

This encoder can encode {{{audio/x-raw}}} S16LE to {{{audio/x-alaw}}}.

=== flacenc ===

This encoder can encode {{{audio/x-raw}}} S24LE, S24_32LE, S16LE, S8 to {{{audio/x-
flac}}}.

=== lamemp3enc ===

This encoder can encode {{{audio/x-raw}}} S16LE to {{{audio/mpeg}}}.

=== Examples ===

This will take in an audio source from the mic and save it as an mp3.
{{{#!bash
gst-launch-1.0 alsasrc device="hw:0,0" ! audioconvert ! imxmp3audioenc ! filesink
location=/tmp/mic.mp3
}}}
* Note the audioconvert element is used to convert audio samplerate / bitwidth /
format (via software algorithms) as needed between two elements.

[=#audio-decoding]
== Decoding ==
Decoding is the term used to decode an encoded audio stream to a raw audio stream.

A list of audio encoders on the imx6 platform:

* {{{gstreamer-imx}}} specific
* imxuniaudiodec
* Other GStreamer encoders
* vorbisdec
* ivorbisdec
* a52dec
* mulawdec
* alawdec
* flacdec

=== imxuniaudiodec ===

This decoder can decode {{{audio/mpeg}}} to {{{audio/x-raw}}} S32LE, S24LE, S16LE,
S8.

=== vorbisdec ===

This decoder can decode {{{audio/x-vorbis}}} to {{{audio/x-raw}}} F32LE.

=== ivorbisdec ===

This decoder can decode {{{audio/x-vorbis}}} to {{{audio/x-raw}}} S16LE.

=== a52dec ===

This decoder can decode {{{audio/x-ac3}}}, {{{audio/ac3}}}, and {{{audio/x-
private1-ac3}}} to {{{audio/x-raw}}} F32LE.

=== mulawdec ===

This decoder can decode {{{audio/x-mulaw}}} to {{{audio/x-raw}}} S16LE.

=== alawdec ===

This decoder can decode {{{audio/x-alaw}}} to {{{audio/x-raw}}} S16LE.

=== flacdec ===

This decoder can decode {{{audio/x-flac}}} to {{{audio/x-raw}}} S8, S16LE,
S24_32LE, S32LE.
=== Examples ===
We can decode the MP3 audio saved previously and playback to the default audio
device with the following:
{{{#!bash
gst-launch-1.0 filesrc location=/tmp/mic.mp3 ! mpegaudioparse ! imxuniaudiodec !
alsasink
}}}

[=#audio-format]
== Audio Format Conversion ==
In specify the details of the audio format, you use a GStreamer {{{capsfilter}}}
and to convert it from one format to another you use the GStreamer
{{{audioconvert}}} element.
{{{#!bash
gst-launch-1.0 alsasrc device="hw:0,0" ! "audio/x-
raw,rate=32000,channels=2,depth=16" ! \
audioconvert ! avimux ! filesink location=./audio.avi
}}}

[=#audio-loopback]
== Audio loopback (Useful for testing audio input and output) ==
A simple aduio loopback test can take audio input from an input device and output
it to an output device.

loopback audio from the first audio card:

{{{#!bash
gst-launch-1.0 alsasrc device="hw:0,0" ! alsasink device="hw:0,0"
}}}

To send audio from the sgtl5000 (analog input) to the imxhdmisoc (HDMI output):
* using device/function numbers (ie from {{{aplay -l}}}):
{{{#!bash
gst-launch-1.0 alsasrc device="hw:0,0" ! alsasink device="hw:2,0"
}}}
* or, using names (ie from {{{aplay -L}}}):
{{{#!bash
gst-launch-1.0 alsasrc device="sysdefault:CARD=sgtl5000audio" ! \
alsasink device="sysdefault:CARD=imxhdmisoc"
}}}

To send audio from the tda1997x HDMI receiver (digital input) to the imxhdmisoc
(digital output):
{{{#!bash
gst-launch-1.0 alsasrc device="sysdefault:CARD=tda1997xaudio" ! \
"audio/x-raw,rate=44100" ! alsasink device="sysdefault:CARD=imxhdmisoc"
}}}
* Note here we need to specify the audio sample-rate as it can vary per input
stream (and gstreamer does not validate the rate). This rate must match the source
stream samplrate which can be found via sysfs {{{/sys/bus/i2c/drivers/tda1997x/2-
0048/audmode}}}. If your output device requires a different sample-rate than the
source input device, you need to perform a sample-rate conversion.