Zain version

Abstract;

In this study we explore how different audio distortions affect the quality and uniqueness of speaker
embeddings. By introducing forms of noise changing pitch and speed downsampling and other
manipulations to audio signals we measure the resulting impact, on speaker embeddings. Our main
focus is on measuring the distance between the modified embeddings to understand any changes.
This research offers insights into how resilient speaker embeddingsre against common audio
distortions and provides evidence of residual information encoded within them.

Methodology;

Data Collection;

We selected an audio sample from a speaker and used a pre defined and pre trained model to
convert it into its corresponding speaker embedding.

Model Used;

For extracting the speaker embedding we utilized a trained Pyannote audio model that was trained
on the Voxceleb dataset.

Audio Distortions;

To thoroughly analyze the impact of distortions on embeddings we systematically applied a series of

distortions to the audio. We then analyzed these modified embeddings based on their cosine
distance as visually inspecting their positions in vector space. The details of these distortions are as
follows.

1. Noise Addition; We added types and levels of noise (such as noise or ambient noise) to the audio
to evaluate if it affects the presence of noise conditions, in speaker embeddings.

2. Changing the pitch or speed of the audio without altering the content.

3. Modifying the duration of the audio without affecting its pitch.

4. Adding levels of reverberation to simulate room conditions.

5. Converting recordings to mono or vice versa. Intentionally removing one of the stereo channels.

6. Reducing the sample rate and/or bit depth of the audio which may result in a decrease, in quality.

7. Allowing only specific frequency bands to pass through or deliberately blocking bands using band
pass and band stop filtering techniques.

8. Introducing background music or speech to create scenarios with overlapping sounds.

Evaluation;
Each distorted audio was converted into a speaker embedding and we measured the distance
between the distorted embeddings as our main metric for assessing how each distortion affected
speaker representation.

The distance metric used is cosine distance, which is a measure of similarity between two vectors.
The lower the cosine distance, the more similar the vectors are. The cosine distance is calculated
using the following formula:

\begin{equation}

\cos(\theta) = \frac{A \cdot B}{\|A\| \|B\|}

\end{equation}

Results;

We generated bar plots to represent the impact of each distortion, at levels (factors).

Greater distances indicate a level of distortion impacting the speaker embedding. As depicted in
figures 1 2 downsampling has the impact, on the embedding while Noise and sound overlay have a
relatively moderate effect. On the hand pitch modification seems to have influence on speaker
embeddings. This suggests that there is a correlation between the changes in speaker embeddings
and the encoded information, within them.

Auto version
Considering the nature of the experiments, where various audio distortions are introduced and their
impact on speaker embeddings is measured it seems appropriate to use the term "Feature Ablation."

Feature Ablation

Research Question

In this section we address the research question; "What's the effect of different audio distortions on
the quality and distinctiveness of speaker embeddings?" This inquiry aims to assess how resilient
speaker embeddings are against distortions and understand what residual information these
embeddings encode.

Experimental Design
The methodology involves applying a range of distortions to a representative audio sample. We then
analyze how these distortions affect speaker embeddings extracted using a trained Pyannote audio
model. The distortions include noise addition, pitch and speed alteration time stretching, reverb
addition, channel manipulation, downsampling, bit depth reduction, band pass filtering and
background sound overlap. To measure the impact of each distortion we use distance to compare
distorted embeddings.

Results

When visualized through bar plots our results demonstrate that downsampling has an effect, on
speaker embeddings. Noise addition and sound overlay have an impact. Surprisingly though pitch
modification shows influence.

These findings imply that audio distortions directly impact speaker embeddings suggesting that
specific characteristics are encoded within the embeddings. The report includes representations
illustrating these effects along, with an analysis provided in the Appendix.

Conclusion;

This comprehensive study provides valuable insights into the resilience and vulnerabilities of speaker
embeddings. Understanding how audio distortions affect these embeddings is crucial for enhancing
systems that heavily rely on speaker related tasks. Furthermore it sheds light on the human concepts
embedded in these representations.

Future Directions;

Future research endeavors could involve incorporating a range of distortions exploring alternative
models for generating speaker embeddings and investigating how distortion affects performance in
other related tasks such, as gender recognition and speaker identification.