2020 - AJA-Nadja - Evaluation of HA in Everday Life Using EMA

AJA
Research Article
Evaluation of Hearing Aids in Everyday

Life Using Ecological Momentary
Assessment: What Situations
Are We Missing?
Nadja Schinkel-Bielefeld,a Patricia Kunz,b Anja Zutz,c and Bastian Buderd
Background: Ecological momentary assessment (EMA) because they considered it inappropriate, for example,
is a method to evaluate hearing aids in everyday life that during church service or when engaging in conversation.
uses repeated smartphone-based questionnaires to assess They answered fewer questions in conversations with multiple
a situation as it happens. Although being ecologically valid partners and were more likely to postpone questionnaires
and avoiding memory bias, this method may be prone to when not in quiet environments.
selection biases due to questionnaires being skipped or the Conclusion: Data for social situations will likely be
phone not being carried along in certain situations. underrepresented in EMA. However, these situations are
Purpose: This investigation analyzed which situations are particularly important for the evaluation of hearing aids, as
underrepresented in questionnaire responses and physically individuals with hearing impairment often have difficulties
measured objective EMA data (e.g., sound level), and how communicating in noisy situations. Thus, it is vital to
such underrepresentation may depend on different triggers. optimize the design of the study to find a balance between
Method: In an EMA study, 20 subjects with hearing avoiding memory bias and enabling subjects to report
impairment provided daily information on reasons for retrospectively on situations where phone usage may be
missed data, that is, skipped questionnaires or missing difficult. The implications for several applications of EMA
connections between their phone and hearing aids. are discussed.
Results: Participants often deliberately did not bring the Supplemental Material: https://doi.org/10.23641/asha.
study phone to social situations or skipped questionnaires 12746849
T
ypically hearing instruments are evaluated in well- are relevant for real life. Laboratory tests will not cover the
controlled laboratory experiments, which increases whole spectrum of situations experienced in everyday life,
reproducibility and statistical power of the experi- and the best hearing aid solution for one situation may not
ment. Due to their simplified structure, laboratory experi- be optimal in another, see, for example, Smeds et al. (2018).
ments also are often easier to interpret than real-life studies. Furthermore, laboratory tests often comprise constructed
However, it is not clear to what extent the tested situations situations and artificial tasks that are not typical for every-
day life.
a For tests outside the laboratory, subjects typically
R&D AAA SA ERL, WS Audiology, Erlangen, Bavaria, Germany
b
Faculty of Optics and Mechatronics, Hochschule Aalen - Technik
wear hearing instruments for a specific time and then fill
und Wirtschaft, Aalen, Baden-Württemberg, Germany out a retrospective questionnaire. This covers all situations
c
Department of Applied Natural Sciences, Technische Hochschule encountered during that time, but it is unclear how subjects
Lübeck, Schleswig-Holstein, Germany combine their impressions of different situations, and re-
d
Develappers GmbH, Dresden, Saxony, Germany sults can be impaired by memory bias (e.g., Bradburn et al.,
Correspondence to Nadja Schinkel-Bielefeld: 1987). In addition, specifics of the acoustic environment
nadja.schinkel-bielefeld@wsa.com for the rated situations are unknown to the experimenter.
Editor-in-Chief: Ryan W. McCreery
Editor: Gabrielle H. Saunders
Received October 1, 2019
What Is Ecological Momentary Assessment?
Revision received February 7, 2020 Ecological momentary assessment (EMA) is character-
Accepted April 17, 2020 ized by Stone and Shiffman (1994) as assessing phenomena
https://doi.org/10.1044/2020_AJA-19-00075
Publisher Note: This article is part of the Special Issue: 4th International Disclosure: The authors have declared that no competing interests existed at the time
Meeting on Internet and Audiology. of publication.
American Journal of Audiology • Vol. 29 • 591–609 • September 2020 • Copyright © 2020 The Authors 591
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Downloaded from: https://pubs.asha.org 220.255.167.16 on 09/21/2020, Terms of Use: https://pubs.asha.org/pubs/rights_and_permissions
while they occur using carefully timed repeated assess- the patient’s individual needs during fitting, fine-tuning, and
ments of subjects’ momentary behavior and experiences acclimatization (Timmer et al., 2017a).
in real time in their typical environments. This is particu-
larly useful when assessing rapidly fluctuating processes
as, for example, perception, affect, or pain (see Shiffman Representativeness of EMA for Real Life
et al., 2008, for a review). Typically, subjects report on
their current situation multiple times per day, often using EMA is assumed to be representative for everyday
a smartphone for data collection. Because questionnaires life. This means that (a) subjects assess situations that are
concern the current situation, or one very shortly before, part of their life and that matter to them and (b) the distri-
memory bias is eliminated or minimized. Participants cre- bution of assessed situations reflects the distribution of sit-
ate these reports repeatedly while living their daily lives, uations encountered in everyday life. The importance of
and so this is more representative of their life than labora- the second aspect varies with the research question (RQ).
tory experiments. Given that questionnaires usually do If a hearing aid feature is supposed to be assessed in a cer-
not only enquire about subjects’ experience with their hear- tain kind of situation, then the distribution of situations
ing instruments, but also collect information about the lis- may not matter as long as there are sufficient assessments
tening intent or the acoustic situation in that moment, they of situations in which this feature is active. However, for
are also context sensitive. Effects not visible when averaging example, for an assessment if hearing loss or wearing hear-
over all questionnaires may become significant when ana- ing aids changes the auditory reality of a person, it is criti-
lyzing only specific situations, for example, higher noise cal that EMA results sample all encountered situations
levels (Wu, 2017). Information about the current situation equally. The distribution of situations occurring in real life
can be subjective (i.e., subject responses) or objective, (e.g., is only reflected in questionnaires, if there is no selection
recordings of acoustic parameters). Thus, although EMA bias due to a specific trigger or due to subjects postponing
covers a variety of situations, it is also possible to analyze or not answering questionnaires in certain situations. If
specific acoustic situations separately. questionnaires (or single questions therein) are skipped
Today, modern technology allows for easy data in- at random, only the amount of data is reduced, but the dis-
put in EMA using smartphones and for the collection of tribution of assessed situations stays the same. However,
objective data on the acoustic environment. Many hearing it is important to be aware of any specific situations where
aids classify the acoustic situation in order to show this data the probability of data collection is systematically decreased
logging to the health care professional. The connection to in order to judge the reliability of the results and to mini-
the smartphone enables a continuous readout of these data, mize potential biases in the study design.
which can be timed accurately, such that it can be evaluated It should be ensured that there is no type of (im-
in relation to the subjective data. portant) situation where data are systematically missing.
Situations with social interactions are considered impor-
tant for well-being and for hearing aid research. Although
EMA in Hearing Aid Research
most hearing aids work well in quiet, speech in noise (SiN)
and Clinical Practice is a challenging condition, and many hearing aid algorithms
Hearing aid research using EMA has recently grown are developed specially for these situations. However, in
in popularity. An example of an early pen-and-paper ver- these situations, subjects may skip questionnaires due to
sion was reported by Walden et al. (2004). Further work considering mobile phone usage inappropriate. The majority
evaluated construct validity (Galvez et al., 2012; Timmer of hearing aid wearers are elderly people and may have an
et al., 2017a; Wu et al., 2015) and technical prerequisites to even stronger feeling of smartphone use being inappropriate
collect objective data (Kowalk et al., 2017) or employed in social situations than younger people (Berenguer et al.,
the method to understand more about the acoustic reality 2016; Kadylak et al., 2018). For this reason, possible selec-
of individuals with hearing impairment to design ecologi- tion biases—though also present in other areas—may be
cally valid laboratory experiments (Smeds et al., 2018). particularly critical for hearing aid research.
More recent studies compare different hearing aid modes This article analyzes which situations are likely to
(Jensen et al., 2019; Wu et al., 2019) or try to show the ben- be missed by EMA and explores options to minimize data
efit of hearing aids compared to unaided listening (Timmer loss in situations where smartphone use may be inappropriate.
et al., 2018; von Gablenz et al., 2019). Subjects in an EMA study were asked for situations and
Apart from being a research tool, EMA also has reasons that they may not provide feedback, meaning they
great potential as a clinical tool that helps patients to (a) did not bring the study phone, (b) missed random trig-
observe their difficulties in a structured manner. Publicly ger questionnaires, (c) configured the phone such that there
available apps serve the patients as self-help apps and, at are no questionnaires, or (d) did not initiate a question-
the same time, enable a large-scale data collection for med- naire themselves despite having something to report. The
ical research (e.g., Pryss et al., 2019; Schlee et al., 2016). effects of different kinds of triggers on the distribution of
The collected data can facilitate the communication be- acoustic situations as well as reported user intentions and
tween patients and health care providers and may enable possible effects of selection bias, particularly on the random
the acoustician to better adjust the hearing aid settings to trigger questionnaires, are analyzed.
592 American Journal of Audiology • Vol. 29 • 591–609 • September 2020

Research Questions questionnaire is to allow retrospective reporting for some
time after the event. Thus, it is analyzed:
Though the overall question in this article is whether
RQ4: Does the distribution of acoustic situations and
certain situations are missing in the collected EMA data,
listener intents differ between immediately answered ques-
this can be split in many subquestions. The analysis pre-
tionnaires and those that are delayed by more than 3 min?
sented in this article is divided in two parts: The first inves-
tigates a possible underrepresentation of social or other
situations in the subjective data and the second analyzes Objective Data
this underrepresentation in the objective data collected Objective data are only collected if the phone is in
by the EMA app. the vicinity of the hearing aids and a Bluetooth connection
can be established. Thus, also the continuously collected
objective data in the EMA app can be biased, if there are
Subjective Data situations in which subjects are less likely to carry their
The part on subjective data starts with an overview phone with them. If this is the case, the difference in acous-
of the collected questionnaires to put the following analysis tic situations should also be visible when comparing the
into perspective. Then, the following RQs are investigated: data collected by the EMA app to the long-term data log-
RQ1: What are the main reasons for not filling out ging of the hearing aid, which is independent of phone us-
questionnaires? Are these reasons unspecific or do they tar- age. This leads to the following two questions:
get specific situations and thus could give rise to a certain RQ5: What are the reasons for not carrying the study
situation being underrepresented in the collected data? phone or switching it off? Are these reasons unspecific or are
Here, not filling out questionnaires could mean not they related to specific situations or listener intents?
answering a prompted questionnaire (RQ1a), not using the RQ6: How does the distribution of acoustic situations
possibility to initiate a user-triggered questionnaire (RQ1b), differ between the data collected by the EMA app and the
or configuring the app in such a way that there are tempo- long-term data logging in the hearing aid?
rarily no questionnaire triggers (RQ1c). It is also analyzed All these effects are analyzed separately as they have
if subjects consider questionnaires as more annoying in different consequences for the study design. However, they
certain situations (RQ1d), as this could indirectly lead to may accumulate. Thus, it is analyzed how big the under-
fewer filled-out questionnaires in these situations. Also, representation of certain situation is in questionnaires with
Rintala et al. (2020) find that being disturbed by a ques- high data quality, that is, long questionnaires that were an-
tionnaire trigger is a predictor of lower compliance. swered in the situation:
If reasons for withheld feedback are situation spe- RQ7: How does the distribution of acoustic situations
cific, this should also be visible in the distribution of acous- differ between the entire wearing time measured by long-term
tic situations (as classified by the hearing aid data logging) data logging and randomly triggered, immediately answered,
in which questionnaires are answered. This leads to the long questionnaires?
question: Last, the subjective burden due to the EMA study
RQ2: How does the distribution of acoustic situations was evaluated:
and listener intents vary for different trigger questionnaires, RQ8: How large is the reported burden of subjects
and how does it differ from the objective data collected by due to the number of triggers, the number of questions, and
the EMA app during the entire wearing time? In particular, the need to carry the phone with them all the time?
is the distribution of acoustic situations for the random
trigger questionnaires representative for the entire wearing
time? Method
In the absence of biases, the distribution of acoustic The EMA App
situations in the random trigger questionnaires should fol-
low that of the entire wearing time. A loud environment Participants were provided with Samsung Galaxy S7
trigger was designed to capture more conversations in noise, smartphones (Android Version 7.0) containing the German
and users can initiate questionnaires themselves to report version of the Sivantos EMA app (Version 23e). Figure 1
on situations that are important to them independent of shows screenshots of the app (English version). An app
prompted questionnaires. locker from DoMobile was used to prevent any unwanted
If there is an underrepresentation of a particular situ- change of settings or the use of any other apps. SIM cards
ation, it potentially could be lessened by using a very short provided connection to the Internet for data transfer, but
questionnaire that can be filled out quickly and does not no minutes for calling.
interrupt the current activity of the subject as much. This
is analyzed by the question: Questionnaire: Triggers
RQ3: How does the distribution of acoustic situa- The EMA app prompted subjects at eight random times
tions and listener intents differ between short and long per day to fill out a questionnaire. The prompt consisted of
questionnaires? an Android notification including the standard notifica-
Another way to potentially facilitate giving feedback tion sound, vibration alarm, LED display, and flash noti-
on situations where it may seem inappropriate to fill out a fication, but no streaming into the hearing aids and no
Schinkel-Bielefeld et al.: Hearing Aid Evaluation in EMA: Missing Situations 593

Figure 1. Screenshot of (a) the welcome screen, (b) a single choice question, and (c) a multiple-choice question in
the Sivantos EMA app (English version). EMA = ecological momentary assessment.
repetition. Subjects could control the volume of the notifi- be determined as self-initiated triggers cancelled random
cation sound. The notification stayed on the display until triggers under certain circumstances). Extrapolating from
the questionnaire was opened by tapping on the notification this and from experience with four pilot subjects, this com-
or until the trigger was timed out. Subjects had the possibil- bination of triggers and instructions was expected to yield
ity to configure “do not disturb”-times where no trigger on average approximately four randomly and four subject-
was issued. To decide on the random trigger time, the remain- triggered assessments per subject per day.
ing minutes of that day, excluding the 15 min after the last
questionnaire and the “do not disturb”-times, were divided Current Situation Questionnaire
by the number of triggers remaining for that day. Then, a The app contains two kinds of questionnaires: one
random time within the first interval was chosen. Up to four about the current situation and a diary-like “end-of-day
additional “loud environment triggers” were only issued if questionnaire” where subjects recall the entire day. The
subjects were in environments with > 65–dB SPL broadband “current situation questionnaire” is divided into two parts:
level. The mean broadband level was measured every min- A mandatory short part consists of up to seven questions
ute by the hearing aids. Every 15 min, these data were col- about satisfaction with the hearing aids, the sound source
lected by the smartphone, and if the median of the mean people are listening to, possible background noises, and
broadband levels for the last 5 min exceeded 65 dB SPL, if the situation changed due to filling out the questionnaire.
data collection about active hearing aid algorithms was If participants had time, there were up to 20 additional op-
started and a loud environment trigger was issued 3 min later. tional questions in a longer questionnaire. See Appendix A
There was a minimum of 75 min between two “loud envi- for questions relevant to this article and the Supplemental
ronment triggers.” For all triggers, there was a minimum of Material S1 for a complete overview of all questions. The
15 min between the last questionnaire and a new trigger. questionnaire was adaptive, omitting nonrelevant follow-
Fifteen minutes after the trigger notification, the question- up questions.
naire could no longer be answered and the subject was asked
for the reason the assessment was missed (see Appendix A). End-of-Day Diary Entry
If not answered, the “missed trigger” question was discarded In order to evaluate possible reasons for missing data,
at the next current situation trigger, so that no records exist if subjects were asked to fill out an end-of-day questionnaire
several triggers in a row were missed. every evening, detailing connection problems and reasons
In addition to random triggers, subjects could initi- for not carrying the phone with them or not filling out
ate questionnaires themselves whenever they wanted to. questionnaires (see Appendix A). Subjects were reminded
Instructions asked them to fill out five self-initiated ques- by the app at a specific time of their choice, but could initi-
tionnaires per day (“It would be nice if you could initiate ate the end-of-day questionnaire at any time.
5 questionnaires per day yourself”). In a pilot study (Kerner
et al., 2018), compliance rates of 80% for self-initiated Retrospective Reporting and Timeouts
questionnaires and at least 50% for randomly triggered As questionnaires can be inconvenient or dangerous
questionnaires were obtained (exact numbers could not to fill out in the situation, subjects were given the possibility

to report on past situations. A timeout after 30 min for be overwritten 15 min later. This requires subjects to carry
retrospective reporting was chosen based on a previous the study phone with them to enable continuous data
EMA study by Kerner et al. (2018), where assessments of collection. If the connection is lost due to technical failure,
conversations with several people were sometimes delayed restarting the app, phone, or hearing aids will bring it back.
by up to 20–30 min, but only in 5% of all cases more than To ensure objective data are collected during the question-
that (see Figure 2). naires, random triggers were rescheduled if the Bluetooth
Despite retrospective reporting, objective data should connection between hearing aids and mobile phone was
be collected for the assessed situations. Asking subjects interrupted. Self-initiated questionnaires were independent
for an estimate of the time passed since the assessed situ- of the connection.
ation is not a reliable method of matching rated situations
to the objective data across time. Hence, subjects were
Long-Term Data Logging
instructed to start data collection by opening the question-
Inside the hearing aids, the acoustic situation is clas-
naire. They then had 30 min to finish the activity that was
sified once every minute and summed up over the entire
keeping them from responding directly and could fill out
wearing time. The summation of acoustic situations and
the questionnaire afterwards. This way, the time stamps of
of the wearing time is updated every 15 min. If the hearing
the objective data can be matched to the initiation of the
aid is restarted before these 15 min are complete, the data
questionnaire.
of the unfinished package are discarded (analogous to the
15-min package with information for each minute that are
Programs and Volume collected by the EMA app). When connecting the hearing
The hearing aids contained three hearing programs aids with the fitting software (here Connexx 8.5.15), the
differing in their directionality settings. The app automati- wearing time since the first fit together with the distribu-
cally changed the hearing program each night, such that tion of acoustic situations is displayed on the data logging
subjects had a different program each day. Subjects were page. Comparison of this long-term data logging with the
aware of the daily program change but had no knowledge data collected by the app reveals the amount of objective
of the current program and could not change programs data missing in EMA data and any resulting changes in
or volume themselves. Here, the data for the different pro- the distribution of acoustic situations. An overview to which
grams is pooled as any program comparison is beyond the RQs the objective EMA data and the long-term data log-
focus of this article. ging contributed is given in Appendix B.
Connection to Hearing Aids and Objective Data

Hearing instruments were connected to the mobile Participants
phone via Bluetooth with an icon in the upper right corner The EMA study was conducted in Erlangen and in
of the app (see Figure 1a) indicating the current connec- Lübeck with a total of 20 German-speaking experienced
tion state. Hearing instruments classify the acoustic envi- hearing aid wearers with moderate to severe hearing loss.
ronment into Speech in Quiet, SiN, Noise, Music, Car, or Twelve subjects (eight men, four women) were recruited
Quiet. Every minute, the distribution of the classified scenes from the Sivantos GmbH subject database and eight (six
is determined with a 4-s resolution and stored in 15-min men, two women) from the database of the Technische
packages. These packages are collected continuously Universität Lübeck. The study was approved by the ethics
throughout the wearing time. If the app does not collect committee of the Faculty of Medicine of the Friedrich-
the data in time due to a lost Bluetooth connection, it will Alexander-Universität Erlangen-Nürnberg (Application
Figure 2. Distribution of listening intent depending on the time between the situation and self-initiated reporting in a pilot ecological momentary
assessment study. Unpublished plot based on (Kerner et al., 2018).

327_18B) and the ethics committee of the Technische Uni- Experimental Procedure
versität Lübeck (application for a behavioral study with
Hearing Aids
adults with hearing impairment from October 11, 2018).
Subjects were fitted with a pair of Signia 7Nx M
Inclusion criteria were German-speaking adult hear-
hearing aids and closed click sleeves. Based on the Con-
ing aid wearers with at least 2 years of bilateral hearing
nexx insitugram, a hearing aid fitting with the proprietary
aid experience and a mean hearing loss in the frequencies
Nx fitting formula and experience level “experienced” as
0.5, 1, 2, and 4 kHz (pure-tone average [PTA]) between 35
well as training for own voice processing was performed.
and 75 dB HL (see Figure 3 for mean audiograms). Eigh-
Fine-tuning was added where necessary. Subjects wore the
teen of the 20 subjects also owned a private mobile phone.
hearing instruments for approximately 3 weeks. Depending
During the phone call to recruit subjects, the EMA proce-
on the availability of the subject and the experimenter,
dure was explained, and only subjects that specifically indi-
the number of days varied slightly. There was one early
cated that they felt comfortable with the mobile phone
termination after 7 days, because the subject considered
usage required for this study were invited.
participation in EMA too time consuming.
Characteristics for subjects on both sites can be
seen in Table 1. Due to logistical constraints, it was not
possible to recruit younger participants on both sites, Introduction of Subjects to the App
and it was prioritized to include some younger listeners During the first appointment, subjects completed
at least on the Erlangen site rather than to age match informed consent, were fitted with hearing instruments,
the two groups. Given that future hearing aid users will and were introduced to the EMA app. In addition to oral
likely be more accustomed to mobile phone usage in instructions, a detailed written instruction of all app and
public, a larger age range was considered more represen- mobile phone functions relevant to the study were given
tative for future studies. to the participant, as well as the phone number of the ex-
Participants were paid 20 cents for every short ques- perimenters to receive help in case of difficulties. Many
tionnaire and additional 50 cents when answering a long EMA-related publications stress the importance of training
questionnaire, but no more than 10 Euros per day. Experi- to ensure that subjects understand the questions and can
menters in Erlangen and Lübeck were in close alignment operate the app, see, for example, Christensen et al. (2003).
regarding their manner of instructing subjects by frequent For training and familiarization, subjects filled out the
discussions on the phone and by exchanging written notes current situation questionnaire in the laboratory for six
on what to say to subjects in order to assure that test con- different sound scenarios (one speaker in quiet, one speaker
ditions were as similar as possible between the two labora- in noise, and a group conversation in noise, each for two
tories and across subjects. Furthermore, subjects received different hearing programs) with the possibility to ask ques-
written instructions. tions and receive help from the experimenter.
Despite this, Lübeck participants in contrast to Er- After a week, subjects were asked if they had any
langen participants often did not switch off their hearing problems handling the app. Furthermore, the data were
aids when not wearing them. Thus, the eight subjects from continuously monitored to check that subjects had indeed
Lübeck are excluded from any analysis that comprises the filled out questionnaires. After 3 weeks, subjects returned
acoustic environment during the complete wearing time the study phone and filled out a final questionnaire inquir-
(i.e., Figure 7a, Figure 11, and Table 2). ing about their experience with the EMA app.
Analysis and Statistics

Figure 3. Subjects’ average hearing loss. Error bars denote 95% When analyzing the objective data of the acoustic
confidence intervals. situations for a questionnaire, only the 3 min before the
start of the questionnaire were taken into account, because
the process of answering the questionnaire may have changed
the situation, making the time before the questionnaire
more informative. Indeed, even when subjects indicated that
the situation did not change, there was a significant drop
in level the first minute after the start of the questionnaire.
When analyzing distributions of situations or ques-
tionnaire responses, the distributions for each subject were
computed separately and then averaged over all subjects.
This way, each subject is weighed equally, and the general
tendency of subjects to avoid phone usage and answering
questionnaires in certain situations can be assessed.
All significance tests were performed using one-
sided paired-samples permutation tests using bootstrapping
(Efron, 1982) comparing mean values. For multiple com-
parisons, Benjamini–Hochberg corrections were performed

Table 1. Subject characteristics at the two sites.
Erlangen (N = 12) Lübeck (N = 8)

Characteristics M Range M Range
Age (years) 58.3 24–79 71.9 60–82

Hearing aid experience (years) 21.6 2–33 14.0 11–32
Mean pure-tone average (dB HL) 54.1 41.9–69.4 55.5 35.6–75.0
to control for the familywise error rate. Error bars denote questionnaires with no subject having less than 59% of
95% confidence intervals determined using bootstrapping long questionnaires.
with 10,000 repetitions. Effect sizes (Hedges’ g) were com- Median duration of an assessment ranged from 14 to
puted using the effect size toolbox (Hentschke & Stüttgen, 111 s (average: 47.1 s) for short questionnaires and from
2011). 52 to 436 s (average: 140.2 s) for long questionnaires. On
average, this lies within the 3-min duration recommended
by Stone and Shiffman (2002). Larger values may also be
Results caused by subjects initiating the questionnaire and answer-
After an overview of the collected data, different ing it after the completion of their current activity.
ways in which responses may lead to a distorted picture of The average distribution of triggers comprised 53% of
the subject’s life are presented. Reasons for missed subjec- assessments from random triggers, 14% from loud environ-
tive data due to subjects systematically delaying responses, ment triggers, and 33% from self-triggered ones.
answering only few questions, or not answering at all in
certain situations were analyzed, as well as missing objec- Loss of Subjective Data
tive data when subjects do not take the smartphone with
them. There could be several reasons for missing subjective
data resulting in a distorted picture of a subject’s auditory
reality, for example, subjects not answering questionnaires
EMA Participation and Collected Subjective Data in certain situations.
Overall, 3,752 questionnaires for the current situa-
tion were collected. The number of answered self-triggered Missed Questionnaires (RQ1a)
and random trigger questionnaires varied greatly between When a random trigger or a loud environment trig-
subjects (see Figures 4a and 4b). The average number of gered questionnaire was not answered within 15 min, sub-
daily questionnaires ranged from 2.6 to 12.8 (M = 9.0). jects were asked for the reason it was missed (see Figure 5a).
The subject that answered the least questionnaires stated “It was inappropriate to fill out a questionnaire in that situa-
technical difficulties as the reason for the low number. tion” was the most frequent reason for not filling out a ques-
With an average of 85%, there was a large fraction of long tionnaire (46%), triggering the follow-up question in what
Figure 4. (a) Number of daily questionnaires for the current situation and (b) distribution of the different questionnaire triggers per subject.

Figure 5. (a) Reasons a notification to fill out a questionnaire has not been answered and (b) specification in which way filling out a questionnaire
may have been inappropriate. HIs = hearing instruments.
way it would be inappropriate (see Figure 5b). The notion reveal to what extent social situations are underrepresented
of being impolite was a much more common reason than in the questionnaire responses. If there was no bias in the
having no time or not being in the mood, though this last response behavior, the distribution of acoustic situations
answer may be avoided as it is socially unfavorable (Van in random questionnaires should equal that of the entire
de Mortel, 2008). In 35% of cases, safety reasons, such as wearing time. Thus, the distribution of acoustic environ-
driving a car, prevented subjects from answering question- ments determined by the classifier of the hearing aids was
naires. Unheard questionnaire triggers are responsible for compared between the entire wearing time recorded by the
36% of missed questionnaires (see Figure 5a). EMA app and the 3 min before the random questionnaires.
Data for the 3 min before are available for 73% of the random
“Do Not Disturb”-Times and Withheld Feedback (RQ1b–d) trigger questionnaires, 67% of the self-initiated question-
While missed prompted questionnaires seem to naires, and 82% of the loud environment questionnaires.
cause underrepresentation of social situations in random Averaging the mean ratios for each subject and com-
trigger questionnaires due to mobile phone usage being paring them via permutation testing (see Figure 7a) showed
impolite, there could be further data missing in case sub- a significant underrepresentation of SiN and Car in ran-
jects configure “do not disturb”-times for specific situations. dom trigger questionnaires compared to the total data re-
Similarly, self-triggered questionnaires could systemati- corded by EMA ( p ≤ .05, Hedges’ g = 0.80 and g = 0.71,
cally lack a particular kind of situation if subjects do not respectively). Overrepresentation of Quiet in these question-
feel comfortable filling out questionnaires under certain naires is only trend significant after a Benjamini–Hochberg
circumstances. correction ( p = .06, Hedges’ g = 0.38). Although it is un-
This was evaluated by the end-of-day questionnaire. certain which situations are classified as SiN, many social
When subjects reported either not having given feedback situations are expected to fall into this category. Hence,
despite their desire to report something, to have configured underrepresentation of the SiN and Car classes are consis-
a “do not disturb”-time or to have considered answering tent with the subjective responses in the end-of-day ques-
questionnaires annoying, they were asked for a reason. tionnaire, indicating that, in social situations and for safety
Perceiving questionnaires as annoying does not directly reasons/while driving, feedback was often withheld (see
lead to data loss, but subjects may be less likely to report Figures 5 and 6).
when they find reporting burdensome. The obtained text The loud environment trigger was introduced to obtain
comments were clustered into categories (see Figure 6). This more conversation in noise situations. Indeed, it increased
shows that social situations are by far the most frequent the fraction of SiN- and Car-classed situations compared
reason for missing questionnaire data. A few responses to both other triggers and the entire wearing time (see
were also withheld when subjects did sports or when it was Figure 7a). Thus, it can help to compensate the lack of
not safe to use a mobile phone. Nonspecific reasons such SiN-classed situations in the random trigger.
as wanting quiet, which do not target specific situations, Similarly, different triggers lead to different distri-
only comprised a small fraction of the missing data. butions of listening intents, that is, responses to the ques-
tion “Whom or what are you listening to?” (see Figure 7b).
Representativeness of Questionnaire Data Compared to the random trigger, the loud environment
for Different Triggers (RQ2) trigger comprises more conversations with more than one
The end-of-day questionnaire showed that subjective person ( p ≤ .01, Hedges’ g = 0.99) and fewer TV/radio
data are missing primarily in social situations, but did not ( p ≤ .01,Hedges’ g = 0.77). This is consistent with the

Figure 6. Responses from the end-of-day questionnaire regarding situations in which questionnaires were not answered or were annoying.
overrepresentation of SiN in the loud environment trigger. situations (Hedges’ g = 0.61) and a larger fraction of long
Compared to the random trigger, self-initiated question- questionnaires for Music (Hedges’ g = 0.47). However, de-
naires tend to give fewer situations without active listening spite its intermediate size effect, after a Benjamini–Hochberg
(Hedges’ g = 0.30) and more “other” responses (Hedges’ correction, this is no longer significant.
g = 0.49). However, this is no longer significant after a In the same way, the question “Whom or what are
Benjamini–Hochberg correction. you listening to?” was analyzed (see Figure 8b). Except for
the answer “not actively listening to anyone/anything,” all
Short and Long Questionnaires (RQ3) responses indicate active listening. Though not significant,
If subjects answer only the short questionnaire in conversation with several people is the only active listening
certain situations, this results in less detailed information situation underrepresented in long questionnaires (Hedges’
about the situation and the hearing aid performance therein. g = 0.31), whereas “music,” “radio/TV,” and “conversa-
The following analyzes situations in which subjects are tion with one person” are significantly overrepresented
more likely to answer only the short questionnaire. ( p ≤ .05, Hedges’ g = 0.50, 0.95, and 0.73, respectively).
Averaging the obtained distribution of acoustic situa-
tions over subjects and performing pairwise comparisons Delayed Questionnaires (RQ4)
between short and long questionnaires (see Figure 8a) Selection bias can also be caused by subjects delaying
indicate a larger fraction of short questionnaires in SiN their answer in certain situations until the situation has
Figure 7. (a) Distribution of the acoustic environment determined by the classifier inside the hearing aids in the 3 min before the questionnaire
and throughout the entire data collected by ecological momentary assessment (for the 12 Erlangen subjects only). Only significant differences
between random trigger questionnaire and wearing time are marked. (b) Distribution of different listening intents (as indicated in the questionnaire)
for different triggers (all 20 subjects). SiQ = Speech in Quiet; SiN = Speech in Noise.

Figure 8. (a) Average fraction of short and long questionnaires filled out in different acoustic situations and (b) for different answers to the
question “Whom or what are you listening to?” SiQ = Speech in Quiet; SiN = Speech in Noise.
changed. Of all answered random trigger questionnaires, Analysis of different listening intents only revealed signifi-
70.6% were answered in the first minute after the trigger. cantly more immediate questionnaires for “TV/radio” ( p ≤
This varied between subjects, but no participant answered .05, results not shown). Corresponding to the overrepre-
less than 40% of all filled-out questionnaires in the first sentation of “Car” in the acoustic situation, the same anal-
minute (see Figure 9a). ysis for subjectively reported background noises (data not
To analyze the situation dependence of the delay in shown) shows more “traffic noise” and “machinery” in
answering, questionnaires delayed by more than 3 min were delayed questionnaires ( p ≤ .05, Hedges’ g = 0.86 and
contrasted with questionnaires filled out within the first 0.31, respectively). In “music/TV” and “silence,” more im-
3 min after the trigger. The 3-min threshold was chosen as mediate responses were obtained (p ≤ .05, Hedges’ g = 0.34
a trade-off between a longer delay where changes in the and 0.47). An increase of “voices/other people” in the back-
distribution of acoustic situations may be stronger and ground of delayed questionnaires is not significant (Hedges’
a short delay such that there are sufficient questionnaires g = 0,31).
that are delayed. The choice was independent of the ex-
pected memory bias for the delay. Fractions of different
acoustic situations for immediate and delayed questionnaires Objective Data
averaged over all subjects show an overrepresentation of The EMA app requires a Bluetooth connection to
“Quiet” (p ≤ .05, Hedges’ g = 0.82) and a trend for under- collect data from the hearing aids. This can be interrupted,
representation of “Car” (p = .06, Hedges’ g = 0.61) in the especially as subjects do not always have the study phone
immediately answered questionnaires (see Figure 9b). with them. The average daily wearing time of hearing
Figure 9. (a) Fraction of all received random trigger questionnaires for the current situation versus time passed since trigger notification.
(b) Distribution of acoustic situations for random trigger questionnaires that were answered immediately or with more than 3-min delay
after the trigger notification. SiQ = Speech in Quiet; SiN = Speech in Noise; t.s. = trend significant.

aids according to long-term data logging was 13.2 hr (SD = From the objective data, it is not possible to distinguish
3.3 hr) for the Erlangen subjects. The measured 19.4 hr between data loss due to interfering sources or due to the
(SD = 6.3 hr) for the Lübeck participants do not reflect distance between phone and hearing aids. On average on
their actual wearing time as some of them did not switch 80% of all days, subjects did not report noticing any loss
off hearing aids at night. However, this value can still be of Bluetooth connection. If a loss occurred, usually the con-
used to determine the ratio of data collected by the EMA nection was easily reestablished. Only two subjects reported
app, which, averaged over all 20 subjects, is 75% of having trouble reestablishing the Bluetooth connection,
the wearing time with no significant difference across the affecting in total only 2% of all study days. A loss of Blue-
two sites. tooth connection only reduced the number of random
triggers if it persisted for such a long time that it was
Data Loss Due to the Phone Being Switched Off not possible to reschedule the remaining trigger for that
or Not in the Vicinity (RQ5 + 6) day. When asked if Bluetooth failures occurred more
If the data loss was only due to forgotten smartphones, often in a specific situation, it was primarily reconnection
it should be random and affect all situations equally. How- problems after switching on hearing aids or after returning
ever, if subjects deliberately do not bring the study phone to the vicinity of the phone that were mentioned. Thus,
to specific situations, then the collected data are not rep- there is no indication that data loss due to technical fail-
resentative of the distribution of experienced situations. ures contributed to the underrepresentation of SiN in the
According to subjects’ responses in the end-of-day ques- collected data.
tionnaire, nonspecific reasons such as forgetting the phone
or wanting quiet are important reasons, but the most fre- Relation Between Acoustic Situation and Listening Intent
quently missed situations are again social situations (see The classifier data on the acoustic situation does
Figure 10). not measure the listening intent, but there is some relation
Comparing the long-term data logging (entire wearing to it: In the 3 min before the current situation question-
time) to the complete objective EMA data collected over naires, 55% of SiN situations correspond to conversations
the course of the study (see Figure 11) reveals overrepresen- (33.5% to conversations with several). However, given that
tation of quiet environments and underrepresentation of subjective responses imply that social situations are under-
SiN situations in the EMA data ( p ≤ .01, Hedges’ g = 0.22 represented in the questionnaire responses, presumably
and 0.26). While the effect is small on the group level, for more than 55% of all SiN situations encountered through-
single subjects, up to 71% of the data for SiN situations out the wearing time correspond to social situations.
are missing. As social situations usually comprise SiN sit-
uations, this is consistent with the subjective responses in Accumulation of Effects (RQ7)
Figure 10. So far, different ways of missing information or
receiving lower quality information have been analyzed
Instability of Bluetooth Connection separately, as they have different implications for possible
A loss of objective data could also be caused by a countermeasures. However, all these effects accumulate,
technical failure leading to a loss of the Bluetooth connection. and compared to the long-term data logging, immediately
Figure 10. Reasons for the mobile phone being switched off or not in the vicinity of the hearing aids
as indicated in the end-of-day questionnaire.

Figure 11. Distribution of acoustic situations for the total wearing the bulky protection case, study phones did not fit into their
time (long-term data logging) and the data recorded in the EMA app.
Analysis for the 12 subjects from Erlangen only. EMA = ecological
pockets.
momentary assessment; SiQ = Speech in Quiet; SiN = Speech in
Noise.
Discussion
After a summary of results and a discussion of
the limitations of this study, the implications for differ-
ent EMA RQs are analyzed and methods to minimize
biases are proposed. Finally, the potential of EMA is
discussed.
Underrepresentation of Specific Situations

This study analyzed possible reasons for missing sub-
jective data. The results indicate that, in social situations,
subjects are more likely to skip random trigger question-
naires (RQ1a), to not initiate self-triggered questionnaires
(RQ1b), to configure “do not disturb”-times (RQ1c), to
find it generally annoying to fill out a questionnaire (RQ1d),
and to not take the study phone with them (RQ6). This
selection bias is visible in the responses of the end-of-
answered, randomly triggered long questionnaires contain day questionnaire (RQ1b–d and RQ5; see Figures 6 and
only a third of the feedback expected for SiN situations 10, respectively) as well as in the lack of SiN situations in
(see Table 2). When combining random and loud environ- the objective EMA data for the wearing time (RQ6; see
ment triggers, this increases to two thirds of the expected Figure 11) and when answering questionnaires (RQ2; see
SiN situations. Figure 7a). As a result, high-quality responses, that is, long
Also, when averaging the fractions over subjects, questionnaires that were answered immediately after the
SiN and Noise are underrepresented ( p ≤ .01, Hedges’ g = random trigger, contain only a third of the SiN situations
1.45 and 1.02) and Quiet is overrepresented ( p ≤ .01, that were to be expected if the distribution of acoustic situ-
Hedges’ g = 0.94) in the immediately answered, randomly ations reflected that of the entire wearing time (RQ7; see
triggered long questionnaires compared to the entire wear- Table 2). Generally, subjects were not annoyed by the num-
ing time. ber of triggers and questionnaires but by having to carry
the phone at all times (RQ8; see Figure 12).
In the short questionnaire, there was a trend toward
Subjective Burden Due to Study Participation (RQ8) a higher fraction of SiN situations and more conversa-
After the study, the subjects’ burden due to EMA tions with several people than in the long ones, whereas
was evaluated by a questionnaire. Most subjects considered most other active listening conditions were more frequent
the number of triggers and questions to be “just right” (see in the long questionnaires (RQ3; see Figure 8). If subjects
Figures 12a and 12b). This is consistent with the large frac- encounter hearing problems, they might be eager to report
tion of long questionnaires (see Figure 4). However, it was them in detail in the long questionnaire. This could explain
perceived as being a burden to carry around the smartphone why most active listening conditions are more frequent in
at all times (see Figure 12c). the long questionnaire. However, while there are likely
Though not an explicit question in the exit ques- problems to report in conversations with more than one
tionnaire, several subjects spontaneously reported that the person, this may not be visible in the fraction of long ques-
3-week study period was too long. Subjects would prefer tionnaires as clearly, as it is counterbalanced due to the
the app on their own phone and complained that, due to feeling of inappropriateness of using a smartphone in these
situations.
Questionnaires are more likely to be filled out imme-
Table 2. Odds ratios for situations observed in the total wearing diately after the trigger when subjects are in a quiet envi-
time and directly before a long questionnaire that was answered
within the first minute after the trigger. ronment or watching television but tend to be postponed
when traveling in a car (RQ4; see Figure 9b). Therefore,
Trigger SiQ SiN Noise Music Car Quiet allowing retrospective responses (at least some time after
the trigger) and using short, fast-to-fill-out questionnaires
Random 0.93 0.33 0.54 0.92 0.68 1.77 can be methods to acquire more responses in situations
Random + loud environment 0.92 0.65 0.87 0.81 1.55 1.18 where mobile phone usage may be considered inappropriate.
Note. Analysis for the 12 subjects from Erlangen only. SiQ = Speech However, both the answers in the end-of-day question-
in Quiet; SiN = Speech in Noise. naire (RQ5; see Figure 10) and the comparison of the objec-
tive EMA data with the long-term data logging (RQ6; see

Figure 12. Responses of the exit interview regarding the burden due to participation in the ecological momentary assessment study.
Figure 11) suggest that subjects often do not bring the study insights into the lifestyle of hearing aid wearers, see, for
phone to social situations and thus cannot fill out any example, Timmer et al. (2017b). The dominance of con-
questionnaires. versation in SiN situations observed in this study makes
Here, the lack of feedback in social situations is it likely that the lack of SiN for the questionnaires is
emphasized, as hearing well is particularly important and caused by subjects’ tendency to avoid answering question-
challenging in these situations. However, it is also difficult naires in social situations.
to obtain responses when subjects are doing sports, running
errands, driving, or operating heavy machinery (RQ1 and Complete Avoidance
RQ5; see Figures 6 and 10). This is consistent with Rintala As long as subjects do not avoid a situation completely,
et al. (2020) who, in a lagged analysis of seven EMA stud- there is at least some information available. However, cer-
ies with a total of more than 1,300 subjects, found lower tain situations may be avoided entirely based on the belief
compliance when subjects are outside the home. In the that hearing aid performance will be insufficient in that
current study in about a third of the cases, people gave a situation. Other situations might be experienced but never
nonspecific reason for not bringing the phone, for example, reported, for example, because mobile phones are not allowed
“phone forgotten” or “wanting peace.” Assuming this af- at work. Two subjects reported wearing their own hearing
fects all situations equally, this should not distort the image aids instead of the study devices, for example, for streaming
of the subject’s acoustic reality. music (not possible with study devices) and during concerts
without trying the study devices for those situations. In
all these cases, information on relevant situations is not
Limitation of the Study available to the experimenter at all.
Reliability of Subjects’ Responses Situations in everyday life are so diverse and depend
Here, subjects’ responses in the end-of-day ques- on so many factors, including acoustic environment, physi-
tionnaire were used to judge which situations are missing. cal and cognitive ability, personality, intent, and mood of
Users may not always have responded honestly. For exam- the participant, that even the large amount of data collected
ple, one subject did not indicate any behavior that led to in EMA will never fully characterize a situation. Hence,
data loss. Due to the tendency to give socially acceptable even if data are obtained for each situation characterized
responses (Van de Mortel 2008), it seems more likely that by the EMA app, that does not guarantee that no type of
these events were not reported than that they did not situation is missed completely.
occur.
Implications for Different Goals of an EMA Study
Interpretation of the Classifier The impact of systematically missing situations de-
This study relies on the classifier inside the hearing pends on the RQ.
instruments to indicate underrepresentation of certain situ-
ations in the questionnaires. Classifiers are based on acous-
tic parameters and do not measure listening intent (despite Analysis or Comparison of Acoustic Reality
displaying relations to it). However, objective data provide When analyzing the frequency of certain situations,
the only possibility to receive continuous feedback from experimenters need to consider that (depending on the
hearing aid wearers without interfering with their behavior, subject’s personality) some situations will likely be under-
and it is typically used by hearing aid companies to gain represented. Finding prototype situations for laboratory

tests may not require an exact representation of the occur- a situation where filling out the questionnaire is not appropri-
rence of situations. However, comparisons of the distribution ate and then complete it afterward did not work well: Many
of acoustic situations for different user groups (e.g., normal subjects reported to not have understood this concept during
hearing and hearing impaired) could be strongly distorted the first appointment, probably due to the large amount of
by missing data. new information. After a repeated explanation at the second
appointment, several subjects tried this but reported that the
Comparison or Evaluation of Hearing Aids timeout after 30 min was too soon. With a later time out
When comparing subjective ratings for hearing aid and more training for the subjects, it may be a possibility to
features that are primarily helpful in social situations, achieve correctly timestamped retrospective responses that
statistical power will be reduced by the lack of feedback can be matched to the corresponding objective data.
in these situations. EMA studies often result in a large Memorized questionnaires and memory bias. Several
number of ratings. However, pooling over all situations subjects reported knowing the questions by heart, answering
will typically not show any significant differences between them mentally in the situations, and then entering the re-
hearing aids due to the large variance in situations and sponses later. This likely results in less memory bias than
ratings, and differentiation according to situation often when having to recollect the situation without having mem-
leaves little data for each individual situation. Given that orized the responses. Unless cognitive overload prevents
ratings are not missing in all SiN situations, underrepresen- subjects from answering questions mentally in the situation,
tation of SiN situations may be (at least partially) com- instructing subjects to use this strategy could work to reduce
pensated for by a loud environment trigger that prompts memory bias for retrospective responses in future EMA
subjects more often in such situations. studies. Jenstad et al. (2018) found that a secondary task
Comparing subjective ratings is only one way of eval- has no influence on ratings. However, noise was stated as
uating hearing aids. Several internal studies have shown being louder when rated in the situation as opposed to
that, when their devices use a narrow directional micro- shortly afterward. Other ratings were not affected.
phone beam, hearing aid users tend to spend more time Mehrotra et al. (2015) discuss possibilities to issue
in loud situations. Analysis of subjects’ tendency to avoid questionnaires in situations when they are less interruptive.
particular situations with certain hearing aid settings will Although their method may increase response rates, it is
be decreased in statistical power if objective data for situa- not clear that it yields higher quality information on the
tions that are prone to avoidance is missing. Hence, data situations relevant for the RQ. Furthermore, the charac-
collection should be ensured also when the phone is tempo- ter of a conversation can be changed by the presence of
rarily not in the vicinity of the hearing aids. a mobile phone (Dwyer, 2017; Misra et al., 2016). Thus, if
input from subjects is required, there is always the risk of
EMA as a Consumer App altering the analyzed situation.
When using EMA as a tool to facilitate communica- As questionnaires should interrupt real life as little
tion between hearing aid wearers and health care profes- as possible, they need to be short and cannot use multiple
sionals, it is probably sufficient to have some example questions to evaluate a single construct, as it is often done
situations where difficulties with hearing aids arise rather in retrospective questionnaires. Thus, to obtain accurate
than complete coverage of the persons’ everyday life. EMA information, it is important to ensure that subjects under-
can help to obtain a very precise picture of some problem- stand the questionnaire before starting the home trial.
atic situations, and the acoustician can fine-tune the hearing
devices accordingly. To obtain an overview of the lifestyle Objective Data
of the hearing aid wearer and their needs, it might be easier In order to recover the objective data lost when sub-
to use the data logging usually shown in the fitting soft- jects do not carry the phone, the information on the acous-
ware or to ask him or her directly. While the EMA app is tic situation should be saved inside the hearing aid for an
expected to help customers in choosing the optimal device extended period of time, such that it can also be retrieved
and settings, presumably the number of triggers and ques- if the connection between phone and hearing aids is lost
tions has to be reduced in order to not burden them too for several hours. Subjects’ motivation for wearing their
much throughout the testing period of their hearing devices. own devices instead of study devices can be minimized by
providing a good fitting, including careful fine-tuning and
using study devices that provide at least the functionality
Methods to Minimize Biases of their own devices.
Subjective Data App on own phone. Subjects wished to have the app
Allowing subjects to report with some delay should on their own phone such that they do not need to carry a
enable them to also assess situations where they cannot or second device. This was also found by Burke et al. (2017).
do not want to use their mobile phones. However, this may Providing the app on the subjects’ phones would lead to a
increase memory bias and makes it difficult to relate the stronger incentive to take the phone with them. Further-
times of reported situations to the objective data. more, their private smartphone probably is selected for the
Recovering the time of the reported situations by individual needs of each subject regarding phone size, font
instructing subjects to already initiate the questionnaire during size, and so forth. Hicks et al. (2010) show that familiarity

with the phone increases data quality. However, an app that References
runs reliably on many different smartphones requires a
Berenguer, A., Goncalves, J., Hosio, S., Ferreira, D., Anagnostopoulos,
larger development effort, and interaction with other personal T., & Kostakos, V. (2016). Are smartphones ubiquitous? An
apps on the phone could impair the stability of the EMA in-depth survey of smartphone adoption by seniors. IEEE
app (Burke et al., 2017). Consumer Electronics Magazine, 6(1), 104–110. https://doi.org/
10.1109/MCE.2016.2614524
Bradburn, N. M., Rips, L. J., & Shevell, S. K. (1987). Answering
Potential of EMA autobiographical questions: The impact of memory and infer-
ence on surveys. Science, 236(4798), 157–161. https://doi.org/
Although several shortcomings of the EMA method 10.1126/science.3563494
are described here, it remains one of the best tools to ensure Burke, L. E., Shiffman, S., Music, E., Styn, M. A., Kriska, A.,
that study results are valid in the everyday life of hearing Smailagic, A., Siewiorek, D., Ewing, L. J., Chasens, E., French,
aid wearers. No research method is free of biases. Under- B., Mancino, J., Mendez, D., Strollo, P., & Rathbun, S. L.
standing potential biases can assist in designing research (2017). Ecological momentary assessment in behavioral research:
studies to keep biases to a minimum and in drawing the Addressing technological and human participant challenges.
Journal of Medical Internet Research, 19(3), e77. https://doi.org/
correct conclusions from the results.
10.2196/jmir.7138
Compared to retrospective questionnaires, memory
Christensen, T. C., Barrett, L. F., Bliss-Moreau, E., Lebo, K., &
bias will be reduced in EMA even if questionnaires are an- Kaschub, C. (2003). A practical guide to experience-sampling
swered after the assessed situation. In paper-based diaries, procedures. Journal of Happiness Studies, 4, 53–78. https://
back filling can be problematic (Stone et al., 2002), which is doi.org/10.1023/A:1023609306024
easily detectable in EMA. Furthermore, the objective data Dwyer, R. (2017). Smartphone use undermines enjoyment of face-
provide a large amount of information to the experimenter to-face social. The Faculty of Graduate and Postdoctoral Studies,
without any extra burden on the subject. This information The University of British Columbia. https://doi.org/10.1016/
facilitates the interpretation of subjective responses com- j.jesp.2017.10.007
pared to traditional questionnaires where no objective in- Efron, B. (1982). The jackknife, the bootstrap, and other resampling
formation on the acoustic environment of the assessed plans (Vol. 38). Siam. https://doi.org/10.1137/1.9781611970319
Galvez, G., Turbin, M. B., Thielman, E. J., Istvan, J. A., Andrews,
situation is available.
J. A., & Henry, J. A. (2012). Feasibility of ecological momen-
tary assessment of hearing difficulties encountered by hearing
aid users. Ear and Hearing, 33(4), 497–507. https://doi.org/
Conclusion 10.1097/AUD.0b013e3182498c41
Hentschke, H., & Stüttgen, M. C. (2011). Computation of mea-
We showed that social situations are underrepresented sures of effect size for neuroscience data sets. European Journal
in the objective and subjective data collected with the EMA of Neuroscience, 34(12), 1887–1894. https://doi.org/10.1111/
app. Depending on the RQ, this could distort the results j.1460-9568.2011.07902.x
of a study. Hearing aid problems in social situations may Hicks, J., Ramanathan, N., Kim, D., Monibi, M., Selsky, J., Hansen,
be underestimated, and no exact picture of the acoustic M., & Estrin, D. (2010). AndWellness: An open mobile system
reality of a subject is obtained. Countermeasures against for activity and experience sampling. In I. M. Jacobs, P. Soon-
Shiong, E. Topol, & C. Toumazou (Eds.), Wireless Health 2010
this could be (a) to store objective information in the hear-
(pp. 34–43). Association for Computing Machinery. https://
ing aid long enough such that it can be retrieved after a doi.org/10.1145/1921081.1921087
situation where subjects did not bring their mobile phone, Jensen, N. S., Hau, O., Lelic, D., Herrlin, P., Wolters, F., & Smeds,
(b) to give subjects a possibility to report retrospectively, K. (2019). Evaluation of auditory reality and hearing aids using
and (c) to allow subjects to report the most important in- an ecological momentary assessment (EMA) approach. In
formation quickly. Although special triggers such as a loud M. Ochmann, M. Vorländer, & J. Fels (Eds.), Proceedings of
environment trigger do not necessarily make the data more the 23rd International Congress on Acoustics (pp. 6545–6552).
representative for real life, they help to collect more data Aachen, Germany, 9–13 September, 2019.
for conversation with several other people or SiN situations. Jenstad, L. M., Gillen, L., Singh, G., DeLongis, A., & Pang, F.
Despite these potential shortcomings, EMA is a prom- (2018). A laboratory evaluation of contextual factors affecting
ratings of speech in noise: Implications for ecological momen-
ising tool for receiving accurate information on many situa-
tary assessment. Ear and Hearing, 40(4), 823–832. https://
tions in the everyday life of subjects by combining subjective doi.org/10.1097/AUD.0000000000000686
and objective information. Kadylak, T., Makki, T. W., Francis, J., Cotten, S., Rikard, R. V.,
& Sah, Y. J. (2018). Disrupted copresence: Older adults’ views
on mobile phone use during face-to-face interactions. Mobile
Media & Communication, 6(3), 331–349. https://doi.org/10.1177/
Acknowledgments 2050157918758129
We are grateful to all the subjects who participated in this Kerner, M., Giese, U., Holube, I., & Schinkel-Bielefeld, N. (2018).
study. Furthermore, we thank Markus Kallinger for his support Vergleich von TruEar und direktionalem Mikrofon im Alltag
of the student project and the ethics application in Lübeck, Sarah mit einer Methode des Ecological Momentary Assessment.
Gotowiec for fruitful discussions, and Rosa-Linde Fischer and [Comparison between TruEar and directional microphone
Alastair Manders for carefully reading the article. in everyday life with a method of ecological momentary

assessment]. Paper presented at the 21st Annual Meeting of Stone, A. A., & Shiffman, S. (1994). Ecological momentary assess-
the German Audiological Society, Halle (Saale), Germany. ment (EMA) in behavioral medicine. Annals of Behavioral
https://www.dga-ev.com/fileadmin/dga2018/site/data/final/ Medicine, 16(3), 199–202. https://doi.org/10.1093/abm/16.3.199
0108.pdf Stone, A. A., & Shiffman, S. (2002). Capturing momentary, self-
Kowalk, U., Kissner, S., von Gablenz, P., Holube, I., & Bitzer, J. report data: A proposal for reporting guidelines. Annals of
(2017). An improved privacy-aware system for objective and Behavioral Medicine, 24(3), 236–243. https://doi.org/10.1207/
subjective ecological momentary assessment. In S. Santurette, T. S15324796ABM2403_09
Dau, J. Christensen-Dalsgaard, L. Tranebjærg, T. Andersen, & Stone, A. A., Shiffman, S., Schwartz, J. E., Broderick, J. E., &
T. Poulsen (Eds.), Proceedings of the International Symposium Hufford, M. R. (2002). Patient non-compliance with paper
on Auditory and Audiological Research (Vol. 6, pp. 25–30B). diaries. British Medical Journal, 324(7347), 1193–1194. https://
The Danavox Jubilee Foundation doi.org/10.1136/bmj.324.7347.1193
Mehrotra, A., Vermeulen, J., Pejovic, V., & Musolesi, M. (2015). Timmer, B. H. B., Hickson, L., & Launer, S. (2017a). Ecological
Ask, but don’t interrupt: The case for interruptibility-aware momentary assessment: Feasibility, construct validity, and
mobile experience sampling. In K. Mase (Ed.), UbiComp/ future applications. American Journal of Audiology, 26(3S),
ISWC’15 Adjunct: Proceedings of the 2015 ACM International 436–442. https://doi.org/10.1044/2017_AJA-16-0126
Joint Conference on Pervasive and Ubiquitous Computing and Timmer, B. H. B., Hickson, L., & Launer, S. (2017b). Hearing aid
Proceedings of the 2015 ACM International Symposium on use and mild hearing impairment: Learnings from big data.
Wearable Computers (pp. 723–732). Association for Comput- Journal of the American Academy of Audiology, 28(8), 731–741.
ing Machinery. https://doi.org/10.1145/2800835.2804397 https://doi.org/10.3766/jaaa.16104
Misra, S., Cheng, L., Genevie, J., & Yuan, M. (2016). The iPhone Timmer, B. H. B., Hickson, L., & Launer, S. (2018). Do hearing
effect: The quality of in-person social interactions in the pres- aids address real-world hearing difficulties for adults with mild
ence of mobile devices. Environment and Behavior, 48(2), 275–298. hearing impairment? Results from a pilot study using ecologi-
https://doi.org/10.1177/0013916514539755 cal momentary assessment. Trends in Hearing, 22, 1–15. https://
Pryss, R., Schlee, W., Reichert, M., Kurthen, I., Giroud, N., doi.org/10.1177/2331216518783608
Jagoda, L., Neuschwander, P., Meyer, M., Neff, P., Schobel, Van de Mortel, T. F. (2008). Faking it: Social desirability response
J., Hoppenstedt, B., Spiliopoulou, M., Langguth, B., & Probst, bias in self-report research. Australian Journal of Advanced
T. (2019, July). Ecological momentary assessment based dif- Nursing, 25(4), 40–48.
ferences between android and iOS users of the TrackYour- von Gablenz, P., Kowalk, U., Bitzer, J., Meis, M., & Holube, I.
Hearing mHealth crowdsensing platform. 2019 41st Annual (2019). Individual hearing aid benefit: Ecological momentary
International Conference of the IEEE Engineering in Medi- assessment of hearing abilities. In A. A. Kressner, J. Regev,
cine and Biology Society (EMBC) (pp. 3951–3955). IEEE. J. Christensen-Dalsgaard, L. Tranebjærg, S. Santurette, &
https://doi.org/10.1109/EMBC.2019.8857854 T. Dau (Eds.), Proceedings of the International Symposium on
Rintala, A., Wampers, M., Myin-Germeys, I., & Viechtbauer, W. Auditory and Audiological Research (pp. 213–220). The Danavox
(2020). Momentary predictors of compliance in studies using Jubilee Foundation.
the experience sampling method. Psychiatry Research, 286, Walden, B. E., Surr, R. K., Cord, M. T., & Dyrlund, O. (2004).
112896. https://doi.org/10.1016/j.psychres.2020.112896 Predicting hearing aid microphone preference in everyday lis-
Schlee, W., Pryss, R. C., Probst, T., Schobel, J., Bachmeier, A., tening. Journal of the American Academy of Audiology, 15(5),
Reichert, M., & Langguth, B. (2016). Measuring the moment- 365–396. https://doi.org/10.3766/jaaa.15.5.4
to-moment variability of tinnitus: The TrackYourTinnitus smart Wu, Y.-H. (2017). 20Q: EMA methodology—Research findings
phone app. Frontiers in Aging Neuroscience, 8, 294. https:// and clinical potential. Audiology Online, Article 20193.
doi.org/10.3389/fnagi.2016.00294 Wu, Y.-H., Stangl, E., Chipara, O., Hasan, S. S., DeVries, S., &
Shiffman, S., Stone, A. A., & Hufford, M. R. (2008). Ecological Oleson, J. (2019). Efficacy and effectiveness of advanced hear-
momentary assessment. Annual Review of Clinical Psychol- ing aid directional and noise reduction technologies for older
ogy, 4, 1–32. https://doi.org/10.1146/annurev.clinpsy.3.022806. adults with mild to moderate hearing loss. Ear and Hearing,
091415 40(4), 805–822. https://doi.org/10.1097/AUD.0000000000000672
Smeds, K., Wolters, F., Larsson, J., Herrlin, P., & Dahlquist, M. Wu, Y.-H., Stangl, E., Zhang, X., & Bentler, R. A. (2015). Con-
(2018). Ecological momentary assessments for evaluation of struct validity of the ecological momentary assessment in audio-
hearing-aid preference. The Journal of the Acoustical Society logical research. Journal of the American Academy of Audiology,
of America, 143(3), 1742. https://doi.org/10.1121/1.5035685 26(10), 872–884. https://doi.org/10.3766/jaaa.15034

Appendix A ( p. 1 of 2)
Questionnaires and English translations
As questionnaires were extensive, the table shows only the English translation for questions discussed in this article. The original
questionnaire was in German. English translation of the questions below has been verified by backtranslation. The first column
indicates the nature of the question: SC = single choice, MC = multiple choice, TB = text box. Opt means that questions were
only posed if previous questions indicated that this is suitable, for example, text box questions if, in the previous question, the
option “other/further details” was chosen.
No Translated question
Current situation questionnaire (short)

MC02 Whom or what are you listening to right now?
• I do not actively listen to anyone
• Music
• Radio/TV/audiobook/talk
• Conversation with one person
• Conversation with several people
• Telephone
• Other/further details
TB03 (opt) Please describe whom or what you are listening to right now.
MC04 What sounds are audible in the background?
• Voices/other people
• Traffic noise
• Household noise
• Music/television
• Engines/machinery/ventilation
• Wind
• Silence
• Other/further details
TB05 (opt) Please describe which sounds are audible in the background.
SC06 To what extent has the situation changed by filling out the questionnaire?
• Strongly, e.g., I leave the room/switch off the TV
• Slightly, e.g., the conversation slows down
• Not at all
SC07 Do you have time for further questions right now?
• Yes, I have time
• No
Questionnaire in case of missed current situation questionnaire
SC99 It can always happen that a questionnaire is missed. Please tell us why you missed the last questionnaire, so we can
improve our study design.
• Filling out the questionnaire was not appropriate in that situation
• Alarm not heard
• Hearing aids not worn
• Mobile phone not with me
• (Technical) problems when filling out the questionnaire
• Other
SC88 (opt) In what way was it inappropriate to fill out a questionnaire in this situation?
• It would have been impolite
• For safety reasons
• I had no time/was in a hurry
• I did not feel like it
• Other
End-of-day diary entry
SC65 Were there situations today in which you would have liked to give us feedback but did not start a questionnaire yourself?
• Yes
• No
TB66 (opt) In which situations/why did you not start a questionnaire yourself, despite the fact that you would have liked to give
us feedback?
SC47 Was the connection sign in the app (right upper corner) today more than 5 min green, despite the switched-on hearing
aids being in the vicinity of the mobile phone?
• Yes
• No, I did not notice it
• I did not pay attention
(table continues)

Appendix A ( p. 2 of 2)
Questionnaires
(Continued). and English translations
No Translated question
MC48 (opt) Did you try anything to reestablish the reconnection?
• Hearing aids restarted
• Batteries changed
• App restarted
• Phone restarted
• I did not do anything like that
SC49 (opt) What led to a successful reconnection? (connection icon is green)
• Restarting the hearing aids
• Changing the batteries
• Restarting the mobile phone
• Nothing helped
TB50 (opt) I noticed disruptions in the following situations: (e.g., at specific places/next to other devices/after switching on, etc.)
SC51 Were there situations today in which you did not wear the hearing aids? (except when sleeping or showering, etc.)
• Yes
• No
TB52 (opt) Please describe for what reasons you did not wear the hearing aids.
SC53 Were there situations today in which you did not have the mobile phone with you?
• Yes
• No
TB54 (opt) Please describe for what reasons you did not have the mobile phone with you.
SC64 Were there situations today in which you had switched off the mobile phone?
• Yes
• No
TB67 (opt) Please describe for what reasons you had the mobile phone switched off.
SC57 Were there situations today in which you used the ‘please do not disturb’-function? (except for sleeping)
• Yes
• No
MC58 (opt) For what reason did you activate the ‘please do not disturb’-function today?
• It would have been impolite
• For safety reasons
• I wanted to have peace
• I had no time or was in a hurry
• Other
TB60 (opt) Please describe in more detail in which situations you activated the ‘please do not disturb’-function.
SC62 Was it bothersome to you today to fill out the questionnaires?
• Yes
• Rather yes
• Rather no
• No
TB63 (opt) In which situations/for what reason were you disturbed by filling out the questionnaire?

Appendix B
Overview of Measures Used and Main Results
The table below gives an overview of which question within the different questionnaires and which objective data contribute
to each research question, including the resulting main findings. An overview and motivation of research questions (RQ1–RQ8)
can be found in the section “Research Questions” in the introduction. All relevant questions of the current situation and end-
of-day questionnaires are listed in Appendix A. The table below refers to the question identifiers (e.g., SC99, TB66) given in
Appendix A.
Research
question Used measures Main findings
RQ1a • Questionnaire in case of missed current situation • ”Filling out questionnaires inappropriate/ impolite” and “Alarm
questionnaire (questions SC99, SC88) not heard” are common reasons for skipped questionnaires
RQ1b–d • End-of-day questionnaire (Questions TB66, TB54, • Subjective data are primarily missing in social situations.
TB60)
RQ2 • Objective EMA data: Classification of acoustic • SiN and Car are underrepresented in random trigger
situation (a) continuously collected and (b) before questionnaires compared to continuously collected data.
self-initiated, random trigger and loud environment • Loud environment trigger gives more situations classified as
trigger questionnaires SiN than the other triggers.
• Current situation questionnaire: Listening intent • Loud environment trigger increases the number of reported
(question MC02) for different triggers conversations with several and reduces “TV/radio” compared
to the other triggers.
RQ3 • Length of current situation questionnaire (short/long) • Trend towards more SiN situations in short questionnaires
• Classification of acoustic situation shortly before than in long questionnaires.
questionnaire • All active listening conditions except conversation with
• Current situation questionnaire (Question MC02) several are more common in long questionnaires than in short
ones.
RQ4 • Delay between trigger and start of questionnaire • Questionnaires delayed more than 3 min contain more
• Classification of acoustic situation shortly before situations classified as quiet than questionnaires answered
questionnaire within the 3 min after the trigger.
RQ5 • End-of-day questionnaire (Questions TB54, TB67) • Main reason for not carrying a mobile phone are social
situations.
RQ6 • Classification of acoustic situation in long-term data • Situations classified as SiN are underrepresented, and those
logging classified as quiet are overrepresented in the EMA collected
• Classification of acoustic situation in continuously data compared to the long-term data logging.
collected EMA data
RQ7 • Classification of acoustic situation in long-term data • Objective data corresponding to random trigger long
logging questionnaires filled out within 3 min after the trigger contain
• Classification of acoustic situation shortly before only a third of the SiN situations one would expect when
immediately answered, randomly triggered long extrapolating from the long-term data logging.
questionnaires • When taking random and loud environment triggers together,
• Classification of acoustic situation shortly before this raises to two thirds of the expected SiN situations.
immediately answered, loud environment triggered
long questionnaires
RQ8 • Exit questionnaire: Questions regarding annoyance of • Carrying the mobile phone is annoying.
trigger, number of questions, and carrying the phone • The number of triggers and questions is ok.


2020 - AJA-Nadja - Evaluation of HA in Everday Life Using EMA

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

2020 - AJA-Nadja - Evaluation of HA in Everday Life Using EMA

Uploaded by

Copyright:

Available Formats

AJA

Evaluation of Hearing Aids in Everyday

592 American Journal of Audiology • Vol. 29 • 591–609 • September 2020

Downloaded from: https://pubs.asha.org 220.255.167.16 on 09/21/2020, Terms of Use: https://pubs.asha.org/pubs/rights_and_permissions

Schinkel-Bielefeld et al.: Hearing Aid Evaluation in EMA: Missing Situations 593

594 American Journal of Audiology • Vol. 29 • 591–609 • September 2020

Downloaded from: https://pubs.asha.org 220.255.167.16 on 09/21/2020, Terms of Use: https://pubs.asha.org/pubs/rights_and_permissions

Connection to Hearing Aids and Objective Data

Schinkel-Bielefeld et al.: Hearing Aid Evaluation in EMA: Missing Situations 595

Analysis and Statistics

596 American Journal of Audiology • Vol. 29 • 591–609 • September 2020

Downloaded from: https://pubs.asha.org 220.255.167.16 on 09/21/2020, Terms of Use: https://pubs.asha.org/pubs/rights_and_permissions

Erlangen (N = 12) Lübeck (N = 8)

Age (years) 58.3 24–79 71.9 60–82

Schinkel-Bielefeld et al.: Hearing Aid Evaluation in EMA: Missing Situations 597

598 American Journal of Audiology • Vol. 29 • 591–609 • September 2020

Downloaded from: https://pubs.asha.org 220.255.167.16 on 09/21/2020, Terms of Use: https://pubs.asha.org/pubs/rights_and_permissions

Schinkel-Bielefeld et al.: Hearing Aid Evaluation in EMA: Missing Situations 599

600 American Journal of Audiology • Vol. 29 • 591–609 • September 2020

Downloaded from: https://pubs.asha.org 220.255.167.16 on 09/21/2020, Terms of Use: https://pubs.asha.org/pubs/rights_and_permissions

Schinkel-Bielefeld et al.: Hearing Aid Evaluation in EMA: Missing Situations 601

Underrepresentation of Specific Situations

602 American Journal of Audiology • Vol. 29 • 591–609 • September 2020

Downloaded from: https://pubs.asha.org 220.255.167.16 on 09/21/2020, Terms of Use: https://pubs.asha.org/pubs/rights_and_permissions

Schinkel-Bielefeld et al.: Hearing Aid Evaluation in EMA: Missing Situations 603

604 American Journal of Audiology • Vol. 29 • 591–609 • September 2020

Downloaded from: https://pubs.asha.org 220.255.167.16 on 09/21/2020, Terms of Use: https://pubs.asha.org/pubs/rights_and_permissions

Schinkel-Bielefeld et al.: Hearing Aid Evaluation in EMA: Missing Situations 605

606 American Journal of Audiology • Vol. 29 • 591–609 • September 2020

Downloaded from: https://pubs.asha.org 220.255.167.16 on 09/21/2020, Terms of Use: https://pubs.asha.org/pubs/rights_and_permissions

Current situation questionnaire (short)

Schinkel-Bielefeld et al.: Hearing Aid Evaluation in EMA: Missing Situations 607

608 American Journal of Audiology • Vol. 29 • 591–609 • September 2020

Downloaded from: https://pubs.asha.org 220.255.167.16 on 09/21/2020, Terms of Use: https://pubs.asha.org/pubs/rights_and_permissions

Schinkel-Bielefeld et al.: Hearing Aid Evaluation in EMA: Missing Situations 609

You might also like