Hillary Oberpeul

ENGL 7001
12/12/13

Nurse Concierge Brochure Effectiveness: A Concurrent Think

Aloud Study
Introduction
I became interested in creating a study that explored the most effective brochure design for
introducing a new business to its potential clients earlier in the fall. This idea came about when I
was approached by a family friend, Carolyn Bargero, RN BSN, to help her with her promotional
materials for a new company. Nurse Concierge, LLC. Nurse Concierge is a company that
provides patients with a nursing service to supplement medical care by offering the interpretation
of information at doctors visits including diagnosis and medication, care planning and
implementation, and organization of bills. The current brochure design is very plain, has a lot of
text, and Carolyn is wondering if it is an effective way to advertise her business.
The goal of this study was to determine which pieces of important information defined by
Carolyn were actually understandable to readers of the brochure. It tested the readers abilities to
articulate what exactly Nurse Concierge does after each participant read through the brochure
and offered comments. This is a small study was focused on two different age groups of potential
users for this brochure. It involved two different people that fell close to the target markets
defined by Carolyn (elderly patients and those caring for elderly patients) who were asked to
perform a Concurrent Think Aloud (CTA) protocol of the brochure. The results were then
compiled into tables of salient quotes pulled out of the video recordings.

Expected Results
I did not expect participants to be able to determine the key points that Carolyn defines as her
message from the current brochure. I expected them to say that the current brochure design has
too much text, not enough color, and confusing language. I thought that they would comment on
the long paragraphs and lack of color in the brochure making it difficult to understand the
message of the brochure. I expected participants to be interested in the service itself, but I
anticipated that they would say that it takes them too long to understand the salient information
in the brochure.

Methods
The Concurrent Think Aloud (CTA) method is a useful protocol when testing usability because it
allows the researcher to hear what the participant is thinking as they experience the material
being tested (Van Den Haak). There is not a large amount of literature on CTA protocols dealing
with brochure design and information retention. However, there is one study that I found to be
useful when designing the protocol. The study was done to assess the understanding of third

graders reading difficult texts. It used video cameras, notes on computers, and probing questions
along with the CTA method to glean the greatest amount of information from the participants.
(Schellings) I chose to use this method in my study because it provides video documented
evidence of the positive and negative aspects of this brochure as well as a report of my findings
for Carolyn to use. She will be able to take the results and apply them to her marketing materials,
knowing that users in her target audiences have tested them.
The target audiences listed by Carolyn for Nurse Concierge are older patients and families caring
for elderly patients. The list of key points that she defined as important for people to pull out of
the brochure are listed below. Nurse concierge does the following:
o Goes with the patient to doctors visits to help them interpret the information given to
them
o Helps with medication management
o Helps organize and consolidate bills
o Helps the patient understand their insurance policy
In order to test the effectiveness of this brochure, I asked two people, Elaine S. (75) and Paul T.
(23), to sit down with me and read the brochure, noting points that confused them, points that
were very clear, and eventually tell me the most important points they had gathered from the
brochure. Both Elaine and Paul have a Bachelors degree, though Elaine is a graduate of the
Professional Writing program at the University of Cincinnati as well.
Both participants were asked to be a part of a 30 minute long video-recorded session in which
they read the brochure for Nurse Concierge out loud. When the participant arrived, he or she was
given a consent form (Appendix A), then asked to sit down and read the brochure. I asked them
to read the brochure out loud and noted when they stopped or read in a different manner than
before. The participants were both asked open-ended questions like Why did you stop? or
What is confusing here? so that I could get his or her reaction on video. The full list of
potential questions that were asked during the session is listed in Appendix B.
During the time that the participant was reading, I took notes on my computer to include small
reactions and for insurance if the video happened to fail. At the end of the session, while still
recording, I asked the participants to tell me what they think Nurse Concierge does and whether
or not they would use this service and why. At the end of the session, I read each participant a
debriefing document (Appendix C) and asked them if they had any further questions or
comments about the brochure or the study itself.

Results & Discussion

The results of this study were simultaneously surprising and expected. Both Paul and Elaine
made comments on the various aspects of the brochure as they read through the brochure. Many
of the comments focused on the layout of the brochure, which was noted to be busy and lacking
in white space. These types of comments were expected. See the tables below for quotations of
the participants comments.

Table 1: Quotes from Paul T.

Nurse Concierge is basically your
management team for everything.
It would be cool if you had a picture
of a meeting actually happening. So
youre showing the patient sitting
there with the doctor and Nurse
Concierge.
Talking about the visual guide: You
need to show exactly what theyre
doing.
The pictures are fine, but do they
really fit the business? The nurses
hat thing on the front page seems
really old.

I understand all of the content, and

theres a nice little overview on the
back.

The brochure looks like something

someone would have done as a school
project.

Talking about if he would use Nurse

Concierge: Personally, unless I had
dementia or something, I would not. I
can make my own medical decisions
with my own wherewithal, especially
since I have my family.
Talking about the text: Its really
close to the edges. Maybe put a
border around everything so the text
doesnt fall off the page.
Ok, this lady (points at woman in
hospital bed) looks like shes about to
die. That is NOT the picture you want
on your brochure, unless its for a
funeral home.

Talking about if he would use Nurse

Concierge: Now if I was all by
myself, old, had a job and no family,
or was just with my significant other,
then maybe I would use it, just so I
didnt have to deal with it.
I wouldnt pick this up at a doctors
office or something. Its so plain and
fades into the background.
Who is this woman [Carolyn]? You
have no idea. Shes just this floating
head kind of entity that is Nurse
Concierge. Its so distant.

Table 2: Quotes from Elaine S.

Information shared and information
received whos doing the sharing
and whos doing the receiving?

Talking about the tone of the text:

That sounds a little formal.

Talking about a long sentence: Well

thats a mouthful, isnt it?

Patient education that sounds a

little condescending.

Talking about a long sentence:

Thats kind of a long sentence with a
lot of stuff stuffed into it.

Does the Nurse Concierge go to the

appointments and sit there? And if
she goes to the appointments, that
suggests that the patients privacy
might be an issue.

That sounds like shes giving orders.

That needs to be softened up a bit.

So the family members have

appointments with the doctors? I
dont think thats usually the case.

Thats in bold italics. She doesnt

use that anywhere else.

I dont know what BSN means.

The first page is cut off on the left,

so theres not enough white space. It
seems crowded.

She needs to make it seem as though

the patient is an adult, and still
competent, even if they need some
help.
Talking about if she would use Nurse
Concierge: I guess there are people
who would feel the need for it. I
wouldnt, but then Im stubborn.

Talking about content: I think its

fairly clear, but its kind of obfuscated
by the way the sentences are
constructed.
Talking about who would use Nurse
Concierge: I guess some
overwhelmed old lady might have
room for someone to help her.

Heres another one of those long

sentences. That needs to be broke
down into manageable, bite-sized
pieces.
I dont think it would be a bad idea
for her to have her picture in here
somewhere, to make her familiar with
the patient.
The text looks kind of dense.

Both Paul and Elaine spent a large amount of time talking about ways that the information in the
brochure would be easier for a reader to understand. The visuals and layout of the brochure, tone
of the text, and sentence structure are all parts of the brochure that are discussed at length by the
two participants. Elaine had more questions about the viability of Nurse Concierge as a whole,

which may be something that can be attributed to the age difference between Paul and Elaine.
Elaine is 75, whereas Paul is 23. It seems that the younger generation in this study is more
accepting of a business made up of a woman who helps with a patients health decisions. Elaine,
as someone who falls into the target age group for Nurse Concierge, needed more convincing of
why she should use it. This is an important point for Carolyn to note. The message is being
conveyed for the most part, but its not convincing as it stands.
One of the most interesting part of the results was the fact that most of the comments on the
brochure were focused on the layout of the brochure, suggesting that the content itself was not
the issue for the readers. Both Paul and Elaine were able to adequately define the goals of Nurse
Concierge in the terms that Caroline provided, even with the chaotic layout and long, wordy
sentences. This suggests that the current brochure is not ineffective because of the words used (as
I thought it would be) but because the layout hinders its readability. Elaine even says about the
content, I think its fairly clear, but its kind of obfuscated by the way the sentences are
constructed.
Another surprising result of this study has to do with the comments that were not made during
this session. All of the comments on layout, sentence structure, and tone of voice had to do with
their effect on the reader, who was assumed to be a patient. At the end of both video sessions,
Paul and Elaine separately agreed that they would not use this service themselves, even though
they could understand the brochure. Both participants noted that they would probably have
someone to help them with such things or that they were competent enough to handle doctors
appointments on their own. These types of comments point to a large issue with the current
Nurse Concierge brochure: it does not address the caretakers of a patient in a way that convinces
readers that it is a worthwhile service.
One of the audiences that Carolyn presented in her speech to me about Nurse Concierge was the
caretaker of an elderly patient. She wanted to show that she, as the Nurse Concierge, would be
able to assist someone who normally goes to doctors appointments with their parent or aunt or
other elderly relative. Because neither participant was able to understand and rearticulate this bit
of information after reading through the brochure, it would seem that its not well represented in
the content. This area of the brochure needs to be addressed if it is going to be as effective as
Carolyn wants it to be. Multiple brochures may be a better way to address the multiple audiences
rather than one, especially if there isnt room to talk about all of the information in one brochure.
Limitations of the Study
There are some limitations that come with doing a study this small. Because the people I had
access to were all connected to the UC population or my friend group, they have both received at
least a bachelors degree, which may not describe all of the users of this brochure. Both
participants are at opposite ends of the age spectrum, which could cause for some missing
information on the usefulness of this brochure. The gap between the ages of the two participants
is 52 years, which can make a difference in interpreting information.
Some bias could also have been introduced because one of the participants was a friend of mine
and may not have wanted to speak as freely because of that fact. Having only two participants
makes the results useful to Carolyn Bargero, but in a larger sense could only serve as a pilot

study. If the study were to be replicated, it would make sense to use a larger sample size of
participants to get a broader view of the information presented. Because the study is not
quantifiable, the information gathered will be more useful in a sense of suggestion rather than
generalization of the information
Future Research
It would be interesting to expand this research study to include more people from both of the age
groups used in this small study, but also to include a few people from the median age group that
was not used in this study. This would allow access to information from a group that may be
currently caring from an elderly patient. It would also be able to test the finding from this study
that users could understand what Nurse Concierge does for the patient, but the current brochure
does not address what Nurse Concierge can offer those caring for an elderly patient.
If future researchers wanted to create a study that resulted in more generalizable information, it
would be interesting to see a Concurrent Think Aloud protocol done using two different versions
of a health brochure (such as the one used in this study). Each group of participants could do a
CTA protocol, first on one brochure (that had longer sentences and layout issues similar to the
Nurse Concierge brochure) and then another that was more concise and laid out well. These
results could pinpoint some of the issues with audience because the layout issues would be taken
care of.

Appendices
Appendix A: Consent Form
Nurse Concierge CTA Protocol
You are being asked to participate in this interview protocol to test the usability of the Nurse
Concierge, LLC brochure. I, Hillary Oberpeul, will be asking you to read through the entire
brochure out loud and comment on sections you find confusing or clear. You should navigate
through the brochure in whichever way seems most logical to you. As you make decisions on
where to navigate, read aloud, and comment, I will be taking notes on what you say or how you
react. I will also have a backup audio recording device running in case the video fails. I may
prompt you with questions as you read the brochure. Please answer them truthfully and comment
as you see fit. If you have any questions or concerns, please let me know now. If not, please fill
out the rest of this form.
Date:
Time:
Name of Participant (First name and first initial of last name only): _______________________
Please read the following statements and sign on the line below them:

I have voluntarily agreed to be a part of this focus group discussing the effectiveness of
brochures for Nurse Concierge, LLC.
I will follow the guidelines of the discussion leader, Hillary Oberpeul.
I have had a chance to ask questions and understand how the session will proceed.
I authorize Hillary Oberpeul to capture my opinions with a video recording device, a
sound recording device, and personal notes.
I permit her to use these responses in a written report of the findings.
I understand that only my first name and the first initial of my last name will be used
when referring to any opinions I have expressed during the time of this protocol.

Participant Signature: _________________________________________________

Appendix B: Questions During Protocol
This question will be asked at the beginning of the protocol:
1. Quickly flip through the brochure without reading it. What is your initial impression of
it?
The following questions can be asked in any order, at any point in the study, to elicit more
information from participants as they read the brochure.
1. Why did you stop reading?
2. Can you explain that?

3. What do you find unclear/confusing? (If they say they are confused and dont explain)
The following questions will be asked at the end of the protocol:
1. What does Nurse Concierge do?
a. Can you get this information from this brochure?
b. Why/why not?
2. Would you use this service?
Appendix C: Debriefing Document
Thank you for participating in this CTA protocol examining the Nurse Concierge, LLC brochure.
I will use your questions, comments, and suggestions to prepare a research report conveying
findings from this protocol. They will help me make suggestions to Carolyn Bargero about
creating a more informative and succinct piece of marketing information for her new company. If
you have any final questions or comments, please share them with me now.
Appendix D: References
Schellings, Gonny, Cor Aarnoutse, and Jan van Leeuwe. "Third-graders think-aloud protocols:
Types of reading activities in reading an expository text." Learning and Instruction 16
(2006): 549-68. Web. 18 Nov. 2013.
Van Den Haak, Maaike J., Menno D. De Jong, and Peter J. Schellens. "Retrospective vs.
concurrent think-aloud protocols: testing the usability of an online library
catalogue." Behaviour & Information Technology 22.5 (2003): 339-51. Academic Search
Complete. Web. 22 Nov. 2013.