Professional Documents
Culture Documents
(MASTER) CX QA Playbook
(MASTER) CX QA Playbook
Why Test?
Testing is the gate that ensures a developed solution meets our customer's
expectations. Testing should begin after the conclusion of the build phase of
your project.
hat is resolved should be retested. Use the status column to mark each defect appropriately
d as signed off by both you and the customer. Using this has 2 benefits:
Qualtrics Recodes
Qualtrics Active Survey
Qualtrics Survey Security
Qualtrics User Setup
Qualtrics Data flow
iQ Directory / D
[CLIENT] Survey Distribution Check
Qualtrics iQ Directory
Qualtrics Distribution Rules
Qualtrics Translations
Website Fe
Qualtrics Creatives
Qualtrics Intercepts
Confirm that Teams / users are setup correctly in the Ticketing Tool. Research Core
Confirm that Actions / Ticket Tasks are setup correctly Research Core
Confirm that conditions on the ticket match the design document Research Core
Confirm ticket owners are notified of ticket creation if this option is
selected Research Core
Data Cleanup
Remove all test responses in Research Core for every survey Research Core
Remove all test Mailing Lists iQ Directory
Remove all test Directory contacts in iQ Directory iQ Directory
Verify that all response data has been removed in the relevant dashboard CX Dashboard
Remove all test Email distributions for every survey Research Core
Ensure all test tickets have been removed from the system Research Core
Business
Re-confirm all relevant stakeholders within the business are informed Customer
about the launch (INTERNAL)
Customer
Any further internal comms that need to happen (INTERNAL)
Technical
Customer
To be populated by the customer if required (INTERNAL)
Customer
To be populated by the customer if required (INTERNAL)
Customer
To be populated by the customer if required (INTERNAL)
Customer
To be populated by the customer if required (INTERNAL)
Customer
To be populated by the customer if required (INTERNAL)
Qualtrics Sign Off Name Qualtrics Date Customer Sign Off Name Customer Date
For questions 2 and 3
the question text is in a
ID Test Script ID Product Area Issue
Question Text Description
different font style to
B-01 UAT-001 Formatting the rest of the survey
B-02
B-03
B-04
B-05
B-06
B-07
B-08
B-09
B-10
B-11
B-12
B-13
B-14
B-15
B-16
B-17
B-18
B-19
B-20
Status Reported By Actions
01.01.2020 - action to be Comment
Open Customer Team assigned during next call
Test Case Description
https://www.qualtrics.com/support/survey-platform/survey-module/editing-questions/formatt
https://www.qualtrics.com/support/survey-platform/survey-module/using-logic/
https://www.qualtrics.com/support/survey-platform/survey-module/look-feel/look-feel-overvi
https://www.qualtrics.com/support/survey-platform/survey-module/editing-questions/validat
https://www.qualtrics.com/support/survey-platform/survey-module/survey-options/survey-op
https://www.qualtrics.com/support/survey-platform/survey-module/survey-flow/standard-ele
https://www.qualtrics.com/support/survey-platform/survey-module/survey-tools/translate-su
https://www.qualtrics.com/support/iq-directory/getting-started-iq-directory/getting-started-wi
https://www.qualtrics.com/support/vocalize/user-admin-cx/roles-cx/
https://www.qualtrics.com/support/vocalize/user-admin-cx/user-attributes-cx/
https://www.qualtrics.com/support/vocalize/dashboard-theme-cx/
Test Case Description
Desktop Look and Feel Ensure Desktop Creative only appears on desktop
Ensure survey is readable on Desktop and Look and Feel
Desktop Look and Feel are as expected
Mobile Look and Feel Ensure Mobile Creative only appears on mobile
Ensure survey is readable on Mobile and Look and Feel are
Mobile Look and Feel as expected
Ticket owner/team Check to see that ticket owners and teams are receiving
assignment tickets when they should
https://www.qualtrics.com/support/survey-platform/survey-module/editing-questions/formatt
https://www.qualtrics.com/support/survey-platform/survey-module/using-logic/
https://www.qualtrics.com/support/survey-platform/survey-module/look-feel/look-feel-overvi
https://www.qualtrics.com/support/survey-platform/survey-module/editing-questions/validat
https://www.qualtrics.com/support/survey-platform/survey-module/survey-options/survey-op
https://www.qualtrics.com/support/survey-platform/survey-module/survey-flow/standard-ele
https://www.qualtrics.com/support/survey-platform/survey-module/survey-tools/translate-su
https://www.qualtrics.com/support/iq-directory/getting-started-iq-directory/getting-started-wi
https://www.qualtrics.com/support/iq-directory/lists-tab/managing-contacts-in-a-mailing-list/#
https://www.qualtrics.com/support/iq-directory/directory-settings-tab/contact-frequency-rule
https://www.qualtrics.com/support/website-app-feedback/common-use-cases/using-tag-ma
https://www.qualtrics.com/support/website-app-feedback/intercepts-tab/navigating-intercep
https://www.qualtrics.com/support/website-app-feedback/creatives-tab/navigating-creatives
https://www.qualtrics.com/support/website-app-feedback/intercepts-tab/edit-intercept-sectio
https://www.qualtrics.com/support/vocalize/user-admin-cx/roles-cx/
https://www.qualtrics.com/support/vocalize/user-admin-cx/user-attributes-cx/
https://www.qualtrics.com/support/vocalize/dashboard-theme-cx/
https://www.qualtrics.com/support/survey-platform/actions-module/ticketing/tickets-task/
https://www.qualtrics.com/support/survey-platform/actions-module/ticketing/ticketing-teams
https://www.qualtrics.com/support/survey-platform/actions-module/ticketing/follow-page-op
https://www.qualtrics.com/support/survey-platform/actions-module/ticketing/ticket-settings/
Test Case Description
Desktop Look and Feel Ensure Desktop Creative only appears on desktop
Ensure survey is readable on Desktop and Look and Feel
Desktop Look and Feel are as expected
Mobile Look and Feel Ensure Mobile Creative only appears on mobile
Ensure survey is readable on Mobile and Look and Feel are
Mobile Look and Feel as expected
Ticket owner/team Check to see that ticket owners and teams are receiving
assignment tickets when they should
https://www.qualtrics.com/support/survey-platform/survey-module/editing-questions/formatt
https://www.qualtrics.com/support/survey-platform/survey-module/using-logic/
https://www.qualtrics.com/support/survey-platform/survey-module/look-feel/look-feel-overvi
https://www.qualtrics.com/support/survey-platform/survey-module/editing-questions/validat
https://www.qualtrics.com/support/survey-platform/survey-module/survey-options/survey-op
https://www.qualtrics.com/support/survey-platform/survey-module/survey-flow/standard-ele
https://www.qualtrics.com/support/survey-platform/survey-module/survey-tools/translate-su
https://www.qualtrics.com/support/iq-directory/getting-started-iq-directory/getting-started-wi
https://www.qualtrics.com/support/iq-directory/lists-tab/managing-contacts-in-a-mailing-list/#
https://www.qualtrics.com/support/iq-directory/directory-settings-tab/contact-frequency-rule
https://www.qualtrics.com/support/website-app-feedback/common-use-cases/using-tag-ma
https://www.qualtrics.com/support/website-app-feedback/intercepts-tab/navigating-intercep
https://www.qualtrics.com/support/website-app-feedback/creatives-tab/navigating-creatives
https://www.qualtrics.com/support/website-app-feedback/intercepts-tab/edit-intercept-sectio
https://www.qualtrics.com/support/vocalize/user-admin-cx/roles-cx/
https://www.qualtrics.com/support/vocalize/user-admin-cx/user-attributes-cx/
https://www.qualtrics.com/support/vocalize/dashboard-theme-cx/
https://www.qualtrics.com/support/survey-platform/actions-module/ticketing/tickets-task/
https://www.qualtrics.com/support/survey-platform/actions-module/ticketing/ticketing-teams
https://www.qualtrics.com/support/survey-platform/actions-module/ticketing/follow-page-op
https://www.qualtrics.com/support/survey-platform/actions-module/ticketing/ticket-settings/
Test Case Title Description
Ensure question text formatting is as expected (format,
Question formatting colour,
Ensure size etc.)skip and branch logic providing expected
display,
Survey Logic survey
Ensure experience for respondent
Survey theme, progress bar, page transition,
Desktop Look Look
Mobile/Tablet and Feel
and header/footer etc. appearing
Ensure Survey theme, progress as expected
bar, pageon Desktop
transition,
Feel header/footer etc. appearing as expected on Mobile/Tablet
Validation Ensure that all validation is performing as expected
Survey Options Ensure survey options are configured to requirements
End of Survey Confirm
Complete end of survey
survey in all experiences are configured
configured languages correctly
checking
Translations spelling for survey text, validation messages and logic
Expected Result Actual Result Tester
Questions
Respondent appear as expected
experiencing the survey
correctly based on logic defined
Look and feel is as expected on
Desktop
Look and feel is as expected on
Mobile/Tablet
Validation messages, look and feel
Back button, Partial Completion, base
and logic
language, working as
anonymise expected
response etc.
All paths to the end of the survey
are configured
produce correctly
the correct end of survey
experience
All languages look and perform as
expected
Status Date Support Link
https://www.qualtrics.com/support/survey-platform/survey-module/editing-questions/formatt
https://www.qualtrics.com/support/survey-platform/survey-module/using-logic/
https://www.qualtrics.com/support/survey-platform/survey-module/look-feel/look-feel-overvi
https://www.qualtrics.com/support/survey-platform/survey-module/editing-questions/validat
https://www.qualtrics.com/support/survey-platform/survey-module/survey-options/survey-op
https://www.qualtrics.com/support/survey-platform/survey-module/survey-flow/standard-ele
https://www.qualtrics.com/support/survey-platform/survey-module/survey-tools/translate-su
Test Case Description
Ensure a contact list is created in Expected Result Actual Result
Contact List the correct
Ensure foldercreated in the
contacts Contact list created successfully
Contacts correct
Ensure contact
contact list
lists in the correct Contacts added to list
Library library
Ensure contact embedded data is Contact list in correct library
Embedded Data stored
Ensureas expected
contact transaction data is Embedded Data set correctly
Transaction
Contact Data stored
Ensureas expected
contact frequency rule(s) Transaction data created
Contact Frequency correctly
behaving as
Frequency behaving as configured
Ensure deduplication rule(s) expected
Deduplication behaving as configured deduplication behaving as expected
Tester Status Date Support Link
https://www.qualtrics.com/support/iq-directory/getting-started-iq-directory
Test Case Description Expected Result
SMS 2-way Check to ensure SMS messages are worded
Content as configured Message wording is a expected
Opt Outs Ensure
Ensure contacts can opt translation
survey message out working as Opt outs working
Translations expected
Validate message length does not exeed Messages translated as expected
Message length desired message count Messages take up 1 message each
https://www.qualtrics.com/support/survey-platform/distr
https://www.qualtrics.com/support/survey-platform/distr
Test Case Description
Check to make sure the correct email message
Library messages is selected
Content Content should look
The appropriate good for
language for respondents
a respondent
Translations should appear
The email styling should be appealing to
Look and feel respondents
Respondents should be able to opt out if they
Opt Out wish
Expected Result Actual Result Tester Status
Correct library message being piped into email
Email content looking and reading as expected
Language translations appear and read as expected
Email look and feel as expected
Confirm opt-out functionality works as expected
Date Support Link
https://www.qualtrics.com/support/survey-platform/distributions-module/email-distribution/emails-overview/
Test Case Description
Ensure all user access is configured to give Expected Result
Roles users
Ensurethe correct data view Assigned
Assigned roles showing the correct
thepages
Ensure all
thatuser access is configured
all dashboard content is to give user attributes filtering data for
User Attributes users the correct data view
displaying the results that each audience each user group as expected
Dashboard content signed off by dashboard
Content needs
Ensure audiences
Ensure that
pagethe
and dashboard aesthetics
widget filters meet
are correct Dashboard looks and feels as desired by
Look and Feel user requirements
with drop-downs populating and filtering as dashboard audience
Filters expected Filters behaving as expected
Actual Result Tester Status Date Support Link
https://www.qualtrics.com/support/vocalize/user-admin
https://www.qualtrics.com/support/vocalize/user-admin
https://www.qualtrics.com/support/vocalize/dashboard-
Test Case Description
Testing to see if the call Expected Result Actual Result
[API CALL 1] executes successfully
Test to see if return data is as
[API CALL 1] expected
Testing to see if the call
[API CALL 2] executes successfully
Test to see if return data is as
[API CALL 2] expected
Tester Status Date Support Link
https://api.qualtrics.com/
Test Case Description
Code execution Code executes
Network requestasforexpected in tag appearing
Site Intercept manager in
Code execution network tab (use
Check multiple Q_DEBUG
pages where
to ensure necessary)
deployment
Code execution running
Validateon expected
intercept pages correctly on target
executing
Intercept evaluation pages
Validate intercept executing but not appearing on
Intercept evaluation selected pages for
Check intercept exclusion
does - up until
not appear to 3 3 seconds
Intercept evaluation have passed
Confirm survey does not appear if user returns to
Supression Rules page where intercept already seen
Intercept evaluation Confirm sampling rules are being applied
Desktop Look and Feel Ensure
Ensure Desktop
survey isCreative
readableonly appears and
on Desktop on desktop
Look
Desktop Look and Feel and Feel are as expected
Mobile Look and Feel Ensure
Ensure Mobile
survey Creative only
is readable onappears on mobile
Mobile and Look and
Mobile Look and Feel Feel are as expected
Tablet Look and Feel Tablet Look and Feel
Tablet Look and
Respondent Feel
Survey
Tablet Look and Feel
Ensure survey function performs as expected (i.e.
Experience logic working as expected, buttons clickable etc.)
Check Embedded Data and Survey Data to
Language Ensure
confirm Survey
all fieldslanguage appears
are passing as expected
through to the
Data Flow - Qualtrics Qualtrics Data and Analyis tab correctly
Tests to repeat for different browsers (Chrome,
Browser Firefox, Safari, IE) on both Mobile and Desktop
Expected Result Actual Result
Code running as expected in tag manager
Code executing on target webpage
Code running successfully
Intercept logic evaluated and survey appears
Survey does not appear on selected exclusion pages
Survey does not appear until after 3s
Survey does not appear
Survey does not always appear
Desktop-sized Creative appears on desktop but not on mobile or tablet
Look and feel are acceptable
Mobile-sized Creative appears on mobile but not on desktop or tablet
Look and feel are acceptable
Ensure Tablet Creative only appears on tablet
Ensure survey is readable on Tablet and Look and Feel are as expected
Survey Function working as expected
Spelling, wording and question layout all as expected
All data fields appearing in Data and Analysis tab as expected
Intercepts perform as expected across all browsers
Tester Status Date Support Link
https://www.qualtrics.com/support/website-app-feedback/common
https://www.qualtrics.com/support/website-app-feedback/intercep
https://www.qualtrics.com/support/website-app-feedback/creative
https://www.qualtrics.com/support/website-app-feedback/intercep
Test Case Description
Confirm that Salesforce.com account is linked
Verify that Qualtrics automatically sends surveywithin
Linking Accounts
Trigger and Qualtrics brand
invitation when workflow rule is met in Salesforce
Email Survey
Response Expected
Verify that answers provided in Qualtrics survey map
Mapping back to Salesforce record
Salesforce.com account is linked within
Survey-->Tools or appears in drop-down under
Expected Result
Actions-->Add Actions-->Add Task-- Actual Result Tester
When Action condition is met, survey response
>Salesforce
Recipient receives
mapping should survey
create, email or
update, invitation
delete
when workflow rule is met in Salesforce
Salesforce record as specified or post to
chatter.
Status Date Support Link
This checks to see if tickets have
been created successfully as per
Test Case Description
the conditions configured in the Expected Result
Check to see that ticket owners
Tickets created
Ticket owner/team Actions tabare receiving tickets
and teams Tickets are created successfully
assignment when
Confirm they should
ticket owners and teams Tickets being assiged as expected
Ticketing App acess have
Check access to
to ensure the ticketing
owners app Owners have access to the app
Checking to ensure ticketare
owners
Notifications notified
have when a ticketdatais created Ticket notifications received
Checkthe to necessary
ensure root causesto action
and
Ticket Data the ticket
follow up questions appearing as Ticket data fields
Root Causes and populated correctly
follow up as
Root Cause and Follow Up configured expected
Actual Result Tester Status Date Support Link
https://www.qualtrics.com/support/survey-platform/actio
https://www.qualtrics.com/support/survey-platform/actio
https://www.qualtrics.com/support/survey-platform/actio
https://www.qualtrics.com/support/survey-platform/actio
Confirm business logic rules
Test Case Description
behaving as specified. This Expected Result
Business logic in automation
Business logic in automation includes processing rows under
Mailing lists being created in the correct Confirm correct libraryas
being behaves as expected
the correct conditions wellused
as Mailing lists by day and
library
Contacts are being populated in the to house
forcing mailing
required lists
columns
Ensure contacts are being survey
All contacts populated the
correct
Contactsmailing listof the correct
have all created
Contactsinbeing
the desired
createdlocation
with correct mailing
All contacts listthe correct
have
Embedded Data
File with links areetc.
placed in the correct correct
Ensure data
the correct automation embed data fieldsput on
File successfully
automations folder folder being used automation server in IN folder
Survey distribution Correct survey being distributed Correct survey is distributed
Actual Result Tester Status Date
Support Link
Test Case Description
Check that source system connected to
File drop from the customer
File validation (UTF-8 encoding, correct automation
Confirm folder
Confirm file formatlogic
business will be accepted
rules behaving by as
the
column headers, delimiter etc.) automation
specified. This includes processing rows
Business logic in automation
Mailing lists being created in the correct under
Confirmthecorrect
correctlibrary
conditions
being as welltoas
used house
library forcing
mailing required
lists columns
Contacts are being populated in the Ensure contacts are being created in the
correct
Contactsmailing listof the correct
have all desired location
Contacts being created with correct data
Embedded Data etc.
Expected Result Actual Result Tester Status
File successfully put on automation server
in correct target folder
File validation has the right format
Business logic in automation behaves as
expected
Mailing lists by day and survey
All contacts populated the correct mailing
list
All contacts have the correct embed data
fields
Date Support Link