SEARCH QUALITY RATING QUALIFICATION PROGRAM

Side-by-Side
What is this What do we want How can I
task about? to achieve? get it right?

Rating A Side-by-Side rating task

asks you to compare two
sets of results and then,
decide on which side is
SxS better using the scale:
much better, better, slightly
better, or about the same.
We want to know which side
is better, respecting the
user’s intent. Users should
follow an order and they
should have many options
and choices.
For most SxS projects,
you will complete each
task by indicating which
side you like better (and
how much better), then
explaining why.

REMINDER But...why do
we want it?
In the exam, keep this in mind
to succeed in your ratings! To satisfy results for users
and provide them with
different options since not
everyone likes the same
websites or types of content.
KEY CONCEPTS
SCRB
Special Content Result Block

LP UNDERSTANDING SS
Landing page: web page which
serves as the entry point for a website
or a particular section of a website.
1 Users should find what they are looking for. Respect their intent!
R1
This R stands for the “right side”
of the page.
2 Users should get the best results in Remember!
L1 the most helpful order. Order matters
This L stands for the “left side”
of the page.

Especially on mobile devices

with smaller screens.

3 Users should have many options and choices. Not everyone likes the same
websites or types of content.

RATING PHILOSOPHY
Results should feel right for the user Use your knowledge and
4
location and language. rate from a local perspective.

1 Describe the user intent for the given query.

Describe why you prefer one side over the other, or

COMMENT'S PART 2 (optionally) why you think both sides are about the same.

Main Content (MC) ADS

Describe any particular result(s) that led
3 you to prefer either side.

KEY QUESTIONS AND CONCEPTS

Ask yourself all these questions and do a little research based on

them before submitting any rating:

Figure out the intent of the user and

What is the intent of the query? focus on how well each side fullfills
that intent.

Which side of NM and PQ rating is more helpful?

Compare which side is providing the most

How does the order of results leave an Impact on selecting sides? accurate/most helpful answer at the top,
this leave an impact on selecting sides.

In SXS, SCRBs play an important role if

Are the SCRBs helpful and at the top? the SCRB result is satisfying and at the top
in one side (that side will be much better).

Time Sensitive queries deserve

How fresh are the results? results that are up to date and
recent.

Are the results with the dominant intent more

on one side than the other?

Such as Websites, Apps, SCRBs,

Are their diverse results in different forms? News Articles, Videos, etc.

Minor, common, and dominant

Does one side have more diversity in terms of interpretations? interpretations add diversity and
helpfulness to the user.

Prefer higher quality result sets

to lower quality result sets

Some queries might request Results should feel right for

recent/fresh info the user’s location

The set of results must be

considered as a whole PAGE QUALITY Prefer results sets with helpful
content in result blocks

FRESHNESS USER LOCATION

AND LANGUAGE

HOW HELPFUL IS CONTENT

THE SET OF RESULTS? RESULT SET CONSIDERATIONS IN THE RESULT
BLOCK

ORDER NUMBER
OF RESULTS OF RESULTS

DIVERSITY
OF RESULTS

Important! Better results should be More results are not

higher up in the list of results necessarily better

Which side has more diversity

to address different aspects of the
query interpretation/user intent?

RATINGS STEPS AND DETAILED SIDE’S EVALUATION

STEPS

1 2 3 4 5

• Understand the query.
• Think about likely user
intents and query inter-
pretations.
• What are users looking
for when issuing the
query?
• Assign Needs Met and
Page Quality ratings to
all results on each side.
• Note that some special
content result blocks
may not ask for a Page
Quality rating.
• Also, assign flags and
mark duplicate results as
needed.
• Decide which set of
results isbetter for users
in the rating language,
locale, and user location.
• This may be difficult
when there are lots of
factors to consider, so use
your best judgment.
• Choose an overall SxS • Write a comment to
rating based on the explain your rating.
results.
• After deciding which
set of results is better,
choose a SxS rating.
• The rating depends on
how much better one
side is for users issuing
the query.

Your rating quality for these tasks is evaluated based on your

Needs Met ratings, Page Quality ratings, overall SxS rating, and
the reasoning behind your ratings as explained in your comments.

L SIDE L SIDE L SIDE ABOUT R SIDE R SIDE R SIDE

MUCH BETTER BETTER SLIGHTLY BETTER THE SAME SLIGHTLY BETTER BETTER MUCH BETTER
Very Helpful SCRBs
at the top on one side Yes Maybe No No No Maybe Yes

More Diversity on one side Maybe Yes Yes No Yes Yes Maybe

More Helpful results No Maybe Yes Maybe Yes Maybe No

at the bottom

Both sides are duplicates No No No Yes No No No

Better order of helpful results Maybe Maybe Maybe No Maybe Maybe Maybe
on one side
One side has more recent Yes Yes Maybe No Maybe Yes Yes
results at the top

Send feedback about this infographic to:

searchqualityqc@welocalize.com

Confidentiality Notice:
This file is intended solely for the individual to whom they are delivered within Search Quality Rater Training Program and contains
2023 C

confidential or legally privileged information. If you are not the intended recipient, or an employee or agent responsible for delivering version 3.0
the file to the intended recipient, you are hereby notified that disclosing, copying, distributing or taking any action in reliance on the 04/28/2022
contents of this information is strictly prohibited.