How to Perform the Task

A search engine should provide clear, relevant, and trustworthy results which align with the
intent of the query. A query is a search term someone types into Google or another search engine
to find related content on the internet. In addition to related content, search engines promote
content using ads. Ads should be relevant to the query and add value to the page.

Process Steps:

1. Understand the query, the intent and expected results:

Read the Query and use your intuition to determine what the user is searching
for.
2. Read the Location from where the query was made and assess whether the
query results would change if it were made from a different location.
3. Determine which side shows better results for the given query by taking potential
user intent(s) into consideration.
4. Choose ‘Same’, LeftBetter or RightBetter as the answer. If you see a blank
screen instead of a web page please open in a new tab, if it does not open in the
new tab either, or does not exist, please click on 'Page Not Found'.

* "Page Not Found" is for cases when the comparison can not be made because one of
pages does not exist.
* If you are not sure or the page is in a language you do not understand, you should skip
the task.

Some tips:

1. A beautiful layout does not make a feature relevant, always double check the
content!
However, if both sides have equivalently relevant content, a more well organized,
pleasing design should be preferred.
2. When providing a score, please evaluate both sides FAIRLY. When you are not
certain, click through the web links to see which side provides more appropriate
and relevant results. We appreciate your attention to detail and your thoughtfully
written comments and justification.
3. The losing side may contain more repetitive or useless results. The winning side
may contain more trusted or recent results.
4. the losing side fails to answer the question or gets the intent wrong while
winning side completely satisfies the query intent
 5. You may see SERPs or pages with very few web results or only web results. These
are not broken pages. They are valid and should be judged normally.
6. If you see hits where the SERPs are very similar - this is not an error! There is
typically a minor difference between those two SERPs.
7. Please carefully note the intent likelihood of user location vs a location specified
in the query. If the user has stated they want a specific location in the query, that
trumps the user's location for relevance. Pay close attention to local or map
answers to make sure they match the user's intent.
8. Remember, taking breaks and stepping away from judging for a few moments
can be just as helpful as carefully investigating pages. Take a few minutes to
stretch and relax if you find yourself getting stuck on queries!
9. PLEASE ENSURE YOU DO NOT HAVE AN AD BLOCKER PLUGIN INSTALLED OR
ACTIVE. This actively prevents you from seeing important aspects of the SERPs
that may negatively impact your Spam Score.
10. Details are important. Pay attention to everything on the page—it’s often the
small things that enhance or ruin the overall user experience.
11. Please be fair to both search engines. Above all else, we value your honesty and
objectivity when evaluating the results and features on the page.
12. Please remember that results from a specific website are more desirable when a
user asks for them, such as “BBC coverage of world cup” or “vimeo funny cat
video.”
13. Please remember that results covering a specific topic are considered more
helpful for queries with a narrow subject matter, such as “season for planting
tulips” or “steps for tying a tie.”

Examples:

LeftBetter

Query Location Url1 Url2 Comment

what 5 letter Omaha, Left Right The intent of
word has the Nebraska, Websearch Websearch the query is to
Results Results
most vowels United States learn what 5
letter word has
the most
vowels. The
Right side has
an irrelevant
instant answer
with a 6-letter
word which is
 misleading
and
significantly
downgrades
the usefulness
of the page,
while the Left
side provides a
QnA with the
list of 5 letter
words with the
most vowels
which satisfies
the query
intent directly.
Since the
query intent is
only satisfied
fully on the
Left side, it is
rated
LeftBetter.
russian figure New York, Left Right The intent of
skater United States Websearch Websearch the query is to
Results Results
find
information
about a
Russian figure
skater. The
right side
provides an
irrelevant news
answer about
skaters from
other
countries,
while the left
side promotes
results about
Russian figure
 skaters which
is exactly what
the user asked
for. Since the
winning side
does a better
job answering
the query
intent, the Left
side is rated
LeftBetter.

RightBetter

Query Location Url1 Url2 Comment

how many El Paso, Texas, Left Right The intent of
pack of United States Websearch Websearch the query is to
Results Results
cigarettes can learn how
usc cross many packs of
cigarettes you
can cross
through US
customs. The
Left side has a
QnA with a
Canadian's
Guide to
bringing
cigarettes
which is
misleading
and
significantly
downgrades
the usefulness
of the page,
while the Right
side provides a
government
 website for US
Customs and
Border
Protection that
contains rules
about tobacco
products for
United States.
Since the
query intent is
only satisfied
fully on the
Right side, it is
rated
RightBetter.
womens Fairport, New Left Right The intent of
olympic figure York, United Websearch Websearch the query is to
Results Results
skating results States find
information on
women's
Olympic figure
skating results.
While the Left
side fails to
answer the
query intent,
the Right side
answers the
query intent
perfectly. For
example, the
Left side
provides a rich
caption from
Wikipedia that
has results for
a short
program which
is misleading,
whereas the
QnA on the
Right side
provides
information
about the final
results in the
Olympics that
took part in
Beijing 2022.
Since the
query intent is
satisfied on
the Right side,
it is rated
RightBetter.