Yukon Qualification Exam

AT A GLANCE: PART 3

WHERE IS IT?
Using your laptop or desktop computer, go to raterhub.com/evaluation/rater and log
in with the gmail account you provided to Appen. Click on the "Acquire if available"
button to start.

WHAT'S IN IT?
7 tasks requiring you to assign a Needs Met rating based on FailsM SM MM HM FullyM
how well the result meets the user intent
You must enter both Page Quality (PQ) and Needs Met (NM)
ratings, but only the NM rating will be scored

HOW DO I PASS IT?

OPEN THE GUIDELINES USE THE NM RATING GRID

The exam is open book, so refer to the The Needs Met (NM) Rating Grid in the
General Guidelines regularly during Learning Center will provide you with
the exam. suggested ratings for the tasks.

UNDERSTAND WHAT YOU'RE RATING

Web Search Results Block (WSRB)
Includes only a title, landing page URL, and
brief snippet
Should be rated solely on the landing page
Typically not rated above HM unless they
or dual intent. satisfy
or dual a website or dual intent
intent.
SEND TO DEVICE Special Content Results Block (SCRB)
You'll need to use your Provides "at a glance" result without
mobile device to rate needing to go to the landing page
parts of the Needs Met Should be rated solely on the content block
tasks. Look for the "Send Can be rated as high as HM+ or FullyM so
to Device" instructions in long as it meets all the requirements for the
the Learning Center. rating

QUICK RATING GUIDE

Start from Moderately Meets (MM) and move the slider based on your answers to both of the questions below.

FailsM FailsM+ SM SM+ MM MM+ HM FullyM

(Fails to Meet) (Somewhat Meets)
(Moderately Meets)
(Highly Meets) (Fully Meets)

Is the result close enough to be helpful for a user in this location?

Yes, the result is close to the user's location: +1 (MM+ to HM)
Yes, but closer results would be better: +0.5 (MM+)
Location not important for this query: No change (MM)
Sort of, the result is near the user but too far to be convenient: -1 (SM to SM+)
No, the result is nowhere near the intended location: FailsM (skip next question)

Does the result meet the query intent?

Yes, the result is highly relevant, accurate, up-to-date, highly credible, and gives the user
exactly what they are looking for and more: +2 (HM to FullyM*)
Yes, it's highly relevant, accurate, up-to-date, and highly credible: +1 (MM+ to HM)
Yes, it's relevant and meets the bare minimum of the query intent: +0 to 0.5 (MM to MM+)
Sort of, it is relevant but slightly out-of-date or has minor inaccuracies: -1 (SM to SM+)
Not really, it is on-topic but too broad, inaccurate, or outdated to be useful: -2 (FailsM+ to SM)
No, itisn't what the user is looking for and would satisfy no one: FailsM

ALWAYS RATE THESE AS FAILS TO MEET FINAL TIPS & REMINDERS

Harmful to Self or Other Individuals (Section 4.2)
Harmful to Specified Groups (Section 4.3)
Harmfully Misleading Information (Section 4.4)
Useless results despite a high PQ rating
Untrustworthy (Section 4.5)
Spammy (Section 4.6)
Porn (Section 15.1)
When unsure between two ratings, choose the
lower rating
News or encyclopedia information must be
accurate, up-to-date, and highly credible
Medical or science information must align with
well-established medical or scientific consensus

A project by