Answer Relevance: Instructions

You might also like

Download as doc, pdf, or txt
Download as doc, pdf, or txt
You are on page 1of 2

Answer Relevance

Instructions
This hit provides a Query, Answer and Url. We expect the judges to rate the
Answer to the given Query as Excellent, Bad, Fair or Detrimental.

The Answer is Excellent if it is accurate, reasonably complete, well


formatted and answers the question directly.

The Answer is Bad if it does not meet any of the above criteria.
If answer is inaccurate or has bad formatting they should be immediately flagged
as bad. If answer is incomplete and does not help the user get the direct answer
he is looking for then too it should be marked bad.

The Answer is Fair if it you think it is accurate, slightly incomplete but user can
infer the core information he is looking for from the answer. It can also be marked
Fair if the intent is ambiguous and the answer is something specific but broadly
common interpretation.

You might also like