Professional Documents
Culture Documents
Page-Question Interesting (Single) Hitapp: Instructions
Page-Question Interesting (Single) Hitapp: Instructions
Instructions
This hit provides a document (namely a web page) and a question answer pair.
Our goal is to provide interesting questions to help users understand the
document better. So we want the judgers to label the interestingness of question
and answer.
There are three levels of interesting:
Good:
• Contribute to a better understanding of the whole document.
• Sparks user's curiosity. Surprises user.
Fair:
• Contribute to a better understanding of a part of the document.
• Contribute to a better understanding of the whole document, the answer is
too obvious from the document. See Example 4.
Bad:
• The question is unrelated to or cannot be answered by the document. See
Example 6.
• Question is trivial or the answer is a trivial fact. See Example 5.
• Is a controversial question (conspiracy, offensive, etc).
• The question is hard to understand (maybe grammar error or not
authentic).
• The question is a common sense. Common sense consists of two situations:
o It is a common sense for general people.
For example, Who is the Michael Jordan?
o It should be a common sense for who is visiting the website.
For example. For who is visiting a website which talks about
supporting the US president Joe Biden, a question like Which party is
Biden from is a common sense for him.
Examples
Example 1
Document:
https://www.goarmy.com/ranger/join-the-army-rangers/requirements-to-join.html
Question:
Answer:
In order to qualify for service in the 75th Ranger Regiment, you must: 1 Possess U.S. ... 2
Volunteer for assignment and be on active duty 3 Have a General Technical Score of 105 or
higher 4 Pass physical requirements, including the Ranger 5 Fitness Test (58 push-ups, 69 sit-
ups, five mile run in 40 minutes or less, six pull-ups), the Water Survival Assessment, and a
12-mile march with a 35-pound rucksack and weapon ... More items...
Label: Good
Comment:
The document is talking about the qualifications for the 75th Ranger Regiment and how to
join. So Question and Answer are exactly the main topic of it. It is helpful for the users quickly
understand the document;
Example 2
Document:
https://developers.google.com/apps-script/guides/sheets/macros
Question:
Answer:
Editing macros. You can edit macros attached to a sheet by doing the following: In the Google
Sheets UI, select Tools > Macros > Manage macros. Find the macro you want to edit and
select more_vert > Edit macro. This opens the Apps Script editor to the project file containing
the macro function. Edit the macro function to change the macro behavior. ...
Label: Good
Comment:
The page talks about how to edit macro. The question contributes to better understand of
whole document, so its label belongs to top level
Example 3
Document:
https://mollymaid.com/cleaning-tips/schedules-charts-and-checklists/house-cleaning-
checklist
Question:
Answer:
Label: Good
Comment:
Although the qualifier “new” is added, it belongs to a kind of the “house” that the page talks
about, and the answer can be obtained from it.
Example 4
Document:
https://developers.google.com/apps-script/guides/sheets/macros
Question:
Answer:
You can also activate the macro from the Google Sheets Tools > Macros menu. When you
record a macro, Google Sheets automatically creates an Apps Script function (the macro
function) that replicates the macro steps. The macro function is added to an Apps Script
project bound to the sheet, in a file titled macros.gs.
Label: Fair
Comment:
The Question and Answer is about the page. But the answer is too obvious: the whole
document is talking about how to edit, the answer must be yes.
Example 5
Document:
https://www.commonsensemedia.org/book-reviews/wish
Question:
Answer:
Label: Bad
Comment:
The question asks about trivial information of the document.
Example 6
Document:
https://support.google.com/mail/troubleshooter/2650727?hl=en-gb
Question:
Answer:
Troubleshooting steps 1 Update your Gmail app. To get the latest fixes on problems with
sending or receiving email, update your Gmail app. 2 Restart your device 3 Check your
settings. Try opening a website. ... 4 Clear your storage. If your phone or tablet is running out
of space, syncing won't work. 5 Check your password. On your computer, open Gmail. ... 6
Clear your Gmail information. ...
Label: Bad
Comment:
This question is rather vague, and the expression of "not working" is not clear enough, so
that the answer cannot be obtained directly from the document