Guidelines: Cucamonga - Saliency

Project Overview

In this project, you will review a post to determine if the post is or is not related to a
specified topic. This will help us to serve users better, more relevant content to
improve their experience. On the left side of the screen you will see a post. On the
right side of the screen you will see a topic, a definition of the topic, and a question.
Please provide the best answer and click submit.

Instructions

1. In the right side panel, review the topic and the definition provided. Spend 5-
20 seconds to make sure you understand it.
a. The topic you are evaluating is the phrase in the header. In the
example above the topic is 'Spice Girls'.
b. Below the header you will find a wiki page and wiki preview of
the topic. This is helpful in cases where two topics may have the same
name but mean very different things (for example, if a sports team is
named after a city, the wiki page will make it clear that the topic in
question is the sports team).
2. Look at the post on the left and spend 5-20 seconds to make sure that you
understand it.
a. It's important to consider any text in conjunction with any media - for
example, think about how a photo might be interpreted in
combination with any text that's present. If the posts contains...
 i. ...a photo, consider the photo(s) (including text overlaid on it),
as well as any text appearing above the photo.
ii. ...text only, consider the text (including emojis) in the post.
iii. ...a link, click on the link and quickly scan the content on the
page, then also consider any accompanying text in the post
itself.
iv. ...a video, watch the video for up to 60 seconds (if the video is
long watch first 20-30 seconds, click through 4 other spots and
watch 5-10 seconds at each) then also consider any
accompanying text.
b. If there are names or phrases in a post that you do not understand,
you should look them up. Similarly, if you think a post may be
referencing a topic, but you are not sure, do a search to find out. You
can also look up the author of the post for more context.
3. In the right side panel select 'Yes' or 'No' as appropriate for the question 'Is
this post about the entity described in the above Wiki?'
a. See section below called 'What does it mean to be 'about' a entity?' for
additional guidelines.
b. Again, make sure to answer the question for the specific entity, and
not the parent topics.
4. Click 'Submit'.
Note: If you encounter an error that prevents you from evaluating the post, then use
the appropriate 'Skip' button, according to the descriptions below.
• The tool doesn't load → Click 'Reject : Post not loaded'
• The post is in a language that you don't understand → Click'Reject :
Incorrect Language'
• You see sensitive content → Click ‘Reject: Sensitive Content’. See below
section for information about sensitive content.
Sensitive Content
“Sensitive” is defined as containing or potentially containing any of the following
content:
• Child Exploitation and/or Child Nudity
• Self Injury and Suicidal Content
• Credible Threats, Violence or Calls to Violence
• Adult Sexual Content and/or Nudity
• Hate Speech
• Acts of Terrorism
• Human Trafficking
• Bullying and Harassment

Escalation Protocol
1. Immediate Escalation: If the content contains imagery or text/voice
indicating or soliciting Child Exploitation, Child Nudity or Human
Trafficking, escalate the Job ID immediately to your manager. CEI should
 never be screenshot or replicated in any way as it only further exacerbates
the issue. Please always look to use task/job number for issue identification.
a. Child Exploitive Imagery refers to imagery (images, videos) depicting
the sexual exploitation of a child.
b. Human Trafficking includes the recruiting, transporting, or harboring
of people by means of threat, coercion, or fraud for the purpose of
exploitation. That exploitation can come in many different forms,
including sexual exploitation, forced slavery, slavery, servitude, or the
removal of organs.
2. If you are not comfortable reviewing the content you can skip the job after
escalating.

What does it mean to be 'related'?

If it would makes sense to see the post in a Facebook feed about topic X, then the
post is related to the topic.

How 'related' does it have to be?

It is not sufficient for a post to merely mention a topic. Some posts explicitly
mention a topic, but do not address the topic such that it would make sense to show
in a search for that topic. Conversely, a post can be strongly related to a topic
without calling it by the same name.
• Post is a news article of a boy winning a school competition. There is photo of
the boy and his family and dog in front of their house with the caption:
“Competition winner, pictured with his parents, sister and dog” → this is not
about 'Dogs'.
o While a dog is shown and mentioned, the post is not about dogs and
this would not make sense to show in a feed about 'Dogs'.
• Post is a meme about “Man's best friend” → this is about 'Dogs'.
o While the word “dog” does not appear, the phrase “Man's best friend”
refers to dogs so this post is about 'Dogs'.

What if the post is about many topics?

A post can be about more than one topic. In this workflow you only need to consider
if it is about the specified topic.

Consider if the post is focused on the specific topic asked or on a broader parent
category.
• Post is photo with a bunch of different types of dogs of which one is a Golden
Retrievers → This is NOT about Golden Retrievers
o The post is not about the individual dog types, but rather dogs in
general.
 • Post is photo of bunch of dogs with a list of the 10 best types of dogs of which
Golden Retrievers is one→ This IS about Golden Retrievers
o The post is about specific types of dogs, not just dogs in general.
If the post links to an article, video, or slideshow that is long and includes a lot of
subtopics, scan to see if there are topics it focuses on most. If it has a focus on your
topic, select yes. Otherwise, select no.
• Post is a 22 minute video of a sports show. You click at 5 places throughout
and all 5 are talking about NY Yankees → This IS about 'NY Yankees'.
• Post is a 22 minute video of a sports show. You click at 5 places throughout
and all 5 are talking about a different sports headlines, of which one is
about NY Yankees → This is NOT about 'NY Yankees'
• Post is a 22 minute video of a sports show covering many headlines, but the
caption of the post says “NY Yankees make biggest trade ever, Monday night
football highlights, and more” and the cover image is of a Yankee player →
This IS about 'NY Yankees'

How do we tag posts about a subset of the topic?

Only select “Yes” if the post is about a subset the specific topic asked.
• Post is about Barack Obama → This IS about 'US Presidents'
• Post is about McDonald's → This is about 'Fast Food Restaurants'
• Post is about Classic Rock → This is about 'Rock Music'
However, do not assume that every post which mentions an entity is about that
entity. For example, a video about the Olympics might show the logo of its sponsor,
McDonald's, but that isn't enough to consider the post related to McDonald's (or to
'Food & Drink').

Select “No” if the post is about a something broader than the specified topic (and the
specified topic is not a focus)
• Post is about US presidents → Post is is NOT necessarily about 'Barack
Obama'
• Post is about Fast Food Restaurants → Post is is NOT necessarily about
'McDonald's'
• Post is about Rock Music → Post is is NOT not necessarily about 'Classic Rock'

What about low quality / personal posts?

Make your selection without regard for quality, professionalism, or the size of the
audience for the post.
• Post is a professional video of Beyoncé concert → This is about Beyoncé
• Post is a cellphone video of a Beyoncé concert → This is about Beyoncé
• Post is a cellphone video of a 3 year old singing a Beyoncé song → This is
about Beyoncé
• Post is a meme about Beyoncé → This is about Beyoncé
Do we tag topics related to the author of the post?

If the author of a post is a well known individual, tag topics relevant to the author as
well as the content of the post. This ONLY applies to individuals and not any other
types of authors.
• Barack Obama writes a post saying the Chicago Bulls are the best team
ever → This is about 'Politics' as this post is expressing Barack Obama's
opinion and Barack Obama is an individual strongly associated with politics.
• The Financial Times writes a post saying the Chicago Bulls are the best team
ever → This is NOT about 'Business, Finance & Economic'. While the
Financial Times is an entity strongly related to finance, it is not an individual.

How do we tag Memes?

Memes often use content from one topic to make a statement about another topic. In
this case, tag both topics.
• Meme of Batman looking sad with the caption “How I feel when my dog
walks away from me” → This is about 'Batman', it is also about 'Dogs'

Research

If there are names or phrases in a post that you do not understand, you should look
them up. Similarly, if you think a post may be referencing a topic, but you are not
sure, do a search to find out. You can also look up the author of the post for more
context. You can use the links provided below the post, however those links are auto
generated, so use your judgment to determine if they are relevant.

Be careful to look closely!

Often the post is going to have some connection to the topic, remember to consider
if it would makes sense to see the post in a Facebook feed about the topic.

Sometimes the post and the topic are going to be very similar and at quick glance it
may seem they are the same. One of the main reasons for this workflow is to catch
those cases so we are counting on you to be diligent. For example, the job below
asks about “Nebraska Cornhuskers Football”. The post is talking about a Nebraska
Cornhuskers game, but it is about the volleyball team, not the football team. So this
post is NOT about Nebraska Cornhuskers Football.

Also note that in the job below the google link section includes a link to Nebraska
Cornhuskers Football. Careful not to rely on those links for clues as to what the post
is about. They are there to help you do research if needed, but they are also auto
generated so you have to use your judgment to determine when they are relevant.
Also keep an eye out for topics where the name can have multiple meanings. For
example, if the topic 'Squash' is referring to the vegetable (per the wiki), then
content about Squash the racket sport would NOT be relevant.