Guidelines: Post Tagging Round 2 - IG Rater Copy

Contents
1. Project Overview
2. Round Two Rating Steps
a. Details Steps
b. Sensitive Content
3. Key Rating Concepts
a. Relevance
b. Granularity
c. Quality
d. Entities
4. Research Tips
a. Side Searches
5. Mixed Examples - Rating Scenarios

Project Overview
In this project, you will review a social media post and then label all of the topics to which it's related. The labeling flow
consists of two rounds:

● Round One: You will view a random social media post and then select from a group of broad topics (e.g. Sports,
Food). 
● Round Two: The posts are then routed to the appropriate topic-specific Round Two queue(s), based on the topics
selected in Round One. In Round Two, you will focus on posts that were previously assigned to just one of the
broad topics, and then choose among a large group of much more specific subtopics which relate to it. 

These Guidelines are for Round Two of the project. In Round Two you will be assigned to a number of the Round Two
topic queues, based on topic knowledge (as demonstrated in qualification) and task availability.
These Guidelines provide a summary of the rating process, and things to consider when working in Round Two.

Round Two Rating Steps

Detailed Steps

1. Review Post Content

Look at the post on the left (1) and spend 30-60 seconds to make sure that you understand it. It's important to
consider any displayed text in conjunction with any media - for example, you'd want to think about how a photo might
be interpreted in combination with any text that's present. In addition to the text itself, if the post contains...

a. ...a photo, consider the photo(s) (including text overlaid on it), as well as any text appearing above the
photo.
b. ...text only, consider the text (including emojis) in the post.
c. ...a link, click on the link and quickly scan the content on the page, then also consider any accompanying
text in the post itself.
d. ...a video, watch the video for up to 60 seconds (if the video is long watch first 10-15 seconds, click
through 5 spots and watch 5-10 seconds at each) then also consider any accompanying text.

2. Conduct Research if Needed

A Side Search can be used to help determine what the content of a post is about when it is unfamiliar. Side searching
involves searching for key terms or names featured in the post via a search engine in a separate browser tab. It won’t be
necessary to do this for all posts, since many can be understood simply by reviewing the content. You will also
encounter posts which feature content that may be familiar to you already. However, in some instances side searches
can be highly beneficial in helping you provide more accurate labels for unfamiliar content.

See the ‘Research Tips’ section of these Guidelines for more detailed examples of how to use Side Searches to conduct
research.

3. Use the Typeahead to Select the Subtopics that the Post is about
Using the Typeahead, enter any subtopics or entities that are relevant to the post. Please try to be as thorough as
possible and enter all that apply, regardless of whether you’re in the queue for that subtopic. When you do so, the
Typeahead will automatically auto-populate with any parent subtopics as well:

Selecting a subtopic from the suggested list will show the different levels of subtopics:
Important: Include all subtopics that are relevant to the post. If there are subtopics that relate to the post, but are not
part of the queue topic, it is still helpful to include these as well. These can only be added from the Subtopics
Typeahead. For example, in the Relationships queue, you should select a relevant subtopic for the post (here LGBTQ
Relationships), and if the post is also related to a Social Issue, you can add the relevant Social Issues subtopic in the
additional subtopic box (here, the subtopic Gender Identity & Sexual Orientation (LGBTQ) Rights & Discrimination was
added).

4. Use the Subtopic Picker

You are able to enter all of the subtopics that are relevant using the Typeahead, but it is also recommended to check
the Subtopic Picker afterwards, to see if there is anything you may have missed.

 In the tool, after you type in and apply all the labels in the Typeahead, click  the ‘next’ button to see the Subtopic
Picker populate. If you hover directly over the label, you’ll see the definition of the subtopic. Review this right before
selecting to make sure it is the right subtopic, paying special attention to the “Excludes” categories listed. Remember
that if a post is related to the subtopic, you should select it even if none of the subtopics under it apply.

 Topics are organized in different sections for easier navigation. General subtopics show up first, followed up by
groupings of subtopics. For groupings of entities (i.e. specific people, organizations, etc.) which often have many
subtopics, the tool will show all labels. Click the caret (^) next to the Parent Label to expand or collapse the list of
subtopics”.

1. You should carefully review the available subtopics here and click any that apply based on your review of the
content.
2.

3. Once you do so, click ‘Next’ and you will be provided a list of deeper subtopics (where available), from the
subtopics you selected.

4.

5. You should continue to click ‘Next’ and select any relevant subtopics from the list of deeper subtopics available
until you exhaust the available lists. At this point you will see the message below. Any subtopics you have
selected will be shown in the ‘Selected’ row. 
6.
Important: If the post is not directly related to any of the listed subtopics but is still appropriate for the queue, you
should select ‘No Topic’, and click on ‘Next’:

5. Check Specificity/Granularity
After selecting the subtopic, take a moment to check the specificity widget. The goal is to choose the most specific
subtopic possible. When the most specific subtopic is chosen, the progress bar will be filled completely and the widget
signals that the specificity is “Very High”. If you notice that the subtopic chosen is not the most specific, please
brainstorm more specific subtopics that are central to the post. Please do your best to choose the most granular
subtopics possible.
6. Check and Submit
 After selecting all the subtopics that are related to a post from the Typeahead and Subtopic Picker (or ‘No Topic’ if no
subtopic applies), take a moment to review all of the subtopics you have selected. If there are any that you no longer
feel should be selected, you can click on the “X” button to remove them.

● NOTE: Removing a subtopic will also remove any of its children. When you find a situation where it seems like
the subtopic should apply without the parent topic, please use your best judgement. If you think there is an error
in the taxonomy, please make note of it in the comments section.

Once you are confident that you have selected all relevant labels, you can press the Submit button at the bottom of
the screen to move to the next Post.

7. Skip Content
If you encounter an error that prevents you from evaluating the post, then use the appropriate 'Reject' button,
according to the descriptions below.

● The tool doesn't load → Click 'Reject : Post Not Loaded' 

● The post is in a language that you are not assigned to work on. → Click 'Reject : Incorrect Language'
○ Note: If majority of the post (>50%) includes the correct language, either in the text in the post, caption of
the post, or spoken in the post, do not reject and use what is understood by the predominant language. 
● The post is not about the high level topic of the queue → Click ‘Reject : Incorrect Queue’
 ○ Note that sometimes a post will belong to the right queue, but there won’t be any additional subtopics
that apply. In this case, do not use the ‘Reject : Incorrect Queue’ option, but rather the ‘No Topic’ checkbox

● You see sensitive content → Click ‘Reject: Sensitive Content’. See below section for information about sensitive
content.

Sensitive Content
“Sensitive” is defined as containing or potentially containing any of the following content:

● Child Exploitation and/or Child Nudity

● Self-Injury and Suicidal Content
● Credible Threats, Violence or Calls to Violence
● Adult Sexual Content and/or Nudity
● Hate Speech
● Acts of Terrorism
● Human Trafficking
● Bullying and Harassment

For these types of content, select 'Reject: Sensitive Content'.

Escalation Protocol 

1. Immediate Escalation: If the content contains imagery or text/voice indicating or soliciting Child Exploitation,
Child Nudity or Human Trafficking, escalate the Job ID immediately to your manager. CEI should never be
screenshot or replicated in any way as it only further exacerbates the issue. Please always look to use task/job
number for issue identification.
a. Child Exploitive Imagery refers to imagery (images, videos) depicting the sexual exploitation of a child.
b. Human Trafficking includes the recruiting, transporting, or harboring of people by means of threat,
coercion, or fraud for the purpose of exploitation. That exploitation can come in many different forms,
including sexual exploitation, forced slavery, slavery, servitude, or the removal of organs.”

Key Rating Concepts

Relevance

If it would make sense to see the post in a Instagram feed about the subtopic, then the post is related to the subtopic
and you should select it. However, it is not enough for a post to merely mention a topic. Some posts explicitly mention
a topic, but do not address the topic such that it would make sense to show in a search for that topic. Conversely, a
post can be strongly related to a topic without calling it by the same name. Consider the following examples - 

● Post is a news article of a boy winning a school competition. There is photo of the boy and his family and dog in
front of their house with the caption: “Competition winner, pictured with his parents, sister and dog” → do NOT
tag ‘Dogs’.
 ○ While a dog is shown and mentioned, the post is not about dogs and this would not make sense to show in
a feed about dogs.
● Post is a meme about “Man's best friend” → Tag ‘Dogs’.
○ While the word “dog” does not appear, the phrase “Man's best friend” refers to dogs. 
Remember to Tag ALL relevant subtopics that apply. Pay particular attention to posts that are about one thing that is
related to multiple topics.

● Post about someone using a new fitness routine to improve a health condition→ Tag 'Health & Medical
Treatments' + ‘Gym Workouts and Fitness Activities’
● Post provides reviews of the best restaurant three friends found while traveling through Italy → Tag ‘Restaurants
and Dining’ + ‘Food Tourism’ + 'Travel Advice’ + 'Friendships’ + ‘Food Awards & Reviews’ + ‘Restaurant Reviews &
Ratings’

Granularity

In Round 2, some posts can be related to a very large number of subtopics. It is sometimes tricky to know how
‘granular’ to be when selecting subtopics (i.e. how deep or thorough should you be when a post has dozens of
potentially relevant subtopics).

Generally, you should strive to be as thorough as you can be and select all subtopics which meet the relevance
definition above. However, there may be cases where a post relates primarily to a broader subtopic and it is not
necessary to tag all the deeper subtopics which are mentioned. Consider the difference between the following
examples -

● Post is photo with a bunch of different types of dogs → Tag 'Dogs' but do NOT tag each dog type
○ The post is not about the individual dog types, but rather dogs in general so only the broad category 'Dogs'
should be tagged.
● Post is photo of bunch of dogs with a list of the 10 best types of dogs → Tag 'Dogs' and tag all 10 specific dog
types
○ The post is about specific types of dogs, so each dog type should be tagged.

If the post links to an article, video, or slideshow that is long and includes a lot of subtopics, scan to see if there are
topics it focuses on most. If so, select those. If not, select the parent category and anything specifically mentioned or
depicted in post itself. 

● Post is a 22 minute makeup tutorial. You click at 5 places throughout and all 5 are talking about eye makeup →
select 'Eye Makeup'
● Post is a 22 minute makeup tutorial. You click at 5 places throughout and all 5 are talking about a different kind
of makeup → select “Makeup' but do NOT select each makeup type
● Post is a 22 minute makeup tutorial covering many types of makeup, but the caption of the post says “Eye
makeup and other beauty tips” and the cover image is of someone applying eye shadow → Tag 'Eye Makeup'

A few tips to keep in mind about the following subtopics:

● Music Roles [Music & Audio]: We would use these labels if the post were about the roles in an industry
perspective.
○ Post is about Beyoncé. We would not label ‘Singer’ unless it’s about Beyoncé and her experience in the
profession
 ● Photography [Visual Arts, Architecture & Crafts]: We would use this label if the post were about the art and
profession of photography
○ This would exclude photos that are shared for a purpose other than appreciating the photo itself

Quality

The quality of a post does NOT impact whether a post is relevant to a subtopic or not. You should always tag all of the
subtopics the post is about without regard for quality, professionalism, or the size of the audience for the post. For
example– 

● Post is a professional video of a Beyoncé concert → Tag 'Musical Performances’ (Plus any additional subtopics
that apply)
● Post is a cellphone video of a band performing in a bar → Tag 'Musical Performances’ (Plus any additional
subtopics that apply)
● Post is a cellphone video of a 3-year-old singing in the car → Tag 'Musical Performances’ (Plus any additional
subtopics that apply)
● Post is a meme about a concert → Tag 'Musical Performances’ (Plus any additional subtopics that apply)

Entities 

Entities are unique people, places or things. You should pay special attention to whether the post is related to any
entities. If the post is about an entity, and that entity is strongly related to a topic, then you should use the Typeahead
to assign the applicable subtopic label to that topic. The Typeahead will auto-populate the parent subtopics after you
select the applicable child subtopic label.  For example - 

● Post is about McDonald's → Tag ‘McDonald’s’ and ‘Restaurants & Dining’ and ‘Fast Food’ will also populate
● Post is about The Beatles → Tag ‘The Beatles (music group)’ and ‘European Music’, ‘Rock Music’, ‘British Music’,
‘60’s Rock Music’ and ‘British Invasion (Music)’ will also populate
● Post is about Beyoncé → Tag ‘Beyoncé (actor, music artist, producer)’ and ‘R&B – Soul Music’, ‘Contemporary R&B
– Soul Music’, ‘Contemporary R&B’ will also populate.
○ Do NOT use the label ‘Singer’ unless the content is specifically about the profession of singing

However, note that just because an entity is strongly associated with a topic, doesn’t mean the post is about the topic. 

Additionally, do not assume that every post which mentions an entity is about that entity. For example, a video about
the Olympics might show the logo of its sponsor, McDonald's, but that isn't enough to consider the post related to
McDonald's.

Do we tag topics related to the author of the post?

The content of the post must be strongly associated with or about the author who wrote it for the author to be applied
as a label, the post can’t just be from the author: 
 

○ A post by Joe Biden about a baseball game is not about Joe Biden.
○ A post by Joe Biden presenting his political platform is about Joe Biden.

 
Please note the post you are labeling must also belong to the queue you are rating in or “reject: incorrect queue”
would need to be applied. 

Research Tips
Side Searches

If there are names or phrases in a post that you do not understand, you should look them up (i.e. the name of a
famous person, company/organization or concept). You can also look up the author of the post for more context (if
relevant to the content of the post) – they may be well known for writing about a particular topic. 

Below the post there is sometimes a section called “Google Links”. The links provided will open a new tab to Google
search results for the specific term. These can be a great shortcut, but keep in mind the links are auto generated based
on phrases in the post. Use your judgement to determine if they are relevant.

Example 1
Post is by CIZE. It may not be initially clear to someone unfamiliar with this workout what it is. A quick side
search shows that all the results relate to this dance workout – see the repetition of these key phrases in the search
results.

● Dance and Gym Workouts & Fitness Activities can be selected from the Subtopic Picker or Typeahead. As it is a form
of rhythmic cardio fitness, Aerobics would also be applicable, as would Fitness Workout Plans, since Cize is a
workout plan you can purchase.

Example 2
This post features ‘Wild Horses’ at the Wilbur. For those unfamiliar, it may be unclear who Wild Horses are. It would be
easy to incorrectly guess that they might be a music group and tag as such. As we are unsure, we should conduct a side
search to check.
A side search for just ‘Wild Horses’ is too broad and shows many results that aren’t a match for this group. We can make
the search more specific to this post by adding another key term from the post, ‘Wilbur’. 
Doing so brings up many results indicating that they are an improv comedy team who and that this performance is part
of a comedy festival in Boston.
 ● Comedy and Improv Comedy can be selected from the Subtopic Picker or Typeahead. As this is also a type of
festival, a search for Festival in the Typeahead shows that Performing Arts Festivals would be an appropriate topic.
Vacations & Leisure Activities and Festivals & Cultural Events are automatically selected as well as parent topics.

Example 3

● The post features a long video (over 1 hour) of a religious service. It may not be immediately obvious what
religious denomination the service is for. However, we are given the name of the minister – and a quick side
search gives results on Wikipedia and his official website that confirms he is a pastor for a Pentecostal Christian
church, a type of evangelical Protestantism. 

● Wikipedia Entry – Note the references to Pentecostal Christianity and Evangelical Christianity.
● Pentecostalism and Evangelical Protestantism can be selected from the Typeahead, which also tags the parent
topics, Christianity and Protestantism. As this is a religious service on a spiritual channel, Prayers and Blessings and
Religion & Spirituality TV & Movies also apply.