Climate Science Classification

Guidelines

5 l
25 tia
1. Introduction
1.1 Overview and Context

The goal of this workflow is to determine whether or not a Facebook or Instagram

post is related to climate science topics. The data from this workflow will be used to

22 n
measure the prevalence of climate on our platforms, and to train our classifier to
recognize such content.
e
In each job, you will see a post which may include some text, a photo, a video, a link,
and the author. You may need to do some additional side searches to better
understand the intent and content of the post, after which you will determine if the
post is related to climate science topics and related subtopics.
56 id
1.2 Understanding the Task
1.2.1 The Rating UI in SRT
nf

[insert finalized rating UI screenshot]

1.2.2 The Rating Process

1. Review the post and perform side searches as needed. Spend 30-60
seconds on the post. Perform side searches on Google and/or Facebook if
Co

you do not understand what the post is about.

2. Determine Ratability:
a. If you can rate the job, choose “Yes” and continue.
b. If you cannot rate the job, choose the appropriate reason and click
“Submit”.
3. Answer the core questions based on definitions listed in Section 2:
a. Q1: Is this related to climate science/climate change?
i. Yes
ii. No
b. Q2: [If Q1 = Yes] What subtopics would this climate content fall
under?
i. Biodiversity
ii. Climate Activism
 iii. Transportation
iv. Energy System
v. Industry/Economics
vi. Food/Agriculture
vii. Greenhouse Gas Emissions

5 l
viii. Renewable/Clean Energy

25 tia
ix. Electrification
x. Carbon Sequestration
xi. Land Use
xii. Human Adaptation
xiii. Government Policy
xiv. The science of water/oceans
xv. Physical and Health Related Climate Risks
xvi. Air Quality

22 ne xvii. Hurricanes / Cyclones

xviii. Wildfires
xix. Heat waves
xx. Droughts
xxi. Floods
xxii. Cold Snaps / Winter Storms
xxiii. Disease Prevalence
56 id
xxiv. Crop Failure / Famine
xxv. Other [Text]
c. Q3 [If Q1 = Yes] If you selected Climate, select whether the post
has a [neutral / positive / negative] sentiment towards the topic
i. Neutral
nf

ii. Positive
iii. Negative (includes hoaxes, conspiracies)
iv. Mix or Unclear
d. Please select all that apply to the content:
i. Content is not supporting either side
ii. Content is simply informational
Co

iii. Content expresses confidence for a particular topic

iv. Content expresses confirmation or general agreement
v. Content expresses support for a particular topic
vi. Content expresses denial or disagreement
vii. Content expresses skepticism or questioning
viii. Content expresses minimization
ix. None of the above
1.2.3 How to Review a Post

A post may contain only text, or text with a photo/photo albums, a link, or a video.
It's important to review the post in its entirety — that is, consider any text in
conjunction with any media to fully understand the post's meaning. For example,
 think about how a photo might be interpreted in combination with any text that's
present.

If the posts contains...

1. ...a photo/photo album, consider the photo(s), including any overlaid text,

5 l
as well as any text appearing above the photo/album.

25 tia
2. ...text only, consider the text (including emojis) in the post.
3. ...a link, click on the link and quickly scan the content on the page, then also
consider any accompanying text in the post itself. All non-hyperlinked URLs
should be reviewed as well.
4. ...a video, follow this guidance:
a. For videos 90 seconds (1:30) or less in length, watch the entire
video. For videos longer than 90 seconds, you should watch at least
the first 30 seconds, middle 30 seconds, and final 30 seconds.

22 ne b. Consider the video content you watched in conjunction with the

following:
i. Review the cover photo of the video for content related to
Political/Social Issues Topics.
ii. Skim thumbnails (if available) at the bottom. If anything
appears to be related to Political/Social Issues Topics, jump to
that timestamp and watch about 15 seconds to verify.
56 id
iii. If the video transcript is available, scan the text for
Political/Social Issue Topics. You may also jump to the
timestamp and watch about 15 seconds to verify.
iv. Always consider the video content in conjunction with any
accompanying text in the post itself.
nf

1.2.4 Side Search Tips

Side search is an essential part of evaluating any post. Here are some tips on how to
conduct side searches:
• If there are names or phrases in a post that you do not understand, you
Co

should look them up using Google or Bing. Similarly, if you think a post may
be referencing a topic, but you are not sure, do a search to find out.
• You can also look up the author of the post on Facebook for more context.
• When a post contains only a photo without any caption, you may need to do a
Google Images search and read the accompanying text of the results to gain
context.

1.2.5 Reasons Why a Job Cannot be Rated

If you answer “No” to the question “Can you rate this job?” please choose the
appropriate reason below:
 a. No content to review: When there is no content available under
“Content Under Review.” (i.e. no images, no videos, no captions, or no
URLs).
b. Broken image: The post has loaded, however the image associated
with the post has not loaded.

5 l
c. The video is not playing: The post has loaded, however the video

25 tia
associated with the post is not able to play.
d. Broken URL: The post has loaded, however the URL associated with
the post is broken and raters are unable to access the external page.
e. The post is in a foreign language: The post is in a language that you
don't understand. See section 1.2.6 - Foreign Language below for
more information.
f. The job keeps loading: The post continues to load on the screen and
is unable to display content to review.

22 ne g. Sensitive content: The post contains sensitive content. See section

1.2.7 - Sensitive Content below for more information.
h. Hashtag baiting: The post includes more than 10+ hashtags and is
not topically related to the hashtag.
i. Reason not listed here: You encounter a different problem that
prevents you from labeling.
56 id
1.2.6 Foreign Language

Occasionally, jobs will feature words, letters and/or phrases from languages outside
of your queue's target language. Please review all job components (ex. caption,
image, URL, video, etc), as instructed above, and comprehend the overall message or
story of the post.
nf

Do not choose “No - Foreign Language” if the post contains proper names and is in
Latin characters (e.g., “Real Sociedad de Fútbol”), or if only a small part of the result
is in a foreign language and you can understand the content from the remaining
portion. Only choose “No - Foreign Language” if the post is predominantly in a
Co

foreign language and you cannot understand it enough to rate. If a video is in a

foreign language, but has English subtitles, it is ok to rate.

1.2.7 Sensitive Content

“Sensitive content” refers to posts that contain, or potentially contain, any of the
following information:
• Child Exploitation and/or Child Nudity
• Self Injury and Suicidal Content
• Credible Threats, Violence or Calls to Violence
• Adult Sexual Content and/or Nudity
• Hate Speech
• Acts of Terrorism
 • Human Trafficking
• Bullying and Harassment
• Unlawful Sale of Animals: Consider the type of animal being sold as well as
the seller. Sale of animals by private individuals (rather than by businesses)
is considered sensitive

5 l
• Violent, gruesome, or repulsive content, intended to shock users, for example

25 tia
dismembered body parts or visible internal organs (unless in a medical
setting)
• Personally Identifiable Information (PII): This includes names, residential
addresses, and phone numbers of private individuals. This does NOT include
business addresses and phone numbers, or business names (e.g., “Becky
Smith Travel Agency” or “Law Offices of Perry R. Clark”).
Note: Instructions for Immediate Escalation — If potential Child Exploitation

22 n
and/or Child Nudity or Self-Injury/Suicidal Content or Time-Sensitive Credible
Threat is the reason for marking a job as “sensitive," escalate the Job ID immediately
to your manager. DO NOT TAKE A SCREENSHOT. Please refer to the Escalation
Guidance handout for further details.
e
56 id
2. Key Definitions
2.1 Climate Science Topics

Content related to the scientific study of climate, scientifically defined as weather

nf

conditions averaged over a period of time. This modern field of study is regarded as
a brand of the atmospheric sciences and a subfield of physical geography, which is
one of the Earth sciences.
• Includes -
o solar energy products pros/cons.
Co

o information on climate change and discussions in context with

society, politics, economy, jobs etc.
o air pollutions, fossil fuels, coal energy, renewable energy, natural
resources when it relates to climate
o Leading advocates & scholars on environmental topics. Does not
include the CXO or founders of energy companies.). Does include NGO
organizations working on environmental topics.
Excludes - Car/boat/plane racing and other motorsports unless specifically about
the vehicles (code as Sports), Cycling when focused on recreation/sport (code as
Sports).

Climate Topics can include (but are not limited to):

 2.1.1 Climate Change
• Includes:
o Topics that directly mention “climate change”, “climate science”, or
“global warming”
o Climate impact, climate mitigation, climate adaptation, climate

5 l
solutions

25 tia
o Climate modeling/prediction & carbon accounting
o Education (sans editorialization), by content partners, about climate
science, impacts, or solutions, including what a treaty, bill, etc. says
o How different communities are impacted disproportionately by
climate change (i.e. Climate justice)
• Excludes:
o Ozone, which is related to the broader environment but not directly
related to climate change

22 ne o Content supporting or opposing pieces of legislation, executive orders,

or other government actions.
o Content responding to, praising, condemning, or otherwise
characterizing a political actor or actors.
o Content that (1) trivialises climate change, (2) opposes or supports
specific state actions in response to climate change, or (3) places the
burden of fixing/mitigating/fighting climate change solely on
56 id
individual actors.
2.1.2 Greenhouse Gas (GHG) Emissions
• Includes:
o Types of gases/emissions and their sources including particular
industries and their effect on humans and/or the environment.
nf

o GHG potency and how they affect climate change over different time
periods- e.g. methane is a ‘stronger’ GHG than carbon dioxide in the
short run, while CO2 is stronger over longer time periods.
o The scale of the emissions e.g from different sources both
geographical and industrial
Co

o How GHGs trap heat and affect the climate e.g. how global warming
actually works
o GHG gas capture and storage
2.1.3 The Science of Water + Oceans
• Includes:
o Ocean acidification, coral die off/destruction and related biodiversity
impacts increased ocean temperatures
o Water stress (including scarcity, accessibility, quality), and flooding
o Rising sea levels due to melting of polar ice caps and glaciers
o Changes in the location, timing, and duration of precipitation events
(e.g., the Indian Monsoon)
o Climate change impacts on water currents and the interplay between
the oceans and climate
 • Excludes:
o Fish species decline due to overfishing
o Coral and marine ecosystem damage from tourism
2.1.4 Physical and Health Related Climate Risks (Climate

5 l
Resilience/Adaptation)
• Includes:

25 tia
o Increases in extreme weather exacerbated by climate change: Heat
waves (including drought), water stress (including water scarcity,
accessibility, quality), flooding, wildfires, hurricanes, typhoons
o Rising sea levels and its effect on climate migration and displacement
o Rise in infectious diseases due to rising temperatures
o Health problems and death associated with heat stress and other
extreme weather event

22 ne o Malnutrition due to crop/market failures due to rising temperatures

and disruptions to global supply chains
o Damage/destruction of infrastructure associated with the above
o Examples of how different communities are impacted
disproportionately by climate change (i.e. Climate justice)
• Excludes
o Weather, generally
56 id
2.1.5 The Science of Energy/Transport

Renewable Energy, Solar Power, and Energy Efficiency, etc by themselves are
climate adjacent, but when discussing how renewable energy will help mitigate
some of the effects of climate change they can be climate-related.
nf

• Includes:
o The energy grid/infrastructure and technology requirements to
respond to decarbonization
o Different transport methods and how they contribute i.e the
contribution of cars/driving/public transport/shipping/air
travel/freight etc to carbon emissions
Co

o New sources of energy technology being developed and potential

benefits to the climate
o Energy efficiency
o Topics related to technologies intended for climate such as geo-
engineering, carbon capture, carbon sequestration
• Excludes:
o Renewable Energy or Fossil Fuels where only discussing energy
security or other context outside of climate

2.1.6 Food/Agriculture

Forest clearing/fires, the dairy industry, and other aspects related to how our
agricultural and food systems are having a direct impact on climate change are all in
 scope whereas topics such as organic vs. non-organic are only relevant when the
research is related to atmospheric emissions and climate change.
• Includes:
o Dairy and meat contribution to carbon emissions
o Emissions associated with food waste

5 l
o Lower carbon intense diets such as veganism

25 tia
o Soil and its role in carbon sequestration and management
o The collapse of forests, land, soil, soil health and its climate impacts
• Excludes:
o Organic/non-organic w/out tie to emissions
2.1.7 Waste/Pollution

Waste and Pollution content by itself is “climate adjacent” and is not in scope.. But

22 n
when tied to its effects on climate, or the impact of climate change on waste and
pollution, it could be considered. Below are examples that could be considered in
scope: but should be considered when also discussing the contribution of
production industries (from mining to tech to fashion) on climate change. Again,
e
issues and research on these aspects should be limited to where there is reputable
scientific research related to how it is driving or contributing to climate change.
56 id
• Includes:
o The impact of production/overproduction and various industries’
waste on carbon emissions/climate change
o How waste and waste management responds to climate change and
the potential for recycling and the circular economy to bring these
nf

emissions down
o How rising sea levels (due to climate change) will release toxins from
polluted land areas into waterways
o Air pollution’s impact on climate change, or climate change’s impact
on air quality (e.g. air pollution in the form of carbon dioxide and
methane raises the earth’s temperature and other types of air
Co

pollution are then worsened by that increased heat)

o Examples of how different communities are impacted
disproportionately by climate change (i.e. Climate justice)
• Excludes:
o Microbeads, microplastics, and other marine litter.
o Littering and waste or recycling process
2.1.8 Biodiversity

Biodiversity issues are not always related to climate change. For example, species
loss from poaching or urban sprawl is not necessarily related to species loss from
climate change changing the habitat of a particular species. It's important that the
emphasis of any news/content is grounded on the principle of climate change as
opposed to other environmental issues.
 • Includes
o Climate change’s negative impacts on biodiversity
o Amazon rainforest absorbing carbon dioxide
o How wildlife and species’s behavior, habitats, and migration patterns
are changing due to climate change

5 l
• Excludes:

25 tia
o Biodiversity itself as a topic
o Conservation and species decline due to poaching/hunting
o Fish species decline due to overfishing
o Wildlife and animals generally
2.1.9 Climate Subtopics

Common examples of climate subtopics, according to the above criteria, include (but

22 n
are not limited to):
1. Biodiversity
2. Climate Activism
3. Transportation
e
4. Energy System
5. Industry/Economics
6. Food/Agriculture
56 id
7. Greenhouse Gas Emissions
8. Renewable/Clean Energy
9. Electrification
a. The process of replacing technologies that use fossil fuels (coal, oil,
and natural gas) with technologies that use electricity as a source of
energy
nf

10. Carbon Sequestration

a. The process of capturing produced carbon dioxide and subsequently
storing it safely, away from the atmosphereLand Use
11. Human Adaptation
a. Adaptation and mitigation due to adverse impacts of climate changes
Co

12. Government Policy

13. The science of water/oceans
14. Physical and Health Related Climate Risks
15. Air Quality
16. Hurricanes / Cyclones
17. Wildfires
18. Heat waves
19. Droughts
20. Floods
21. Cold Snaps / Winter Storms
22. Disease Prevalence
23. Crop Failure / Famine
24. Other [Text]
 2.2 Neutral / Positive / Negative Sentiment
• Neutral: Select Neutral if the content is not supporting either side or if the
content is simply informational.
• Positive: Select Positive if the content expresses confidence or support for a
particular topic

5 l
• Negative: Select Negative if content expresses denial, disagreement, or

25 tia
refusal for a particular topic
o Common areas of denial/myths:
▪ Recent global warming is caused by natural events, not
humans
▪ Too much carbon dioxide in the atmosphere does not harms
earth's plant life
▪ Overall, polar bear populations are not declining because of

22 n
•
e global warming
▪ The severity of recent droughts and wildfires is not driven by
the changing climate
▪ The cause of climate change is not widely agreed upon in the
scientific community
▪ The cost of renewable energy is expensive
Unsure - Conflicting Sentiment: Select this option if you’re unsure of the
sentiment of the post or if the content expresses both positive and negative
56 id
sentiment about multiple topics.

2.2.1 Additional Characteristics Around Sentiment

• Content is simply informational
• Content is not supporting either side
nf

• Content expresses confidence for a particular topic

o Content is fully confident in mainstream scientific consensus that the
global climate is changing due to human activity
o This option only pertains to denial of scientific issues, not
disagreements about policy or other responses to climate change
• Content expresses confirmation or general agreement
Co

o Content agrees with most facts pertaining to climate, no skepticism.

• Content expresses support for a particular topic
o This option should be selected for support for policy or other
responses to climate change
• Content expresses denial or disagreement
o Content must explicitly rebuke or reject, the mainstream scientific
consensus that the global climate is changing due to human activity
o This option only pertains to denial of scientific issues, not
disagreements about policy or other responses to climate change
o Ex: “Now that the pandemic is ending, climate change is just the latest
way to keep us scared, but it’s completely fake. Human’s can’t alter the
climate.”
• Content expresses skepticism or questioning
 o Content calls into question the science of climate change without an
explicit denial
o This option also does not pertain to skepticism or criticism of policies
or other responses to climate change, only skepticism regarding the
science

5 l
o Ex: “I just don’t think we know anything for sure about climate

25 tia
change. We don’t have enough information to know humans are the
cause - much less to predict the future!”
• Content expresses minimization
o Content attempts to downplay the impact or scope of climate change
or downplay the role of human activity as a cause of climate change
o Ex: “Maybe humans put some CO2 into the air, but it’s all a drop in the
bucket compared to the natural carbon in the atmosphere. Hardly
seems like humans are to blame for global warming.”

22 n
•
e o Ex: “They say the Earth’s going to get a few degrees warmer, and I’m
supposed to be concerned? I’m looking forward to it! It’ll be nice and
warm in March soon!”
None of the above
o Any content that does not meet criteria for options A-H above.
o Important note - this option includes any content that contains the
type of content describe in A-C, but solely for the purpose of reporting
56 id
on the claim by a third party, or for the purpose of debunking the
content
▪ Ex: “My grandma doesn’t ‘believe in’ climate change, because
we don’t ‘know enough’. How do I tell her it’s science, not the
Tooth Fairy, it doesn’t matter if she ‘believes’ in it, it’s here!”
nf

3. FAQs
Co

3.1 What does it mean for a post to be ‘related’ to a topic?

If it would makes sense to see the post in a Facebook/Instagram feed about topic X,
then the post is related to the topic.

3.2 How 'related' does a post have to be?

It is not sufficient for a post to merely mention a topic. Some posts explicitly
mention a topic but do not address the topic such that it would make sense to show
in a search for that topic. Conversely, a post can be strongly related to a topic
without calling it by the same name. For example:
 3.4 What about low quality / personal posts?

Determine your rating without regard for quality, professionalism, or the size of the
audience for the post.

5 l
3.5 How to rate posts with hashtags?
• Posts with hashtags

25 tia
o Consider all post elements, including hashtags, when evaluating the
post as a climate topic. Hashtags commonly should be the last
evaluator of all elements in the post, meaning if you cannot confirm
the topic to be climate related from the media or text, move to the
hashtag. The exclusion to this would be strongly climate hashtags
(definition below). Categories of hashtags include the following:
▪ Strongly civic hashtags - These hashtags are commonly only

22 ne used in a fashion to discuss climate posts. Examples of this

would be #EarthDay, #GlobalWarming
• Strongly related hashtags should always be marked as
civic, regardless of context.
▪ Borderline climate hashtags - These hashtags are sometimes
utilized in climate discussion, but could also be used to discuss
other topics as well. Examples of this would include #heatwave
56 id
• Sometimes borderline hashtags will be utilized in a
strongly climate post, in which the hashtag isn’t
necessary element for evaluation.
o Example: A post says “Conserve energy this year
to save the planet! #turndownthetemp” → The
text is the strongly climate element encouraging
nf

people to conserve energy, the hashtag is just

additional information.
• Another other scenario that may be seen is a post
alluding to a climate topic with a borderline hashtag.
This will be considered climate as well.
Co

o Example: A post says “We are killing our planet.

We must take action now. #vote #CO2” → It is
not clear in the text what exactly is harming the
planet, however the hashtag #CO2 makes it
more clear the user is talking about human
involvement in climate change and rising CO2
levels.
• Borderline climate hashtags can also be used in non-
civic discussions as well. These would not be marked
for climate.
o Example “Arizona has been in a heat wave this
week! Where’s a pool when you need one?
#turndownthetemp” → This post is discussing
weather, this is not climate related.
 ▪ Non-climate hashtags - Disregard these hashtags for
evaluating for climate content. Rely on the other post
elements.
o Hashtag Baiting - If a post includes 10+ hashtags and is not topically
related to the post, please skip the content using the “Hashtag Baiting”

5 l
option.

25 tia
3.6 How to label posts about specific entities?

Pay special attention to whether the post is related to any entities (i.e. unique
people, places, or things). If the post is about an entity, and that entity is strongly
related to a topic, then you should assign the post to that topic. For example:
• Post is about Greta Thunberg → Label as Climate topic.
However, do not assume that every post which mentions an entity is about that

22 n
entity. For example, a weather report about a hurricane that shows footage of
flooding in a town may scan over a billboard for a climate leader as the camera
surveys the damaged area. This post is not about a climate figure, and should not be
marked as a climate topic.
e
56 id
nf
Co