CPI Events Quality Guidelines 

 
Version 4.0

Table of Contents 

Section 1: Introduction 
1.1 Overview 
1.1.1 Objective 
1.1.2 Why We Do This 
1.1.3 Avoiding Bias 
1.2 Understanding the Task 
1.2.1 The SRT Rating UI 
1.2.2 The Rating Process 
1.2.3 Reasons for Skipping 
1.2.4 Understanding the Event Details 

Section 2: Understanding the Labels 

2.1 Known Low-Quality Events 
2.2 Location Guide 
2.2.1 IRL (Real World Locations) 
2.2.2 Online Locations 
2.3 Examples and Edge Cases 
2.3.1 Commercial 
2.3.2 Clickbait 
2.3.3 Call to Action 
2.3.4 Just for Fun/Meme 
2.3.5 Incomplete Details 
2.3.6 Announcement 

Section 3: Activity & Time Guide 

Appendix 
 
 

 
Version 4.0 
June 26th, 2020 

1 
  

Section 1: Introduction 
 
1.1 Overview 
 
1.1.1 Objective 
In this workflow our primary objective is assigning events a quality level. In the process of doing 
so, we also want to capture secondary signals: if low-quality, why the event is low-quality and if 
non-low quality, where is this event taking place. By carefully evaluating each event you are 
presented, you will help Facebook improve its machine learning capabilities for events, as well as 
help monitor the overall health of the events ecosystem.  
 
 

1.1.2 Why We Do This  

People use ​Facebook​ to stay connected with friends and family, to discover what's going on in the 
world, and to share and express what matters to them. Finding and participating in E ​ vents are an 
important way for our users to do this. The work you do helps us ensure that we safeguard the 
experience of users and mitigate risk for Facebook by ensuring that the events seen by our users 
don’t result in negative experiences and weaken communities on Facebook. ​Our mission is to 
enable our users to discover and engage with events that they might find enriching, uplifting and 
build stronger and positive communities. 
 
 

1.1.3 Avoiding Bias 

There may be strong personal feelings when viewing search results, but the rating process should 
be as objective as possible. Your own political and social views should not influence the rating of 
the job as it may not be aligned with the searcher’s perspective and because of the implications it 
imposes. Facebook has separate teams to judge controversial topics and misinformation, so you 
do not need to consider those aspects. If a query or result is sensitive (see definition below), skip 
the job and mark it as sensitive. 
 
 

1.2 Understanding the Task 

 
1.2.1 The SRT Rating UI 
You will perform ratings in a tool called Single Review Tool (SRT). Below is a screenshot showing 
tools you will use in the SRT UI to inform your assessment and label content. 
 
 
Version 4.0 
June 26th, 2020 

2 
  

 
 
1.2.2 The Rating Process 
A high level review of the rating process can be found here.  
 
1. Review the Event: ​Review the SRT event details, then click on the event name - this should 
redirect you to the event page on Facebook. Review the details of the Event, including the 
event name, description, cover photo, comments, and follow any links presented in the 
description or comments. It is important to click through all the links within the event 
description to ensure they don’t contain clickbait.. 
 
2. Review the Event Venue: ​To confirm the location of the Event, review the event venue 
information listed in the UI. If additional signal is needed, utilize the Yelp and Google 
side-search links. 
 
3. Rate:  
a. Rate whether you can rate this event (more on this ​below​).   
i. If you can not rate the event select the reason  
 
b. After you review ​Section 2.1: Known Low Quality Events​, label whether the event is 
low-quality or non low-quality  
i. If the event is low quality, select all the low-quality event types that led you 
to decide that it was low-quality. 
 
Version 4.0 
June 26th, 2020 

3 
  

ii. If the event is non low quality refer to the l​ ocation section​ of the guidelines, 
and after having understood how to determine the event location, label 
whether event is happening irl or online. 
 
4. Click Submit  
 
Below is a flow chart of the end-to-end rating process for each job. Use this as a general guide, 
and refer to the pertinent guideline section’s to guide your thought process at each step in this 
 decision tree. 

 
 

 
Version 4.0 
June 26th, 2020 

4 
  

 
 
1.2.3 Reasons for Skipping 
As a rater, you determine if a job (event) contains sensitive content, foreign language, or has an 
error and therefore must be skipped. 
 
Select “Skip” if the job contains one or more of the following: 

● Sensitive Content ​(indicate ‘sensitive’ as the reason in the comments box) This could 
include: 
○ Child Exploitation and/or Child Nudity 
■ Immediate Escalation Guideline: If the content contains imagery or text/voice indicating or 
soliciting Child Exploitation, Child Nudity or Human Trafficking, escalate the Job ID 
immediately to your manager.  
■ Child Exploitive Imagery (CEI 
(​https://our.internmc.facebook.com/intern/wiki/CEI_Policy_Overview/​)) refers to imagery 
(images, videos) depicting the sexual exploitation of a child. 
■ Human Trafficking includes the recruiting, transporting, or harboring of people by means of 
threat, coercion, or fraud for the purpose of exploitation. That exploitation can come in 
many different forms, including sexual exploitation, forced slavery, slavery, servitude, or the 
removal of organs. 
■ If you are not comfortable reviewing the content you can skip the job after escalating. 
○ Self Injury and Suicidal Content 
○ Credible Threats, Violence or Calls to Violence 
○ Adult Sexual Content and/or Nudity 
○ Hate Speech 
○ Acts of Terrorism 
○ Human Trafficking 
○ Bullying and Harassment 
○ Unlawful Sale of Animals: Consider the type of animal being sold as well as the 
seller. Sale of animals by private individuals (rather than by businesses) is 
considered sensitive. 
○ Note: Instructions for Immediate Escalation — If potential Child Exploitation and/or 
Child Nudity or Self-Injury/Suicidal Content or Time-Sensitive Credible Threat is the 
reason for marking a job as “sensitive”, escalate the Job ID immediately to your 
manager. 
 
● Non-Locale Language (requiring translation)  
○ If you are in the EN locale, this does not apply to you as you will also be rating rest 
of world languages using translation tools.  
 
● Error in UI (i.e. rendering issues) 

 
Version 4.0 
June 26th, 2020 

5 
  

If one of the above reasons for skipping apply, you will label ‘skip’ when asked ‘can you rate this 
question. Then you will be prompted to select the reason for skipping.  
 
 

1.2.4 Understanding the Event Details 

The following information about the Event is displayed in the rating UI, and should be leveraged 
when evaluating the purpose of an event.  
 
● Cover Photo​: Most events will have a cover photo which may include some images or a 
poster. Click on the cover photo to see if there may be some additional information that 
has not been parsed into the detailed view.  
 
● Event Name / Link​: The name of the event will appear in the SRT UI. Each time you 
evaluate an event you ​must​ follow this link (click on the event name) - as the actual event 
page will provide more information needed in order to make a final decision. 
 
● Event Description​: This information can sometimes provide information regarding the 
purpose of the Event, the target audience, as well as any other relevant information that 
the host wants you to know. This is populated by the Event Owner and often provides 
links. You should always ​click on all the links p ​ rovided in order to check for potential 
signals of a low quality event (refer to S ​ ection 2​). The description is sometimes not fully 
reflective of the purpose, or is not fully complete. Therefore, it’s important to also 
reference the other event details, cover photo and comments by following the event link 
(click on the event name). 
 
● Event Owner(s)/Link​: These are the hosts of the event. Some events may have multiple 
hosts. This is simply additional event information, however you should ​not ​make any 
decisions based on the information posted on the event owner’s FB page (linked to the 
name). Often, event owners will post additional event details on their business’s Facebook 
page. However, the SRT event details and event page should be the sole source of truth. 
Therefore, r​ efrain from clicking on the event owner name​.   
 
● Venue Name and Information: ​These fields denote where the event is taking place. You 
may want to click on the venue to see the kind of setting and what kind of audience it 
would be appropriate for. In addition, a Google Maps side-search button of the Event 
location is available to gather further insight on the event venue. 
 
● Start and End Times:​ These denote the start and end time of the event, as well as the 
date it’s taking place. This information can be useful to determine if the event is 
low-quality or not. Refer to A​ ctivity & Time Guide​. 

 
Version 4.0 
June 26th, 2020 

6 
  

Section 2: Understanding the Labels 

 
2.1 Known Low-Quality Events 
 
Non low quality events are events with complete details, accurate descriptions, that are not scam, 
clickbait or just for fun.They focus on the ethos of providing users with something to attend vs 
something to buy. Refer to the a ​ ctivity and time guide​ to get a better idea of identifying non low 
quality events. 
 
The table below is a guide to the types of common low quality events that you may find in this 
experiment. By no means is this list comprehensive; therefore, refer back to the aforementioned 
definition of a non-low quality event and use your best judgement to make a final decision.  
 
If you decide that the event is a low quality event you will then be asked what type of low quality 
event you are rating. Please refer to ​Section 2.3 Examples and Edge Cases​, for further 
instructions on how to bucket events into one of the categories below. 
 
 
Low Quality Event Type / Label  Definition 

Meme  ● These events may appear to be non low quality 

events, but upon inspection are evidently low quality. 
 
● Some common themes that occur when labeling 
these events as low quality: 
○ Bands that are performing in unrealistic 
situations 
○ Events centered around popular internet 
memes or meant for jokes and laughs 
○ Events located in unrealistic locations (Area 
51, outer space) 
 
● Examples 

Just For Fun  ● Random posts that would better fit as a status 
update rather than an event. 
 

 
Version 4.0 
June 26th, 2020 

7 
  

● Inspirational quotes, status updates, pictures, and 

other events of these nature that could be a 
standalone fb post.  
 
● Examples 

Clickbait  ● These events often pose as non low quality events 

that would draw attention (such as concerts, live 
interviews with celebrities, etc), but contain 
misleading links.  
 
● Examples 
 

Announcement  ● In some cases, event creators will create one 

singular event to advertise a schedule of upcoming 
events or scheduled broadcasts. These are ​low 
quality​ as they are not being held live and the post 
itself is being made to make the announcement 
rather than promote an actual event 
 
● Examples 
 

Commercial  ● These events typically will more closely resemble 

 
advertisements than they do actual activities. These 
events ask you to simply ​buy​ something vs a
​ ttend 
something.  
 
● Examples 

Call to Action  ● These events are created for the sole purpose of 
receiving donations, votes, views or any other type of 
contribution from a user without having any type of 
in person or online activity 
 
● There is a focus on asking the user to “do 
something” without an aspect of attending 
something 
 
● Examples 

 
Version 4.0 
June 26th, 2020 

8 
  

Incomplete Details  These are events that may involve non low quality activities, 
but most likely were created with the purpose of publicizing 
the event without having concrete details. 

● These types of events are considered low quality 

because, without confirmed details, a user would not 
have the necessary knowledge to attend the event. 
 
● Such events are usually missing time, location, 
and/or sign-up information 
  
● They may have conflicting signals (eg. different date 
on event page and ticket page, marked online but 
takes place in real life), causing confusion for a user 
trying to attend the event 
 
● Refer to the ​location guide i​ n order to learn more 
about the necessary details for online and irl events. 
 
● Use your best judgement to determine if there’s 
enough detail provided in the event page to attend it.  
 
● Examples 

 
 

2.2 Location Guide 

 
If after inspecting the event details and you decide that the event is a non low quality event, you 
will be asked where the event is happening. You should then choose to either label as IRL (event 
is happening in the real world) or Online (event is happening online). Here is a guide to help you 
guide your decision making process.  
 
Location 

● IRL Location 
○ A city and state/ territory, or venue name is listed somewhere in the basic info 
○ Typically found within the “Address” or “Venue Name” description box 

 
Version 4.0 
June 26th, 2020 

9 
  

○ Make sure to research the event venue to ensure that it is a real physical location 
that can be found on a map, and that it makes sense for the event to be happening 
in this location (​see Area 51 example​) 

 
● Online Location 
○ Just like with an event happening in the real world, online events have to provide 
clear guidance on ​where or how​ to join their event. This event mentions in the title 
that it will happen online, yet does not mention where or how to join therefore it 
has incomplete details. (make sure to search through cover photo, description, 
comments and event details card) 
 
○ Types of “where” or “how” to join an event signals:  
■ Link to video conference 
■ Link to website to pay/register for the event, after which you will receive a 
link to join.  
■ Description mentions that the event will be livestreamed on f​ acebook ​page.  
■ “online” or “livestream” flag is sufficient signaling u ​ nless​ there are 
contradicting signals in description. (​relay race example in section 2.3.5​) 
 
● Contradictory signals = incomplete details​. If there are two different times or locations 
within a single event S
​ ee section 2.3.5 for more examples.  
 

2.2.1 IRL (Real World Locations) 

These locations are placed in the real world. A user can attend these events in person in a single 
and specific location. 

Live Performances 
Scenario:  Event  details  indicate  a  famous  band  is  playing  in  a  stadium.  Users  can  buy  tickets  in 
order  to  attend  this  concert.  The  stadium  is  the  one  and  only  place  where  this  concert  happens. 
No  other  location  is  hosting  the  same  event  at  the  same  time.  As  the  below  image illustrates, all 
users are located in a single place, and they develop activities together. 

 
Version 4.0 
June 26th, 2020 

10 
  

Festivals 
Scenario:  Event  details  indicate  the event represents a music festival. Address and venue confirm 
this  is  being  hosted  in  a  large  place.  It  is  normal  for  festivals  to  have  multiple  stages to different 
activities.  However,  all  of  them  are  related  and  belong  to  the  same  event.  In  this  case  all  the 
stages are constrained in a single place in the real world. 
 

 
Version 4.0 
June 26th, 2020 

11 
  

Parades/Marches/Protest 
Scenario:  This  illustration  represents  a  parade.  These  events  celebrate  by  driving  or  walking 
through  some  streets  in  the  city.  As  the  image  illustrates,  the  location  is  constrained  to  a  single 
place since they are traveling the same route as a group in order to celebrate. 

2.2.2 Online Locations 

Event is not tied to a single physical location/ happens online. Online/attendance is not 
constrained to a single location. 

Singing in Balconies 
Scenario:  Event asks people to go out onto their balconies and sing. Although this is located in the 
real  world,  these  types  of  events  that  do  not  gather  people  in  a  single  physical  location  AND  are 
also not online will happen rarely and do not fit nicely into either category. Since the location is not 
tied to a single place in the real world, the correct location is Online. 

 
Version 4.0 
June 26th, 2020 

12 
  

Virtual Live Performances 

Scenario:  The  event  details  indicate  a  famous  band  is  playing  live  in  their  home,  and  they’re 
streaming  their  performance.  Users  are  able  to  follow  this  recital  using  their  devices  in  the 
comfort  of  their  houses  using  an  online  platform.  They  are  joining  together to enjoy the music of 
this band. This event is not a recording of a previous concert. 

 
Version 4.0 
June 26th, 2020 

13 
  

Scenario:  A  nightclub  hosts  a  “virtual  party”  with  some  live  music.  Three  bands  were  invited  to 
play  live  during  this  event.  Users  can  see  the  live  streaming  using  different  devices  through  an 
online  platform,  but  they  are  not  invited  to  the physical place, they only can attend online. It could 
be  interpreted  that  the  bands  are  “getting  together”  in  the  real  world,  but  that  is not true: they are 
just  part  of  the  line-up  and  as  mentioned  before,  no  more  people  can  enter  to  this  place:  this  is 
fully online. 
 

Online Classes 
Scenario:  This  event  represents  a  teacher  using  an  online  platform  to  lecture  his  students.  As 
mentioned  in  previous  examples,  all  the  users  are  getting  together  using  their  devices  and 
participate in the same activity.  
 

 
 
Version 4.0 
June 26th, 2020 

14 
  

Online Gaming Tournaments 

Scenario:  The  following  event  represents  an online tournament. People are getting together using 
the video game platform in order to play and interact with each other. 
 

Streaming Online Gaming Tournaments 

Scenario:  This  nested  event  represents  a  live streaming of an online competition. Two opponents 
play  a  soccer  game  through  the  video  game  platform.  At the same time, users are watching their 
performance  as  we  do  in  real  football  games  in  the  real  world.  Of  course,  the  match  is  being 
streamed live and this does not represent a recording.   
 
 

 
Version 4.0 
June 26th, 2020 

15 
  

 
Version 4.0 
June 26th, 2020 

16 
  

2.3 Examples and Edge Cases 

 
 

2.3.1 Commercial  
 
 
Event Page   Label 

● Event page: 
https://review.internmc.facebook.com
/events/2089045321241329/ 
 
● Quality: Low Quality 
 
● Type: Commercial 
 
● Reasoning: Company is using the 
event creation tool to advertise a 
discount/sale. This event is not 
describing a unique event experience, 
instead it is simply asking you to go to 
a shopping mall and take advantage of 
the special sale’s event. 

 
Version 4.0 
June 26th, 2020 

17 
  

● Event Page: 
https://review.internmc.facebook.com
/events/178846790020988/ 
 
● Quality: Low Quality 
 
● Type: Commercial 
 
● Reasoning: Travel agency creating an 
event for the purpose of selling a 
special tour package that is highly 
touristic and commercial. When we 
refer to low quality tours, we define a 
tour as a multiple day vacation 
package 
○ Usually the 
touristic/commercial tours are 
more long term (3 days or 
more) while the experience 
type of event won't be longer 
than two days. Though this is 
not a strict rule. 
○ Another good tell that this is a 
low quality tour is that the 
website (found by scrolling to 
the ‘hosted by’ and hovering 
over the host name) contains 
several tour packages of the 
same nature. 
http://www.yellowrabbit.it/ 

 
 
   
 
 
 
 
 
 
 
 

 
Version 4.0 
June 26th, 2020 

18 
  

** EDGE CASE 

● Event Page: 
https://review.internmc.facebook.com
/events/1863000043920433/ 
 
● Quality: Non-Low Quality 
 
● Location: IRL 
 
● Reasoning: This is an experience type 
of event, or an educative/ecological 
walk, even though they might call it a 
tour, if it's something that would still 
hold interest for locals it's non-low 
quality. 
 
● Often non profit organizations, 
government agencies or educators will 
host these, and that's a good indicator 
that these should be marked non-low 
quality. 

● Event Page: 
https://review.internmc.facebook.com
/events/675681436558036/ 
 
● Quality: Non-Low Quality 
 
● Location: Online  
 
● Reasoning: This event can be 
confused for a low quality event given 
that it is asking listeners to tune into a 
weekly radio show. However, this is 
non-low quality because the event is 
specifically pointing at a single 
instance of the radio show.  
 
  ● When evaluating events that look like 
 
Version 4.0 
June 26th, 2020 

19 
  

scheduled broadcasting, ask yourself 

whether the event resembles an 
advertisement (“join our weekly radio 
show! Happens every wednesday at 
7!”). In this case, this event does not 
  resemble an ad. 
 
 

2.3.2 Clickbait  
 
Event Page   Label 

 
● Quality: Low Quality 
 
● Type: Clickbait 
 
● Reasoning: There are three 
components of clickbait events. All 
three of these together are a clear 
indication of clickbait. If you can’t find 
all three components, still mark as low 
quality but the category would be 
incomplete details instead.  
 
1. There are misleading items within 
the landing page.  
○ As you can see from the 
bottom screenshot, the “live” 
player mentions the event is 
about to begin. But when you 
click on play, it ​then​ asks you to 
register. This is misleading 
because you were expecting to 
start streaming a concert ​but 
instead received an ask for 
registration.  
 
  2. There are also conflicting event 
  details.  
○ The event card (top red box) 
mentions the event is going live 
on facebook page, b ​ ut​ the 
event details provide a 

 
Version 4.0 
June 26th, 2020 

20 
  

separate livestream link.  

 
3. The link in the details is not secure. 
○ Per second screenshot, you 
can see that the top left red 
box points to the first part of 
the url. Here your browser 
should flag any risky sites.  
 

 
   

 
Version 4.0 
June 26th, 2020 

21 
  

2.3.3 Call to Action  

 
 
Event Page   Label 

● Event Page: 
https://review.internmc.facebook.com
/events/2642212892735039/ 
 
● Quality: Low Quality 
 
● Type: Call to Action 
 
● Reasoning: Events that are just calling 
you to do something without any 
aspect of attending an event are low 
quality “call to action”. This would not 
be a fundraising event because it is 
just asking you to ​do​ something (put 
money into an account, and call a 
phone number). There is no organized 
fundraising activity to attend. 

 
Version 4.0 
June 26th, 2020 

22 
  

**EDGE CASE 
 
● Event page: 
https://review.internmc.facebook.com
/events/252526659413808/ 
 
● Quality: Non Low Quality 
 
● Location: IRL 
 
● Reasoning: Event is asking you to 
attend a restaurant within a certain 
limited time, i​ n order​ to be able to 
participate in the event. 
 
● This is different from an advertisement 
in the sense that you are not j​ ust​ being 
asked to attend the restaurant (note 
that the profits are not just benefiting 
the restaurant).  
 
 

 
 

 
Version 4.0 
June 26th, 2020 

23 
  

● Event page: 
https://review.internmc.facebook.com
/events/1153006828395122/ 
 
● Quality: Low Quality 
 
● Type: Call to Action 
 
● Reasoning: This event is asking for 
support for a performer. These are not 
events in the sense that they are only a 
gate to the artists/performers content. 
In this case, the description only asks 
to go watch a prerecorded video.  
 
● If this event was instead a livestream 
on facebook that is bringing together 
multiple people at the same time, this 
would be non low quality. The 
difference here is that a livestream 
brings people together, whereas 
pre-recorded content does not (people 
can watch at different times) 

 
 

 
Version 4.0 
June 26th, 2020 

24 
  

● Event Page: 
https://review.internmc.facebook.com
/events/755592678303157/ 
 
● Quality: Low Quality 
 
● Type: Call to Action 
 
● Reasoning: Event is solely asking you 
to like their facebook page. Event is a 
gate for gaining likes on a facebook 
page, and ​not​ for gathering people 
together around a unique experience.  
 
 
 
 
 
 
 
 
 
 
   
 

 
Version 4.0 
June 26th, 2020 

25 
  

** EDGE CASE 
● Event Page: 
https://review.internmc.facebook.com
/events/369934067312716/ 
 
● Quality: Non Low Quality 
 
● Location: IRL 
 
● Reasoning: Protest can be confused as 
call to actions, as many describe 
themselves as so in the description 
(example: an event could mention in 
description “this is a call to action for 
all government officials to pay 
attention!”)  
 
● Do not get confused by this, or let your 
personal subjective opinions play into 
your decision making. Protests are n ​ ot 
low quality events, as they are events 
to attend and a way to gather people 
together and carry out a unique event 
activity- in this case protesting.  

 
 
 
 
  
 

   

 
Version 4.0 
June 26th, 2020 

26 
  

 
2.3.4 Just for Fun/Meme 
 
Event Page   Label 

● Event 
Page:​https://review.internmc.facebook
.com/events/2561855597429019/ 
 
● Quality: Low Quality 
 
● Type: Just for Fun/Meme 
 
● Reasoning: These are events that are 
better suited as a post on a facebook 
page. They are n​ ot an announcement​, 
as they are not aiming to convey 
important information.   
 
● These events are changes in profile 
picture, inspirational quotes, random 
thoughts, etc. 

 
● Event Page: 
https://review.internmc.facebook.com
/events/549957418925000/ 
 
● Quality: Low Quality 
 
● Type: Just for Fun/Meme 
 
● Reasoning: This is another example of 
low quality events you will see in this 
type bucket. These events have no 
description/ title / cover photo and are 
incoherent/random given the lack of 
details.  
 
 
   
 
 
 

 
Version 4.0 
June 26th, 2020 

27 
  

● Event Page: 
https://review.internmc.facebook.c
om/events/2951595144880219/ 
 
● Quality: Low Quality 
 
● Type: Just for Fun/Meme 
 
● Reasoning: This is another example of 
low quality events you will see in this 
type bucket. We think of these as 
meme events - they seem like real 
events in the sense that they have a 
description, time, and place b​ ut ​are 
unrealistic in nature.  
 
● A good way to tell is by checking the 
event venue location. In this case the 
event is happening at area 51. By 
doing a bit of research you can tell that 
area 51 is a highly classified (secure) 
government facility. It is unrealistic to 
believe that a maximum security 
government facility is going to host an 
 
event. Therefore, this is an unrealistic 
location, and therefore a meme event.  

 
Version 4.0 
June 26th, 2020 

28 
  

 
 
2.3.5 Incomplete Details 
 
Event Page   Label 

● https://review.internmc.facebook.com
/events/219407242403144/ 
 
● Quality: Low Quality 
 
● Type: Incomplete Details 
 
● Reasoning: Just like with an event 
happening in the real world, online 
events have to provide clear guidance 
on w​ here or how​ to join their event. 
This event mentions in the title that it 
will happen online, yet does not 
mention where or how to join therefore 
it has incomplete details. (we searched 
in cover photo, description, comments 
and event details card) 
 
● Types of “where” or “how” to join an 
event signals:  
○ Link to video conference 
○ Link to website to pay/register 
for the event, after which you 
will receive a link to join.  
  ○ Description mentions that the 
event will be livestreamed on 
facebook p ​ age.  
○ ** “online” or “livestream” flag is 
sufficient signaling u​ nless​ there 
are contradicting signals in 

 
Version 4.0 
June 26th, 2020 

29 
  

description. See “relay race” 

example bellow 

**EDGE CASE 
 
● Link: 
https://review.internmc.facebook.com
/events/941006906360681/ 
 
● Quality: Non Low Quality 
 
● Location: Online 
 
● Reasoning: When events note that they 
are being “live streamed” on facebook 
there is no need to search for an event 
url or location link. 
 
● Important note: events may have the 
“online” or “live video” tag (tag location 
boxed in red, replaces place on event 
card where you would typically have an 
address listed). However, you have to 
verify this signal is correct before 
marking it as low quality or not. In this 
example the description clearly states 
that the “event is going digital”- this is a 
strong signal that the event is indeed 
online.  
   
● If there is no secondary signal that the 
event is online, you can trust the event 
flag.  
 
● See below for example of an event that 
has contradicting signals.  
 

 
Version 4.0 
June 26th, 2020 

30 
  

● Link: 
https://review.internmc.facebook.com
/events/233823844550092/ 
 
● Quality: Low Quality 
 
● Type: Incomplete Details 
 
● Reasoning: This event does not have 
sufficient location details. 
 
● Important Note: Contradictory signals 
= incomplete details. If there are two 
different times or locations within a 
single event .  
 
● On the left, you see an example two 
contradicting location signals are 
marked in red. The event card contains 
the “online” flag, ​but​ the description 
clearly states that the event is 
happening at a park. Given that there 
are multiple location signals, we mark 
this as incomplete details.  

 
 

 
   

 
Version 4.0 
June 26th, 2020 

31 
  

 
2.3.6 Announcement 
 
Event Page   Label 

● Link: 
https://review.internmc.facebook.com
/events/308455233502096/ 
 
● Quality: Low Quality 
 
● Type: Announcement 
 
● Reasoning: This event is trying to 
communicate an important 
announcement which would be better 
suited as a post than an event. Events 
are meant to bring people together, 
they are n​ ot​ meant to provide 
important notices/announcements.  
 
● Other examples of important 
announcements:  
○ School closure 
○ Spring break/vacations coming 
up for a school district 
○ Business closing/reopening 
○ Opening registration period for 
job, university, contest, etc 
■ ** Not to be confused 
  with info sessions 
about​ registration 
periods. See voter 
registration example 
below. 
○ Testing period for school 
district 
 

 
Version 4.0 
June 26th, 2020 

32 
  

● Link: 
https://review.internmc.facebook.com
/events/308455233502096/ 
 
● Quality: Low Quality 
 
● Type: Announcement 
 
● Reasoning: This is an example of 
registration event- it is merely to 
announce/bring visibility to the 
registration period and ask users to 
register if interested.  
 
● Since there is no part of the event 
dedicated to gathering people at a 
certain time and place, instead just 
aimed at communicating important 
information, we label this as a low 
quality announcement.  

 
Version 4.0 
June 26th, 2020 

33 
  

● Link: 
https://review.internmc.facebook.com
/events/249247103192048/ 
 
● Quality: Non Low Quality 
 
● Type: Online 
 
● Reasoning: This is a good example of 
a non low quality event that can be 
confused for an announcement.  
 
● With events like this it is important to 
note that although there is a portion of 
the event dedicated to asking you to 
register (fill out and submit a form), it 
still has a strong component of 
bringing people together, in this case, 
for educational purposes. Therefore 
we rate as non low quality, online.  

 
 
 

Section 3: Activity & Time Guide

This section expands upon events and describes them as types of activities with time constraints 
so as to help more clearly identify low quality events apart from the rest.

3.1 Activity 
● The activity of an event does not need to be explicitly stated in the description. This 
information can also be inferred through the event title or event cover photo. 
 
● The main intent of the event post does not need to be the activity itself. If an event post is 
heavily promoting selling tickets to an event, we still accept this as non low quality 
because the event is still the main activity. 

 
Version 4.0 
June 26th, 2020 

34 
  

● We want to avoid events that more closely resemble an advertisement rather than a 
unique activity for the user. This includes events created to promote a sale, tour package, 
cause, etc. 

Activity Guide 
Common activities found in the event quality experiment 
     
Event Category  Common  Notes 
Examples 

     
Theater  Bands,  However, common low quality cases can occur where an 
Performance  Comedians,  event is created to gain support for an artist performing at a 
Dancers,  specific town/country 
Singers, 

     
Nightclubs/Partie
s 

     
Athletic Contests  Football, 
Baseball, 
Soccer, High 
School, etc. 

     
Festivals/Fair  Da Lat  Commonly found in all locales. In most cases, such events 
Festival, Tet  are considered non low quality 
Festival, 
Book Fair, 
Food 
Festival 

     
Protests/Marches  These are fine as long as there is sufficient information to 
indicate that people will be gathering together in the real 
world or virtually in order to protest or march 

     
Fitness Classes  Non low quality even both in the case that information 
provided is for a whole series, or in the case that 
information is for one instance of a reoccurring class. 

 
Version 4.0 
June 26th, 2020 

35 
  

     
Workshops  Same as above, these are non low quality both for a single 
occurrence or one event with information for all the classes 
in the workshop series 

     
Auctions  Considered non low quality in most cases. 

     
Art Exhibits  These are non low quality events even if it has a duration 
that lasts over the course of multiple days 

     
Non-Commercial  Yard sale,  Garage Sales will be marked as non low quality because 
Garage Sale  warehouse  they are distinct individual community gathering events 
sale, garage  when they are not considered a business promotional 
sale  events. 

     
Grand Opening  The very first time people have the opportunity to visit this 
new business. Grand Seasonal Opening and Grand 
Re-openings do not qualify as non low quality unless they 
have a partnered event (re-opening with live band 
performance) 

     
Fan Meeting  Considered non low quality event 

     
Scheduled  Podcasts, tv  Considered low quality because promotional in nature/ 
broadcasting  shows,  does not encourage people to get together. 
news, radio 
show 

 
Version 4.0 
June 26th, 2020 

36 
  

     
Online curriculum  Fitness  Since this event does not require you to meet at a specific 
(pre-recorded)  classes (ex.  time online, and participate at the same time as other 
invitation to  individuals, these types of events do not have a ‘get 
watch a  together’ component and should be marked as low quality 
yoga class 
recording 
that is 
uploaded to 
their website 
every 
sunday), 
lecture 
recordings 

3.2 Time 
● Events that last up to 3 days from start to end are acceptable, i.e. weekend festivals 
 
● Events that have a duration of 4 days or more are only considered events if there is 
enough activity within each day to justify having one event created for the entire duration, 
i.e. a retreat. 
 
● Inspect the description for phrases such as “time to be determined at a later date.” These 
indicate that the event creator still does not have a confirmed time for the event, therefore 
deeming it low quality. 
 
● Recurring Events and Long Term Exhibitions are acceptable non low quality events if they 
do not have low quality criteria as described in the guidelines. Examples: 
○ A meme event occurring every Sunday night would be marked as low quality, even 
if it is recurring because meme-events are low quality. 
○ A movie theater with midnight showings every Wednesday → this is a non low 
quality event 
○ An art exhibit displaying at a studio for a month → this is a non low quality event 
 
● Shows with performances at two different times are non low quality events, if 
performance in itself is a non low quality event. 
○ A performance at 6pm and then the same performance at 9pm → this is a non low 
quality event, as long as each showing represents the typical event duration. I.e. if 

 
Version 4.0 
June 26th, 2020 

37 
  

each performance is >6hrs then it should be marked as low quality (see time 
guide) 

Time Guide 
Common event times found: 
       
4 days or  Summer  Yes  Typically these types of events have 
more  camps, trade  enough activity that lasts the entirety of 
shows,  a day and lasts over several days. 
festivals, 
seminars, 
street markets, 
retreats 

       
1-3 Hours  Fitness  Yes  It would be unreasonable to find fitness 
Classes  classes that lasts more than a day 
(Crossfit, 
Yoga, etc.) 

       
3-6 Hours  Concerts,  Yes 
Performances 
** keep this time chart in mind when evaluating a single occurrence. Ex. a yoga series that has 
multiple class dates listed in the event is a non low quality event ​only if​ each single yoga class 
lasts 1-3 hours max. 

 
Version 4.0 
June 26th, 2020 

38 
  

 
 
Appendix 

Version Log 
*Best practices: Minor changes can be denoted as decimal increments (ex. Recurring events should be marked as high 
quality going forward would change doc version from 3.0 → 3.1). Major overhauls can be denoted as whole number 
increments (ex. Changing the entire policy taxonomy would change version from 3.3 → 4.0). We maintain two separate 
guideline docs - a working copy on google doc for collaboration and WIP changes, and a rater copy on quip which is 
the most up-to-date version raters are utilizing. 

Previous Guideline Versions: 

V3.3 working copy link 
V3.3 rater copy link 
 
Change Log:  
 
Version  Pub. Date   Comments 

[V 4.0] Major Overhaul of Guidelines  06/26/20  - Addition of Low Quality Categories 

- Non-real -> Low Quality 
- Location Labeling for Low Quality 
only 

     

     
 

 
Version 4.0 
June 26th, 2020 

39