Guidelines for Video - Similarity

Video Similarity - Rating guidelines

Introduction
In this document, We explain how to judge the similarity between 2 pieces of video content.
Similarity between content is used in multiple interfaces on the Apple TV app to recommend movies
and TV shows for users and help them discover content based on their interests.

The importance of your work as a Rater

The data we receive from you in the form of high quality similarity ratings will be used to build and
improve artificial intelligence systems such as search and recommenndation algorithms and
machine learned rankers that power the user experience for Apple TV users globally.
Our ultimate goal is to surprise and delight our customers by improving video search and
recommendation quality and enhancing customer satisfaction, and you play an important role in this.

Stay curious about the relations between the seed content and the related item. Results will not all
be in the same genre, the same subject matter or the same cast. The connections between the seed
content and the related candidate could be all three of those factors, or it could be one or two.

Your attention to detail, research and language skills as well as your cultural knowledge of the
market are all critical to the success of our projects.

Rating Instructions
In each task you will be given two pieces of content to evaluate and assess their similarities.

Similarity will be assessed based on 4 specific aspects:

1. Target audience
2. Factual aspects
3. Theme
4. Style
Each aspect assessment is independent and is not related to the other aspect.
Two pieces of content can be similar in one or more aspect/s and different in one or more aspect/s.

Rating Scale
For each aspect you need to rate similarity according to the scale:
1. Very Similar
2. Slightly similar
3. Not similar
4. Problem: Other (Technical issue with performing the task).
Getting more information
Most of the content that you will require to rate, you probably have not seen yourself.
How do you go about learning more about the content in order to make an educated judgement?

Under each content (seed item and target item) we provide you with links to destinations where you
can learn more about the content.

banker.png 211 KB

1. AppleTV product page (Click on the title)

e:g: https://tv.apple.com/US/movie/umc.cmc.2f8qhsa039voq5x0iwn1eixj1
You can find age and genre information as well as cast &crew information

You can also watch the trailer to get more acquainted with the content.

trailer.png 97.4 KB

1. Other links in the content tile will lead you to additional web destinations where you can
learn more about the content.
IMDB , Youtube, Google universal search and Google play video search.

Rating examples
A few complete rating examples before we dive into the details of each aspect

1. Rating for very similar content

Seed item: Wonder Woman
Target item: Wonder Woman 1984

ww.png 105 KB

ww2.png 72.4 KB

Target audience: Very similar

Factual aspects : Very similar
Theme : Very similar
Style : Very similar

1. Rating for Not similar at all content

Seed item: The silence of the lamb

Target item: Big
 collection.png 76

KB big.png 34.2 KB
Target audience: Not similar
Factual aspects : Not similar
Theme : Not similar
Style : Not similar

1. Rating for similarity based on factual aspect

Seed item: Grease
Target item: Pulp fiction

grease.png 59.9 KB
 pulp.png 90.1 KB

Target audience: Not similar

Factual aspects: Very similar --> Lead actor: John Travolta
Theme : Not similar:
Style : Not similar:

1. Similarity based on several aspects. Target audience and style.

Seed item: Survivor
Target item: Keeping up with the Kardashians

survivor.png 81.5 KB

kard.png 43 KB
Target audience: Slightly similar --> Both are reality TV but fall under different sub genre
Factual aspects : Not similar
Theme : Not similar
Style : Slightly similar --> Both reality TV style of interviewing the characters about the events
We will now dig deeper into each aspect and explain what they mean and how to categorize and
rate.

Target audience and Genre

For this aspect you assess similarity based on 2 things:
• Audience
• Genre

Audience
Is it a kids movie? what age? Is it a movie for adults? Is it for all ages?

Age groups and examples of movies and TV shows for these age groups:
Made for young kids, Content for kids under 8 : Sesame Street, Doc McStuffins, Peppa Pig, Dora
the Explorer, Bob the Builder, Teletubbies, PAW Patrol, Beauty and the Beast, VeggieTales, ,
Despicable Me, Finding Nemo, Toy Story

Made for older kids /tweens, Content for kids older then 10 but not allowed to see PG-13
content: Adventure Time, Andi Mack, Boy Meets World, Chopped, Little Mosque on the Prairie,
MasterChef Junior, High School Musical, Wonder, My Life as a Zucchini, To
Kill a Mockingbird, A League of Their Own. iCarly, School of
Rock, Andi Mack, Boy Meets World, Little Mosque on the Prairie, Bill & Ted's Excellent Adventure,
Back to the Future, The Goonies, Holes, Napoleon Dynamite
Note: Teenagers would be able to see PG-13 movies, but “tweens” the term for kids that are
between the ages of 10-12 are those kids that are not recommended to see PG-13 content without a
guardian or adult.

Not for kids.

Movies and TV shows that use a filmmaking style that is traditionally used for children productions,
but the subject matter is not for children at all. this can be animation that features mature content,
puppets that do adult things, etc
Examples: Heavy Metal, Team America, The Happytime Murders, Meet the Feebles, Batman:
Assault on Arkham, Akira, South Park, Family Guy, The Simpsons, Rick and Morty, Mr. Pickles,
Archer

Adults. Content for adults

How to find information about target audience's age?

On the Apple TV product page (click on the poster image of the title) there is often information from
Common Sense Media that gives a suggested appropriate audience age for the content.
This age number will be next to the Common Sense Media logo of a green circle with a check mark,
seen in this screenshot below.
https://tv.apple.com/US/movie/umc.cmc.4y1lybtx807iptcviuugevxti (15+)

drama.png 14 KB
Genre
What are the genres and sub-genres for this content?

Here is a list of genres and examples of sub-genres that can help you categorize the content. Please
note that genres are listed under the title of the content, but subgenera are not.

Genres / Sub-genres
ACTION - Stories that have the protagonist faced with a series of physical challenges that could
include violence, physical feats or frenetic chases.
Some sub-genres include action adventure, action comedy, action horror, war action, crime
action, spy action, sci-fi action, martial arts

ADVENTURE - Stories that involve the protagonists moving outside of their regular life and traveling
to different locales, this often could include some action packed scenes, but an adventure can
happen without any action
Some sub-genres include ancient epic, animal story, escape story, exploration adventure,
family-friendly adventure, fantasy adventure, historical adventure, sci-fi adventure

ANIMATION - Stories that are animated which could include traditional hand-drawn cell-type
animation or with new computer-animated technology or with stop-motion.
Some sub-genres include animated action, animated adventure, animated comedy,
animated drama, animated musical

ANIME - Animated movies and tv shows that originate from Japan with specific animation
characteristics. The stories are often fantastical but could also portray real world slice of life dramas
or comedies.
Some sub-genres include isekai, josei, bishounen, bishoujo, shonen, shoujo, slice of life,
mecha

COMEDY - A story that has the main purpose to make the viewer laugh
Some sub-genres include dark comedy, family comedy, historical comedy, late night tv,
satire, sex comedy, sitcom, social comedy, workplace comedy

CRIME - Stories that have a main plot focused on a crime, committing it, solving it or investigating a
past crime.
Some sub-genres include Crime drama, crime comedy, crime documentary, crime thriller,
film noir, neo-noir, prison story, police stories, legal stories.

DOCUMENTARY - A nonfiction story that is about a true factual account that has the impetus to
educate and enlighten the viewer.
Some sub-genres include biographical documentary, historical documentary, medical
documentary, music documentary, nature documentary, political documentary, pop culture
documentary, science documentary, arts & culture documentary, human observation

DRAMA - a scripted story that is often presented in a non-humorous tone that intends to illicit an
emotional response and connection.
Some sub-genres include biopic, based on a true event, coming of age story, faith-based,
family drama, historical drama, legal stories, medical drama, political drama, social drama,
workplace drama

FANTASY - a story that mainly uses magical elements, mythology, supernatural elements, folklore,
or exotic fantastic worlds
 Some sub-genres include fantasy action, wuxi, parallel worlds, animated fantasy,
supernatural horror, magic story

HISTORY - A story that revolves around the exploration or depiction of a true event that has
happened in the past
Some sub-genres include historical documentary, historical drama, based on a true event

HOLIDAY - a movie that centers around the notable December holidays including Christmas and
Hannukah
Some sub-genres include holiday romance, holiday comedy, holiday reality

HORROR - a story that elicits the emotions of being scared and frightened
Some sub-genres include vampire story, werewolf story, body horror, slasher, supernatural
horror, survival horror, monster story, horror comedy

KIDS & FAMILY - a story that is made for the intent for kids to enjoy - these stories could also be
enjoyed by older people
Some sub-genres include family-friendly comedy, family-friendly drama, family-friendly action,
family-friendly adventure, family-friendly documentary, family-friendly animation, educational

LGBTQ - A story that focuses on the plight of lesbian, gay, bisexual, transgender or queer people.
Some sub-genres include lesbian, gay, bisexual, transgender

MUSIC - A story that centers around a music-oriented theme - this can be a musical, music
documentary, music biopic, concert film or fictionalized account of a musician.
Some sub-genres include, concert film, dance story, musical, music documentary, fictional
music story

MYSTERY - a story that is revolved around a protagonist (sometimes a detective) who is attempting
to solve a crime or mysterious circumstance that remains mysterious until the end of the story.
Some sub-genres include Whodunit, film noir, Neo-noir, detective stories, procedurals

NEWS - tv show content that is made to inform the viewer of extremely recent, current notable
cultural, societal events.
Some sub-genres include daily news show, primetime news show, news magazine, weekly
news show, current news analysis

REALITY - a genre of tv that presents real life situations in an unscripted format to mainly entertain
audiences.
Some sub-genres include reality competition, reality soap operas, funny reality, reality
adventures, makeover/improvement, reality romance, social experiment

ROMANCE - a story that centers around the love relationships of the main characters.
Some sub-genres include romantic adventure, romantic comedy, romantic drama, tragic
romance, workplace romance, star-crossed lovers, marriage stories, dating stories

SCI-FI - a story that mainly features fictional science-based scenarios.

Some sub-genres include cyberpunk, sci-fi action, sci-fi comedy, sci-fi horror, space adventures,
alien invasion stories

SPORTS - a tv show or movie that centers around sports - this could be a fictionalized story about
sports or a documentary about sports
Some sub-genres include sports documentary, football stories, baseball stories, basketball
stories

TALK SHOW - a tv show that features a host or a panel of hosts that discuss topics like current
events, pop culture or other social issues.
Some sub-genres include culture & conversation show, late night talk shows, morning talk
shows, political panel discussions, émissions

THRILLER - a story that gives viewers heightened feelings of suspense, excitement, surprise or
anticipation and anxiety. The story gives you thrills.
Some sub-genres include heists, paranoia thriller, political thriller, psychological thriller, techno
thriller whodunit

WESTERN - a story that historically centers around life in the 19th century in the American Old
West, often centering on the life of a cowboy or in a small rural town.
Some sub-genres include revisionist western and spaghetti western.

When trying to assess similarity between 2 contents please try to identify the sub genre for a strong
similarity signal.

Rating guidelines:
• If genre and sub-genre are the same, rating should be Very Similar, Unless the audience is
completely different.
• If just the genre is similar it can be either Very Similar or Slightly similar , depending on
the case, You can use your judgement.

Example 1: Slightly similar

whitney.png 37 KB

apollo.png 53.8 KB
Target audience : Slightly similar

Reason: Both are documentaries, One is musical and the other is historical. Target audience may be
the same.

Example 2: Very similar

whitney.png 37 KB

marley.png 80 KB
Target audience : Very similar

Reason: Both are musical documentaries. The target audience may be the same.

Factual aspects
Evaluate similarity between the the 2 pieces of content based the following:
• Cast and Crew: Same actor or actress, director, studio, writer etc ...
• Location
• Time period
• Sequels-prequels

Rating guidelines:
1. If similarity in 2 or more categories (presented above) --> Very similar
2. If similarity for 2 or more cast&crew members --> Very similar
3. If similarity among lead actor or director --> Very similar
4. If similarity in one category (presented above) --> Slightly similar
 5. If no similarity --> Not similar

• Make sure not to confuse theme (What the movie is about) to facts when choosing your
rating.

Rating example:1

pp.png 67 KB
Factual aspects : Very similar

emma.png 86.3 KB
Reason: Location: England
Author: Jane Austen

Rating example:2

dallas.png 89.6 KB
 fri.png 87.7 KB
Factual aspects: Slightly similar
Reason: Location: Texas, USA

Theme
The theme aspect aims to answer the question what the movie is about. Evaluate similarity
between the 2 pieces of content based the following:

• Subject: What is the movie about?

For example:
Love story, Biography about a person, football, world war 2 , A super hero saving the world, animals
in nature etc ..

Rating guidelines:
• Give more weight to the subject when making your judgement.
• If the subject is completely different between the 2 movies. Rating should be Not similar
• When it’s clear that the it’s the same subject, rating should be very similar regardless of the
mood.
• If you find the mood a very important aspect of the 2 movies (Sad movies) rating can
be Very similar, Otherwise, Mood similarity only merits a Slightly similar rating.
Example:1

dallas.png 89.6 KB
 fri.png 87.7 KB

Theme : Very similar

Reason: Both contents are about football in Texas.

Example:2

grey.png 78.3 KB

ryan.png 68.8 KB
Theme : Very similar
Reason: World war 2, navy combat, army

Example 3:
 grey.png 78.3 KB

wwII.png 73.4 KB
Theme : Very similar
Reason: World war 2 navy combat vs World war 2 history.

Style
In contrast to the theme (presented above) style disregards the happenings of the content and
focuses on the question how the movie is produced. Cinematography is the focus of this aspect.
Evaluate similarity between the the 2 pieces of content based the following:

Cinematography Animated movie, Black and white / color, Silent movie,

Stylistic choices: stand up comedy , live show, panel/talk show

Where can you get more information about the style:

1. IMDB: Motion Picture Rating

Pulp fiction: https://www.imdb.com/title/tt0110912/?ref_=fn_al_tt_1
Rated R for strong graphic violence and drug use, pervasive strong language and some sexuality

2. IMDB parental guide: https://www.imdb.com/title/tt0077631/parentalguide?ref_=tt_stry_pg

 grease.png 59.9 KB

pulp.png 90.1 KB

Style : Not similar

Reason: There is no similarity with style between Grease and Pulp Fiction. Grease is a musical
with a traditional linear story structure and Pulp Fiction is crime drama with a lot of violence and a
nontraditional story structure