Guidelines for Query Interest Tagging

[CONFIDENTIAL - DO NOT DISTRIBUTE]

Table of Contents
Last Updated: 01/18/2023

Intro to Taxonomy Node Tagging 2

0. Welcome 2
0.1 Glossary 2
0.2 Steps 3
1. Review the Entities 3
2. [OPTIONAL] Select a Checkbox and leave comments 3
3. Taxonomy Node Association 4
1.0 Query Entities 6
1.1 Vertical Examples 6
1.2 Ambiguous examples 8
2.0 L1 Verticals 15

Intro to Taxonomy Node Tagging

0. Welcome
Our goal is to find the best entities to represent the meaning of nodes in our Taxonomy

We will do this by reviewing Entities (queries) and selecting the most appropriate node(s) (or
category(-ies)).

0.1 Glossary
● Entity: In this workflow an entity is a search query.

● Vertical: a vertical is a top-level category in the taxonomy.

○ For example, Food & Drink is a vertical.

● Node: a node is a category in the taxonomy.

○ For example, Alcoholic Drinks is a node.

● Intent: Intent is what the average user is likely looking for, thinking about, researching,
or wanting to know more about when they searched for a particular query.
 ● Parent/Children/Grandchildren: Used to identify relations in the taxonomy. A node’s
children are the different node(s) that exist under it. E.g. Tequila is the child of liquor and
Liquor is the parent of Tequila. Similarly Alcoholic Drinks is Tequila’s grandparent.
○ Food and Drinks > Drinks > Alcoholic Drinks > Liquor > Tequila

0.2 Steps

1. Review the Entities

○ Click on the linked entity and review the Google search results.
○ Identify the entity's main L1 verticals or top level categories based on its overall
content and intent (i.e. the purpose/reason of the entity).
i. NOTE: If an entity is not known to you, research it.
○ Think about the overall intention of the vertical you’re considering and ask
yourself:
i. What types of queries are intended for the vertical?
ii. Which pinners in which state of mind are likely to engage with these
queries?
iii. What are the expected results upon searching for a specific query?

2. [AS NEEDED] Select a Checkbox and leave comments:

○ The link for the entity is broken or there is nothing to label.

■ If the entity is not available to judge (dead page, blank page, etc), check
this box and move on.

○ The entity is too ambiguous.

■ If the query results are a mix of intents and you’re unable to mentally
summarize potential intents.

○ This entity does not fit with any node at all.

■ If the entity is not ambiguous, but there is no taxonomy node available
that you feel represents this entity, check the box, leave a comment
noting what node is needed, and skip labeling.

○ I do not understand the language of this entity.

■ If the topic of the entity is in a non-local language different from English or
a language you do not understand, check the box and skip labeling.

■ NOTE: Without spending too much time, you should attempt to do your
best to understand the entity (via Google translate) before selecting this
check box.

○ This entity is unsafe.

 ■ Review search results. If the content of the entity is unsafe, check the
box and skip labeling.

■ The query has antagonistic, explicit, false or misleading, harmful, hateful,

or violent content or behavior.
■ Examples of unsafe queries: aex party, naked atheletes, underslung
shotgun, crotch shots, make a rifle stock, lolitas

3. Taxonomy Node Association

○ Select ALL of the most granular and appropriate node(s) under the identified L1
categories. Remember, think about the overall intention of the vertical.

i. A node with name “X” under vertical “Y” should be selected when the
entity is related to pinner’s need to solve the following: “I want to do
something about X (the node) under the intent of Y (the L1 vertical)”
ii. Keep the intent of the L1 Vertical in mind when selecting a node. Both the
L1 vertical and the selected node need to make sense for the entity in
order to be selected.
1. Example: A query about “how to grow Cilantro” makes sense in
the Gardening vertical, but not in the Food and Drinks vertical
because it’s not about how to use Cilantro in recipes.

We would not select the node with the name “Cilantro” under
vertical “Food and Drinks” here; that is not helping them solve the
following “I want to do something about growing cilantro in
Gardening.”

The two appropriate nodes to select would be the granular nodes

“Planting Ideas” and/or “Herb Garden” in the vertical Gardening as
by selecting these, the pinner will solve “I want to do something
about growing cilantro in Gardening.”

iii. For queries, it means the queries are what pinners might search when
they face the problem.

○ Children of each parent node are displayed directly below the node name (in the
section “Additional related interests under it”) to help you feel confident you are
selecting the most appropriate node for the query.

○ When you find a node(s) that represents the intent of the query, click on the
“Select this category” link. You should see your node selection appear below the
selection tree box. If needed, you can click on the “x” or click on “Deselect this
category” to remove the associated node.
i. When you select a node, its parent, grandparent, etc., are automatically
included, as you’ll see below the selection tree box. No need to select the
parent, grandparent, etc., individually.
1.0 Query Entities
● Queries will point to Google search results. Google provides a good, relatively unbiased
and comprehensive view of what a query is about and what people most likely want to
see given the query.

● One to two word queries are often very difficult to categorize (“distraction,” “nice boy”)
○ If you can not categorize the query, iit may be most appropriate to select the
“Ambiguous” checkbox for these and not label.

● If a query could have multiple meanings, but you still have an idea what the intent is,
please follow the guidelines and “Select nodes within as many verticals as the
intents of the entity.” Some examples:
○ “therapeutic activities” - labelers can select Physical Health AND Mental Health.
○ “cat and man” - labelers can select Entertainment AND Photography (as google
search shows those to be the likely reason to query that)
○ “yellow ico” could be a misspelling of ‘yellow icon’ or could be the Final Fantasy
character ‘ico yellow.’ Labelers can select Digital Art AND Entertainment >
Fantasy Games

● Use your best judgment, but as a general guideline only focus on:
○ The FIRST PAGE of Google search results; and
○ How many times something appears (e.g. ‘ico yellow’ only appears once)

● Not every query with a keyword belongs in that node.

○ For example not all queries with “x images” belong in Photography and not all “y
video” belong in Video.
○ Also, even if the keyword “images” is not used, you should still be thoughtful
about when you select Art (Photography, Illustration, etc). See the examples
“alone in street” vs “jawa poster” below.

● While we do want all relevant nodes selected, it is OK to label on the stricter side.
○ For “dog drawing,” they’re interested in dogs, but they’re looking for a drawing, or
wanting to learn how to draw one. In this example, only a node for ‘drawing’ is
needed.
○ For “womens skirt pattern,” they’re interested in getting a pattern to make a skirt.
They want a pattern, and in the end, they want a skirt. A node(s) for the pattern
and a node for the skirt are needed.

● If there is a crafting element to a query’s results, such as a handmade tablecloth, a

parent-made game, or handcrafted jewelry, the most granular node within DIY and Crafts
should also be selected due to the crafting element even if the intent isn’t DIY. For
example, jewelry from etsy isn’t DIY, but it is a craft as it’s handcrafted.
1.1 Vertical Examples
Note: The nodes in red in the examples below are the most granular selectable nodes. The
grayed out nodes can be found under “additional related interests under it” in Appen (ADAP)
and are provided for context
Query jawa poster Art > Poster Design
Home Decor > Wall > Wall
Art > Posters, Prints, &
Visual Artwork
Query crocheted afghan DIY and Crafts > Fabric
Crafts > Knitting and
Crochet > Crochet >
Crochet for Home > Crochet
Blanket
Home Decor > Room Decor
> Bedroom > Bed Linens >
Blanket
Query alone in street Art > Photography
Query afternoon tea london Travel > Restaurant
best
Travel > Travel Destinations
> Europe Destinations > UK
and IE Destinations > UK
Destinations > London
Query yellow ico Art > Illustration > Icon
Illustration
Design > Web and App
Design > App Icon Design
Search tells us jawa is a
motorcycle brand or a
character in Star Wars.
This query is about
someone looking for or
wanting to create a poster
of a motorcycle, not a
motorcycle. Select labels
for the poster only.
Search results show ones
to make and ones to buy.
This query is more
ambiguous, but search
tells us it has an artistic
element to the query, so
selecting L1 Art is ok here.
Google takes it a step
further and specifies
Photography.
This query is about finding
the best place in London
for afternoon tea. They
want to be in London, and
they want a tea parlor.
Search results show this is
most likely a misspelling of
“yellow icon.” We see one
result of a game character
with a similar name, but
overall, this query seems

Query cristiano ronaldo sem Sport > Soccer > Soccer
camisa Players > Cristiano Ronaldo
Entertainment > Celebrities >
Celebrity Photos
Query maternity photography Art > Photography >
race car Photography Subjects >
People Photography >
Family Photography >
Maternity Photography
Query new puppy Event Planning > Personal
announcement Celebration
Query boho cale Food and Drinks > Desserts
> Cake > Cake Design
Query etsy earrings Women’s Fashion >
Women’s Jewelry and
Accessories > Women’s
Jewelry > Ear Jewelry
DIY and Crafts
Query 1st birthday party DIY & Crafts > DIY Event >
DIY Birthday
Event Planning > Hosting
Occasions > Kids’ Party
Event Planning > Personal
Celebration > Birthday > 1st
Birthday
Query how to grow cilantro Gardening > Planting >
Plants > Planting Ideas
Food and Drinks >
Condiments > Herbs >
Cilantro
to be looking for icons.
seeking images of the
soccer player, shirtless.
This query is not related to
maternity health or to
Motor Sports racing.
They’re either looking for
ideas for a maternity shoot,
or want to look at photos
with a maternity theme.
The intent of this query is
not to look at images of
dogs.
This query is most likely a
misspelling of “cake.” This
is a query looking for the
dessert. Do not assume
Events Planning or any
other node.
Search results show etsy
earrings should be
handcrafted, but it’s not
clearly DIY, so stop at DIY
and Crafts instead of going
deeper to DIY Jewelry.
Google shows ideas for
throwing a birthday party
for a 1 yo ranging from
buying supplies to creating
decorations.
This fits the Gardening
vertical since the intent is
to grow it.
It doesn’t make sense in
Food and Drinks even
though you’ll find “Cilantro”
there, because it’s not

Query womens skirt pattern Women’s Fashion >
Women’s Bottoms > Skirts
DIY and Crafts > Fabric
Crafts > Sewing > Sewing
Patterns
DIY and Crafts > Fabric
Crafts > Clothes > Clothes
Pattern
Query 49ers sweatshirt Women’s Fashion >
Women’s Top > Sweatshirt
Men’s Fashion > Men's
Shirts and Top
Query repurpose mens ties DIY and Crafts > Fabric
Crafts
Men's Fashion > Men's
Accessories > Men's Tie
Query snoopy Entertainment
Food and Drinks > Nuts >
Peanuts > Snoopy
Query penne alla vodka Food and Drinks > World
Cuisine > Italian Recipes >
Pasta
Food and Drinks > Drinks >
Alcoholic Drinks > Liquor >
about using it in a recipe,
so it doesn’t fit under Food
and Drinks.
User is looking for a
pattern to make a skirt.
They want a pattern
because they want a skirt.
Choose both concepts.
Results show men’s and
women’s, so include both
paths.
Same as with the jawa
poster, they’re after an
item, a sweatshirt, so we
don’t select NFL.
Results are about using
men’s ties for reasons
other than men’s fashion,
turning them into
something else. The intent
is fabric craft.
Although they’re using
men’s ties, the intent isn’t
about Men’s Fashion, so it
is not selected.
For a node that has an
incorrect L1 Vertical, the
best action for labelers is
to select the best L1. The
correct L1 here is
Entertainment, not Food
and Drinks.
Here, Vodka is a main
ingredient, although it is
not used as a Drink. It is
still OK to select Pasta and
Vodka, as the intent is
about the L1 Food & Drink
 Vodka and both Pasta and Vodka
nodes are in this vertical.

Query valentines day design Event Planning > Holiday > This query is about graphic
graphic Valentine’s Day design for Valentine’s Day.

Design > Graphic Design

1.2 Ambiguous examples

In the exemples below select “The entity is too ambiguous.”

Query crushi no clear, repeated

possibilities in results.

Query spending time maybe the song, or a

book, or general way of
life…

Query black full set nails? bedding? key caps?

unclear.

Query maladive misspelling of Maldives?

‘sickly’ in French,
uncommon word for sickly
in English. Unclear.

A note on Interior Design vs Home Decor:

● An Interior Designer works with architects to improve a space’s functionality, and with the
client to make the space work for the client. They work with changing the
rooms/building’s structure, not just the contents, and they also decorate.
● A Decorator works with the existing structure.
2.0 L1 Verticals

Below are the different verticals or L1 Categories of the taxonomy.

Animals
Architecture
Art
Beauty
Children's Fashion
DIY & Crafts
Design
Education
Electronics
Entertainment
Event Planning
Finance
Food and Drinks
Gardening
Health
Home Decor
Men's Fashion
Parenting
Quotes
Sport
Travel
Vehicles
Wedding
Women's Fashion