Find Brand & Purchase Related Terms

These guidelines contain confidential and proprietary information belonging to Microsoft Corporation.
The recipient understands and agrees that these materials and the information contained herein may not be used
or disclosed without the prior written consent of Microsoft Corporation.

Overview
Goal:

The goal of this task is to identify Brands from a user query, as well as working out if there is a desire to
make a purchase.

What is a Query?

A query is a line of text that a user enters into a search engine to satisfy their information needs.

Overview:

As a judge working on this task, you will be provided with queries that users type into search engines.
You will be responsible for identifying a Brand from a given query and, in addition, to decide whether
there is an intent to purchase based on the content of the query through the presence of terms related
to making purchases, performing product research or looking for ranking language (best, cheapest etc.)

There are a couple of important points to note:

➢ Please be aware of the fact that not every query will contain a Brand/Purchase Intent term. This
is expected behavior.
➢ We strongly encourage reviewing both the instructions for each rating and the examples
provided. Reviewing the examples will greatly increase your likelihood of success on the project.

What is a Brand?
Brand
A business creates a brand identity to help set themselves apart from competitors and make themselves
memorable to customers.

• This includes companies that make products (ex. Apple, Microsoft etc.), companies that sell
products (Amazon, Walmart etc.), service providers (Comcast, Puget Sound Energy, Marriot) and
commercial websites (YouTube, Facebook).

➢ A brand is a non-generic name of a product which indicates a specific manufacturer or source

for the product. Let’s take a look at the following examples:
o In the query “dove soap” the brand “dove” tells us the user wants soap made by a
specific company (Dove) and not its competitors (Dial, Ivory, Zest etc.)
 o
In the query “verizon iphone” the user wants to obtain a service from a specific provider
(Verizon) and not their competitors.
o In the query “kids sleeping bags walmart” the user is specifying they want to shop at
Wal-Mart and not their competitors.
➢ Brand names may also be used in the case where the product is non-generic.
o Example: “Apple iPhone”, “Microsoft Windows”, “Toyota Camry” etc.
➢ Here are a selection of brands you may recognize – consider other such examples:

Apple IPhone Nikon D750 VW Golf GTI 2015 KitchenAid stand mixer

➢ You may not always be familiar with the brands users are searching for or users may make typos
or spelling mistakes. Use research to make sure you don’t miss any brands.

Query Entities Comment

Puma Brand: Puma Puma is an athletic brand that sells a broad range of clothing, shoes
and accessories for a wide range of sports.
otterbox defender Brand: otterbox Otterbox is the brand of the product the user is looking for – note
apple ipad air 2 that the user is looking for the case, not the ipad – so apple is not
considered a brand.
laundry shelli segal Brand: laundry shelli segal Research shows that this entire query is part of a brand name, so
“shelli segal” isn’t Personal Name.
woodworkershardware Brand: Research shows this is a site that sells hardware.
com woodworkershardware
com
scotts lawn care Brand: scotts Scott’s is a company that sells a broad range of lawn care products
such as mulch, fertilizer, weed killer etc.
holland america line Brand: holland america Service providers are classed as Brand.
line
winberies restaurant Brand: winberies Research shows the name of the restaurant is “Winberie’s
bar oak park il restaurant bar, opentable Restaurant & Bar”. While free to the user, OpenTable generates
opentable revenue through ad services and paid services for restaurants.
tours the grand at Brand: the grand at moon Research shows “the grand at moon palace” is the name of a hotel.
moon palace cancun palace A hotel is a service provider so should be treated as a brand.

south dakota Brand: south dakota This is a purchasable educational service. In the context of the query
university online university “South Dakota University” is the service provider for this program.
Service providers are treated as Brands.
electrical engineering
degree

Purchase Intent
These terms fall into the following three categories:

➢ Terms that indicate the user specifically wants to purchase a product or service, such as:

“Buy” “sale” “price” “clearance” “wholesale” “purchase” “shop” “get”

➢ Terms that refer to the type of research a user might complete pre-purchase to find a product
that meets their specifications, including:

“reviews” “ratings” “charts” “specs” “guide” “chart” “size” “width”

➢ Terms users might use to rank products or indicate desirable attributes when trying to find the
product that best suits their needs.
o These types of terms are usually attributes that can be ranked on a spectrum. Examples
as follows:

“best” “cheap” “older” “newest” “affordable” “cool” “discount” “high end”

o Products can be ranked on a spectrum of best to worst, cheapest to most expensive,

oldest to newest, coolest to least cool, high end to low end, etc.

Query Entities Comment

google chrome Brand: google In the context of the query “Google” refers to the company that
download Purchase: download makes the queried browser. “Chrome” can be considered a
“purchasable” product, even though it is free. In the context of a
product, “download” indicates the desire to obtain that product.
home depot promo Brand: home depot A promo code indicates a purchase intent.
code Purchase: promo code
krumkake vs pizzelle Purchase: vs “Vs.” indicates the desire to find reviews and other information.
open checking account Purchase: open “Open” indicates the desire to obtain a service, in the context of the
online query.
 Terms that are not useful for Advertising

➢ This category includes terms that can’t be understood (junk, non-English) and terms that are
unrelated to making a purchase and are not useful for advertising.

Not Sure
• This is used when the term cannot be understood at all.
o Example: non-English language terms, junk, etc.

6t4oudpeztdql6 NotSure: 6t4oudpeztdql6 This term returns no search results – there is no way of determining
the meaning of this term.
billige adidas NotSure: billige Non-English language terms that cannot be understood in the context
laufschuhe mit laufschuhe mit of the query.
kostenlosem versand kostenlosem versand
Brand: adidas If there are terms in the in-market language you can classify them –
for example here we can clearly see the brand term “Adidas”.
However, if you are unsure – better to choose NotSure.
louis vuitton damier Brand: louis vuitton “Damier” is French for checkerboard. Research shows “damier
graphite district mm graphite” is one of the color options for this bag. The term has
meaning in the context of the query and not in English

• This should also be used to capture terms that don’t have any purchase intent and don’t have any
usefulness in identifying ads.
o To clarify, we define useful terms as things like brands, merchants, products, services,
company names, organizations, locations, movie/song/book titles etc. These are terms or
entities we can link to ads.
o We define Not Sure terms as those with no purchase intent.

Query Entities Comment

lien holder NotSure: lien holder Dictionary term. If you are not familiar with a query term, it is your
responsibility to research the term. It is very easy to find a definition
of this phrase with a quick search determine it isn’t related to
purchasable products or services.
wedding gifts am i NotSure: am i wrong Exclamation/Opinion - “am I wrong here” doesn’t provide any useful
wrong here here information to help find ads related to wedding gifts.
coughing tickle throat NotSure: coughing tickle Non-purchase related information/research. The term remedy in this
remedy throat remedy query most likely refers to tips or home remedies – the user does not
want to make a purchase. We cannot use this term to help find
advertising.
ssi disability NotSure: ssi disability Non-purchase related information/research. The query term is for a
unemployment unemployment government benefit. It is not purchasable so we cannot use the term
to find advertising.
Aegean check luggage NotSure: check luggage Aegean is a useful term because it is an airline and so we could use
this term to find good advertising related to flying, However, “check
luggage” refers to an activity unrelated to a purchase.
address lookup NotSure: address lookup The user wants to find an address directory. This is not a purchasable
service.

• Extra articles (a, an, the, and) etc. that aren’t part of any useful term.

hotels near the space NotSure: the Hotels are a purchasable service and “Space Needle” gives us
needle important information about where the hotel should be. These are
useful terms. But “the” isn’t part of any useful term.

Labeling Overview
Important note on Brand identification:

➢ A Brand should only be identified when it specifically applies to the Product the user is looking
for – see example 2 for details.

There are 3 categories you need to consider when judging

➢ Brand
➢ Purchase Intent
➢ Not Sure

Let’s walk through a couple of examples:

Example 1: Query: “aldi weekly ad”

•
Does the user indicate a specific brand in the query (specifically for the
Is there a specific product the user is looking for)?
brand? • Research shows that Aldi is a merchant which sells groceries.
Example 2: Query: “acme tools•” BRAND: Aldi

• Is there wording in the query which suggests the user is looking to make
Is there a purchase a purchase?
• The term “weekly ad” indicates the user intends to make purchases
intent?
from this merchant
• PURCHASE INTENT: weekly ad

• Are there any terms which are not understandable in the context of the
Are there any terms
query (e.g. foreign language/junk) or non-purchase related terms?
which are not • No such issues with language in this example
understood? • No such non-purchase terms in this example
 Example 2: Query: “insten leather case strap apple ipod classic”

• Does the user indicate a specific brand in the query (specifically for the
product the user is looking for)?
• Research shows that insten is a company that makes leather cases for
Is there a specific ipods.
brand? • BRAND: insten
• Note: since the product the user is looking for is a case, the only
relevant Brand is that of the case maker. The brand ‘Apple’ in Apple
ipod is to be ignored in this example.
•

Is there a purchase • Is there wording in the query which suggests the user is looking to make
a purchase?
intent? • No such purchase-related terms stated

• Are there any terms which are not understandable in the context of the
Are there any terms
query (e.g. foreign language/junk) or non-purchase related terms?
which are not • No such issues with language in this example
understood? • No such non-purchase terms in this example

The judging interface

Here is what you will encounter when judging:

Judging Process
➢ Select the appropriate words from the drop-down list for each term (Brand/Purchase Intent/Not
Sure), where applicable.
➢ Only select Brand if it applies to the Product the user is looking for (see Example 2 {insten
leather case strap apple ipod classic} above).
➢ Use the research panel on the right if and when needed to help identify a Brand unknown to
you.
➢ Click Submit
➢ A further Query will then be populated for the next judgment.
 ➢ To reiterate, not every query will contain a Brand/Purchase Intent/Not Sure term. In that
scenario you are to leave the input fields blank and click Submit
➢ There are also tool tips available to help you remember details – you can access them by
hovering over the question mark in the interface.

Judging Process
➢ Select the appropriate words from the drop-down list for each term (Brand/Purchase Intent/Not
Sure), where applicable.
➢ Only select Brand if it applies to the Product the user is looking for (see Example 2 {insten
leather case strap apple ipod classic} above).
➢ Use the research panel on the right if and when needed to help identify a Brand unknown to you
➢ Click Submit
➢ A further Query will then be populated for the next judgment
➢ To reiterate, not every query will contain a Brand/Purchase Intent/Not Sure term. In that
scenario you are to leave the input fields blank and click Submit