UPDATED-2022-12-08

[Updated notes in Purple]

Introduction: This Hitapp checks the quality of search results. The judge must scan the
whole page and see if the search results and the carousel answer are good, relevant and
meets the user query intent. The judge essentially has to check that carousel and search
results are talking about the same topic or not.

[Definition: User intent, otherwise known as query intent or search intent, is the
identification and categorization of what a user online intended or wanted to find when
they typed their search terms into an online web search engine]

What is a query?
- A query is the search query, as highlighted below in Orange (B)
What are Search Results?
- Search results refer to the list created by search engines in response to a query, as
highlighted in Purple (E)
 <Image: Important Keywords>

What is a carousel?
- A carousel is the list of answers, usually with images. For example, below is a
carousel with harry potter movies. Highlighted in red (D) in the picture.

What is the carousel Title?

- Carousel title is the title above the carousel answer as highlighted by Green (A) in
the picture above.
What is carousel content?
- Carousel content is nothing but the actual answer, highlighted by the Blue (C) in
the picture
 <Image: HitApp Screenshot>

How to Judge?
There are 2 options for judging the search results. The question is, ‘Is
Carousel consistent with the search results:”
1. Yes
2. No
 Let us cover examples of each one by one
1. Answer YES when the Carousel is consistent with the search results, for instance,
- The carousel title is related to the query.
- Carousel answer is related to the query
- Search results below the carousel match the query intent.
- A carousel could be a text carousel or image carousel.
- If everything else is accurate, it is okay if the carousel images are not
loading/missing.
- If everything else is accurate, there could be few results in the carousel.

2. Answer NO when the Carousel is inconsistent with the search results, for instance,
- When carousel title is not related to the query.
- The top few search results(5-10) and carousel answer have nothing in common.

Let’s cover each one of them in detail with examples and screenshots.
When to answer Yes:
- Carousel title is related to the query, maybe not exact match but the carousel title
should have the main keywords in the query
- The carousel answer is related to the query, and has correct, up to date content
i.e. list of characters, movies, tv series books etc, basically whatever user was
searching for.
- When the search results match the query intent. Like in the example below, both
a, b and c are met, so the search results are good. Judge “Yes on the hitapp”
In the above image, we are searching films in Japanese, and we are getting a movie
carousel with a mix of local movies as well as global movies. This carousel is consistent
with the search results as the search results also talk about the movies. It is okay if movies
link in the search results talk about different set of movies as long as there are movies and
the carousel captures the correct intent.
Above carousel is consistent too, because we are showing Spanish movies here, and the
list of movies in the search results doesn’t have to exactly match the movies in the
carousel.
- A carousel could be a text carousel or image carousel: Sometimes the carousel
content might be text carousel instead of image carousel, that is fine if the
information is correct. Like the cast of the movie “Forest Gump”, mentions all the
actors with their character names.

- If everything else is accurate, it is okay if the carousel images are not

loading/missing: Sometimes images could be missing from the carousel answer,
that is acceptable if the content is correct and the search results show results from
similar categories.
If everything else is accurate, there could be few results in the carousel.
Sometimes there might be very few items in the answer, it is acceptable if the
information is correct, and carousel answer and search results are in sync.
It is okay if there are fewer than expected items in the carousel, Say YES.
It is okay if there is less local content, we are only judging whether carousel and
the search results are talking about the same thing.
-
-
The carousel below is consistent with search results and has no images as we are
just searching for genre of the Movie Morbius, which is correct as search results
also talk about the same movie.
When to answer No:
- When carousel title is not related to the query. In the example below, the user
searched for the “the windfall cast”, the search results point towards the correct
movie called “Windfall” but the carousel is being triggered for a movie with a
different name, as shown in the title. Here, Carousel content is also incorrect.
- Mismatch in search results and carousel answer: When the carousel answer is not
related to the query and not the search results. A very good sign is when the
search engine results do not match the carousel, the title may or may not match.
For instance, below we are showing characters of some movie called “Generator”
but that does not match the search results below as you scroll down. “Character
generator” is a software and that is what the user was searching for, and not the movie by
the name “Generator”
The carousel in the next image is inconsistent with the search results, as we are showing
carousel for “xxx” movie genre but the search results are not talking about “xxx” film.
In the above image, the query translates to a non-film query and we are still triggering a
films carousel which is clearly inconsistent with the search results.