Uolo Weekly Errors - Missed Evidence (ES-MX)

Week Ending 11/11/2022

Based on our random audits, we have identified the following errors for Missed Evidence. The
purpose of releasing this is to provide some examples that most raters are missing for a
certain week and how to approach it moving forward. If you have received a quality notice
for the week, we hope that these examples will help you improve in your primary area of
misunderstanding.

What is Missed Evidence?

A thing to note with the Negative Factor for which you have been identified, “Missed
Evidence”, that this type of quality error can be as a result of 2 factors, sometimes this will be
because you selected “No Evidence” for a job and the majority of other raters found
evidence, and sometimes because you selected “No Claim” or “Satire that is not likely to
Mislead” and the majority of other raters identified a claim and found evidence.

For more information about jobs where people have typically Missed Claims, please refer to
the Identified Negative Factor – Missed Claims document.

https://connect.appen.com/qrp/images/document-builder/2646/Uolo_Negative_Factor_-
__Forming_A_Central_Claim.pdf

If you were cited for the Negative Factor “Known Missed Evidence”, this means that you
missed a claim and/or evidence in a result that was found to be Misinformation by Third
Party Fact Checkers.

How to find evidence

General tips and reminders for finding evidence

Most importantly, your central claim matters. Finding evidence for all or most of the jobs
you rate all starts with a properly formed and accurately reflective central claim. If you
form your claim improperly, for example, “This is news about Covid in Nur-Salan” and you
search for “Covid news in Nur-Sultan” you are not likely to yield results that are specific
enough to directly address the real claim you have in mind.

Page 1 of 12
Step one is always form your claim to be specific and in your own words.

• Never use broad terms like “news about”, “Covid vaccination”, “Smailov said” or
“Biden lied”.
• Never copy and paste the post text by author to use as your central claim.
• Use a specific central claim that directly addresses the main thing you will be
researching, which is the main thing the post is about.

Reverse image search is very useful

With so many photo claims in the market, you will often find the best way to find evidence is
to do a reverse image search for any photos on the post.

**Tip** You can also use reverse image search on videos by snipping a portion of the video
thumbnail or one of the frames as it plays.

You can do that by following these steps:

1. Click on the image contained within the post to enlarge it so it fits the full
screen, and then right click with your mouse and the following box appears.

2. Select “Search Google for Image”.

For further help on finding evidence, please refer to the following clarifications from the
Quality Hub:

Page 2 of 12
https://connect.appen.com/qrp/images/document-

builder/1771/Uolo_Finding_Evidence_and_Judging_Reliability_PH_Market_2.15.22.pdf

https://connect.appen.com/qrp/images/document-

builder/1771/Uolo_How_To_Reverse_Image_Search_and_Use_Google_Lens.pdf

https://connect.appen.com/qrp/images/document-

builder/1771/Guidance_on_Finding_Proper_Evidence.pdf

For more Missed Evidence examples please also find the following clarification:

https://connect.appen.com/qrp/images/document-builder/2646/Uolo_Negative_Factor_-

__Finding_A_Proper_Evidence.pdf

Top 1 Error: Failing to find evidence

Screenshot

Page 3 of 12
 Common Rating
No Evidence
Misunderstanding
Acceptable Rating Not Supported, Fully Contradicted

Practicable Approach to Post

Potential Evidence Source: https://www.biobiochile.cl/noticias/biobiochile-english/english-

chile/2019/05/30/tornado-sweeps-through-city-in-southern-chile-injures-16-people-and-
causes-huge-devastation.shtml

Potential Central Claim: Video shows a tornado spawned by Hurricane Ian in Florida.

Tips/Reasoning and Things to Watch out for:

• Performing a reverse image search on photos or key frames of videos and including
the term “fact check” in your keywords is helpful in finding evidence from reliable
sources.
• It is rare to find no evidence, although it may be difficult in some cases to find reliable
evidence. If your initial search does not yield reliable evidence, try using a different
search engine such as Duck Duck Go for keyword searches, or TinEye for reverse
image searches.
• For this example, an initial reverse image search linked to a social media post about a
tornado in Chile. Performing a second search with this information led to a news
article about a tornado in Los Angeles, Bio Bio, Chile, which includes the same video
in this job.

According to the evidence link above, the claim is not true:

Page 4 of 12
Rationale for Final Evaluation:
Based on the available evidence, it would be appropriate to label this job as "Not
Supported, Fully Contradicted”.

Page 5 of 12
Top 2 Error: Missed Evidence as a result of Missing a Claim

Screenshot

Page 6 of 12
 Common Rating
No Claim
Misunderstanding
Acceptable Rating Not Supported, Fully Contradicted

Practicable Approach to Post

Potential Evidence Source: https://www.reuters.com/article/factcheck-oms-time/fact-

check-portada-de-soldado-con-logotipo-de-la-oms-es-un-trabajo-artstico-no-una-
publicacin-oficial-de-la-revista-time-idUSL1N2U52E8

Potential Central Claim: This is an authentic photo of a Time Magazine cover featuring a
UN soldier wearing a new type of uniform.

Tips/Reasoning and Things to Watch out for:

• If you disqualify this job from further review, but another rater rates the post as
having a claim, this post gets passed through to Task B and evidence can be
matched to evidence, you are effectively pre determining that there is no evidence
for this post.
• Performing a reverse image search and including the term “fact check” in your
keywords can be helpful in finding evidence from reliable sources
According to the linked evidence, the claim is not true:

Rationale for Final Evaluation:

Based on the available evidence, it would be appropriate to label this job as "Not
Supported, Fully Contradicted”.

___________________________________________________________________________________________

Thank you for taking the time to go over these examples. It is our hope this clarifies
the concepts for you, and you can use the information to boost your quality moving

Page 7 of 12
forward. Please let us know if you have further questions. We are always happy to
help.

Page 8 of 12
 Weekly Errors Practice Quiz for Missed Evidence
1.

Part 1 Does this content have a central claim?

a) Yes - This content has a central claim.

b) Cannot determine, Wrong language or lacking cultural context
c) No - The content does not have a central claim.

Part 2. Which of the following would be the most acceptable way to perform a search for
evidence?

a) Perform a search in Google on your written central claim

b) Perform a search in DuckDuckGo on your written central claim
c) Highlight the text with your mouse, right click and perform a quick Google Search
d) Perform a reverse image search in Google Lens on the image.

Page 9 of 12
 2.

Does this content have a central claim?

a) Yes - This content has a central claim.

b) Cannot determine, Wrong language or lacking cultural context
c) No - The content does not have a central claim.

______________________________________________________________________________________________

Page 10 of 12
 Weekly Errors Practice Quiz Answers

Answer Question 1 Part 1: Does this content have a central claim?

a) Yes - This content has a central claim.

b) Cannot determine, Wrong language or lacking cultural context
c) No - The content does not have a central claim.

Explanation: The photo in this job contains factual claims that need to be evaluated. The
central claim of a job can be in the post text, media component (photo/video), linked article,
or a combination.

Answer Part 2. Which of the following would be the most acceptable way to perform a
search for evidence?

a) Perform a search in Google on your written central claim

b) Perform a search in DuckDuckGo on your written central claim
c) Highlight the text with your mouse, right click and perform a quick Google Search
d) Perform a reverse image search in Google Lens on the image.

Explanation: They are ALL acceptable methods of searching for a job, there is no one
answer to an acceptable search, some methods are more effective than others for specific
job types. Here All options will find results, but that isn’t always the case and just because
one method fails doesn’t mean you should not consider using a different approach.

Potential Evidence Source: https://www.reuters.com/article/factcheck-aprobacionamlo-71-

idUSL2N2X12R6

Answer Question 2. Does this content have a central claim?

a) Yes - This content has a central claim.

b) Cannot determine, SRT technical issue
c) No - The content does not have a central claim.

Explanation: The authenticity of the photo is the claim for this example. The main purpose of
this post is to imply that Beatriz Gutiérrez Müller, wife of AMLO, was photographed holding a
book critical of her husband, and encouraging people to steal the book.

Page 11 of 12
According to the evidence we found, the claim is not true:

Potential Evidence Source: https://www.reuters.com/article/factcheck-gutierrezmuller-

librocash-idUSL1N315253

Page 12 of 12