CYBERPSYCHOLOGY, BEHAVIOR, AND SOCIAL NETWORKING

Volume 00, Number 00, 2019

ª Mary Ann Liebert, Inc.
DOI: 10.1089/cyber.2018.0475

The Blue Check of Credibility:

Does Account Verification Matter When
Evaluating News on Twitter?

Stephanie Edgerly, PhD1 and Emily K. Vraga, PhD2

Abstract
Downloaded by East Carolina University from www.liebertpub.com at 03/12/19. For personal use only.

The increased reliance on social network sites for news and the proliferation of partisan news have refocused
scholarly attention on how people judge credibility online. Twitter has faced scrutiny regarding their practices
in assigning the ‘‘verified’’ status to Twitter accounts, but little work has investigated whether users apply this
cue in making assessments for information quality. Using an experimental design, we test whether the Twitter
verification mark contributes to perceptions of information and account credibility among news organizations.
We additionally consider how account ambiguity and account congruence with political beliefs condition this
relationship. Our results suggest little attention is paid to the verification mark when judging credibility, even
when little other information is provided about the account or the content. Instead, account ambiguity and
congruence dominate credibility assessments of news organizations. We propose that Twitter may need to
revise their verification badges to increase their salience or provide more information to users. Currently, users
appear to rely on other cues than the verification label when judging information quality.

Keywords: Twitter, credibility, partisan news, social media

Introduction on Twitter. We use an experimental design to address the

following questions: to what extent does account verification

S ocial media websites are increasingly spaces for en-

countering, consuming, and sharing news.1 One attribute
of these sites is context collapse.2 A diverse array of sources—
impact perceptions of credibility? And if so, how do partisan
cues and account ambiguity affect this process?

from political to news and commercial—populate users’ social

media news feeds. No matter their history, reputation, or
content shared, these sources are visually similar. An under-
studied potential difference between these online sources is the
account verification (shown as a check mark) that social media
sites, like Twitter, provide to accounts that either apply for
verification or are internally granted verification. According to
Twitter, ‘‘The blue verified badge on Twitter lets people know
that an account of public interest is authentic.’’3
In principle, the badge provides a mark of authenticity. It
signals that Twitter has confirmed the accounts’ legitimacy
and, as such, its presence (or absence) should impact how
users make credibility judgments. It is this rationale that
prompted Twitter to pause its public verification process in
late 2017, amid concerns that Twitter users ‘‘think of it as
credibility, like Twitter stands behind this [account].’’4
Our study explores how powerful this relatively subtle cue
is for users when they encounter and evaluate partisan news
Hypotheses and Research Questions
Credibility is an audience-based concept reflecting the
extent to which individuals find a source or message be-
lievable, accurate, and trustworthy.5 A long line of research
finds that people rely on simple cognitive heuristics, or
mental rules-of-thumb, to quickly render credibility judg-
ments.6 Twitter account verification appeals to a number of
these heuristics, especially endorsement and expectation
violation.
Endorsement offers ‘‘conferred credibility,’’ transferring
credibility from another source. Twitter, when granting an
account-verified status, provides an endorsement that the
account is legitimate and authentic. In contrast, expectation
violation refers to basic ‘‘surface characteristics of web-
sites and sources’’6 that, when absent, can signal that a
source or message is not credible. It may be that having a
verified Twitter account is expected of accounts that brand

1
Medill School of Journalism, Northwestern University, Evanston, Illinois.
2
Department of Communication, George Mason University, Fairfax, Virginia.

1
 2 EDGERLY AND VRAGA
themselves as news. Twitter users may look for this norm
violation, much like they look for misspellings or the design
layout of websites,7 as a quick cue about the source or
message credibility. Given this, we expect:
H1: The presence of a verification check will result in higher
evaluations of (a) tweet credibility and (b) account credibility.
We also consider whether verification intersects with two
additional source-related factors. First, we expect verification to
have a differential impact based on level of source ambiguity.
Under conditions of low ambiguity—when Twitter users have
ample knowledge of an account—verification should have
limited influence. The credibility of prominent mainstream
news sources, for example, will be uninfluenced by verification
because these sources have name recognition and reputations
useful in assessing credibility.8 It is under conditions of in-
creased ambiguity that verification should matter. High ambi-
guity exists when Twitter users encounter an unknown source;
Downloaded by East Carolina University from www.liebertpub.com at 03/12/19. For personal use only.
while medium ambiguity exists when a source is unknown, but
the account name provides cues about the source (i.e., the name
‘‘Conservative Tribune’’ gives users information about the
source, even if users are unfamiliar with it).
H2: The presence of a verification check will have a larger
impact on evaluations of (a) tweet credibility and (b) ac-
count credibility when combined with medium and high
ambiguity accounts, than when it is combined with low
ambiguity accounts.
RQ1: Will a verification check influence (a) tweet cred-
ibility and (b) account credibility more for high versus
medium ambiguity accounts?
Second, people often judge sources and messages that are
congruent with their political ideology to be more credible.6
We explore how this process interacts with verification sta-
tus. On one hand, conditions of congruency and incon-
gruency may reach a sufficing principle—these are primary
and powerful heuristics that allow people to make credibility
judgments.9 As such, verification status may not matter. On
the other hand, subtle ‘‘proximal cues’’ that accompany news
stories (e.g., timestamps) can increase judgments of credi-
bility, but only for low-credibility news sources.8 Extending
this logic, a verification check may increase credibility, but
only for incongruent sources (typically seen as less credible).
Given competing expectations, we ask:
RQ2: Will a verification check influence (a) tweet cred-
ibility and (b) account credibility more for congruent versus
incongruent accounts?
Our final research question tests whether verification de-
pends both on ambiguity and congruence.
RQ3: What influence do various combinations of ac-
count cues (verification status, ambiguity, congruency)
have on tweet and account credibility?
Methods
To test these hypotheses, data were collected from 616 par-
ticipants recruited from Amazon Mechanical Turk using the
Turk Prime interface on February 5, 2018.* Participants were
offered $0.60 for participating in a 6-minute survey. Participants
were 36 years old (M = 36.13, SD = 11.42), 50% male, and 80%
white, with 49% of the sample having at least a 2-year degree.
After answering a short pretest questionnaire, participants
were randomly assigned to 1 of 12 conditions across three
experimental factors. Across all conditions, participants
viewed a single tweet, with the content held consistent and
ideologically neutral, and then answered a series of questions
about that specific tweet (Fig. 1). First, we manipulated ac-
count verification of the source of the tweet, as indicated by
the presence (or absence) of the blue check mark used by
Twitter. Second, we manipulated the ideological positioning
of the account that posted the tweet such that three sources
favored Republicans, whereas three sources favored Demo-
crats.{ We then matched these sources with participants’ party
affiliation (N = 319 Democrats, N = 167 Republicans), as re-
ported on a 7-point scale in the pretest questionnaire, to create
a measure of congruence (e.g., source matched participants’
affiliation) versus incongruence. We classified those who
‘‘leaned’’ toward a party as partisans, and excluded true In-
dependents (N = 130) from our analyses. Third, we manipu-
lated the ambiguity of the account that posted the tweet: low
ambiguity, defined as mainstream partisan sources (Fox
News, MSNBC), medium ambiguity, defined as lesser-known
sources with partisan cues (Conservative Times, Forward
Progressives), and high ambiguity, defined as lesser-known
sources without partisan cues (Daily Caller, Palmer Report).
Measures
Tweet credibility. A series of 7-point semantic differen-
tials asked participants to evaluate the ‘‘content of the tweet’’
as: complete/incomplete, accurate/inaccurate, unbiased/bi-
ased, trustworthy/not trustworthy, credible/not credible, and
tells the whole story/not tells the whole story to measure
tweet credibility. These items were averaged to form a scale
(M = 3.92, SD = 1.13, a = 0.82).
Account credibility. Participants next evaluated the
account that posted the tweet as: unbiased/biased, trustworthy/
untrustworthy, unprofessional/professional, not credible/
credible which we averaged (M = 4.11, SD = 1.31, a = 0.87).
Manipulation checks
To ensure people were paying adequate attention, we
asked participants at the end of the posttest questionnaire to
report the account that posted the tweet they just saw out of
nine possible sources. Overall, 79.4% of the sample correctly
identified the source of the tweet. A logistic regression
suggests that correct identification of the source was not
predicted by verification or congruence, but low ambiguity
sources were more often identified than high ambiguity
*This sample was extracted from a larger experimental design,
which included three non-partisan news sources that were outside of
the scope of this article. These conditions were not crossed with the
design here and were excluded from all analyses (N = 322).
{
The ideological positioning of the sources was taken from
the media bias chart, available here: www.allgeneralizationsare
false.com/media-bias-chart-3-1-minor-updates-based-constructive-
feedback. The sources were all classified as ‘‘most extreme’’ or
‘‘hyper-partisan’’ in liberal or conservative positioning.
 THE BLUE CHECK OF CREDIBILITY 3
Downloaded by East Carolina University from www.liebertpub.com at 03/12/19. For personal use only.

FIG 1. Example of Twitter

verification manipulation. (a)
Unverified tweet from the
Conservative Tribune. (b)
Verified tweet from the
Conservative Tribune. Note:
Both these posts are also
classified as medium in ac-
count ambiguity and ideo-
logically congruent (among
Republican participants) and
incongruent (among Demo-
crat participants). All con-
tents of the post, including
engagement metrics, were
kept constant across all
conditions.

sources (B = 0.96, SE = 0.29, p = 0.001, odds ratio = 2.62).
We limit all subsequent analyses to those who correctly
identified the source of the tweet (N = 489).
A second manipulation check asked whether participants
had heard of the account source before. Logistic regression
confirms that low ambiguity accounts were more well known
than the high ambiguity accounts (B = 5.22, SE = 0.58,
p = 0.000, odds ratio = 184.54), and that medium ambiguity
accounts were less known than the high ambiguity accounts
(B = -2.27, SE = 0.64, p = 0.000, odds ratio = 0.104). For
example, over 95% of participants had heard of the low
ambiguity accounts, and under 10% had heard of the low and
medium ambiguity accounts.{
{
The one outlier among medium and high ambiguity accounts
was The Daily Caller, in which -27% of participants had heard of
the source previously.
 4 EDGERLY AND VRAGA

Table 1. Analysis of Variance Testing the Effects of Account Verification, Congruence,

and Ambiguity on Tweet Credibility and Account Credibility
Degrees of freedom F p Partial g2
Tweet credibility
Verification 1 0.19 0.67 0.000
Congruence 1 12.04 0.001 0.031
Ambiguity 2 1.75 0.18 0.009
Verification X congruence 1 0.02 0.89 0.000
Verification X ambiguity 2 0.54 0.58 0.003
Congruence X ambiguity 2 5.85 0.003 0.030
Verification X congruence X ambiguity 2 0.07 0.93 0.000
Account credibility
Verification 1 0.28 0.60 0.001
Congruence 1 56.92 0.000 0.132
Ambiguity 2 4.47 0.01 0.023
Verification X congruence 1 0.26 0.61 0.001
Verification X ambiguity 2 0.38 0.68 0.002
Congruence X ambiguity 2 10.40 0.000 0.053
Downloaded by East Carolina University from www.liebertpub.com at 03/12/19. For personal use only.

Verification X congruence X ambiguity 2 0.09 0.91 0.000

A final manipulation check asked participants whether
the account had a verified Twitter account. A logistic re-
gression confirms that our manipulation of verification in-
creased the likelihood that participants correctly identified
the account as verified (B = 1.07, SE = 0.22, p = 0.000, odds
ratio = 2.92), from 39% when it was unverified to 61%
when it was.
Results
We test H1 using an analysis of covariance, which en-
tered verification as the experimental factor while control-
ling for the six sources examined in this study. Our first
hypothesis is unsupported. Verification did not exert a main
effect on tweet credibility, F(1,489) = 0.07, p = 0.79, partial
g2 = 0.000, or account credibility, F(1,489) = 0.05, p = 0.82,
partial g2 = 0.000.
The next hypothesis and research question test whether the
effects of verification are conditioned by characteristics of
the source itself—namely, the ambiguity of the account and
its congruence. A series of three-way analysis of variance
with verification, ambiguity, and congruence as factors
confirm that verification still does not have a main effect on
tweet or account credibility (Table 1). Next, we test H2,
which predicts that verification matters more for high and
medium ambiguity accounts, compared with low ambiguity
accounts; and RQ1, which compares high and medium am-
biguity accounts. We find verification is not conditioned by
source ambiguity for either type of credibility.x
The final research questions explore whether verification
depends on congruence, as well as account ambiguity. We
find no support for these expectations; congruence does not
x
There is a significant main effect of ambiguity on perceptions of
account credibility, with medium ambiguity accounts (i.e., unknown
sources with partisan cues) seen as the least credible (M = 3.78,
SE = 0.12) compared to high ambiguity accounts without partisan
cues (M = 4.29, SE = 0.12, p = 0.01 using Bonferroni adjustment),
with low ambiguity accounts falling between (M = 4.04, SE = 0.10).
This effect is not significant for tweet credibility.
moderate the effects of verification (RQ2), nor is there a
three-way interaction between these factors (RQ3).** Instead,
there is a main effect of congruence and an interaction with
ambiguity.{{
Discussion
This study tests the prevailing assumption that Twitter
account verification badges influence how people make
credibility judgments. Our results consistently show that
verification, on its own or combined with other source-
related cues, did not influence evaluations of a news tweet or
its source. Even when people were unfamiliar with the source
(and were provided no partisan cues), the presence of a
verification mark did not alter credibility. Instead, account
ambiguity and congruency were more powerful cues in as-
sessing credibility.
These findings may underscore the lost value of Twitter
verification badges. The number of Twitter accounts granted
verification status doubled, from 150,000 to 300,000 accounts,
in <3 years.10 Such growth parallels Twitter opening (and then
closing) the application process to the general public. What was
once a useful heuristic for assessing credibility may now be
unhelpful due to its prevalence. Of course, this assumes people
**
We also ran these analyses excluding the high ambiguity
conditions because these conditions lacked cues about congruence.
However, these analyses are substantively similar to those re-
ported in the paper, suggesting these conditions were not muting
our ability to perceive an interaction between verification and
congruence. Please contact the lead author for more details on
these analyses.
{{
Consistent with previous research, people rate the tweets and
accounts as more credible when they are congruent (tweets: M = 4.06,
SE = 0.08; accounts: M = 4.54, SE = 0.10) rather than incongruent
(tweets: M = 3.66, SE = 0.08; accounts: M = 3.54, SE = 0.09) with their
party affiliation. In addition, a significant interaction between ambi-
guity and congruence shows that low ambiguity congruent accounts
were seen as the most credible, whereas low and medium ambiguity
incongruent accounts were seen as least credible. Please contact the
authors for more details on these analyses.
 THE BLUE CHECK OF CREDIBILITY
notice the presence (or absence) of a verification mark. Our
study found that only 61% of our participants correctly noticed
this cue when it was present, whereas 39% said a source was
verified when it was not—although for some participants,
source knowledge (especially for low ambiguity sources) may
have led to misreporting.{{
Results from this study suggest innovations to verification
appearance is needed. One option is to attempt to restore the
value of a verification badge. This could occur by being more
selective—something Twitter is currently doing by elimi-
nating bot-verified accounts and accounts that have broken
terms of service. But equally important is helping users no-
tice this distinction—for example, by making the verification
badge bigger or through public campaigns. However, there is
some indication that Twitter plans to provide verification
status to every user.4 In this situation, it might be useful to
have verification badges that have different colors—perhaps
signifying individuals verses companies, or different types of
Downloaded by East Carolina University from www.liebertpub.com at 03/12/19. For personal use only.
organizations (e.g., nonprofits, government, media, news).
By making these account cues more prominent and infor-
mative, Twitter can draw boundaries between sources and
better appeal to users who process online information heu-
ristically.6
There are several study limitations that should be con-
sidered. The use of an MTurk sample limits our ability to
generalize the findings to the population at large, although
emerging research suggests experiments on MTurk often
produce similar results to general population samples.11
Additionally, more research is needed to determine if our
results apply to other social media sites like Facebook.
Lastly, we focused on source-related factors and did not
manipulate tweet content. An overtly opinionated tweet or
one that appeals to certain news values may produce an in-
fluence of verification on credibility. Ultimately, there is still
much work to be done in this area. While people undoubtedly
rely on heuristics in making credibility judgments online, it
appears that the verification mark on Twitter is currently not
one of those cues.
Acknowledgment
The authors thank the Medill School of Journalism at
Northwestern University for providing funding for this study.
Author Disclosure Statement
No competing financial interests exist.
{{
People were significantly more likely to report low ambiguity
accounts had a verified account as compared to high ambiguity
accounts (B = 0.74, SE = 0.26, p = 0.004, odds ratio = 2.10). Among
the medium and high ambiguity accounts (which on average 10%
had heard of previously), roughly 37% said the unverified account
was verified, compared to 55% when the account was verified.
5
References
1. Shaerer E, Gottfried J. News use across social media
platforms 2017. Pew Research Center: Journalism &
Media. September 7, 2017. www.journalism.org/2017/09/
07/news-use-across-social-media-platforms-2017 (accessed
February 4, 2019).
2. Marwick AW, boyd d. I tweet honestly, I tweet passion-
ately: twitter users, context collapse, and the imagined
audience. New Media & Society 2010; 13:114–133.
3. Twitter. About Twitter verified accounts. https://help.twitter.
com/en/managing-your-account/about-twitter-verified-accounts
(accessed February 4, 2019).
4. Spangler T. Twitter CEO wants to open up verified accounts
to everyone. Variety. March 9, 2018. https://variety.com/
2018/digital/news/twitter-verified-account-open-everyone-
1202722587 (accessed February 4, 2019).
5. Meyer P. Defining and measuring credibility of newspa-
pers: developing an index. Journalism & Mass Commu-
nication Quarterly 1988; 65:567–574.
6. Metzger MJ, Flanagin AJ, Medders RB. Social and heu-
ristic approaches to credibility evaluation online. Journal of
Communication 2010; 60:413–439.
7. Flanagin AJ, Metzger MJ. The role of site features, user
attributes, and information verification behaviors on the
perceived credibility of web-based information. New Media
& Society 2007; 9:319–342.
8. Sundar SS, Knobloch-Westerwick S, Hastall MR. News
cues: information scent and cognitive heuristics. Journal of
the American Society for Information Science and Tech-
nology 2007; 58:366–378.
9. Chaiken S. (1987) The heuristic model of persuasion. In
Zanna MP, Olson JM, Herman CP, eds. Social Influence:
The Ontario Symposium 5. Hillsdale, NJ: Erlbaum, pp. 3–39.
10. Kamps HJ. Who are Twitter’s verified users? Medium. May
25, 2015. https://medium.com/@Haje/who-are-twitter-s-
verified-users-af976fc1b032 (accessed February 4, 2019).
11. Coppock A, Lepper TJ, Mullinix KJ. The generaliability of
heterogenous treatment effect esitmates across samples.
PNAS 2018; 115:12441–12446.
Address correspondence to:
Dr. Stephanie Edgerly
Medill School of Journalism
Northwestern University
1870 Campus Drive
Evanston, IL 60208
E-mail: stephanie.edgerly@northwestern.edu