See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/262026246

The Principal-Agent Problem in Peer Review

Article  in  Journal of the Association for Information Science and Technology · June 2014
DOI: 10.1002/asi.23169

CITATIONS READS

15 1,763

3 authors:

Jose A. Garcia Rosa Rodriguez-Sánchez

University of Granada University of Granada
145 PUBLICATIONS   849 CITATIONS    118 PUBLICATIONS   576 CITATIONS   

SEE PROFILE SEE PROFILE

J. Fdez-Valdivia
University of Granada
138 PUBLICATIONS   742 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

The problem of authors and reviewers who suffer from confirmatory bias View project

A strategy for quality assurance in peer review View project

All content following this page was uploaded by Jose A. Garcia on 24 January 2019.

The user has requested enhancement of the downloaded file.

The Principal-Agent Problem in Peer Review
Jose A. García, Rosa Rodriguez-Sánchez, and Joaquín Fdez-Valdivia
Departamento de Ciencias de la Computación e I. A., CITIC-UGR, Universidad de Granada, 18071 Granada,
Spain. E-mail: jags@decsai.ugr.es; {jfv, rosa}@decsai.ugr.es
In economics, the principal-agent problem is the diffi-
culty in motivating one party (the agent), to act in the
best interests of another (the principal) rather than in his
own interests. We consider the example of a journal
editor (the principal) wondering whether his or her
reviewer (the agent) is recommending rejection of a
manuscript because it does not have enough quality to
be published or because the reviewer dislikes effort and
he/she must work to acquire in-depth knowledge of the
content of the manuscript. The reviewer’s effort pro-
vides him or her with superior information about a
manuscript’s quality. If this information is not correctly
communicated, the reviewer has more information when
compared with the journal editor. This inherently leads
to an encouragement of moral hazard, where the editor
will not know whether the reviewer has done his or her
job in accordance to the editor’s interest. Prescriptions
need to be given as to how the journal editor should
control the reviewers to curb self-interest. Besides the
associate editors monitoring the peer-review process,
incentives can be employed to limit moral hazard on the
part of the reviewer. Drawing on agency theory, we
examine the incentives motivating the reviewers to
expend effort to generate information about the quality
of submissions. This model predicts that for reviewers
early in their careers, promotion-based incentives may
mean there is no need for within-job incentives, but also
that within-job rewards for a referee’s performance
should depend on individual differences in ability and
promotion opportunities.
Introduction
Refereeing is employed to maintain standards of quality,
provide credibility, and determine the suitability for publi-
cation of a manuscript (Campanario, 1998a, 1998b; Lee,
Sugimoto, Zhang, & Cronin, 2013). However, scholarly peer
review in its current form is relatively recent. For example,
the formal peer review system of Nature was instituted
Received September 16, 2013; revised October 30, 2013; accepted October
30, 2013
© 2014 ASIS&T • Published online 2 May 2014 in Wiley Online Library
(wileyonlinelibrary.com). DOI: 10.1002/asi.23169
JOURNAL OF THE ASSOCIATION FOR INFORMATION SCIENCE AND TECHNOLOGY, 66(2):297–308, 2015
in 1967. Einstein’s revolutionary papers published by
Annalen der Physik in 1905 were not peer reviewed by
anyone other than Planck (the journal editor) and Wien (the
co-editor).
Currently, even though the decision whether to publish a
scholarly article lies with the journal editor, he/she usually
refers to the opinion of a panel of experts in making the
decision. This is because the editor cannot devote sufficient
time to each of the articles submitted, and he/she cannot be
expected to be sufficiently expert in all areas covered by a
single journal. A reviewer’s report usually includes a recom-
mendation of what to do with the submitted manuscript
(e.g., acceptance, minor or major revision, or rejection). The
role of the reviewers is advisory, and the journal editor is
typically under no formal obligation to accept their recom-
mendations. To encourage innovation, Armstrong (1997)
proposed that referees do not make a recommendation about
acceptance or rejection, but only comment on papers and
offer suggestions about how they might be improved,
leaving final decisions to editors.
Some sociologists of science argue that peer review
makes the ability to publish susceptible to control by knowl-
edge communities that regulate themselves and to personal
jealousy (Eriksson, 2004; Higgs, 2007). This raises the pos-
sibility that reviewers may serve as gatekeepers (Bradley,
1981). Peer review is often censured for its conservativism,
that is, bias against groundbreaking and innovative research
(Braben, 2004; Chubin & Hackett, 1990; Wesseley, 1998).
Conservativism in peer review may suppress dissent against
“mainstream” theories (Anonymous, 2003; Campanario &
Martin, 2004; Martin, 1997).
In the psychological literature, confirmation bias is the
tendency to gather, interpret, and remember evidence in
ways that affirm rather than challenge one’s already held
beliefs (Lee et al., 2013; Nickerson, 1998). In the context of
peer review, confirmation bias is understood as reviewer bias
against conclusions that contradict the theoretical perspec-
tive of the reviewer (Jelicic & Merckelbach, 2002; Mahoney,
1977). On the contrary, reviewers evaluate more favorably
the submissions of authors who belong to similar schools of
thought (Travis & Collins, 1991). At the same time, senior
scientists are more likely than less established scientists to
be sought out as reviewers, particularly by journals of high
prestige. In this case, manuscripts that align with the view of
established scientists are more likely to be published and to
appear in top journals than are revolutionary manuscripts,
which is in accord with observations about the structure of
scientific revolutions by Kuhn (2012). A comprehensive
description of the major findings in peer-review studies can
be found in Burnham, (1990), Chubin and Hackett, (1990),
and Lee et al. (2013).
Here we describe a conceptual study exploring scientific
journal peer review through the analytical lens of agency
theory (Eisenhardt, 1989). In this theory, the principal-
agent problem represents the difficulty in motivating one
party (the agent), to act in the best interests of another (the
principal) rather than in his or her own interests. One
common example of the principal-agent problem is the
relationship between poorly motivated middle managers
in large, complex firms (agents) and shareholders (princi-
pals). For another example, consider a journal editor (the
principal) wondering whether his or her reviewer (the
agent) is recommending rejection of a submitted manu-
script because the paper does not have enough quality to be
published in the journal or because the reviewer disagrees
with the authors, for example, not liking their choice of
theory or their conclusions. Another source of bias in peer
review is when reviewers are in competition with authors,
with rejection being a convenient way to harm competitors
(Anonymous, 2003; Campanario & Martin, 2004; Horton,
2000).
The principal-agent problem may arise if agent and prin-
cipal have different interests and the agent has more infor-
mation than the principal. In this case, the principal cannot
ensure that the agent is acting in the principal’s best interest,
(Bebchuk & Fried, 2004). For example, it may happen if the
agent’s actions are costly for the principal to observe and if
the agent’s efforts that are useful to the principal are costly
to the agent. Then, moral hazard and conflicts of interest
may arise.
The solution to the agent-principal problem is to ensure
the provision of appropriate incentives so agents act in the
way principals wish. Various mechanisms may be used to
align the interests of the agent with those of the principal, for
example, the principal may use piece rates/commissions,
profit sharing, efficiency wages, performance measurement,
the agent posting a bond, the threat of dismissal, promotion-
based rewards, and nonmonetary incentives, among others
(Sappington, 1991). In terms of game theory, it involves
changing the rules of the game so that the self-interested
rational choices of the agent coincide with what the princi-
pal desires.
In this paper, we analyze the editor-reviewer relationship
from the viewpoint of agency theory where reviewers are
seen as agents of the journal editor (the principal). The
agency problem in peer review may arise because the
reviewer dislikes effort or because reviewers who are not
298 JOURNAL OF THE ASSOCIATION FOR INFORMATION SCIENCE AND TECHNOLOGY—February 2015
DOI: 10.1002/asi
knowledgeable (but seem to be) are unnecessarily harsh on
authors (thereby alienating potential contributors) or too
slow or unreliable (Lee et al., 2013; Martin, 1997).
The reviewer must work to acquire knowledge about the
manuscript to be reviewed. His or her effort provides supe-
rior information about the quality of the submission, which
produces asymmetric information between editor and
reviewers if this superior information is not correctly com-
municated. If it takes a few iterations of review invitations
to find reviewers and then one of the reviews is delayed,
this can add up to substantial delays in getting back to the
author, even for journals that strive for rapid turn arounds
(Hickok & Poeppel, 2011). This encourages moral hazard
(Perrow, 1986), where the journal editor will not know
whether the reviewer has done his or her job in accordance
with the editor’s interest. Moral hazard is central to agency
theory (Hendrikse, 2003), and here refers to the actual
effort taken by the reviewer that the journal editor cannot
observe directly. The problem of moral hazard in peer
review leads to costs for the journals associated with
administering the implicit contract between editors and
reviewers. The level of these costs depends on the ability
of the journal editor to find an appropriate solution to
reducing information asymmetries.
The mechanisms for curbing moral hazard in peer review
can be monitoring or incentive contracts (Daily, Dalton, &
Cannella, 2003; Gomez-Mejia, Wiseman, & Dykes, 2005),
where the associate editors (who control the review of sub-
mitted manuscripts) often comprise the main monitoring
mechanism in peer review. The associate editors should act
on behalf of the journal editor and hold foremost responsi-
bility for the functioning of the peer-review process, with the
goal of reducing information asymmetries by ratifying and
monitoring reviewers’ decisions (Heath, 2009; Shapiro,
2005). The more effective the associate editor is in obtaining
information about reviewer behavior, the more likely the
reviewer will be to act in the interest of the journal editor,
and therefore fewer resources need be spent on aligning the
interests of journal editor and reviewers through incentives
(Eisenhardt, 1989).
Besides the associate editors monitoring the peer-review
process, incentives can be similarly employed to limit moral
hazard on the part of the reviewer. Changes in behavior are
driven by changes in incentives (Prendergast, 1999), and a
reviewer’s behavior is directed by maximizing self-interest
under game-theoretical-like conditions (Perrow, 1986). This
article examines the incentives motivating risk-averse
reviewers to expend effort to generate information about the
quality of submitted manuscripts and to make the best deci-
sion during peer review. They align the journal editor and
reviewers’ objectives. Of course, financial incentives for
reviewers can be counterproductive. As psychologists have
shown, when someone is motivated by intrinsic factors, such
as the satisfaction from doing a task, then offering money
or other rewards for doing the job can redirect attention
to rewards and reduce commitment and performance
(Prendergast, 1999). Given that most reviewers are not paid,
it is questionable whether introducing direct payment is a
wise idea. However, we are referring to any kind of rewards,
including promotion-based or nonmonetary incentives.
These rewards can take many different forms, including
implicit promises of future promotion opportunities, praise
from peers, feelings of self-esteem that come from recogni-
tion, and future cash rewards related to performance. For
example, reviewers of Elsevier journals receive 30 days
access to Scopus (which is owned by Elsevier and is avail-
able online by subscription). Some journals have instituted a
reward system for reviewers, for example, if the review is on
time they get entered into a lottery to win a prize (Hickok &
Poeppel, 2011). In many cases, junior scientists end up
reviewing the submitted papers. Senior scientists in the field
tend to be more pressed for time and receive far more review
requests and, thus, the people who end up agreeing to review
(often via referrals from the first choice scientists) are more
junior and have more time and more to gain, for example, in
the form of potential career benefits (Hickok & Poeppel,
2011).
In the following, the Agency Problem section presents
the principal-agent problem in peer review. Drawing on
agency theory, the Model section introduces a model of
incentives for reviewers. Next, the Discussion section pro-
vides a discussion of this model. The main conclusions of
the work are summarized in the last section.
Agency Problem
Following agency theory, peer review can be modelled
like a complex process with the need for external capital (e.g.,
experts in a given field). This combined with an increased
number of submissions, a limit on managerial wealth, and a
need for efficient risk allocation in the selection of submitted
manuscripts (Demsetz, Saidenberg, & Strahan, 1997; Garcia,
Rodriguez-Sanchez, & Fdez-Valdivia, 2014), means an
increase in the diffused responsibility of peer review
decision-making among editors.
As journal editors have a willingness to bear risk in the
selection of submissions but do not necessarily possess the
time to actively review each submitted manuscript, a con-
tractual relationship may be created wherein an agent
(referee) will review the submission on behalf of the prin-
cipal (journal editor). The acceptance of a reviewer to
perform the review of a manuscript creates a contract
between the journal editor and himself. As such, the peer-
review process is reduced to a nexus of (more or less) infor-
mal contracts between journal editors and reviewers and the
separation of decision making and peer review is created.
One factor that can contribute to the appearance of prob-
lems in the selection of high-quality manuscripts is the dif-
ference in attitudes toward the risk of reviewers and journal
editors. For instance, reviewers may turn down submissions
that would be profitable from the point of view of journal
editors because the reviewers perceive the submitted manu-
scripts to be overly risky (e.g., very innovative, original, or
too technical) from the point of view of their own welfare.
JOURNAL OF THE ASSOCIATION FOR INFORMATION SCIENCE AND TECHNOLOGY—February 2015
Instead, journal editors tend to be risk-neutral for two
reasons: (a) The risk of accepting (to be published) a sub-
mission is small relative to the total number of papers pub-
lished by the journal, and (b) a journal may publish so many
risky manuscripts that they on average tend to cancel each
other out (i.e., diversified risk).
Given the separation of decision making and peer review
and the diverging risk profiles of journal editors and review-
ers (Jensen, 1989), it cannot be expected that risk-averse
reviewers will act in the interest of risk-neutral journal
editors as it may not be in the reviewer’s self-interest to
pursue journal wealth maximization (Lan & Heracleous,
2010).
The agency problem in separating decision making and
peer review is the diverging goals of the journal editor and
reviewers (see, e.g., Hendrikse, 2003). From Jensen and
Smith (1985), there are three problems with management (in
particular, regarding the selection of high quality manu-
scripts) that cause conflict of interest: (a) the choice of effort
for reviewers, (b) differential risk exposure of the cooperat-
ing parties (risk-neutral editors and risk-averse reviewers),
and (c) differential time horizon between journal editor and
reviewers. Recall that the acceptance of a reviewer to
perform the review of a manuscript creates a contract
between the journal editor and reviewer. But this contract
(possibly imperfect) in the agency relationship (between
journal editor and reviewers) makes the observation of true
effort (by reviewers) difficult (to the journal editor) and
as such causes the hidden action problem of asymmetric
information well known in agency theory (Arrow, 1968;
Prendergast, 1999). We model the reviewer as choosing
between “reject” and “further consideration” for a submitted
manuscript. The reviewer can select a particular alternative
on the basis of the information signal that his or her effort
generates. The reviewer must work to acquire superior
knowledge about the submitted manuscript. His or her effort
provides superior information about the quality of the sub-
mission he/she faces (in comparison to the level of informa-
tion of the journal editor). This produces asymmetric
information between journal editor and reviewers if the
superior information is not correctly communicated to the
editor. This leads to an encouragement of moral hazard
(Perrow, 1986), where the journal editor will not know
whether the reviewer has done his or her job in accordance
to the editor’s interest (see, e.g., Shapiro, 2005).
To sum up, the agency problem considered is that of
motivating the reviewer to expend effort to generate infor-
mation about the quality of the submitted manuscript. But
the journal editor and the reviewer will not always agree
regarding which submissions should be published by the
journal. What incentives should be provided to the reviewers
to do the right thing?
A Model of Incentives for Reviewers
Economic models of compensation generally assume that
higher performance requires greater effort or that it is in
299
DOI: 10.1002/asi
some other way associated with disutility on the part of
reviewers. To provide incentives, these models predict the
existence of reward systems that structure compensation
so that a reviewer’s expected utility increases with observed
productivity. These rewards can take many different
forms, including implicit promises of future promotion
opportunities, praise from peers, feelings of self-esteem that
come from recognition, and future cash rewards related to
performance.
Drawing on an economic model of agency theory (Gibbs,
1995), we examine the motivation of reviewers in peer
review. Reviewers generally work for research and academic
institutions in which advancement appears to be an impor-
tant part of the incentive system. In addition to promotion-
based incentives, we assume that reviewers can be eligible
for within-job merit raises and bonuses. Following Gibbs
(1995), “within-job” rewards refer to bonuses and merit
raises earned when there is no promotion. Instead,
“promotion-based” incentives are rewards due to a promo-
tion to a new position as well as the option value of potential
rewards from further advancement. The point is that an
optimal incentive system should explicitly account for moti-
vation from promotion and within-job incentives. For
example, those reviewers who are full professors at their
academic institutions are not well motivated by the potential
for promotion, so stronger bonus or merit pay schemes
should be used for such reviewers. However, the combina-
tion of promotion-based incentives and short-term bonuses
for performance can be considered as belonging to the same
FIG. 1. Reviewer’s performance as a function of effort (top) and ability
model of incentives as follows. (bottom).
Reviewers combine ability α (which may include accu-
mulations of special skill, knowledge, or judgement in a
particular area over time) and effort e to make a recommen-
dation (regarding the acceptance/revision/rejection of a where error ε is assumed to be distributed normally, that is,
given manuscript), which produce performance q as ε ∼ N(0, σ ε2 ) .
follows: Figure 2 shows the reviewer’s performance as measured
by the editor, with error ε distributed normally and such
q = μ ⋅α ⋅ e (1) that σ ε2 = 10 (see Figure 2[top]) or σ ε2 = 100 (see
Figure 2[bottom]). The variance σ ε2 is a measure of how far
with μ denoting the differences in marginal product associ-
a set of numbers is spread out. A variance of zero indicates
ated with expertise and effort across reviewers, associate
that all the values are identical. A small variance indicates
editors and other jobs in peer review; and where the perfor-
that the data points tend to be close to the expected value,
mance q represents the efficiency with which reviewer ful-
whereas a high variance indicates that the data points are
fills his or her intended purpose in a peer-review process. A
spread out from the expected value and from each other.
reviewer’s ability is assumed to be distributed normally
Therefore, Figure 2 illustrates the magnitude of the hidden
α ∼ N(α , σ α2 ) .
action problem of asymmetric information between journal
Figure 1 (top) shows the reviewer’s performance q as a
editor and reviewer if information is not correctly commu-
function of effort e, for three different levels of ability α
nicated to the editor.
(marginal product is μ = 1). As can be seen from Figure 1
The idea underlying this model of incentives is that the
(top), the greater the reviewer’s level of expertise, the
journal editor sorts experts into different positions in the
greater the return to effort e. Figure 1 (bottom) shows per-
journal hierarchy (e.g., reviewer/associate editor) based on
formance q as a function of ability α, for three different
learning over time about the experts’ skills. To this end, the
levels of effort e. The lesser the reviewer’s level of effort, the
journal editor makes use of promotions to put higher ability
lesser the return to ability.
experts into positions of greater authority in the journal
We also assume that the editor measures the reviewer’s
hierarchy. However, this process may also include demo-
performance with error ε following:
tions or dismissals as negative rewards. Sustained declining
μ ⋅α ⋅ e + ε (2) of overall journal publishing performance may force the

300 JOURNAL OF THE ASSOCIATION FOR INFORMATION SCIENCE AND TECHNOLOGY—February 2015
DOI: 10.1002/asi

FIG. 2. Reviewer’s performance as measured by editor, with error
distributed normally and variance 10 (top) and 100 (bottom).
unproductive associate editor to leave the journal organiza-
tion. To perform the experts’ sorting, we assume that the
journal editor sets a performance standard, z, and promotes
any and all experts whose performance measure exceeds z.
For instance, z might be a minimum number of high-quality
submissions that were reviewed by the referee. The editor or
a board of editors would identify those accepted submis-
sions of high quality. Hence, it follows that the probability of
winning promotion, p(e), given an effort level e will be:
p(e) = Prob(winning promotion ) = Prob(μ ⋅ α ⋅ e + ε > z )
= 1 − F (z) (3)
with F being the cumulative distribution of measured per-
formance; where F depends on the distributions of ability α
as well as error ε of performance measurement. Hereafter,
the marginal product μ is normalized to 1, assuming that
differences in marginal product across reviewers, associate
editors and other jobs in peer review are not so important.
Promotion (e.g., to associate editor in a prestigious aca-
demic journal) signals to the research and academic institu-
tions in the expert’s field that he/she is more able. This raises
the reviewer’s value to those institutions. Thus, the present
value of this increase in career earnings and potential future
promotions is the reward of advancement, which is to be
denoted by Δ.
We are assuming that the journal also makes use of
within-job rewards to elicit optimal reviewers’ efforts. The
JOURNAL OF THE ASSOCIATION FOR INFORMATION SCIENCE AND TECHNOLOGY—February 2015
FIG. 3. Within-job incentive rewards as a function of the piece rate b.
within-job incentive reward is thus given by (recall that the
marginal product μ was normalized to 1):
b0 + b(α ⋅ e + ε ) (4)
where b is a linear piece rate based on the performance
measure α · e + ε. The piece rate b can take many different
forms such as, for example, nonmonetary rewards, including
praise from peers, professional reputation, and feelings of
self-esteem that come from recognition, among others.
Noise ε in the performance measure α · e + ε makes the
distinction between skill α and effort e difficult. Figure 3
illustrates the within-job incentive rewards as a function of
the piece rate b, for ε distributed normally and such that
σ ε2 = 10 and σ ε2 = 100. Besides, promotion-based rewards
have incentive effects for reviewers because reviewers’ extra
efforts will increase their probability of being promoted as
can be seen from Equation 3. Obviously, here we are inter-
ested in an optimal scheme of reviewers’ incentives which is
able to balance within-job and promotion-based rewards.
Within a journal there is a fixed (small) number of slots
and as a consequence it has little control over promotion
rates, so journals may find it costly to reorganize their hier-
archical structure for peer review to optimize promotion-
based rewards. To simplify, in our model of incentives, we
note that the hierarchical structure within a journal is set to
optimize the distribution of ability and other production
factors. It determines an equilibrium reward distribution
across journal positions driven by external market pressures.
The journal then uses within-job rewards to fine-tune a
reviewer’s performance in peer review, conditional on
promotion-based rewards.
Discussion
The Optimization of Reviewer’s Expected Utility
We postulate that a reviewer, just as any other worker, is
trying to maximize expected income less the cost of effort
301
DOI: 10.1002/asi

FIG. 4. Expected utility, EU (e), as a function of effort e for three different
reviewers with expected ability 0.8 (top), 0.1 (middle), and 0.3 (bottom).
(Gibbs, 1995). As noted previously, performance rewards
can take many different forms, including promotion-based
and nonmonetary incentives.
Let EU(e) be the expected utility, which is defined as
expected income less the cost of effort:
EU (e) = b0 + b ⋅ α ⋅ e + p(e)Δ − C (e)
(5)
where p(e) denotes the probability of winning promotion; Δ
is the present value of the increase in career earnings and
potential future promotions (i.e., the reward on advance-
ment); and C(e) denotes the cost of effort. Figure 4 illus-
trates the expected utility EU(e) as a function of effort e for
three different reviewers with expected ability ᾱ = 0.8 (top),
ᾱ = 0.1 (middle), and ᾱ = 0.3 (bottom). This figure shows
that a more able reviewer (top) has higher expected utility
across effort, even with lower promotion-based incentives
(Δ = 0.1). To achieve high levels of expected utility, a less
302 JOURNAL OF THE ASSOCIATION FOR INFORMATION SCIENCE AND TECHNOLOGY—February 2015
DOI: 10.1002/asi
able reviewer (middle/bottom) should apply more effort
(e = 1).
Therefore, following the basic assumption that a
reviewer is trying to maximize expected income less the
cost of effort, we have that he/she maximizes expected
utility EU(e).
Optima of unconstrained problems are found at station-
ary points, where the first derivative of the objective function
is zero. More generally, they may be found at critical points,
where the first derivative of the objective function is zero or
is undefined. An equation stating that the first derivative
equals zero at an interior optimum is called a “first-order
condition.” For our problem, the first-order condition is as
follows:
C ′(e*) = b ⋅ α + p ′(e*)Δ. (6)
Although the first derivative test identifies points that may
be extrema, it does not distinguish a point that is a
minimum from one that is a maximum or one that is
neither. These cases can be distinguished by checking
the second derivative. The conditions that distinguish
maxima from other stationary points are called “second-
order conditions.” For our problem, the second-order
condition is:
−C ′′ + p ′′ ⋅ Δ ≤ 0. (7)
If a candidate solution e satisfies the first-order condition,
then satisfaction of the second-order condition as well is
sufficient to establish at least local optimality.
From the second-order condition it follows that
C ′′
p ′′ ≤
Δ
that is, the second-order condition of the maximiza-
tion problem requires that the probability of winning pro-
motion p(e) does not increase too quickly when e is
increased.
If first- and second-order conditions are not met, the
reviewer may fail to do what is required and he/she will exert
zero effort to review the submitted manuscript.
The first condition, as given by Equation 6, shows that
total incentives
b ⋅ α + p ′(e*)Δ
C′
are provided by the combination of within-job incentives for
a reviewer’s performance (given by the piece rate b times
the marginal product of effort, which includes an ability
effect):
b ⋅α
and promotion-based rewards for a referee’s performance
(as given by the prize of promotion times the marginal effect
of effort on the chance of winning advancement):
FIG. 5. Optimal within-job piece rate b as a function of reward on advancement, for two different reviewers with expected ability 0.9 (top) and 0.2 (bottom).

p ′(e*)Δ. Given that a successful journal is interested in the optimal

within-job piece rate b*, the journal editor should choose a
manuscript (piece) rate b such that it maximizes a reviewer’s
The Journal’s Optimization expected utility

Hastening the publication of significant findings leads to α ⋅ e* − C (e*)

competition between journals for the best articles. It may
drive a journal’s expected profits to zero, and then, the value subject to the first-order necessary condition for the review-
of reviewer’s performance er’s optimization problem, given in Equation 6

α ⋅ e* C ′(e*) = b ⋅ α + p ′(e*)Δ

should equal his/her expected reward It yields the following equation, where we note that:
b0 + b ⋅ α ⋅ e* + pΔ p′ ⋅ Δ
b* = 1 − (8)
α
so that
where Δ is the present value of the increase in career earn-
α ⋅ e* = b0 + b ⋅ α ⋅ e* + pΔ ings and potential future promotions (the reward on
advancement); p(e) denotes the probability of winning pro-
In this case, following Gibbs (1995), we note that a review-
motion; and ᾱ is the expected reviewer’s ability.
er’s expected utility EU(e) reduces to
Figure 5 illustrates the optimal within-job piece rate b* as
α ⋅ e* − C (e*) a function of reward on advancement Δ, for two different

JOURNAL OF THE ASSOCIATION FOR INFORMATION SCIENCE AND TECHNOLOGY—February 2015 303
DOI: 10.1002/asi
FIG. 6. Optimal within-job piece rate b as a function of expected ability for two different reviewers with reward on advancement 0.2 (top) and 0.7 (bottom).
reviewers with expected ability ᾱ = 0.9 (top) and ᾱ = 0.2
(bottom). In both cases (top/bottom), the optimal piece rate
b* is a decreasing function of Δ. This figure also shows that
a more able reviewer (top) has higher piece rate b* across Δ
values. The optimal piece rate decreases more quickly for a
less able reviewer (bottom) with ᾱ = 0.2.
Figure 6 illustrates the optimal within-job piece rate b* as
a function of expected ability ᾱ for two different reviewers
with reward on advancement Δ = 0.2 (top), and = 0.7
(bottom). In both cases (top/bottom), the optimal piece rate
b* is an increasing function of expected ability ᾱ. This
figure also shows lower values of optimal piece rate b* for a
reviewer with a higher reward on advancement Δ = 0.7
(bottom).
From Equation 8 it follows that the manuscript (piece)
rate b adjusts for changes in promotion-based rewards (as
given by p′ · Δ) in such a way that the reviewer’s total
incentives are set in the right manner. In particular, we have
that b* < 1, but more interesting, it follows that the optimal
piece rate b* may be small, if promotion-based rewards are
large. In such a case, the promotion system generates
304 JOURNAL OF THE ASSOCIATION FOR INFORMATION SCIENCE AND TECHNOLOGY—February 2015
DOI: 10.1002/asi
enough motivation for the reviewers given the effort cost,
and the journal would use only a small piece rate b* to
produce total incentives. But obviously there is the possibil-
ity that b* < 0 should be ignored in the empirical work.
The interesting point about b* is that for reviewers early
in their careers, promotion-based incentives may mean there
is no need for within-job rewards. In fact, from Equation 8,
we note that in the presence of promotion-based motivation,
the piece rate b* will be strictly smaller than it would be
otherwise. If promotion-based incentives are large, then this
may explain why within-job incentives for a reviewer’s per-
formance are in general of minor significance in scholarly
journals. Hence, this model predicts that for young review-
ers, promotion-based incentives may mean there is no need
for within-job incentives.
From Equation 8, another prediction about promotion-
based and within-job incentives is noted: This equation
regarding optimal piece rate suggests that the journal will
switch between the two forms of rewards (promotion based
and within job) based on observable individual differences
in expected ability. To further analyze this point, in the
 FIG. 7. Probability of winning promotion, p(e), as function of effort e (top); the corresponding first derivative (bottom).
following we consider the effect of changing expected
ability ᾱ on piece rate b*. To this end, the optimal piece rate
b* in Equation 8 may be differentiated with respect to an
expected reviewer’s ability ᾱ (Gibbs, 1995):
db* p ′ ⋅ Δ ⎛ p ′ δΔ Δ δ p ′ ⎞
= 2 −⎜ ⋅ + ⋅
dα α ⎝ α δα α δα ⎟⎠
This equation shows that there are two general effects of
changing expected ability ᾱ on piece rate b*. First, there is
a productivity effect, the first term, which is positive:
p′ ⋅ Δ
.
α2
From this first term, we note that it is optimal to give more
able referees greater rewards when effort and ability inter-
act in the review. There is also a substitution effect reveal-
ing that expected ability also affects promotion-based
rewards in two ways, the two components of the last nega-
tive term:
JOURNAL OF THE ASSOCIATION FOR INFORMATION SCIENCE AND TECHNOLOGY—February 2015
⎛ p ′ δΔ Δ δ p ′ ⎞
−⎜ ⋅ + ⋅
⎝ α δα α δα ⎟⎠
.
First, it affects the reward, as higher ability reviewers will
earn more over their careers, therefore the second term in
(9)
Equation 9 will be negative. Second, as in the third term in
Equation 9, it changes the marginal effect of effort on the
opportunity of winning promotion.
How does a shift in expected ability ᾱ change the effect
of extra reviewer effort on the probability of winning pro-
motion p(e)? Because ability and effort are complements,
we may expect that increasing expected ability ᾱ raises the
returns to additional reviewer effort in trying to win promo-
tion. However, it is also true that if expected ability ᾱ (and
thus the probability of winning promotion) is high, extra
effort makes little difference because the promotion is
unlikely to be lost. Increasing ability could then lead to a
smaller marginal effect of reviewer effort. It follows that
those reviewers with lower expected ability ᾱ should gener-
ally be given larger within-job rewards for their performance
305
DOI: 10.1002/asi
 FIG. 8. Optimal piece rate b as a function of effort e for a reviewer with high expected ability (0.9).

as a result of lower promotion-based incentives. At the same
time, though, the marginal productivity effect works in the
opposite direction, reducing within-job rewards for a
reviewer’s performance for those referees with lower exper-
tise. As a consequence of these two effects, we note that
within-job reward for a referee’s performance should
depend on individual differences in ability and promotion
opportunities. These are illustrated in Figures 7, 8, and 9.
Figure 7 shows an example of probability of winning pro-
motion p(e) as function of effort e (top) as well as the
corresponding first derivative p′(e) (bottom). It predicts that,
by increasing effort, the probability of winning promotion
also increases.
Figure 8 shows that, for a talented reviewer with high
expected ability ᾱ = 0.9, the optimal piece rate b* (that
controls within-job incentives) is decreasing with effort e
p′ ⋅ Δ
since increases with effort level e. Recall that from
α slow or unreliable) and he/she must work to acquire the
Figure 7 it follows that, by increasing effort, the probability
of winning promotion also increases. As can be seen from
Figure 9, this also happens for a reviewer having lower
expected ability ᾱ = 0.1.
The comparison of these two figures (Figures 8 and 9)
shows a significant difference between a reviewer having
ᾱ = 0.9 and another with ᾱ = 0.1: For a reviewer having
higher reward on advancement (Δ = 0.9) and lower expected
ability (ᾱ = 0.1; see Figure 9[bottom]), it follows that the
optimal piece rate b* (that controls within-job rewards) will
be almost zero or even negative for any level of effort.
Conclusions
We analyzed the editor-reviewer relationship from the
viewpoint of agency theory, where reviewers are seen as
agents of the journal editor (the principal). The principal-
agent problem in peer review may arise because the
reviewer dislikes effort (or because reviewers who are not
knowledgeable are unnecessarily harsh on authors or too
superior knowledge about the manuscript to be reviewed.
It produces asymmetric information between editor and
reviewer if this superior information is not correctly
communicated.

306 JOURNAL OF THE ASSOCIATION FOR INFORMATION SCIENCE AND TECHNOLOGY—February 2015
DOI: 10.1002/asi
 FIG. 9. Optimal piece rate b as a function of effort e for a reviewer with low expected ability (0.1).
The mechanisms for curbing moral hazard in peer review
can be monitoring and incentive contracts. In this paper,
changes in behavior were driven by changes in incentives,
and a reviewer’s behavior was directed by maximizing self-
interest under game-theoretical-like conditions. These
incentives can take different forms, including implicit prom-
ises of future promotion opportunities, praise from peers,
feelings of self-esteem that come from recognition, and
future cash rewards related to performance.
In this case, we demonstrated that junior reviewers early
in their career are already satisfied with promotion-based
incentives without further consideration of within-job
rewards. If promotion-based incentives are large, then this
may explain why within-job incentives for reviewer’s per-
formance are in general of minor significance in scholarly
journals. In particular, for a reviewer having high reward on
advancement and low expected ability, it was shown that the
optimal piece rate (that controls within-job rewards) was
almost zero or even negative for any level of effort.
We have also shown that, in general, within-job incen-
tives decrease with effort, because the probability of
JOURNAL OF THE ASSOCIATION FOR INFORMATION SCIENCE AND TECHNOLOGY—February 2015
winning promotion also increases with effort. Because
ability and effort are complements, we may expect that
increasing expected ability raises the returns to additional
reviewer effort in trying to win promotion.
However, it is also true that if expected ability is high,
extra effort makes little difference because the promotion is
unlikely to be lost. In this case, increasing ability could then
lead to a smaller marginal effect of reviewer effort.
We have shown that those reviewers with lower expected
ability should generally be given larger within-job rewards.
At the same time, though, the marginal productivity effect
works in the opposite direction, reducing within-job rewards
for reviewer performance for those referees with lower exper-
tise. In conclusion, we have shown that within-job reward for
a referee’s performance should depend on individual differ-
ences in ability and promotion opportunities. It is implied that
the editor should have computerized personnel records for
reviewers of a scholarly journal over a significant time period.
Data should include age, education, institution of employ-
ment, rank, title or position, country of employment, perfor-
mance rating, salary, and bonus. The journal editor needs to
307
DOI: 10.1002/asi
 have enough data to make reliable estimates of promotion
rates and long-term rewards from promotion. Another impor-
tant variable to measure is expected ability of reviewers.
Future work includes empirical data about the effects that this
analysis has on peer-review processes.
Acknowledgments
This research was sponsored by the Spanish Board for
Science and Technology (MICINN) under grant TIN2010-
15157 cofinanced with European FEDER funds. Sincere
thanks are due to the reviewers for their insightful comments
and constructive suggestions. We would also like to thank to
Juan Miguel Campanario for his kindness and comments on
our manuscript.
References
Anonymous. (2003). Coping with peer rejection. Nature, 425(6959), 645.
Armstrong, J.S. (1997). Peer review for journals: Evidence on quality con-
trol, fairness, and innovation. Science and Engineering Ethics, 3, 63–84.
Arrow, K. (1968). The economics of moral Hazard: Further comment.
American Economic Review, 58(3), 537–539.
Bebchuk, L., & Fried, J. (2004). Pay without performance. Cambridge,
MA: Harvard University Press.
Braben, D.W. (2004). Pioneering research: A risk worth taking. Hoboken,
NJ: Wiley-Interscience.
Bradley, J.V. (1981). Pernicious publication practices. Bulletin of the Psy-
chonomic Society, 18, 31–34.
Burnham, J.C. (1990). The evolution of editorial peer review. JAMA: The
Journal of the American Medical Association, 263(10), 1323–1329.
Campanario, J.M. (1998a). Peer review for journals as it stands today Part
1. Science Communication, 19(3), 181–211.
Campanario, J.M. (1998b). Peer review for journals as it stands today Part
2. Science Communication, 19(4), 277–306.
Campanario, J.M., & Martin, B. (2004). Challenging dominant physics
paradigms. Journal of Scientific Exploration, 18(3), 421–438.
Chubin, D.E., & Hackett, E.J. (1990). Peerless science: Peer review and U.S.
science policy. Stony Brook, NY: State University of New York Press.
Daily, C., Dalton, D., & Cannella, A., Jr. (2003). Corporate governance:
Decades of dialogue & data. Academy of Management Review, 28(3),
371–382.
Demsetz, R., Saidenberg, M., & Strahan, P. (1997). Agency Problems and
Risk Taking at Banks. FRB of New York Staff Report, 29.
Eisenhardt, K. (1989). Agency theory: An assessment and review.
Academy of Management Review, 14(1), 57–74.
Eriksson, L. (2004). Love the maverick. New Scientist, 183(2461), 4.
Garcia, J.A., Rodriguez-Sanchez, R., & Fdez-Valdivia, J. (2014). The selec-
tion of high quality manuscripts. Scientometrics, 98(1), 299–313. doi:
10.1007/s11192-013-1034-4
308 JOURNAL OF THE ASSOCIATION FOR INFORMATION SCIENCE AND TECHNOLOGY—February 2015
DOI: 10.1002/asi
View publication stats
Gibbs, M.J. (1995). Incentive compensation in a corporate hierarchy.
Journal of Accounting and Economics, 19(2–3), 247–277.
Gomez-Mejia, L., Wiseman, R., & Dykes, B. (2005). Agency problems in
diverse contexts: A global perspective. Journal of Management Studies,
42(7), 1507–1517.
Heath, J. (2009). The uses and abuses of agency theory. Business Ethics
Quarterly, 19(4), 497–528.
Hendrikse, G. (2003). Economics and management of organizations. New
York: McGraw-Hill Education.
Hickok, G., & Poeppel, D., (2011). Should peer reviewers be paid for their
work? Talking Brains: News and views on the neural organization of
language. Retrieved from http://www.talkingbrains.org/
Higgs, R. (2007). Peer review, publication in top journals, scientific con-
sensus, and so forth. Independent Institute. Retrieved from http://
www.independent.org/
Horton, R. (2000). Genetically modified food: Consternation, confusion,
and crack-up. The Medical Journal of Australia, 172(4), 148–149.
Jelicic, M., & Merckelbach, H. (2002). Peer-review: Let’s imitate the
lawyers! Cortex; a Journal Devoted to the Study of the Nervous System
and Behavior, 38, 406–407.
Jensen, M. (1989). Eclipse of the public corporation. Harvard Business
Review, 67(5), 61–74.
Jensen, M., & Smith, C. (1985). Stockholder, manager, and creditor inter-
ests: Applications of agency theory. In E. Altman & M. Subrahmanyam
(Eds.), Recent advances in corporate finance (pp. 93–131). Burr Ridge,
IL: Irwin Professional Publishing.
Kuhn, T.S. (2012). The structure of scientific revolutions 50th anniversary
edition (p. 264). Chicago, IL: The University of Chicago Press.
Lan, L., & Heracleous, L. (2010). Rethinking agency theory: The view from
law. Academy of Management Review, 35(2), 294–314.
Lee, C.J., Sugimoto, C.R., Zhang, G., & Cronin, B. (2013). Bias in peer
review. Journal of the American Society for Information Science and
Technology., 64(1), 2–17.
Mahoney, M.J. (1977). Publication prejudices: An experimental study of
confirmatory bias in the peer review system. Cognitive Therapy and
Research, 1(2), 161175.
Martin, B. (1997). Suppression Stories. Fund for Intellectual Dissent,
Wollengong. Australia.
Nickerson, R.S. (1998). Confirmation bias: A ubiquitous phenomenon in
many guises. Review of General Psychology, 2(2), 175–220.
Perrow, C. (1986). Economic theories of organization. Theory and Society,
15(1/2), 11–45.
Prendergast, C. (1999). The provision of incentives in firms. Journal of
Economic Literature, 37(1), 7–63.
Sappington, D.E.M. (1991). Incentives in principal-agent relationships.
Journal of Economic Perspectives, 5(2), 45–66.
Shapiro, S. (2005). Agency theory. Annual Review of Sociology, 31(1),
263–284.
Travis, G.D.L., & Collins, H.M. (1991). New light on old boys: Cognitive
and institutional particularism in the peer review system. Science, Tech-
nology, & Human Values, 16(3), 322–341.
Wesseley, S. (1998). Peer review of grant applications: What do we know?
Lancet, 352(9124), 301–305.