Journal of Theoretical Biology 486 (2020) 110103

Contents lists available at ScienceDirect

Journal of Theoretical Biology

journal homepage: www.elsevier.com/locate/jtb

Probabilistic punishment and reward under rule of trust-based

decision-making in continuous public goods game
Yuhang Jiao, Tong Chen, Qiao Chen∗
College of Management and Economics, Tianjin University, Tianjin 300072, PR China

a r t i c l e i n f o a b s t r a c t

Article history: Altruistic punishment and reward have been proved to promote the evolution of cooperation in the public
Received 31 July 2019 goods game(PGG), but the punishers and the rewarders have to pay a price for these behaviors and that
Revised 1 November 2019
results in overall loss of interest. In present work, probabilistic punishment and reward are introduced to
Accepted 2 December 2019
PGG. Probabilistic punishment and reward mean that punishment and reward are executed with a cer-
Available online 3 December 2019
tain probability. Although that will reduce unnecessary costs, occasional absence of execution can lead to
Keywords: distrust. Thus we focus on how to implement punishment and reward efficiently within the structured
Probabilistic punishment and reward populations. Numerical simulations are performed and prove that probabilistic punishment and reward
Trust can promote the evolution of cooperation more effectively. Further researches indicate that there is an
Executing probability optimal executing probability to promote cooperation and maximize reduction of cost. In addition, when
Cooperation the unit cost is high, the PGG with probabilistic punishment and reward still helps the evolution of al-
Public goods game
truistic punishers and rewarders, thereby avoiding collapse of cooperation.
© 2019 Elsevier Ltd. All rights reserved.

1. Introduction various selective incentives, such as reward (Sasaki et al., 2015;

In both nature and human society, cooperative behaviors are
common. Human are inseparable from each other in activities of
production and living. However, not all individuals will take coop-
erative action in a group (Hardin, 1968; West et al., 2006). How
to understand the emergence and maintenance of cooperative be-
havior among selfish individuals has attracted extensive attention
(Nowak and Sigmund, 2004; Sasaki and Uchida, 2013). The evolu-
tionary game theory explores how individuals with bounded ratio-
nality can maximize returns over time in the process of repeated
game (Hauert et al., 2019; McAvoy and Hauert, 2015). At present,
evolutionary game theory is considered to be one of the most
powerful means to study cooperative behavior (Perc and Szol-
noki, 2010; Wang et al., 2015; Gintis, 20 0 0a). Prisoner’s dilemma
game (PDG) (Weibull, 1997; Szolnoki et al., 2008; Wang and Lv,
2019b)and public goods game (PGG) (Hardin, 1968; Wang and Lv,
2019a; dos Santos, 2015) are the most common models for study-
ing cooperative behaviors.
Some experiments have also shown that because of the friend-
liness, revenge, and transparency of Tit-for-Tat, cooperation can
evolve in selfish individuals (Newth, 2009; Kümmerli et al., 2007;
Nowak and Sigmund, 1992; Milinski, 1987). It is well known that
∗
Corresponding author.
E-mail address: chenqiao@tju.edu.cn (Q. Chen).
Szolnoki and Perc, 2010; Sasaki and Uchida, 2014; Dos Santos and
Peña, 2017), punishment (Fowler, 2005; dos Santos and Wedekind,
2015; Brandt et al., 2003; Bettina and Manfred, 2006), or volun-
teer (Semmann et al., 2003; Hauert et al., 2002), have been used
to modify structures of payoff and suppress human selfish behav-
ior. In addition, diversity (Santos et al., 2008; Maciejewski et al.,
2014), information (Liu et al., 2018; Chen et al., 2018; Uchida and
Sasaki, 2013) and emotions (Chen et al., 2016; Wang et al., 2017)
have also been shown to be conducive to promoting cooperation.
Reciprocity (Gintis, 20 0 0b; Nowak and Sigmund, 1998; Boyd and
Richerson, 1988; Uchida, 2010) plays an important role in promot-
ing cooperation. Indirect reciprocity often combines with reputa-
tion (Ohtsuki and Iwasa, 2004). A body of literatures in psychol-
ogy suggest that reward is a better incentive in changing attitude
and behavior than punishment is Kim et al. (2006) and Larsen and
Tentis (2003). Other studies concluded that punishment is more
effective than reward in repeated PGG (Choi and Ahn, 2013; Sutter
et al., 2010). Some researchers have suggested punishment is very
effective for deterring free-riding and, thus, facilitating cooperation
in PGG (Bochet et al., 2006; Carpenter, 2007). Tolerance-based pun-
ishment (Gao et al., 2012) has been investigated and the results
may enhance the understanding of altruistic punishment in the
evolution of human cooperation. However, punishment are costly,
and who bears the costs has become a new problem. “Second-
order free riders” who avoid execution prevail and thus eliminate
the threat of punishment (Szolnoki and Perc, 2017).

https://doi.org/10.1016/j.jtbi.2019.110103
0022-5193/© 2019 Elsevier Ltd. All rights reserved.
2 Y. Jiao, T. Chen and Q. Chen / Journal of Theoretical Biology 486 (2020) 110103
Fig. 1. the whole circulatory process of evolution.
At present, some researchers (Chen et al., 2016) turn to some
practical examples to explore the way of solving the dilemma of
public goods. “Carrots and sticks” are measures that are widely
used in practice to promote cooperation. For example, in order to
raise funds from local villagers for a kind of public cultural goods,
village opera, village officials and local prestigious elders take some
measures of punishment and reward in some areas. However, they
sometimes do not execute it due to their unwillingness to bear
these additional costs. Under this circumstance, cooperation can
still emergence and be sustained a few years later (Village opera
is held once a year). We also find that when they hardly perform
punishment and reward, the effect of promoting cooperation is bad
in the long run because of the distrust of villagers. Thus we ex-
plore whether there is an optimal probability of implementing to
promote and maintain cooperation efficiently. Inspired by previous
literature and realities, this article introduce probabilistic punish-
ment and reward in PGG. It means that individuals perform pun-
ishment or reward with a certain probability (ρ ). That is to say,
the punisher (rewarder) sometimes does not punish (reward) the
defectors (cooperators). If a individual averagely performs punish-
ment (rewards) v times per w times public goods games, it can be
said ρ = wv , (0 ≤ ρ ≤ 1). Since punishment or reward is costly,
probabilistic execution could reduce unnecessary losses in PGG.
However, this kind of incomplete performance leads to the distrust,
which in turn induces opportunistic behavior causing decreasing of
cooperators. Some researches have shown that expressing trust is
an important social mechanism to promote cooperative behaviors
(Axelrod and Hamilton, 1981). The raise of donation is referred to
as the result of sense of trust and the level of trust changes over
time and with experience (Fehr and Fischbacher, 2003; King-Casas
et al., 2005). This design takes this into account and combines
trust with probabilistic punishment and reward. In general, this
work aims to investigate the role of probabilistic performance in
the emergence and maintenance of cooperation within the struc-
tured populations. The result of the simulation is to provide advice
on how the individuals perform reward and punishment to enable
the public goods to be well supplied.
The rest of this paper is structured as follows. Public goods
game model with probabilistic punishment and reward is intro-
duced in detail in Section 2. The results of the numerical simu-
lation are analyzed in Section 3 and the conclusions are drawn in
Section 4.
2. The PGG model with probabilistic punishment and reward
To preserve comparability with previous works, the public
goods game is staged on a square lattice using periodic bound-
ary conditions. In the network, lattices represent agents and every
agent has ki = 4 neighbors. At each time step, agents on the net-
work attend ki + 1 games groups that are centered on itself and
centered on its neighbors. The group consists of a central lattice
and its four neighbors. There are six types of strategies (Si∗ ), in-
cluding pure cooperator (CN), pure defector (DN), punishing co-
operator (PC), punishing defector (PD), rewarding cooperator (RC)
and rewarding defector (RD). Each punishing individual (rewarding
individual) sanctions (rewards) its nearest defectors (cooperators)
in the group centered on itself (not considering itself) with a fine
(bonus) at a personal cost. Initially, CN and DN respectively account
for 25 percent, and other types of players respectively account for
12.5 percent. Fig. 1 shows the whole circulatory process of evolu-
tion. The important parameters and their meanings in the model
are summarized in the Table 1.
The interaction. Each cooperator i (CN, PC, RC) will contribute
ai = 1 to the group that it attends, while the defectors i (DN,
PD, RD) contribute nothing and ai = 0. Subsequently, the sum of
contributions in a group is multiplied by the synergy factor R. The
 Y. Jiao, T. Chen and Q. Chen / Journal of Theoretical Biology 486 (2020) 110103 3

Table 1
The definitions and descriptions of parameters.

Parameter Definition and description

ki The number of agent i’s neighbors

R The synergy factor
ai The contribution of the agent i
Nc The number of cooperators in a group
i A set of game groups containing agent i
pi The payoff of agent i in the absence of punishment and reward
pi ∗ The final payoff of agent i under punishment and reward
ρ The probability of executing punishment or reward
f The fine
r The bonus
h The unit cost of punishment or reward
u The number of RCt and RDt in the group centered on a cooperator
m The number of PCt and PDt in the group centered on a defector
g The number of cooperators in the group centered on a RCt or RDt
b The number of defectors in the group centered on a PCt or PDt
 The amplitude of environment noise
j The agent i’s nearest neighbors
k The punisher or rewarder who is among j
cik (t) Agent i’s evaluation of credibility of k at time t
i The increment or decrement of cik
CT The tolerance value of credibility of surrounding
Ci (t) Agent i’s overall evaluation of credibility of surrounding at time t
n The number of k
n1 The number of punishers among k
n2 The number of rewarders among k
α1 The weight coefficients of fines for a defector
α2 The weight coefficients of bonuses for a defector
β1 The weight coefficients of fines for a cooperator
β2 The weight coefficients of bonuses for a cooperator

resulting amount is then shared equally among all members of the implementers (PCt , PDt , RCt , RDt ) will bear the corresponding
group, irrespective of their strategy. The agent i’s payoff is the sum costs, while the payoffs of false implementers (PCf , PDf , RCf , RDf )
of shares of all the groups that it belongs to. The pi represents the are equal to that of CN or DN respectively. We use p∗i to represent
payoff of each agent i in the absence of punishment and reward, the final payoff under probabilistic punishment and reward, and
then it can be expressed as: its calculations are on the basis of pi . So p∗i can be expressed as
  ⎧
  R ∗ Nc pi + u ∗ r Si∗ = CN (PC f , RC f )
pi = pij = − ai (1) ⎪
⎪
kj + 1 ⎪
⎪ Si∗ = DN (P D f , RD f )
j∈i j∈i ⎨ pi − m ∗ f
pi + u ∗ r − b ∗ h Si∗ = PCt
Where j represents one of the game groups that player i at- p∗i = (3)
⎪ pi + u ∗ r − g ∗ h Si∗ = RCt
⎪
⎪
tends. R represents the synergy factor. Nc represents the number ⎪
⎩ pi − m ∗ f − b ∗ h Si∗ = P Dt
of cooperators in the group j. ai is the contribution of player i. pi − m ∗ f − g ∗ h Si∗ = RDt
When agent i is a cooperator, ai = 1, and when agent i is a de-
fector, ai = 0. kj is the number of neighbors of centered agent in Where Si∗ represents strategy that is adopted by i. f and r rep-
group j. i represents all the game groups containing player i. resent the fine and bonus respectively. It is assumed that the costs
Probabilistic punishment and reward. At the end of the interac- of punishment and reward are equal, and they are denoted by h.
tion, punishers (PC, PD) and rewarders (RC, RD) execute punish- u (m) represents the number of RCt , RDt (PCt , PDt ) in the group
ment and reward. Because punishment and reward are costly, the centered on cooperator (defector) i (not considering i). g (b) repre-
agent sometimes does not perform them. In this article, the above sents the number of cooperators (defectors) in the group centered
behavior is called probabilistic execution. That is to say, punish- on RCt , RDt (PCt , PDt ) i (not considering i).
ment and reward are performed with a certain probability (ρ ). Initial strategy (Si ). After calculating the p∗i , each agent tries to
We assume that all agents have the same probability of execution maximize its own payoff and imitates others based on the Fermi
(0 ≤ ρ ≤ 1). Thus after an interaction, some of performers authen- formula. We call the strategy by imitation an initial strategy (Si ).
tically punish or reward its nearest neighbors, and others do not. The probability of imitating depends on the difference between the
Accordingly, some of the defectors and cooperators get fines and returns of the two individuals. It can be expressed by
bonuses, and others do not. In order to clearly distinguish differ- 1
ent actions, we divide the punishing and rewarding actions into W Si ← S j = (4)
1 + exp p∗i − p∗j /
two categories according to whether they actually take action. As
shown in the following formula, one is the true implementer who Where t indicates time step. p∗i and p∗j respectively represent
truly perform punishment or reward, denoted by PCt , PDt , RCt , RDt , the final payoffs of agent i and agent j.  represents where the
the other is the false implementer who falsely perform punish- amplitude of environment noise. In this paper,  = 0.1 is a fixed
ment or reward, denoted by PCf , PDf , RCf , RDf . number (Szabó and Hauert, 2002).
 Since the issue of social information is not the focal point, we
true(ρ ) : PCt , P Dt , RCt , RDt
PC, P D, RC, RD (2) assume that agents have local information about strategy types
false(1 − ρ ) : PC f , P D f , RC f , RD f
of its nearest neighbors (Boros et al., 2010). Because individual’s
The calculation of final payoff p∗i . After probabilistic punishment impressions of punishers (rewarders) who did not carry out pun-
and rewards, the payoffs will change. Obviously only the true ishment (reward) become worse, they tend not to believe those
4 Y. Jiao, T. Chen and Q. Chen / Journal of Theoretical Biology 486 (2020) 110103
people in future interactions. In order to explain the impact of the
probabilistic execution on degree of trust, we define cik (t) as agent
i’s evaluation of credibility of punisher or rewarder k (k ∈ j) who
is among its nearest neighbors j ( j = {1, 2, 3, 4}) at time t. cik (t)
is updated at the ene of interactions. If k authentically performs
punishment (reward) at time t − 1, cik (t) will increase by i at
time t. Conversely, if k does not perform punishment (reward) at
time t − 1, cik (t) decreases by i at time t. In order to keep cik (t)
always in interval (0, 1), we set cik (t ) = 1, when cik (t) > 1 and
cik (t ) = 0, when cik (t) < 0 in program. cik (t) can be expressed as
cik (t ) = cik (t − 1 ) ± i , 0 ≤ cik (t ) ≤ 1
Where we set cik (0 ) = 0.5. k (k ∈ j) represents every punisher
or rewarder among i’s neighbors j. Notably, this definition of cik is
agent i’s subjective evaluation or impression of k. Thus, consider-
ing the heterogeneity (Fischbacher and Gachter, 2010), i of each
agent is assigned a random number from a interval (0, 0.5) and re-
mains unchanged in evolution. Each punishing or rewarding agent
builds up its credibility or “a image score” (Fu et al., 2008) by true
execution.
T rust − based rule of decision − making. Probabilistic implemen-
tation affects the degree of trust. Whether individuals trust pun-
ishers and rewarders to carry out them or not influences decision-
making. After initial decision, only when individuals believe that
punishment or reward will be executed, will they believe that
their payoffs vary and make a sensible decisions again. Therefore,
a trust-based rule of decision-making (see Fig. 1) is introduced.
Through two-step judgments, each agent makes final decision (Si∗ )
based on initial strategy (Si ) that is chosen by imitating.
In step one, agent i decide whether to trust the punishers
and rewarders to execute punishment and reward. We define Ci (t)
(0 ≤ Ci (t) ≤ 1) as agent i’s overall evaluation of credibility of sur-
rounding, which is determined by cik (t) (k ∈ j). The calculation of
Ci (t) is given by the following formulas:

cik (t )
Ci (t ) = k
k ∈ {1, 2, 3, 4}
n RD) respectively account for 50 percent. After each initialization,
Where t represents time step. The meaning of k is the same as
above. The sum runs again over all k. n(1 ≤ n ≤ 4) is given by the
total number of k. From the above formula, Ci (t) is the mean of
all cik (t)s of agent i. Therefore, Ci (t) can be kept in (0, 1) because
the value of each cik (t) is within the interval (0, 1). If the value of
Ci (t) is equal or greater than a certain number that is called the
tolerance value CT , agent i trusts the all punishers and rewarders
to sanction and reward. Considering the heterogeneity of the re-
quirements for credibility, CT of each agent is drawn from the uni-
form distribution of interval (0, 1) and remains unchanged in the
process of evolution. If Ci (t) < CT , it means that credibility of sur-
rounding does not meet i’ requirement at time t and, thus Si∗ = Si .
Otherwise, the expected benefit (EB) or expected loss (EL) of agent
i will be calculated to help to determine Si∗ .
In step two, agent i determines Si∗ according to the calculation
of expected benefit (EB) or expected loss (EL). Because individuals
are payoff-driven, they only adopt strategy that is profitable or can
reduce losses. For a defector, if the return is greater than the loss
after transition to cooperative strategy, it would be willing to coop-
erate. Otherwise, it will maintain the initial strategy. Similarly, for
a cooperator, if the loss is greater than the return after transition
to defective strategy, it will maintain the initial strategy. Otherwise,
it will defect. Based on the above analysis, we give the calculation
rules for EB and EL respectively:
 First of all, we need to explore whether the probabilistic ex-
EB = α1 n1 f + α2 ∗ (n2 r − 1 ) Si = D
EL = β1 ∗ (n1 f − 1 ) + β2 n2 r Si = C
Where Si represents the initial strategy. And it is the basis of
determining Si∗ . n1 and n2 respectively represent the total num-
ber of punishers and rewarders among i’s nearest neighbors. The
meanings of f and r are consistent with the above. As shown in
the above formulas, when Si = D, defector i will calculate the ex-
pected benefit (EB) of switching to cooperation based on the up-
per formula. EB should be composed of the benefit that is exempt
from sanction and the added bonus due to transition to coopera-
tive strategy. Since a defector can get rewards only if it firstly con-
tributes ai = 1 to the public pool, the contribution as cost should
be deducted from the bonus n2 r. α 1 and α 2 respectively represent
the weight coefficients of fines and bonuses. The bonus is more
valued than fine for defectors in the process of switching to co-
(5) operation, so α 1 < α 2 . And considering the individual heterogene-
ity, α 1 and α 2 are respectively drawn from the uniform distribu-
tion of interval (0, 0.5) and (0.5, 1). Finally, if the calculated EB is
greater than zero (EB > 0), the defector will transition to cooper-
ator (Si∗ = C). Otherwise the initial strategy will remain unchanged
(Si∗ = Si = D). When Si = C, cooperator i will calculate the expected
loss (EL) of switching to defection based on the lower formula. EL
is composed of the additional fine and the lost bonus due to tran-
sition to defection. Because if a cooperator transitions from coop-
eration to defection, the cost of contributing 1 to the public pool
will be saved, the cost is deducted from the fine n1 f. β 1 and β 2
respectively represent the weight coefficients of fines and bonuses.
Cooperators more mind the fine in the process of switching to de-
fection, so β 1 > β 2 . Considering the individual heterogeneity, β 1
and β 2 are respectively drawn from the uniform distributions of
interval (0.5, 1) and (0, 0.5). Finally, if the calculated EL is less
than zero (EL < 0), then the cooperator will transition to defec-
tor (Si∗ = D), otherwise the initial strategy will remain unchanged
(Si∗ = Si = C).
3. Numerical simulation results and analysis
The simulations are conducted on a square lattice of size N =
10 0 0 0 (L × L = 100 × 100) with periodic boundary conditions. At
the initial moment, cooperators (CN, PC, RC) and defectors (DN, PD,
(6)
the system evolves 10,0 0 0 time steps, and 20 independent simu-
lations ensure reliability of the evolutionary results. We set syn-
ergy factor (R) to be 2.6. We assume that fine (f) and bonus (r)
are equal, and the costs (h) of the reward and punishment are
equal. In addition, in order to explore the impact of probabilis-
tic punishment and reward function more intuitively, we introduce
traditional punishment and reward as a comparison. In the tradi-
tional punishment and reward model, there are six different types
of strategies. The strategy types and their initial proportions are
the same as that in the probabilistic execution model. In the ini-
tial setup of traditional punishment and reward model, the pure
cooperators (CN) and the pure defectors (DN) respectively account
for 25 percent, and punishing cooperator (PC), punishing defector
(PD), rewarding cooperator (RC) and rewarding defector (RD) re-
spectively account for 12.5 percent. However, unlike probabilistic
execution model, the punishment and reward are fully executed in
the traditional punishment and reward model, so the PC, PD, RC,
RD completely correspond to the true implementers (PCt , PDt , RCt ,
RDt ) in the probabilistic execution model, and there are no false
implementers. The calculation of payoff in the traditional model
is also the same as that in the probabilistic execution model. In
addition, players update their strategy solely by imitation learning
based on the Fermi formula, which is different from probabilistic
execution model. The simulation results are as follows.
(7) ecution will affect the emergence and maintenance of coopera-
tion in continuous public goods game. Then, we will further ex-
plore whether there is an optimal probability that can minimize
the executing cost and maximize fraction of cooperators. Therefore,
we compare the PGG of traditional punishment and reward with
 Y. Jiao, T. Chen and Q. Chen / Journal of Theoretical Biology 486 (2020) 110103 5

Fig. 2. The evolution of cooperation over time in the PGG with different probability of execution (ρ = 0.2, 0.4, 0.6, 0.8) and the PGG with traditional punishment and reward.
The PGG with traditional punishment and reward finally reaches the state of global defection. The greater executing probability (ρ ) is, the more favorable it is to promote
cooperation. Other parameter settings:  = 0.1, R = 2.6, f = r = 1, h = 0.5.

Fig. 3. (a)The evolution of the fraction of cooperation over time for high, medium and low f (r) ( f = r = 1.5, 1, 0.5). The cooperation rate increases with the increase of f
(r), but due to the limit of lower ρ , the maximal cooperation rate is only up to 0.45. Other parameter settings: ρ = 0.4,  = 0.1, R = 2.6, h = 0.5. (b)The evolution of the
fraction of cooperation over time for different f (r) ( f = r = 1.5, 1, 0.5). The cooperation rate increases with the increase of f (r). When ρ is large, the maximal cooperation
rate is up to 1. Other parameter settings: ρ = 0.8,  = 0.1, R = 2.6, h = 0.5.

that of probabilistic execution. As shown in Fig. 2, we investigate
the cooperative evolution of PGG model under five different cases.
Here, we set f = r = 1, and h = 0.5. The dark blue curve represents
the PGG with traditional punishment and reward, and the other
four curves respectively represent the PGG with different executing
probability (ρ = 0.2, 0.4, 0.6 and 0.8). It can be clearly seen that
cooperation rate drops to 0 when steady state is reached in tradi-
tional punishment and reward. When ρ is low (ρ = 0.2), the result
is still global defection. The situation improves for ρ = 0.4, but the
cooperation rate is still at a low level. While ρ is large (ρ = 0.6 or
0.8), the result of evolution reaches a state of global cooperation.
Therefore, we can draw the following conclusions: (i)Probabilistic
punishment and reward can promote the emergence and main-
tenance of cooperation in continuous public goods game. (ii) The
greater ρ is, the more favorable it is to promote cooperation. When
ρ = 0.6, the result of evolution is close to the global cooperation.
In this case, about 0.4 of the cost can be saved and efficiency of
supplying public goods is improved.
According to the introduction of the model in Section 2, the
calculation of expected returns and expected losses is also an im-
portant factor that affect the decision-making except the executing
probability. The payoff-driven agent considers changing strategies
only if it is profitable or reducing losses. The expected return and
expected loss are calculated based on the fine and bonus. There-
fore, we continue to study how the different f and r affect the evo-
lution of cooperation in PGG with probabilistic punishment and re-
ward.
Fig. 3 (a) shows the evolution of fraction of cooperation over
time step for different fines and bonuses ( f = r = 1.5, 1, 0.5) on
the condition that the executing probability is at a relatively low
level (ρ = 0.4). It can be clearly seen that when the fine and the
bonus are 1.5 or 1, the cooperation rate first rises to the peak and
then declines and gradually stabilizes over time. And the fraction
of cooperation reaches 0.45 and 0.25 respectively in equilibrium,
both lower than 0.5. When f = r = 0.5, the fraction of cooperation
decreases from the beginning and finally stabilizes ( ≈ 0.1) over
time. It can be concluded that higher f and r are more conducive
to the emergence and maintenance of cooperation than the a lower
fines (bonuses). Fig. 3(b) shows the evolution of fraction of coop-
eration over time step for the three different levels of fines and
bonuses ( f = r = 1.5, 1, 0.5), when the executing probability is at
a high level (ρ = 0.8). From Fig. 3(a) and Fig. 3(b), it can be seen
that under different executing probabilities (ρ ), the level of coop-
eration increases with the increases of fine and bonus. In addition,
for a relatively small ρ (see Fig. 3(a)), even if f and r are very large,
the effect of promoting cooperation is still not significant. But for
a large executing probability (see Fig. 3(b)), as long as f and r are
moderate, system can achieve global cooperation. The possible rea-
son is that when the value of ρ is small, people largely distrust
punishers and rewarders. Therefore, even if f and r are very high, it
will not promote the cooperation well. Moreover, when f and r are
very small, the state of all defection cannot be changed even if ρ is
large. Therefore, it is inappropriate to choose very small ρ and f (r).
In order to better explore the effect of probabilistic execution,
we next focus on how different combinations of ρ and f (r) af-
fect the level of cooperation. Here, we introduce the mean pay-

N
p∗i
off (MP) as a measure. It is defined as MP = i=1
, where p∗i is
N
6 Y. Jiao, T. Chen and Q. Chen / Journal of Theoretical Biology 486 (2020) 110103

Fig. 4. (a)The cooperation rate as executing probability (ρ ) for three kinds of fines and bonuses ( f = r = 1.5, 1, 0.5). For different f(r) values, the cooperation rate increases
with the increase of ρ . (b)The mean payoff as executing probability (ρ ) for three kinds of fines and bonuses ( f = r = 1.5, 1, 0.5). For different f (r), the mean payoff increases
with the increase of ρ . Other parameter settings:  = 0.1, R = 2.6, h = 0.5.

Fig. 5. (a) The cooperation rate as fine (f) and bonus (r) for different executing probabilities (ρ = 0.9, 0.6, 0.3) and traditional punishment and reward. For different ρ ,
cooperation rate increases with increasing value of f (r). (b)The mean payoff as fine (f) and bonus (r) for the different executing probabilities (ρ = 0.9, 0.6, 0.3) and traditional
punishment and reward. For different ρ , the mean payoff increases with increasing f(r). Other parameter settings:  = 0.1, R = 2.6, h = 0.5.

each agent’s final payoff (see Section 2), N is the population (N =

10 0 0 0). Fig. 4(a) and (b) respectively show the variation of frac-
tion of cooperators and the mean payoff with ρ under three differ-
ent levels of punishment and reward ( f = r = 1.5, 1, 0.5). As shown
in Fig. 4(a), the patterns of the three curves are generally analo-
gous. It can be seen that the cooperation rate increases with the
increase of ρ regardless of the value of f and r. When ρ is less
than 0.6, three curves all show an upward trend. In addition, for
f = r = 1.5 or f = r = 1, when the execution probability is 0.6, the
system reaches global cooperation. However, for f = r = 0.5, even
if it is completely executed (ρ = 1), the system cannot achieve
global cooperation. It can be seen that in order to promote co-
operation better, f and r are not suitable to be too low. Accord-
ing to Fig. 4(b), we can draw a similar conclusion. The mean pay-
off increases with the increase of ρ . But when ρ > 0.6, there be
no significant increase. In addition, for a fixed ρ , the mean pay- Fig. 6. The cooperation rate as cost (h) of executing punishment (reward), under
different executing probabilities (ρ = 0.9, 0.6, 0.3) and traditional punishment and
off increases with the increase of f (r), but it does not continue
reward. For ρ = 0.9, 0.6, 0.3, the cooperation rate hardly decreases with the in-
to increase, when f (r) reaches the critical value. Thus, combina- crease of cost (h), but for traditional execution, the cooperation rate drops sharply
tion of moderate execution probability and fines (bonuses), such from h = 0.3 and eventually falls to zero. Other parameter settings:  = 0.1, R =
as ρ = 0.6, f = r = 1, should be adopted in order to achieve a good 2.6, f = r = 1.
effect of promoting cooperation in real life. In this case, cost of 40
percent can be saved. If the cost of execution is relatively high, it
is better to choose a small executing probability (ρ = 0.5) and a the fractions of cooperations increase with the values of f and r.
high f (r) ( f = r = 1.5). The fraction of cooperations in the PGG with ρ = 0.3 is always
In order to explore the critical f (r) values that enable the at a low level. Interestingly, when the executing probability is
system to reach global cooperation under a certain executing high (ρ = 0.9), the critical f (r) value that enables system to reach
probability, we continue to run simulation. Numerical simulation global cooperation is 0.7. However, when the executing probability
results are shown in Fig. 5(a). The blue curve represents the public is a moderate value (ρ = 0.6), the critical f (r) value increases
goods game with traditional punishment and reward. The rest of to 1. Moreover, when the probability is low (ρ = 0.3), the peak
three curves respectively represent the PGG under high, medium cooperation rate is only 0.1 at f = r = 1. The possible reason is
and low executing probabilities. It can be seen from it that when that a low ρ lead to distrust. Fig. 5(b) shows the relation between
f = r < 1, the fraction of cooperations in the traditional PGG mean payoff and the fine (bonus) for different ρ . It can be seen
model is always 0, while in the PGG with ρ = 0.6 and ρ = 0.9, that in the PGG model with traditional punishment and reward,
 Y. Jiao, T. Chen and Q. Chen / Journal of Theoretical Biology 486 (2020) 110103 7

Fig. 7. (a)The fraction of cooperators as synergy factor (R) for different f and r ( f = r = 1.5, 1, 0.5). For both f = r = 1.5 and f = r = 0.5, the levels of cooperation remain
unchanged with the increase of R, and they are maintained at 1 and 0.5 respectively, and for f = r = 1, the level of cooperation increases with the increase of R. Other
parameter settings:  = 0.1, ρ = 0.6. (b)The fraction of cooperators as synergy factor (R) for different executing probabilities (ρ = 0.9, 0.6, 0.3). For ρ = 0.9 and ρ = 0.3, the
levels of cooperation remain unchanged with the increase of R, they are maintained at 1 and 0.1 respectively, and for ρ = 0.6, the level of cooperation increases with the
increase of R. Other parameter settings:  = 0.1, f = r = 1 .

Fig. 8. Snapshots of typical distributions of six types of players at different time steps (t = 0, 10, 10 0 0, 10 0 0 0) for four cases. Colors distinguish CN (blue), DN (red), PC
(green), PD (yellow), RC (pink), RD (orange). The first row shows the PGG with traditional punishment and reward. DN (red) occupy the whole network in the end. In the
second row (ρ = 0.3), RD (orange) take over almost all lattices in the end. In the third row (ρ = 0.6), PC (green) occupy almost all lattices in the end, and a small number
of PD (yellow) still survive in the population. In the fourth row (ρ = 0.9), PC (green) occupy all lattices in the end. Other parameter settings:  = 0.1, R = 2.6, f = r = 1,
h = 0.5. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

mean payoff reaches the peak when f = r = 1.1, whereas in the
PGG model with ρ = 0.9 and ρ = 0.6, the fine (bonus) values
that maximize the mean payoff respectively decrease to 0.7 and
1. Therefore, the critical f (r) value is negatively correlated with
ρ . In real life, if the cost of execution is low, a combination of
low f (r) and high executing probability (ρ = 0.9) could be chosen.
Conversely, if the cost of execution is high, a combination of high f
(r) and moderate executing probability (ρ = 0.6) could be chosen.
Compared with the PGG with traditional execution, achieving
the aim of reducing cost for the rewarder and the punisher is the
8 Y. Jiao, T. Chen and Q. Chen / Journal of Theoretical Biology 486 (2020) 110103
advancement of probabilistic execution. Therefore, we continue to
explore the relation between h and cooperation rate. The results
are shown in Fig. 6. In the PGG with traditional punishment and
reward, the cooperation rate decreases sharply from h = 0.3. When
h = 0.5, the system reaches a state of global defection. However,
when probabilistic execution is introduced, the level of coopera-
tion has been significantly improved. When ρ = 0.9, the rate of
cooperation does not decrease with the increase of h and is always
maintained at a high level ( ≈ 1). When ρ = 0.6, the cooperation
rate shows a slightly downward trend, but it is still close to 1. For
ρ = 0.3, since the executing probability is too low, it directly leads
to distrust. Thus the cooperation rate remains at a low value of
0.1. Therefore, we can draw the following conclusion: When cost is
very high, compared with traditional punishment and reward, the
probabilistic execution is more conducive to the evolution of al-
truistic punishers and rewarders, thereby avoiding the collapse of
cooperation.
Synergy factor (R) is fixed in the above simulations, but it is un-
clear whether the probabilistic execution still plays a role in coop-
erative evolution when R varies. Therefore, we further explore how
R affect the level of cooperation. Fig. 7(a) shows the relation be-
tween fraction of cooperators and the R for different f and r. It can
be clearly observed that there are two horizontal curves, which in-
dicate that for extremely large or small fines and bonuses ( f = r =
1.5 or f = r = 0.5), the level of cooperation is almost independent
of the synergy factor R. Specifically, when they are extremely large
( f = r = 1.5), the fraction of cooperators is always maintained at
1. When they are extremely small ( f = r = 0.5), the fraction of co-
operators is always maintained at a low value ( ≈ 0.5). The other
curve represents f = r = 1, and the level of cooperation increases
with R and finally reaches a stable state of global cooperation.
Fig. 7(b) shows the relation between fraction of cooperators and R
for different ρ . The blue curve represents the PGG with traditional
execution, and R = 2.8 is the critical value that enable the coopera-
tion rate to rise sharply from 0 to 1. The two horizontal curves re-
spectively represent ρ = 0.3 and ρ = 0.9. These indicate that when
ρ is extremely low or high, the level of cooperation is hardly af-
fected by the synergy factor R. Specifically, when ρ is extremely
large, the fraction of cooperators is always maintained at a very
high level of 1 even if R is small. When ρ is extremely small, the
fraction of cooperators is always maintained at a low level ( ≈ 0.1)
even if R is large. The other curve represents the medium proba-
bility of execution (ρ = 0.6). The fraction of cooperators increases
with the R and finally stabilizes ( ≈ 1).
The snapshots of Fig. 8 demonstrate the coarsening process im-
pressively for four cases. In the PGG with traditional punishment
and reward (the first row), DN (red) fast spread in the population
and occupy all lattices in the end. Compared with the first row,
other cases respectively promote the evolution of reward and pun-
ishment. In the PGG with ρ = 0.3 (the second row), although the
rewarders (RD, RC) have gained an advantage of evolution, there
are only a small fraction of cooperators (RC). In the PGG with
ρ = 0.6 (the third row), although a small number of PD (yellow)
still survive in the population, PC (green) occupy almost all lat-
tices in the end. In the PGG with ρ = 0.9 (the fourth row), PC
(green) occupy all lattices in the end, which is similar to previous
researches (Hilbe and Sigmund, 2010).
4. Conclusion
In this article, the probabilistic punishment and reward are in-
troduced to PGG. It means that punishers (rewarders) execute pun-
ishment (reward) with a certain probability (ρ ) due to saving cost.
Thus when individuals interact with them, they first judge their
credibility. If punishers and rewarders are creditable, the expected
benefit or expected loss is further calculated to help individual to
make final decision. Researchers have discovered the existence of
trust in social cooperation in the past study (Fehr and Fischbacher,
2003; King-Casas et al., 2005). This design takes this into account
and combine it with probabilistic punishment and reward. The
present work aims to examines the role of trust-based probabilistic
punishment and reward in continuous public goods games within
structured population. We conducted numerical simulations from
several aspects through agent-based model.
Through numerical simulation, the following conclusions can
be drew. First of all, compared with traditional punishment and
reward, the PGG with probabilistic punishment and reward can
reduce unnecessary costs in the long run and make punishment
and reward more efficient. Secondly, when the ρ is extremely low
(ρ < 0.3), the cooperation rate is at a low level regardless of
the value of f (r). The possible reason is that a very low ρ eas-
ily causes distrust of individuals who interacts with the punisher
(rewarder). When individuals distrust, they will not calculate the
expected benefit or expected loss. Under this circumstance, deter-
rence of punishment and temptation of reward do not work. Sim-
ilarly, when f (r) is extremely low ( f = r = 0.3), global cooperation
cannot be achieved even if ρ is high. The reason is that deterrence
of punishment and the temptation of the reward could not be suf-
ficient to promote global cooperation, instead coexist of coopera-
tors and defectors. Therefore, in practice, a combination of moder-
ate ρ and f (r) could be adopted. When one of them is high, the
other can be relatively low. This kind of combination can signifi-
cantly promote cooperation with minimal cost in interactions. Fi-
nally, through the analysis of results, we find that when ρ = 0.6,
as long as the fines and bonuses are not very small, the emer-
gence and maintenance of cooperation can be achieved. Therefore,
ρ = 0.6 is an optimal choice. Moreover, the result of simulation
suggests that when h is large, cooperative rate in PGG with prob-
abilistic execution is still maintained at a high level. But there is a
significant decrease of cooperation with increase of cost h in PGG
with traditional punishment and reward. This reveals that even
when the cost of punishment and reward is relatively high, the
PGG model with probabilistic execution can help evolution of al-
truistic punishers and rewarders, thereby avoiding collapse of co-
operation. The finding provides a reference for solving the dilemma
in the supply of public goods. It is possible to rely on probabilistic
punishment and reward to aspire people’s enthusiasm for coop-
eration. In addition, the village officials and other social members
should pay attention to the establishment of credibility and avoid
use a low executing probability.
Acknowledgments
We would like to acknowledge funding support from the Na-
tional Social Science Fund of China (NSSFC) (Grant No. 17BZZ006).
References
Axelrod, R., Hamilton, W.D., 1981. The evolution of cooperation. Science 211 (4489),
1390–1396.
Bettina, R., Manfred, M., 2006. The efficient interaction of indirect reciprocity and
costly punishment. Nature 444 (7120), 718–723.
Bochet, O., Page, T., Putterman, L., 2006. Communication and punishment in volun-
tary contribution experiments. J. Econ. Behav. Org. 60 (1), 11–26.
Boros, E., Elbassioni, K., Gurvich, V., Makino, K., 2010. A pumping algorithm for er-
godic stochastic mean payoff games with perfect information. In: International
Conference on Integer Programming and Combinatorial Optimization. Springer,
pp. 341–354.
Boyd, R., Richerson, P.J., 1988. The evolution of reciprocity in sizable groups. J. Theor.
Biol. 132 (3), 337–356.
Brandt, H., Hauert, C., Sigmund, K., 2003. Punishment and reputation in spatial pub-
lic goods games. Proc. R. Soc. London. Ser. B 270 (1519), 1099–1104.
Carpenter, J.P., 2007. The demand for punishment. J. Econ. Behav. Org. 62 (4),
522–542.
 Y. Jiao, T. Chen and Q. Chen / Journal of Theoretical Biology 486 (2020) 110103
Chen, Q., Chen, T., Wang, Y., 2016. How the expanded crowd-funding mechanism of
some southern rural areas in china affects cooperative behaviors in threshold
public goods game. Chaos Soliton. Fractal. 91, 649–655.
Chen, T., Wu, Z., Wang, L., 2018. Disseminators or silencers: the effect of informa-
tion diffusion intensity on cooperation in public goods game. J. Theor. Biol. 452,
47–55.
Choi, J.-K., Ahn, T., 2013. Strategic reward and altruistic punishment support coop-
eration in a public goods game experiment. J. Econ. Psychol. 35, 17–30.
Dos Santos, M., Peña, J., 2017. Antisocial rewarding in structured populations. Sci.
Rep. 7 (1), 6212.
Fehr, E., Fischbacher, U., 2003. The nature of human altruism. Nature 425 (6960),
785.
Fischbacher, U., Gachter, S., 2010. Social preferences, beliefs, and the dynamics of
free riding in public goods experiments. Am. Econ. Rev. 100 (1), 541–556.
Fowler, J.H., 2005. Altruistic punishment and the origin of cooperation. Proc. Nat.
Acad. Sci. 102 (19), 7047–7049.
Fu, F., Hauert, C., Nowak, M.A., Wang, L., 2008. Reputation-based partner choice pro-
motes cooperation in social networks. Phys. Rev. E 78 (2), 026117.
Gao, J., Li, Z., Cong, R., Wang, L., 2012. Tolerance-based punishment in continuous
public goods game. Phys. A 391 (16), 4111–4120.
Gintis, H., 20 0 0. Game theory evolving: a problem-centered introduction to model-
ing strategic behavior. Princeton university press.
Gintis, H., 20 0 0. Strong reciprocity and human sociality. J. Theor. Biol. 206 (2),
169–179.
Hardin, G., 1968. The tragedy of the commons. Science 162 (3859), 1243–1248.
Hauert, C., De Monte, S., Hofbauer, J., Sigmund, K., 2002. Volunteering as red
queen mechanism for cooperation in public goods games. Science 296 (5570),
1129–1132.
Hauert, C., Saade, C., McAvoy, A., 2019. Asymmetric evolutionary games with envi-
ronmental feedback. J. Theor. Biol. 462, 347–360.
Hilbe, C., Sigmund, K., 2010. Incentives and opportunism: from the carrot to the
stick. Proc. R. Soc. B 277 (1693), 2427–2433.
Kim, H., Shimojo, S., O’Doherty, J.P., 2006. Is avoiding an aversive outcome reward-
ing? Neural substrates of avoidance learning in the human brain. PLoS Biol. 4
(8), e233.
King-Casas, B., Tomlin, D., Anen, C., Camerer, C.F., Quartz, S.R., Montague, P.R., 2005.
Getting to know you: reputation and trust in a two-person economic exchange.
Science 308 (5718), 78–83.
Kümmerli, R., Colliard, C., Fiechter, N., Petitpierre, B., Russier, F., Keller, L., 2007. Hu-
man cooperation in social dilemmas: comparing the snowdrift game with the
prisoner’s dilemma. Proc. R. Soc. B 274 (1628), 2965–2970.
Larsen, M.A., Tentis, E., 2003. The art and science of disciplining children. Pediatr.
Clin. 50 (4), 817–840.
Liu, X., Pan, Q., He, M., 2018. Promotion of cooperation in evolutionary game dy-
namics with local information. J. Theor. Biol. 437, 1–8.
Maciejewski, W., Fu, F., Hauert, C., 2014. Evolutionary game dynamics in populations
with heterogenous structures. PLoS Comput. Biol. 10 (4), e1003567.
McAvoy, A., Hauert, C., 2015. Structural symmetry in evolutionary games. J. R. Soc.
Interf. 12 (111), 20150420.
Milinski, M., 1987. Tit for tat in sticklebacks and the evolution of cooperation. Na-
ture 325 (6103), 433.
Newth, D., 2009. Asynchronous Iterated Prisoner’s Dilemma. Adaptive Behavior 17
(2), 175–183.
Nowak, M.A., Sigmund, K., 1992. Tit for tat in heterogeneous populations. Nature
355 (6357), 250.
9
Nowak, M.A., Sigmund, K., 1998. The dynamics of indirect reciprocity. J. Theor. Biol.
194 (4), 561–574.
Nowak, M.A., Sigmund, K., 2004. Evolutionary dynamics of biological games. Science
303 (5659), 793–799.
Ohtsuki, H., Iwasa, Y., 2004. How should we define goodness? Reputation dynamics
in indirect reciprocity. J. Theor. Biol. 231 (1), 107–120.
Perc, M., Szolnoki, A., 2010. Coevolutionary games a mini review. BioSyst. 99 (2),
109–125.
Santos, F.C., Santos, M.D., Pacheco, J.M., 2008. Social diversity promotes the emer-
gence of cooperation in public goods games. Nature 454 (7201), 213.
dos Santos, M., 2015. The evolution of anti-social rewarding and its countermea-
sures in public goods games. Proc. R. Soc. B 282 (1798), 20141994.
dos Santos, M., Wedekind, C., 2015. Reputation based on punishment rather than
generosity allows for evolution of cooperation in sizable groups. Evolut. Human
Behav. 36 (1), 59–64.
Sasaki, T., Uchida, S., 2013. The evolution of cooperation by social exclusion. Proc. R.
Soc. B 280 (1752), 20122498.
Sasaki, T., Uchida, S., 2014. Rewards and the evolution of cooperation in public good
games. Biol. Lett. 10 (1), 20130903.
Sasaki, T., Uchida, S., Chen, X., 2015. Voluntary rewards mediate the evolution of
pool punishment for maintaining public goods in large populations. Sci. Rep. 5,
8917.
Semmann, D., Krambeck, H.-J., Milinski, M., 2003. Volunteering leads to rock–pa-
per–scissors dynamics in a public goods game. Nature 425 (6956), 390.
Sutter, M., Haigner, S., Kocher, M.G., 2010. Choosing the carrot or the stick? endoge-
nous institutional choice in social dilemma situations. Rev. Econ. Stud. 77 (4),
1540–1566.
Szabó, G., Hauert, C., 2002. Phase transitions and volunteering in spatial public
goods games. Phys. Rev. Lett. 89 (11), 118101.
Szolnoki, A., Perc, M., 2010. Reward and cooperation in the spatial public goods
game. EPL (Eur. Lett.) 92 (3), 38003.
Szolnoki, A., Perc, M., 2017. Second-order free-riding on antisocial punishment re-
stores the effectiveness of prosocial punishment. Phys. Rev. X 7 (4), 041027.
Szolnoki, A., Perc, M., Szabó, G., 2008. Diversity of reproduction rate supports coop-
eration in the prisoner’s dilemma game on complex networks. Eur. Phys. J. B 61
(4), 505–509.
Uchida, S., 2010. Effect of private information on indirect reciprocity. Phys. Rev. E 82
(3), 036111.
Uchida, S., Sasaki, T., 2013. Effect of assessment error and private information on
stern-judging in indirect reciprocity. Chaos Solit. Fractal. 56, 175–180.
Wang, X., Lv, S., 2019. The public goods game with shared punishment cost in
well-mixed and structured populations. J. Theor. Biol..
Wang, X., Lv, S., 2019. The roles of particle swarm intelligence in the prisoner’s
dilemma based on continuous and mixed strategy systems on scale-free net-
works. Appl. Math. Comput. 355, 213–220.
Wang, Y., Chen, T., Chen, Q., Si, G., 2017. Emotional decisions in structured popula-
tions for the evolution of public cooperation. Phys. A 468, 475–481.
Wang, Z., Wang, L., Szolnoki, A., Perc, M., 2015. Evolutionary games on multilayer
networks: a colloquium. Eur. Phys. J. B 88 (5), 124.
Weibull, J.W., 1997. Evolutionary Game Theory. MIT press.
West, S.A., Griffin, A.S., Gardner, A., Diggle, S.P., 2006. Social evolution theory for
microorganisms. Nature Rev. Microbiol. 4 (8), 597.