Combinatorial Game Theory | Biological Auction Theory

Ayush Kanodia, Junior undergraduate, CSE, IIT Bombay

Raghav Gupta, Junior undergraduate, CSE, IIT Bombay
Affiliations- Prof Krishnendu Chatterjee, IST Austria
Biological Auction Theory

Combinatorial Game Theory

INTRODUCTION

INTRODUCTION

All-pay auctions- Each player pays his own bid

E.g.- males looking to attract females (some animal species)
Second all-pay auctions- Each player except the winner pays his own bid, the winner
pays the 2nd highest bid
E.g.- war of attrition, for example- between trees competing for sunlight
Classically, only one reward per auction, won by highest bidder

A POMDP has a set of

states, a set of actions, a set
of observations, a
probability distribution over
states for each state-action
pair, and a probability
distribution over states for
each observation

OBJECTIVES
To obtain evolutionarily stable strategies (ESS) for participants in such auctions, with
the number of participants in each auction as a parameter
To extend the existing classical results for the single-reward auction case, to the
setting where there are two unequal rewards, won by the two highest bidders
To establish similarity between the theoretically predicted behavior of the auction
participants and the actual results of computer simulations of these auctions

CONCLUSIONS
Employing the Bishop-Cannings theorem (E(s,I)/s = 0, where I is an assimilated
strategy and s is any pure strategy, E(s,I) is the expected payoff for the pure
strategist), to obtain evolutionarily stable strategies for participants in all-pay and
second all-pay auctions
All-pay auctions-for rewards V1 and V2 (V1 > V2) and n participants in the auction, p(x)
is the density function for an ESS (for 0 <= x <= 1, m = n-1)
(V1 - mV2)p(x)m + mV2 p(x)m-1 = x
Second all-pay auctions-for rewards V1 and V2 (V1 > V2) and n participants in the
auction, p(x) is the density function for an ESS (for 0 <= x <= 1, m = n-1)
mp(x)[(V1 - mV2)p(x)m-1 + (m-1)V2 p(x)m-2] = 1-p(x)m

Simulation results (red) and analytical results (blue) plotted for population size 100,
where a) 5 (left) b) 10 (right) participants are randomly picked for a total of 500 million
auctions to achieve convergence. Here V1 =1 and V2 = 0.9

We considered two different

types of objectives with
which to solve the POMDP

-regular objectives include

Reachability and its dual objective, safety
Bchi and its dual objective, co-Bchi
Parity and Mller
Quantitative objectives include, inter alia, energy objectives and expected total cost
problems

OBJECTIVES
Solving a POMDP with a coBchi objective
Exponential time algorithm to solve almost-sure winning in POMDPs with
coBchi objectives under finite memory strategies
Implement this algorithm along with simplifications which ensure good time
complexity and a fair running time (since the theoretical bound is double
exponential)
Converting a POMDP with a priority objective to one with a coBchi objective
Transform a general POMDP with a Parity Objective to one with a Parity
Objective with just 3 Priorities (0, 1 and 2)
Transform a POMDP with a Parity Objective with the above three priorities to a
POMDP with a parity objective with 2 priorities (1 and 2), which is equivalent
to a CoBchi objective
Approximately solving the Minimum Expected Total Cost (METC) problem for a POMDP
with invisible non-negative rewards
Given a POMDP with costs on each state-action pair and a nonempty subset of
the state space designated as the end-state set
The aim is to obtain a strategy ensuring almost-sure reachability for this endstate set, while minimizing the total expected cost of all the POMDP-edges
traversed

APPROACH

Evolutionary dynamics of biological auctions (Krishnendu Chatterjee, Johannes Reiter,

Martin Nowak)- http://pub.ist.ac.at/~jreiter/publications/bioauctions_2012.pdf
What is Decidable about Partially Observable Markov Decision Processes with omegaRegular Objectives (Krishnendu Chatterjee, Martin Chmelik, Mathieu Tracol)https://repository.ist.ac.at/109/32/What%20is%20Decidable%20about%20Partially%20O
bservable%20Markov%20Decision%20Processes%20with%20%CF%89Regular%20Objectives.pdf

POMDP solution with coBchi objective

Construct belief observation POMDPs for finite-memory strategies
Find the finite memory strategy in the Belief Observation POMDP
Use safety and reachability algorithms in the process
If there exists a finite memory almost-sure winning strategy in G, then there
exists one in the corresponding Belief Observation POMDP and vice versa.
Using the strategy found for the Belief Observation POMDP, construct one for the
original POMDP G.
POMDP objective reduction (parity to coBchi)
Polynomial in Time and Space Complexity
The previous algorithm extends the solution of a general POMDP with a Parity
objective in exponential time
METC for POMDPs
Various settings of the problem- one when costs are non-negative, invisible
Invisible costs costs are on state-actions pairs. Since state is never known
perfectly (only observations are), costs are invisible
One way to solve approximately is to play optimally (i.e. minimizing the
expected total cost) for a finite number of steps (epochs)
This is done till the probability of not having reached a good state goes below a
certain .
This requires us to ensure that at any stage almost-sure reachability is not
violated

ACKNOWLEDGEMENTS

RESULTS

Simulation results (red) and analytical results (blue) plotted for population size 100,
where a) 5 (left) b) 10 (right) participants are randomly picked for a total of 500 million
auctions to achieve convergence. Here V1 =1 and V2 = 0.5

REFERENCES

Krishnendu Chatterjee, Professor, IST Austria (overall internship guide)

Martin Chmelik, PhD Student, IST Austria (project guide for combinatorial game theory)
Johannes Reiter, PhD Student, IST Austria (project guide for biological auctions)

RESEARCH POSTER PRESENTATION DESIGN 2012

www.PosterPresentations.com

New algorithm to approximately solve the METC; software to implement the same
New Java tool to solve POMDPs with Parity and coBchi objectives, for the first time