Halcyon – Bidding Agent Strategy

Eakan Gopalakrishnan
School of Electronics and Computer
Science, University of Southampton.
+447760912219
eg5g09@ecs.soton.ac.uk

ABSTRACT accordingly. F0 queries have lesser conversion rate than F1 and

Internet advertising provides great revenue for publishers and F1 queries in turn have lesser conversion rates than F2. Thus
advertisers annually. The concept of sponsored search is widely depending on this, the bid value should be adjusted.
used by search engines these days and is one of the greatest
sources of revenue for them. Various ad auction mechanisms have 2.4 Distribution Capacity
been proposed and used for advertising with popular search Every advertiser has a distribution capacity which is for a 5 day
engines. This paper presents the strategies used by Halcyon, the window. For example, if within 5 days an advertiser with a
advertising agent in the recent Ad Auction competition held in the distribution capacity of 300 sells out 300 items, then the items run
University of Southampton. The strategies are based on analyzing out of stock and thus the conversion rate decreases, thereby
previous performance of agent and some parameters. increasing the cost of the advertising and reducing the return.
Thus it has to be made sure that the advertiser doesn’t sell out its
1. INTRODUCTION products too quickly, to maintain a good profit level.
In ad auctions and sponsored search, [1] the publishers get paid
on a per click basis. This is a mutually beneficial arrangement and 3. DESIGN AND ARCHITECTURE
both the advertiser and publisher takes of the risk of the uncertain
user behaviours. The position of the ad on the search results page
depends on the Cost-Per-Click rate set by the advertiser by
bidding. Usually online ad auctions are done in a sealed bid [2]
fashion. This paper presents the strategies used by Halcyon, one
such advertising agent.
In Section 2 factors affecting the bid for an ad are discussed.
Section 3 deals with the architecture of Halcyon and section 4
describe the strategies used by Halcyon. In section 5 the results of
the game are analyzed and evaluated.

2. FACTORS AFFECTING THE BID

2.1 Relevant Advertisements
The key to achieving good Click-Through-Rate, is having a top
position in the search results and as well having an ad that is
relevant to the query. This would mean that the ad is probably
what the user is looking for. The first and foremost requirement
would be to ensure that the ads are targeted according to the
query. Thus setting relevant ads is an important factor to consider
3.1 Bid Optimizer
while setting bids for queries. Depending on the content of the
It calculates the optimal bid based on parameters such as
query, the ad is set.
incremental profit percentage, ad positions obtained in the
previous games, distribution capacity and focus levels of the
2.2 Position of the ad queries.
The position of the ad plays an important role in deciding the
CTR. If the ad is not among the 5 slots available for auction, then 3.2 Ad Optimizer
the ad is never shown to the user. Thus the position of the ad if in This module sets optimal ads according to the query and its focus
the top five means that it would be visible to users and there is a level and history information of position.
probability that the user will click on it.
3.3 Spend limit optimizer
This module analyzes the distribution capacity within the five day
2.3 Focus Level of the Queries distribution window to set an optimal spend limit to avoid
In TACAA competition the query consists of a manufacturer and a showing the advertisement after exceeding the spend-limit,
component. A query consists at most two of these strings. Thus thereby reducing the chances of exceeding the distribution
other than allotting the relevant ads for the query, it should be bid capacity.
3.4 Query Analyser
This module is responsible for the analysis of the query reports
that the agent receives on a daily basis.
3.5 Sales Analyser
This module is responsible for the analysis of the sales reports that
the agent receives on a daily basis.
4. BIDDING STRATEGIES
4.1 Strategy - 1
Many of the agents that performed well in the past depended on
prediction of what other agents would bid, what user states would
be etc. which are of course good. But in Halcyon, an attempt was
made in a different direction; by utilizing the information received
from the history, it could be decided whether to increase or
decrease the bid. The key strategy was averaging information
received in a 5 day window and setting bids and ads accordingly.
The factors mentioned in the Section 2 affect the decision of
increasing or decreasing the bid set for a particular ad and query.
A lower position or no position at all would mean that the bid has
to be increased. If the query is a focus level 2 query, the bid is
increased depending on the other factors like position.
So first the conversion rate is calculated:
Conversion Rate is the total number of conversions divided by the
total number of clicks on the advertisement.
Then VPC is calculated. VPC is the expected profit from a current
visitor from a website. It is the product of value per conversion
and conversion rate. This is the cost per ad that will help you
break even in the market. Thus setting the initial bid as the VPC
and then comparing it with ICC would help keeping track of the
bid.
ICC is calculated as the cost of incremental clicks divided by the
number of incremental clicks. Number of Incremental clicks, is
the difference of the clicks on two different bids, on two
consecutive days. The incremental cost is the difference in cost for
the different bids on two consecutive days. Whenever the VPC is
lesser than the ICC it will pay to lower the bid. When it is the
other way round, then the bid has to be increased.
This is an easy way of controlling the bid by maintaining a good
profit and it can be done by just observing the previous
information that the agent has received. All these parameters
could be calculated from the history and allowing good control
over bidding without any complicated mathematical theories or
formula.

The first level of optimization of the bid is done based on the

average profit percentage received by bidding on a particular
4.3 Strategy - 3
query. It is made sure that the bid is placed only on queries which By combining the two previously explained techniques in section
give more returns. Therefore making sure that the agent doesn’t 4.1 and 4.2 we can create a newer version which would give
go on loss. maximum control over the bid. Here queries were categorized into
different classes: One class of queries A that contained the
The spend-limit per query is another factor that has to be manufacturer speciality, another B with all queries of component
considered along with the distribution capacity and the speciality, another C with F2 focus level, and then last category D
distribution window. The distribution capacity is set for a with F1 and F0 focus level, yet another category specifically to
particular distribution window by default in the beginning of hold the both component and manufacturer speciality.
every game. Thus the agent should make sure if the sales in any of
the 5 days is exceeding a certain percentage of the distribution
capacity. This is to ensure that the retailer doesn’t run out of
5. ANALYSIS
The scores of Halcyon were not quite up to the mark which it had
products and thus ensuring that the conversion rate doesn’t come
set during its testing with other agents. Probably the different
down. The spend-limit per queries also depends on the focus level
server versions would have been a factor and the competitors as
of the query. F2 queries should be set a higher spend-limit since
well.
the conversion rate for F2 queries are comparatively higher than
the other level queries. Table 1 Scores and Capacity of Halcyon in the competition
The campaign spend limit is again set according to the different Game Capacity Score Game Capacity Score
distribution capacity levels. High capacity agents have more
campaign spend limits and low capacity agent would have a low 640 S 35190.3 641 M 46184.6
campaign spend limit. 642 M 44202.7 643 L 38058.9
The Bid value is initialized as the Value Per Click if the average 648 L 40409.4 649 M 48272.2
CPC set for that query was 0, otherwise the bid value would be
the average CPC. This is done to ensure that we don’t bid too 650 M 58170.5 651 M 39213.9
high or too low. 652 M 46033.7 653 L 33404.5
654 S 40579.2 655 M 46260
4.2 Strategy – 2
660 M 47603.1 661 L 60974.2
This strategy was taken from Google AdWords [3] tutorial. ICC
and VPC are important factors in deciding whether to increase the 662 S 25368.6 663 M 38100.2
bid or decrease your bid. VPC depends is a function of several Initially, the first three games were played with strategy 2, then
other parameters like revenue, profit, expense, CPC, maximum one game was run with a test version of the ICC strategy with
profitable cost per acquisition that can be made out of selling one slight modifications, and then from the 5th game Strategy 3 was
product. used though with lesser emphasis on ICC.
In the test conducted before the game, Halcyon with a H capacity However, it would also have been useful if bidding was done on
had always scored an average of 60000 in the latest version of the just a selection of queries than on the full spread, most champion
server against competitors like AstonTac, TACTex [4], Brown agents used this strategy. Spend-limit optimization could have
University’s Mega [5].; all of which were top scorers in previous been improved.
years competitions.
Table 2 Scores and Capacity of First and Last ranked agents 7. CONCLUSION
in each game In spite of the shortcomings Halcyon did a good job, never went
on losses and ended up in 9th position scoring very close to the 8th
Game Highest Capacity Least Capacity rank holder Nightmare. In game 662 Halcyon’s connection was
640 60969.3 L 34346.86 L reset and it couldn’t bid for around 12 days, but then from that it
scored 25368, with low capacity. The improvements, if
641 57256.3 L 29473.12 S implemented would boost the profit considerably.
642 63285.2 L 27962.42 S
643 66766.5 L 35222.06 S
8. REFERENCES
[1] Lahaie, S., Pennock, D. M., Saberi, A., and Vohra, R. V.
648 59452.5 M 25720.38 S Sponsored search auctions. In Noam Nisan, Tim
65712.1 L -189136.9 M Roughgarden, Eva Tardos, and Vijay V. Vazirani, editors,
649
Algorithmic Game Theory, pp 699–716. Cambridge
650 73995.9 L 918.13 S University Press, 2007.
651 58848.4 M 1577.57 L [2] Coppinger, V. M., Smith, V. L., Titus J. A., Incentives and
behavious in English, Dutch and Sealed Bid Auctions,
652 59882.4 L -185961.5 L
Economic Inquiry, Vol18, No.1, pp 1-22, 1980.
653 49279.5 M -127052.8 M
[3] Google AdWords Bidding Tutorial.
654 47739.3 L 120579.41 M http://www.youtube.com/watch?v=jRx7AMb6rZ0
655 49999.1 L -72320.73 L [4] Pardoe, David, Chakraborty, Doran, and Stone, Peter.
TacTex09: Champion of the first Trading Agent Competition
660 54863.5 L -37765.7 L
on AdAuctions. Technical Report AI-10-01, Department of
661 60983.1 L -91785.16 M Computer Science, The University of Texas, 2010 (2010).
662 59648.6 L -186091.3 L [5] Design of Agents for Bidding in Sponsored Search Auctions.
http://www.cs.brown.edu/courses/csci2955/ (2010).
663 44721.1 M 0 S
[6] Aggarwal, G, Feldman, J, Muthukrishnan, S, and Pal, M.
On analyzing the game data, it was found that the distribution
Sponsored search auctions with Markovian users. In ACM
capacity had exceeded many times by 20%-40%. This drastically
EC-08 Workshop on Ad Auctions, Chicago (2008).
reduced the conversion rate. In spite of controlling the spend-limit
to avoid excess conversion, the logic didn’t work well.
9. BIBLIOGRAPHY
In games where Halcyon was given high distribution capacity, it [1] Edelman, B., Ostrovsky, M., Schwarz, M. Keywords,
performed poorly when compared to the test results; though it did Internet Advertising and the Generalized Second-Price
well in low and medium capacity situations. Probably the logic Auction: Selling Billions of Dollars Worth of. American
used for spend-limit optimisation wasn’t giving desired results. Economic Review, 97, 1 (2007), 242-259.
[2] Park, S., Durfee, E. H., and Birmingham, W. P. Use of
6. IMPROVEMENTS Markov Chains to Design an Agent Bidding Strategy for
One of the strategies that were looked into but not implemented Continuous Double Auctions. Journal of Artificial
was building a user population estimator [4] [6] which would Intelligence Research, 22, 1 (2004), 175-214.
specifically help the agent to determine how many users search for
[3] Stone, P. Multiagent Competitions and Research : Lessons
different types of queries as done by TACTex [4]. TACTex’s user
from RoboCup and TAC. RoboCup 2002: Robot Soccer
model uses total number of queries for each query type to estimate
World Cup VI (2003), 224-237.
the future population; particle sampling is done using a re-
sampling filter wherein changing states are tracked using weighted [4] The Search Agents. http://www.thesearchagents.com/ (2010).
samples called particles. The particles reflect the user population [5] Garg, D., Narahari, Y., and Reddy, S. S. Design of an
and every day a new set of particles are created from the previous Optimal Auction for Sponsored Search Auction. The 9th
ones. However, owing to time constraints and the level of IEEE International Conference on E-Commerce Technology
mathematics involved, the user population model was not be and The 4th IEEE International Conference on Enterprise
implemented. Markovian Chain [6] user state modelling was also Computing, E-Commerce and E-Services (CEC-EEE 2007)
discussed for implementation. Estimating user state at any given (2007), 439-442.
point in the game would help identify how many users are left in
the Is Searching state and are more likely to make conversions.