CS-TR-0002, May 2006 AN OPTIMAL STRATEGY FOR YAHTZEE James Glenn, Loyola College in Maryland Computer Science Department 4501 N. Charles St Baltimore, MD 21210-2699 Abstract, Solitaire Yahtzee is a complicated game in that the best strategy is not obvious, as it is for simple games like tic-tac-toe, however it is not so complicated that computing the optimal strategy is infeasible. The elementary combinatoric and graph theoretic techniques used to compute the optimal strategy for Yahtzee are described. Once the strategy has been computed, the same techniques can be used to perform statistical analysis of it and other nor-optimal strategies. Some results of this analysis are presented. ‘Keywords: Retrograde Analysis, Computer Games, Computer Strategies and ter at Lafayette College, Easton PA.An Optimal Strategy for Yahtzee James Glenn Department of Computer Science Loyola Colleg: in Maryland 4501 N. Qares St. Baltimere, MD 21210 eamil: jglerécs.loyola.edu May 10, 2006 Abstract: Solitaire Yalitzee & a complicated game in that the best strategy is not obvious, a it & for simple games lle tictartoe, however it is mot so camplicated that computing the optimal strategy is infeesible. The elementary combinatoric and graph theaetic tedunisues used to compute ‘he optimal strategy Sor Yahtbae are described. Once the strategy las been computed, the sazne tediques can be used to peafoum statistical analysis of it aml other nomoptinal strategies Some results of this analysis are mented. 1 Introduction ‘Many games, far example tio-tac-toe, are simple enough that: most young chi dren can master the strategy. Other games, far example chedears and chess, have stich lage state spaces that itis inconceivable that a computer could cam pute the optimal strategy; for thee games we must resort to complex hewis- tics, Ih between stands Yahtzee, a game for which the optimal strategy is not immediately obvious, but can be canputed given even modest computational power, Later we give some reailts of this computation; similar work has boen undertaken by Tom Verhoeff [2] (happily, our computations ended in the same results). ‘Yahivee is a game of sll and chance marketed by Hasbro (a similar game called “Yacht” is described in Hoyle’s Rules of Games [1)). Yahtaee is played \with five sbesided dice. ‘The goal is to roll the dice to make hands resembling poker hands. Different hands are worth different armoxmts of points. The goal of a two-player game is to end up with more points than one’s opponent. Jn this paper we describe how to compute the optitual strategy fax the solitaire game in which the goal is to maximize one’s king term average soore.2 Rules A game of Yahtace lasts thirtean roms. Ench round begins with the player rolling five sbesided dice, After the initial roll, the player may choose to reroll any of the dice, After the first revoll, the pyar is givm a second chance to reall as many’ of the dice as desired. After exhausting the revlls (or choosing to keep all five dice), the player must clasify his five dice in one of thirteen categories, Each catesary may be chosen only ance per game, and ance a roll has ban peed in ane category it may not be changed. Points are awarded based on the roll and the category chosen. ‘The thirteen categories and the comesponding scoring rules are as follows: ones scores one point for each die showing one pip: twos, threes, fours, fives, sixes similar to ones; «¢ three of a kind scores the total pips showing on all the dice if at least three show the sume rumber of pips, zero otherwise; ¢ four of a kind similar to three ofa kind; ‘¢ full house scores 25 points if the roll has three dice showing one rank and ‘two dice showing another, 0 otherwise; ¢ small straight scores 30 points if four dice show consecutive values, 0 otherwise © large straight scores 40 points if all five dice show consecutive value, 0 otherwise ‘¢ chance scores the total of all dice (with no additional restrictions); five of a kind (or “yacht” or “Vahtaee”) scores 50 if all five dice show the same munber of pips, 0 otherwise. Additionally, bonus points are awarded in certain Grounstances. Playas ‘who some at Ieast 63 points in ones through sixes (the “upper categatie?"), eam an additional 35 point “upper bam”. "Thaw is a variation of the ruks that allow players who scare 50 in Yahtare to eam a “Yahitace Bors? of 100 points for each subsequent round that ends with all five dice matching, The player mist sill score the rol in some other categary, and in fact in some cases ‘cat use a second Yahtzee to earn the full score in stall straight, kuge straight, or fll howe (a “Yahtzee Joke”). ‘The strategy for Yahtuoe is more complex than the strategy for poleerin this sere: while the choioss you make in ane poker hand do no affect your chanoes of sgptting a good hand in the next, your choioss in one round of Yahtzee do affect ‘Your chances for scoring wel in the next round: if for example, you choose to sore 111.23 as three of a kind (8 points), you may not later soore 45.6 6.6 there (27 points).We will detamiine how to figure out the best strategy for Yahtzee. Our strategy will tell us which dice to reroll when we have the opportunity, and which category to put our rall in at the end of each round. 3 Method ‘To figure out the best strategy, we envision the game as a graph. Vartices represent states of the game. At some states the player has a choice as to which state to go to next. From those states there will be edges to the subsecnuent states, In our pictures, we join the edges with an are. At other states same random event occurs, The subsequent states will again be indicated by outward edges in the graph. This time, we put weights on the exes indicating the probability that we end up in the destination state, Tp ilkestrate, consider a simpler game, This game is played with two fair coins The player flips both coins simultaneously. After the first flip, the pkyer muy flip either, none, or both of the coins again. After refipping, the player ‘wins 1 dollar if both coins came up heads anc loses 1 dollar if ane enin shows heads and one shows tails; no money changes hans in the remaining, case. ‘There isa vertex in the graph for each state the game can be in. There isan exge from state u to state v if there is an action that will move the game from state u to state v. In this coin taxing game, there are four kinds of states: ane state with indegee zero representing the beginning of the state; three states representing the tive possibilities for the autoome of the fist flip, six states representing the posites for which coins to keeps and three states with ‘uidegree zero representing the three possibilities fr the outcome of the second ‘ip. When drawing the graph, it is canvenient to group each kind of state into column, Edges fitm the initial state to states in the second column and fram the third cohmm to the final states represent the outoome of a random event (flipping the coins); thee edgesare labelled with the probability of the outeame, ‘Edges from the second column to the thin! cohumm represent choices the pkyer ‘can make; in the picture the edges representing all the choices a player hes at. a given state are joined by an are. For exp, in the initial state the player wil flip two coins. There is a sce that the toss will result in two tally a $ chance that the toss vids a head anda tail, anda 4 chance that both eoins land heads up. Having tossed ‘two tails, the player has the choice of reflipping the tail, reflipping the head, or hoxping both coins IF the player choeses to rip the tal, ther isa # chance he will end the game with ovo heads and a 4 chance he will end the gaune with cae head and one tail. The complete graph for this game is in Figure Given the graph, we can compute the strategy that wall maximize average winnings, and figure out the average winnings if the player follows that strate ‘Todo s, we will asign a value V-AL(u) to every state u in the graph. Since the graph has no cycles, we can proceed in arder of a reverse topdlogical sartz all vertices with no outgoing edges are final states of the game and thar values can be computed immediately fim the table of payouts given the the gameFigue 1; State diagram for a coin tossing game, description; the values far other vertices can be computed once the values have been computed for all vertices to which they have outgoing edges. ‘There are two kinds of such vertioas: vertioas: whew a random event oocuns and vertices wha the payer makes a choice. ‘The value of a state u in which a randan event occas is - DY we,v)-VALe). (idee “The value f a state w in which the player hassa choice as to what state to move to next is max, VAL() BEVAO and the best chcice to mille is to move to the state that realized the maxiuum value ‘Mast of the strategy for this game is obvious the player should keep any heads obtained by the first flip, But when the first flip yields two tails there is a decision to be mack: either stand pat and take the draw or relip both eoins in the hopes of winning (the option of reflipping only one cain is dearly infexior because thaw i no chance of winning but there is a chance of losing). From the computed values for each state in Figure 2, we see that, having flipped two tails, the player should stand pat. and take the draw rather trying to go for a win. A little algetaa shows that, keeping the other outcames the same, if the payout for two heads is higher than two than the best strategy is to refip both coins.Figure 2: Values of states for a coin tossing game. 4 Yahtzee State Graph Campating the optimal strategy for a complete 13-round game of Yahtzee is similar to computing the optimal strategy for the ooin tossing game. Once ‘we figure out all the states of the game and all the pessible actions and their ‘consequences, we aan work badswards from the final states (where all categories are used) towart the initial state and figure out, what the ileal strategy is and ‘what car average final soare will be when following that strate 4:1 Structure of the graph ‘Thee will be several different kinds of states in our graph to represent the steps in each tum just as thee war in the graph for the coin taxing game, The gaph for the coin tossing game has states where the player choowes which coins to keep; the graph for Yahtzee has states where the player choos which dice to keep or what category to score a roll in. The graph for the coin tossing game has states where the player fips coins; the graph for Yahtzee has states where the player rolls the dice. Jn addition, states in the Yahtzee graph must keep track of some of the histary of the game, sinoe Yahtzee strategy depends on what: has happened in previous tums, That was not a concer in the coin tossing game becuse there ‘was only one round. ‘We cuuld keep track of all the information about the state of the games that is, for each category what, if anything, has been soared in it, Thee are seven 5possibilities far what has bean soorad in each of categories ones through sixes dither the catesary is empty, or ishas been used farzero, ane, two, three, four or five of the given dice. There are 28 possibilities for chance, three of a kind, and four ofa kind (umised, zero, or anywhere from 5 to 30 points). There are three possibilities for each of the other categories (unused, zero, or nonasro). So the total munber of ways to mark the soarecard § 7° + 28° «34 = 200, 193,098,688. Recording information about that many states uses a prohibitive amount of memory, not to mention time, Obviously we must be more carefil. Observation 4.1 The best strategy does not depend on what has been scored ir cach category (with one caveat), but rather on what categories are stil available. As an extrane exampk, comider what to do if Yahtzee is the only categary Jeft open and after two rolls yau have 5 6 6 6 6. No matter what your scare is the best thing to do is retul the 5. In doing sp you will scare, on average, 4B more points, So instead of keeping track, far each state of the game, what the expected] total spore is, we keep trad of what the expected soar is for the ranaining tums, The average score for the total game woul then be the average score far the rest of the game starting at the initial state, Observation 4.2 The optimal strategy i sensitive to the total score a the ‘upper categories (but not to the distribution of that total among the used cate- gories). This is the caveat to the first observation. Becanse there is a 35 point bors awarded for totaling at least 63 in those categories, the best strategy does depend on the cuxent score in those categories. For example, suppose the cxly renaining categories are ones and Yahtzee, we have finished our next to Jest tum with 5 666 6 and our upper totalis 35. Th that: case we don’t stand to Jose much by sooring za in ones and trying for Yahtzee on the next tum. On the other hand, if our upper total had been 62 then we would want to seone zero in Yahtzee because the we have about a 98.5% chance of rolling a one caring ournext tum and eaming cur 35 paint bams but only about a 4.6% chance of rolling a 0-point Yahtzee, Note, however, that itis the total scare in the upper section that matters and not the distribution of that soore (zero ones and four fours is the saune ass four anes and three fours). So the information we need in each state consists of (1) far each category, whether it i used or unsed (when playing with Yahtzee Bomses, Yahtae is a tristate category: umsed, zero, nonacr0); and (2) the score in the upper section, Thereare 2! pessbilities for the former and 64 for the latter (anything ggeater than 63 is equivalent to 3 since we have already earned our bonus then), so we need 2° = 524,288 states for the beginings of tums (50% more for Yahtvee Bones), better than the original analysis by a factor of about half amilion. These 21° states are the states that represent the start of a tum. Bach tum also neeck states to represent the result of rolls and choices of which dice to keep. The complete graph then consists of 2° widgets, where each widget is constructed as follows:steontsean _ NEBRONES} ST ne) oO a come Wika GiSe Tio sand come wntohaton Wediie(Cigl Se) Figure 3: A small partion of the Yahtzee graph showing some of the states in three widgets and some of their interconnections, See Section 5 for an explana tion of the notation, @ thaw és a sing entry point into the widget that represents the beginning ofa tums (2) that entry point has ecses to states representing the outcomes ofthe initial rll of the dices (3) those states have edexs to states representing the choices the player cum make of which dice to keeps @) exlges from group (3) go to states representing the result of the fist reroll (©) the next group of statesis similar to group (3); and (© the last group of states is similar to group (2) and represents the outcome of the final roll, States in this group are the exit points of the widge:; their outgoing exlges go to entxy points of other widgets or final states of the game. ‘There are no ecees between different widgets. Note that the outgoing edges fram the exit states may differ from widget to widget depending on what categories have been used.‘Table 1: Pattems of dice and mumber of ways to choae kept dice, 42. Sizing up the graph ‘To cout the number of states and exlges in the graph, we nead to Inow the size of each widget. There is clearly one entry point, Counting the manber of states in the other groups nayuires some combinatorics, Each roll coresponds to a bag (or nniltiset) containing 5 munbers chosen frum {1,2,3,4,5,6}- Tn general, there ane ("17") different siaon bags chosen from k possible elements, so thaw are (1°) = 252 posible outcomes when rolling 5 Gsided dice, and so 252 states in gous (2), 4), and (6). Thaw is a state in each both group (3) ‘ann (5) for each partial pall of the dice— bags of size 5, 4 3, 2,1, ar 0. Thee ae (9) + (+O) +@ + GQ) + @ = 462 of those. So in total, each widget, contains 143-252-+2-462= 1681 states ‘Mare impertant are the number of edges betwen thaw states because to figure out the expected value far a given state we need to examine each edge Ikaving it. It is not the case that each state in ane group nist have an alge to ach state in the next group: for example, # you choose to Iiep 234 it is impasitle to end up with 6 6 6.6 6 after rerolling, Certainly each state in groups (2) and (4) has no mare than 2° = 32 outward ‘xlgessinge there are five dice and for each one you can choose to keep it or reroll it, For many states the mumber of outward edges is significantly fewer. For example, if you roll 11 22 2 then rerlling the first die & the same as rerolling the second die, and zerolling the third isthe sume as rerolling the fourth or fifth. What your decision really is in this case is how many ones to reroll and how many {wos to reroll. There are 3 choiees of how many ones (zero, one, ot both) torerdll and 4 choices ofhow many twas to rall, 80 12 distiner choices of which dice to reroll. An initial roll of 3 3.5.5 5 woull be analyzed the same way — the nuunber of distinct chaioes of which dice to rerall depends an the pattern of the dice, Table 1 surmmatizes all the posible pattems (nothing, one pair, two pair, three ofa kind, full house, four ofa kind, five ofa kind), how many times these pattems occur, and how many’ ways there are to chocse which dice to keep for those patterns, So the total mmber of eens fican the states in group (2) (and (4)) is 632+ @- 21+ @)-18+60-16+30- 12+ 30- 10+6-6 = 4568,[Die tep ‘Table Kept dice and number of outcomes of rerolling, Recall that the states in groups (3) and (5) represent the partial rolls that are left after we decide which dice to reroll. The number of outward edges for these states depends an the number of dice that we have decided to keep. This is summarized in Table 2, ‘So there are 1-252+6 - 125+ 21-56+ 56-21 + 126-6+ 252-1 = 4368 edges total from the states in group (3) and the sare number of edges from the states in group (5). (Note that this is the same count we got for the edges from group 2) to group (3) since the edges out are esentially a mimor image of the edges in), Finally, there are ‘edges out of the entry state and at most 13 edges out, of each exit state (one for each category that could be chosen). So the total number of edges in each widget is no more than 252+ 4-4368+13-252= 21000. ‘There are 2!° widgets in the graph, so around 11 billion edges in the graph. ‘Thisis a number that we can reasonably expect to examine in an hour or $0 of CPU time, but it would be helpful to reduce the size of the graph even mar. 4.3 Unreachable States We can recive the manber of widgets substantially since some states are we reachable. For example, it is impossible to have an upper total of 4 if the coaly upper categories used are threes and fives. Figuring out which states are reachable can be solved with dynamic programming, Let S beasubset of {1,2,3.4,5,6} (indicating which of the first six categories have been sed) and n bea natural munber s0 that 0 1 For S#andn> 1, Fins) = {tw HAE SKE MIR aNke Sm A Ren—he,S— (2) After computing R(n, 5) for all approggiate n and S, we find that 1260 of the 4006 values are false, reducing the munber of states and exiges by mare than a quarter,5 Computation We are now ready to desaribe the computations needed to comupute the optimal strategy for solitaire Yahtzee. First, however, we must introduce some notation, Cis the set of categories, {cnes, twos, thnees, fours, fives, sixes, three of a kind, four of a kind, full house, small straight, large straight, chance, Yahtzee}. vill stand for elements of C, C for subsets of C. A Yahtzee state i a triple (C,m, f) where C'is the set of categuries already marked, mis the total in the upper section, and f is aflag indicating whether the next Yahtzee will earn a Yahtave Bonus, that i, whether Yahtzee was marlied ‘with 50 (the flag i, of course, unnecessary when dealing with the gamein which no Yahtzee Boruses are avarded). Yahtave states will generally be denoted by the letter § and the set of all possible Yahtzee states will be denoted by S. Rig is the set of outoomes when rolling i sided dice, ‘The set of rolls we could be left with after choosing which dice toleep is Ry = RooURieUR2U RegURigURog. 1 will stand for dernents of Reg and r’ willstand for dements Of Rg (16 for keeps”). 1. denotes the empty roll. -E(5) is the expected future soare (or “potential”) when starting a tum in state S, E(S.r,n) denotes the potential of state S with roll and nrerollstermaining, (That is, the potential in the middle of tum bagun in state § after one of the ralk of the dice has came up showing r.) -E(5,1°,7) denotes the potential of state S having chosen to keep dice r! with nredks ranaining, P(r! 1) denotes the probability that, havingkept r/, we reroll the ramair- ing dice and end up with r. (5, 7,0) denotes the soore obtained by scoring the roll r in category ¢, given previous game state S. This score will inchide any Yabitzee Boris, but not auy Upper Barus, which we imagine as being award at the end of the game instead of the tum in which the upper total first exooxds 02. ‘u(S,7,0) denotes the additional upper score eamed by scoring roll rin cate egory ¢. Naturally, u(S,7,0) = #(5,75¢) for upper categories and is 0 for lover categories. J(Syr,€) denotes the new value of the Yahtae Bonus flag after scoring 1oll in categay c starting from state 5. f(S,7,¢) i true if and only if either S= (Cymtrue) for sane C and m, ax both = Yahtzee and (5,746) = 50 (thats, we are digitle fr the Yahtave Bomasif we wae previasly digitle for itor if we have just sored 50 in Yahtax). n(Syr5c) denotes the new state after starting in state S and scoring r in category 6. (Com, 756) = (CUfejsm+ul(Cm, f)9r5 6) S (Cam, Dsrs0))« ‘The Procedure for computing the optimal Yahtzee strategy begins by cam- puting the potantial far each final state, If C = C, E(C,m, f) is 35 if m > 63, Ootherwize. Forstates at the end of tums, the expectation is given by the madman overall the used categories c of the score in c plus the potential of the next state: E(Gm, fist,0) =mepxlo(Gm, )ot>6) + En(Gm, 756). For states where the player has just chasen which dice to keep, the expec: tation is given by the weighted average of the expectation of all the states we could be in after the reroll: for n = 1,2, AS.) = YO Pe! 42) -EGynn-D- Re For states representing the result of the initial roll or the first reroll, the ‘expectation is given by the best expectation of the states we could end up in after choosing which dice to keeps for n= 1,2, (S157) = mpc B(Ssrsn, Finally, for states at the beginning of a tum, the expectation is given by the weighted avange of the expectations of all the states we could be in after the initial rol AS)= YD PU 1)-E(,7,2). PCRs ‘The arder of computation is based on how many categuties are filled in a state. By evaluating states with more categories marked before states with fewer categories marked, we esure that we have all the information needed to compute E(S) when we get to S. 6 Implementation Details and Results We have cued the algorithm and associated data structures in abeaut two tho sand lines of C+ code, Only minor modifications need to be made to allow for minor vaiations in scoring rules such as Yahtzee bonuses. After abet one hour of computation timeon an entry-level CPU the mandinmum posible average sare is compaited to be approximately 245.87 without Yahtuee bamuses ar jokers and BA.59 with. A naive implanentation would attempt to keep track of the expected value from each state in each widget. At about one billion states and eight: bytes per eapected value, we would neal SGB of storage. While this is not a problem for today’s supercomputers, it would surdy tax the virtual memory systems of todky’s desktop computers. Fortunately, we do not neal to keep all this information online at once. ‘Note that in each widget, the computation for states in group 6 depends on other states in group 1 in other widgets. The computations for states in groups 2 through 5 depend only on states in the same widget. So, anve a widget is done, we can forget all the information that was computed for it (except for un‘what the ideal strategy is, but that’s lew that one byte per state in groups 2, 4, ane 6, and furthermore anly needs to be written and never read). The only information we neal to keep around for good is that about the 2!° states in soup 1, or about 4MIB (or (MB for the game allowing, Yahtzee boruses) ‘The 4MB table's also good enough to play a gasne using the optimal strates: At the beginning of each tum we will need to recompute the optimal strates for the cument widget. We computed the optimal strategy for 2!° widgets in an hour, 50 the strategy for a single widget cam be computed in a fiaction of a second. Tt should be noted that 4MB of data can be stored easily in the main memory of today’s desktop computers, but it i still a significant chunk of data for a handheld device. We may still wish to have a handheld device play nearly optimal Yahtvee; in the next section we evaluate some strategies through which swe might achieve this, 7 Comparison to Other Strategies Tt is posible to use the graph approach to evaluate the performance of any other Yahtuee strategy by working forward frum the initial state of the game, For the purposes of this evaluation, we view a strategy as a pair of functions, Si (S* Bog x {21} > PL1L,23,4,5}, and fo :(S * Rae) Ce fi tells us, Elven a state of the scomaard, a roll of five dice, and a munber of rerolls let, hich dice to keep. fe determines, given a state of the scorecard and a roll, which category to score the roll in,” (Note that a human phyer’s strategy may not be representable by functions faced with the same situation, a human may randanly choose a move beser] on his ar her mood.) ‘To vate the expected score when following a deterministic strategy, we compute, fir each state, the probability of raaching that state with a cartain score, For this value we waite P(S,n) far states at the bgginning of tua and Prot(Ss15Myk) Of Phoep(Ss79M) for states inthe middle (for the probability of having the scorecard in state S with r showing on the dice, a total scare of n and k rerolls remaining), We always start out with za points at the beginning of the game, 0 for $ = ({},0, false), P(S,0) = 1 and P(S,n) = Ox all'n > 0. E we know P(S,n) for a beginning-oftum state S, we can compte the probability of being in state 5 and having rolled r in the initial rol: Prote(Syrsn52) = P(S,n) + PUL +1. ‘The probability of choosing to hexp rafter our initial rol is then then the sumn overall 1ollsr such that the strategy tells us to kegp rs Peep('Sy1/ 1,2) = > PrailS,1);2) Pe Rag Cnr Knowing the probebility of ending up with each silxull after ane choice allows us to compute the probability of facing our second dhcice with roll r: Proat(S,75M1) = J Preep(S,1!,1%52)+PO! € Rx). cmTae Ege Some [Sid Deviation “Yahtzees Only” Trae O817 “Yahtvees and Straights: 202.51 65.90 Greedy 218.05 A637 Rational Yahtzces 219.86, 6599 Heuristic 2a.6T 6090 ‘Better Heuristic (244.87 5739 Optimal 25450 5061 ‘Table 3: Avaage scores for varias strategies. ‘Repeat for the second and third rolls to obtain Py<(S,r57,0), the probabil- ity of ending a tum in state 5 with n paints and the dice showing r. We then compute c= (5,7), the category the strategy tells us to score r in, and add. Prou(S;r5n,0) to Out running total of P(n(S,r,¢),n + 3(5,¢,7))« After working through the entire graph, the probability of ending the game with score n is given by @ r= [ricerpn-ay+¥ remy]. a se(erue fale} Examining this for all values of n yidds the expected final score as well as the standard deviation of the final soore. ‘The procs is computationally expensive: |weimnst compute P(S,n) far each cambination of on the order of a million states: ane] 1576 possible socres (the maximum score with Boms Yahtaoesis 1575). The table holding P(S,n) requires several gigabytes and the computation currently takes about 24 hours batt nonetheless has ben performed for several different strategies, summarized in Table 3. ‘The strategy dexibed as “Yahtares Only is one that might. be followed by a player who desires to maximize the total Yahtzee Bonus. It dictates that when chocsing Which dice to keep, we keep the anes that came up most often, ‘Dies are broken in favor of highest open category; then by highest rank, When choosing which strategy to soore a roll i, it dictates a sroady strateey; with ties broen arbitrarily. The next strategy (“Vahtzoos and Straights”), 3s designed ‘with the sane goal in mind, but is tempered by the decision to keep a straight if it has been rolled before the tum’s end and the categpry is still open. Even so, the only time this strategy dictates that we chose to keep dice to go for straight instead of Yahtzee is when we already have a stnall straight and both small straight and lage straight are empty. “Rational Yahtzes” is a further refinement of the Yahtzecminded strategy that will, among other things, lap adoubleton instead of a tripleton if the doubleton’s upper category is open and the tripleton’s is not. ‘The est of the strategies are based on estimates of E(S) that can be came uted without using too nich storagespace (remember, the table oF exact values Bof E(S) occupies several megabytes). The greedy strategy uses E((S) = 0, s0 that it sees the potential of each state the same, 90 its goal is abwagss to score the ‘most: points on the current, tum, with ties broken in favor of the more difficult categories (for exaruple, it will fll fouraeekind before threeofekind). ‘The heuristic strategies attempt to more accurately estimate E(S). The first is basa an the expected scores for each category when following the optimal strategy as computed by Tam Verhoef [2). £'(S) in this case has three compo- nents. “The first is the sun of the expected score for each open category. The second is the expected upper bonus, which is computed by assuming that the remaining upper total is normally distiibuted with variance given by the sum of the variance of the individual categories. OF course this is a gross approxima tin since the scares in the upper categories are not independent and so thar ‘variances do not add, but the parfornance is impressive nonetheless. The third component is the expected Yahtzee bomus, which is estimated to be 0. ‘The second heuristic uses larger tables than the previous one but ones that ae still much smaller than those needed by the optimal strategy, One table is obtained by computing the optimal strategy for a simpler version of Yahtage that uses anly the upper categories and awards no upper bas. The other table is obtained the same way but uses only the lower categories, £7(S) is then composed of four components the expected upper total, which & obtained using the first table, the expected lower total, which is obtained using the second table, and the expected upper and Yahtzee bomses, which are computed the same was as for the frst heuristic. 8 Future Work Note that this does not motmize the probability of winning a two-player game, Foreampk, suppose your oppanent’s only open categury is anes, he has already eamed the boms, and he is 40 points ahead of you. Suppese your onby open categories are Yahtzee and chance and you end your next to Jest tum with 11 112, When playing to macimize your seore you would scare zero in Yahtase andl hope for a good roll in chance (the average svare for chance when it i the lest category left i 234 while the average for Yahtzee is 23014). However, if ‘you svore zero in Yahtace you have no chance of winning the game since you head to soore 50 in Yahtzee to overcome the 40 point deficit. IF you score 6 in chance you still have a chance to win by getting a Yahtzee in your last tum, ‘eis not practical to use the method we have for finding the optimal solitaire ‘Yahtace strategy to find a strategy that will macimize the chance of winning a two-pkzyer gamne, for the amount of information that must be kept i too great. Each state would need to reoard the state of the player's scoresheet and thestate Of the opponent's scoresheet and the mumber of points curently separating the two. We have alway seen that there are about 2! states for each scoresheet, ‘The maxinmm sca without, Yahtase bomses is 375, s0 for difference in sare wehave 751 passbilities (behind by 375 to ahead by 375). Examining reachable states will hep us whittle down the mmbas (for example, it is impossible for uane Player to have filled 6 categories whik the other player has filled 10, and it is impossible to be 300 paints ahead after one tum), but as a gross estimate we see that there are on the order of 2 states to contend with. We currently have no means of dealing with either the time or the space that processing so many states would recite. ‘Things become simpler when you consider asortof “bind” two-player gune. Jnstead of taking tums playing and being able to monitar the state af your op- ponent's scoresheet, suppose that: you and your opponent play games in separate rooms and upon completion announce your scores. Tn such a case you would not be concerned with your opponent’s scoresheet or the amount: by which you Jed or trailed since you could not obtain that information, It should be posible to figure out a strategy that will prevail more often than not over someone play- ing the optimal solitaire strategy. In this case, your curent score is important, so the mmmber of states considered will ke dramatically, but the computation should still be within the capabilities of today’s machines. References [1] Albert H. Morehead and Geoffrey Mott-Siith, editors, Hoyle’s Rules of Games. Signet, second revised edition, 1983. [2] Tom Verhoef, Optimal solitaire yahtuse player: Thivia. Jittp:/ Awwwpa.win.tuenl/misc/yahtzee/triviahtml, 1999. Visited Au gust 13, 2001.