Mass Mahjong Decision System Based on Transfer Learning
Yajun Zheng
School of Computer Science, Beijing Information Science
and Technology University; Perception and
Computational Intelligence Laboratory
ABSTRACT
In this paper, we propose a transfer learning to solve the prob-
lem of lacking in data and the difficulty in constructing models
effectively, which is typically represented by Mass Mahjong in the
field of imperfect information. Design and implement the Mass
Mahjong Discard model based on transfer learning. The previously
well-trained Blood Mahjong Discard model on a large dataset is
migrated to Mass Mahjong Discard model in a similar domain. In
the subsequent model optimization, a self-play based approaching
is used to improve the Mass Mahjong Discard model. The experi-
mental results show that the transfer learning-based Mass Mahjong
Discard model performs well in the situation of less data, and can
fit the Mass Mahjong Discard rule. And the model won the sec-
ond prize in the Mass Mahjong event of the National University
Computer Gaming Competition in 2021.
CCS CONCEPTS
• Computing methodologies; • Machine learning; • Learning
paradigms; • Multi-task learning; • Transfer learning;
KEYWORDS
Artificial intelligence, Transfer learning, Mahjong, Machine learn-
ing
ACM Reference Format:
Yajun Zheng and Shuqin LI*. 2022. Mass Mahjong Decision System Based
on Transfer Learning. In 2022 the 6th International Conference on Innovation
in Artificial Intelligence (ICIAI) (ICIAI 2022), March 04–06, 2022, Guangzhou,
China. ACM, New York, NY, USA, 6 pages. https://doi.org/10.1145/3529466.
3529485
1 INTRODUCTION
With the development of deep learning, more and more research in
the field of computer gaming has been broken through. In particular,
the emergence of AlphaGo [1] has allowed computers to beat hu-
mans within the field of Go based on deep learning methods, which
brings the application of deep learning to the field of computer gam-
ing to an unprecedented level. In the field of competitive two against
one, DouZero [2], referring to the ideas of AlphaZero [3], combin-
ing Monte Carlo algorithm [4] with deep learning, has reached the
level of human players in a few days by learning from zero through
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full citation
on the first page. Copyrights for components of this work owned by others than ACM
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,
to post on servers or to redistribute to lists, requires prior specific permission and/or a
fee. Request permissions from permissions@acm.org.
ICIAI 2022, March 04–06, 2022, Guangzhou, China
© 2022 Association for Computing Machinery.
ACM ISBN 978-1-4503-9550-2/22/03. . . $15.00
https://doi.org/10.1145/3529466.3529485
Shuqin LI*
School of Computer Science, Beijing Information Science
and Technology University; Perception and
Computational Intelligence Laboratory
self-playing without any human knowledge. In Riichi Mahjong,
Suphx [5], based on deep reinforcement learning, is currently the
strong Japanese Mahjong AI that can reach levels beyond most
top human players on the Tenhou Mahjong platform. Although
the above research on imperfect information games has achieved
progress of closing to the real level of human players through deep
learning in some games, imperfect information games are diversed
in type, and the differences in the rules of the game can lead to the
differences in the data for training, model inputs and outputs, net-
work structure, and validation methods. Only the game of Mahjong
in China has various from National Standard Mahjong, Sichuan
Mahjong, Riichi Mahjong, etc.. The rules of Mahjong are inherently
complex, and there is also some variability in the rules between
different Mahjong games. And the Mahjong game is a multiplayer
game, including more gaming decision actions, a large action space
and a strong uncertainty. This leads to the development of a new
domain of Mahjong decision model requires huge human and ma-
terial resources, and needs to go through a long process of model
tuning, which not only cannot guarantee a good training effect, but
also cannot guarantee the transferability of research results. And
for some domains where there is a lack of data or less data, it is
not possible to use deep learning for model building, or the domain
model does not generalize well. Therefore, this paper tries to use
transfer learning method to improve the imperfect information
game with less data.
In this paper, using Mass Mahjong as a typical representative,
and with only a small amount of data, we try to transfer the Bloody
Mahjong model [6], which has a large amount of data and is well
trained, to the Mass Mahjong domain through transfer learning to
construct a Discard model applicable to Mass Mahjong. At the same
time, in order to continue to improve the accuracy of this model,
the model is optimized by using a self-play approach.
2 INTRODUCTION TO THE RULES OF MASS
MAHJONG
The game of Mass Mahjong consists of four players and the tiles are
divided into three suits: Dot, Bamboo and Character, with a total of
108 tiles [7]. Among them, the Mass Mahjong common terms are
explained as follows.
Wall: After the initial hand is drawn at the start of the game, the
remaining tiles are the wall of tiles.
Draw: The player draws a tile from the Wall.
Discard: The player chooses a tile from his hand and discards it.
Eat: When a tile discarded by your last player and two tiles in
your hand form a three consecutive tiles of the same suit, then you
can take Eat action.
Pong: If another player discards a tile that is the same as two
tiles in your hand, you can take Pong action.
228
ICIAI 2022, March 04–06, 2022, Guangzhou, China
Kong: There are 3 forms of Kong, divided into Open Kong, Closed
Kong and Repaired Kong. Open Kong is when a tile discarded by
another player is the same as the three tiles in your hand. When
you draw a tile that is the same as three tiles in your hand, you can
take Closed Kong action. If you draw a tile that is the same as a tile
you have already Pong, you can take Repaired Kong action.
Listen: When the hand is one tile short of Win, the hand is then
in a waiting state. The player can choose whether or not Listen.
After Listen, although you can get the score reward directly, but the
hand cannot be changed after that, i.e. whatever tile is subsequently
drawn is discarded until the winning tile in the game appears, then
you can take Win action.
Meld: During the game, the combinations of tiles for Eat, Pong,
Open Kong and Repaired Kong actions.
Win: The player’s hand forms a special combination.
At the beginning of the game, four players seat in the south, east,
north, west, and each player draws 13 tiles in his hand, while the
dealer player draws one more tile in his hand, making a total of 14
tiles. After that, the player in the dealer’s seat starts to discard in a
clockwise direction, and during the game, he can Draw, Discard, Eat,
Pong, Kong, Listen and Win. Among them, Eat, Pong, Kong, Listen
and Win are all directly benefits. And multiple decision actions
may occur at the same moment. The priority of each action is: Win
> Kong > Pong > Eat. When a player wins or the wall of tiles are
empty, the game comes to an end and the total score of each player
is counted based on the score of the winning hand and the score of
the action in the game.
3 OVERALL DESIGN OF THE MASS
MAHJONG DECISION SYSTEM
Due to the rules of Mass Mahjong, Eat, Pong, Kong and Listen
operations can be directly rewarded. Therefore, this part can be
implemented based on knowledge rules. And the quality of the
Discard determines whether the subsequent win and gain. So the
research in this paper focuses on the Mahjong Discard decision.
In previous work [6], the Mahjong Discard decision model was
trained and performed well. Therefore, this paper uses the Bloody
Mahjong decision model to perform transfer learning.
3.1 The similarities and differences between
the rules of Mass Mahjong and Bloody
Mahjong
Mass Mahjong and Bloody Mahjong are similar, the overall rules
are similar but there are some differences, the main similarities and
differences between the two are as follows.
• Similarities: Both are 108 tiles, with only three suits: Dot,
Bamboo and Character. The rules are the same for both the
Pong and Kong actions. The rules for winning are also similar,
as long as the particular tile fits the winning rules.
• Differences: Mass Mahjong does not have the rule of chang-
ing three tiles and there is no requirement for a missing
suit to win. However, two actions have been added to Mass
Mahjong: Eat and Listen. The Eat, Pong, Kong and Listen
actions of Mass Mahjong are all directly rewarded. The game
comes to an end when one player wins, and there is no such
229
Yajun Zheng and Shuqin Li
situation as one player winning and the others continuing
to play.
In conclusion, Mass Mahjong is simpler in rules compared to
Bloody Mahjong and can be seen as a stage in the Bloody Mahjong,
thus providing good conditions for using transfer learning.
3.2 Overall design concept of the Mass
Mahjong decision system
The overall concept of the transfer learning Mass Mahjong Discard
model design is shown in Figure 1. It consists of the following four
main components.
1. Pre-training. Generally, the model that can get better training
on a similar task is selected for pre-training. According to the
characteristics of Mass Mahjong, this paper selects the Bloody
Mahjong Discard model, which is similar to Mass Mahjong, as the
source model and pre-trains it on the Bloody Mahjong dataset.
• Weight transfer. This step mainly uses the trained model
weights and then uses these weights as initial weights in a
new task, which simply means that the parameters of the
trained model are transferred to the new model to help train
the new model. In this paper, we transfer the weights of the
pre-trained Bloody Mahjong Discard model.
• Re-training. According to the characteristics of the current
task, some structures of the original model are fixed or
changed to adapt the model to the new task. In this paper, we
first remove some unique features of Bloody Mahjong from
the model input, such as changing three tiles and missing suit,
and change the input structure of the model, then use data
augmentation, after augmenting less Mass Mahjong data,
and use the transferred weights to retrain Mass Mahjong
to build a transfer learning based Mass Mahjong Discard
model.
• Model optimization. After training the Mass Mahjong Dis-
card model, the model is optimized by fine-tuning. In this
paper, other Mahjong decision models are integrated with
the Mahjong Discard model to form the Mahjong decision
system. According to the rules of Mass Mahjong, a Mass
Mahjong judge system is established to self-play with the
Mass Mahjong decision system, and the data generated by
the self-play is used to update the Mass Mahjong Discard
model to achieve the effect of model fine-tuning and opti-
mization.
4 DESIGN AND IMPLEMENTATION OF MASS
MAHJONG DECISION SYSTEM
4.1 Design and implementation of Discard
model for Mass Mahjong
In previous work, due to a large amount of Bloody Mahjong Dis-
card data, a Bloody Mahjong Discard model is constructed for
pre-training as a source model for transfer learning. For the model
input data representation reference to the method proposed in the
literature [8-10], the game scenario was segmented considering the
completeness and correlation between the data, and the known
information in the current situation was extracted using human
experience based knowledge. Since there are no features in Mass
Mass Mahjong Decision System Based on Transfer Learning ICIAI 2022, March 04–06, 2022, Guangzhou, China

Figure 1: Overall design idea of Mass Mahjong decision system

Table 1: Model input data representation Table 2: Weight transfer parameters

Input Dimension Representation Features Parameter Name Numerical Value

3*9 Player’s current hand num_epochs 50
3*9 Player’s Discard learning_rate 0.001
3*9 Player’s Meld batch_size 64
3*9 Last Player’s Discard depth 100
3*9 Last Player’s Meld growth_rate 12
3*9 Opposite Player’s Discard criterion Cross Entropy Loss
3*9 Opposite Player’s Meld optimizer SGD
3*9 Next Player’s Discard
3*9 Next Player’s Meld
3*9 All Players’ Discard DenseNet network is chosen to be used in Mahjong Discard decision.
3*9 Current Remaining Tiles Since it is a transfer learning and the types of Mass Mahjong and
3*9 Last Hand Position Bloody Mahjong tiles are the same, the previous information needs
3*9 Last Hand Tile to be taken into account when discarding tiles, and only the input
layer of the model needs to be modified. The model output is the
probability distribution of the 27 tile types.
In this paper, we use the Bloody Mahjong dataset for pre-training
to train the Bloody Mahjong Discard model, and after a series
of parametric work, the final accuracy of the Bloody Mahjong
Figure 2: Model output player action representation Discard model on the test set can reach 91.6%. Next, weight transfer
is performed to transfer the parameters of the Bloody Mahjong
Discard model to the Mass Mahjong Discard model, where the
Mahjong such as change three tiles, these features are eliminated specific parameters are shown in Table 2 below.
when constructing the Mass Mahjong Discard model. In this paper, The loss function is chosen as the cross-entropy loss, which is
we choose a feature plane of 3*9 to represent the situation, with 3 expressed in the following equation 1).
rows representing the suits of the tiles and 9 columns representing
c = − x [y ln a + (1 − y) ln a]
Í
the 9 categories of the sequence 1 to 9. For the other known infor- (1)
mation including: the last, next, opposite players have discarded n
tiles and other features are represented by this feature plane. The In Equation 1), y represents the desired output, i.e., the position
specific data representation is shown in Table 1 corresponding to the classification in the label, a represents the ac-
The output of the transfer learning model considers the actions tual output of the neuron, i.e., the position of the multiclassification
as multi-classification, that is, 27 kinds of tile actions, as shown in label predicted by the neural network, x represents the sample, and
Figure 2 below. n represents the total number of samples. The optimization objec-
In previous work, Bloody Mahjong not only considers the current tive is to minimize the loss, and the value of c decreases gradually
situation information when discarding a tile, but also takes the when the actual output of the model is close to the true result.
past tile information into account. And DenseNet [11] network The Mass Mahjong Discard model is obtained by retraining using
establishes the connection relationship between different layers, so the Mass Mahjong Discard data after data augmentation and the

230
ICIAI 2022, March 04–06, 2022, Guangzhou, China Yajun Zheng and Shuqin Li

Table 3: Weight transfer parameters

Action Rules For Use

Eat 1. If the number of tiles in the hand of a Eat meld is 3, not take Eat action.
2. If the Eat meld already formed in the hand, not take Eat action.
3. In the rest of the cases, the Eat action is performed if the Eat is legal.
Pong 1. If the tile number of the Pong is 3 and have single number 1, 2 in the hand, then no take Pong.
2. If the tile number of the Pong is 7 and have single number 8, 9 in the hand, then no take Pong.
3. In the rest of the cases, the Pong action is performed if the Pong is legal.
Kong 1. If the winning hand is a 7 pair, not take Kong action.
2. If the tile of the Kong can be divided into a pair and a meld in the hand, then no take Kong.
3. In the rest of the cases, the Kong action is performed if the Kong is legal.
Listen 1. Listen to a large number of tiles.
2. Listen to the high scoring tile when winning.
3. Listen to the tile is not all discarded.

parameters after weight transfer. The specific algorithm is shown Algorithm 2 Self-play model optimization algorithm
in Algorithm 1 below. input model_Mass, model_Other, epochs
output model_Mass
epoch = 0
Algorithm 1 Mass Mahjong Discard model algorithm based on best_error = 1
transfer learning while epoch < epochs do
data = Self_Play(model_Mass, model_Other)
input data_Bloody, data_Mass
data_win = Select(data)
output model_Mass
data_train, data_dev, data_test = Split_Data(data_win)
model_Bloody = Train(data_Bloody) // Training Bloody Mahjong
model_new = Train(data_train, data_dev)
Discard model
test_error = Compute_Error(data_test, model_new)
data_Mass_ aug= Augmentation(data_Mass) // Mass Mahjong
if test_error < best_error then
data augmentation
best_error ← test_error
model_Mass = Train (data_Mass_ aug) // Training Mass Mahjong
model_Mass ← model_new
Discard model using Bloody Mahjong Discard model parameters
Save(model_Mass)
return model_Mass
epoch ← epoch + 1
return model_Mass

4.2 Design and implementation of other action

decision models for Mass Mahjong
Mass Mahjong has actions such as Eat, Discard, Pong, Kong, Listen,
these actions combined together to form a complete Mass Mahjong
decision system. The Discard model is the one trained by transfer
learning, while the other four models are rule-based models due to
certain constraints in taking actions and the small amount of data
that can be collected. Some of the rules are listed in this paper and
the algorithm is shown in Table 3 below.
The above models of Eat, Pong, Kong, Listen and Discard are
combined into a complete Mass Mahjong decision system.
4.3 Optimized design and implementation of
Mass Mahjong models
After the Mass Mahjong decision system is constructed, the data
is generated through self-play to optimize the Discard model so
that the Mass Mahjong Discard model can be better adapted to the
rules of Mass Mahjong. The training process of the self-play model
optimization algorithm is shown in Algorithm 2.
Figure 3: Raw data format
5 RESULTS AND ANALYSIS
5.1 Data Processing
Due to the small amount of raw data, no further data cleaning is
carried out in this paper in the data processing. The data processing
mainly performs semantic segmentation of the game situation and
feature extraction on the raw data. The Mass Mahjong data used
in this paper comes from the game records of the 2020 National
University Computer Gaming Mass Mahjong Competition. The raw
data is a log in json format, with information including: one’s hand,
one’s meld, one’s seat number, and the game record records so far.
The raw data format is shown in Figure 3 below.

231
Mass Mahjong Decision System Based on Transfer Learning
Figure 4: Transfer learning experiment results
To improve the generalization ability of the model, data augmen-
tation is carried out on the processed data. The three suits of tiles,
Dot, Bamboo and Character, have the same rank and the numbers
one to nine are symmetrical in structure. Referring to AlphaZero’s
data augmentation method, the three suits of tiles can be rotated
and the numbers one to nine are symmetrically processed, with a
total of 48 situations, i.e. there are 47 other positions in a situation
that are equivalent to the current one.
5.2 Transfer learning model experiment
After data processing, these calibration data are divided, and a
total of 900,000 calibration data are obtained. A randomly selected
80% of the data was used as the training set and 20% as the test
set. The model was written with pytorch, using parameters after
weight transfer, and had been trained on a single GPU in the type
GEFORCE RTX 2080. As the number of training increases, the test
set obtained is shown in Figure 4
Although the accuracy of 84% was achieved on the test set. How-
ever, due to the low quality of data, the model needs to be optimized
and improved by subsequent self-playing games.
In order to provide conditions for the subsequent self-play,
this paper constructs the Mass Mahjong judge system. The Mass
Mahjong judge system can simulate the environment of a Mass
Mahjong game and is responsible for interacting with the Mass
Mahjong decision system, which form a complete Mass Mahjong
game together. The Mass Mahjong judge system mainly includes
functions such as drawing tiles, judging winning, calculating scores,
Figure 5: Overall design idea of Mass Mahjong judge system
232
ICIAI 2022, March 04–06, 2022, Guangzhou, China
and judging whether the next decision is legal. The overall design
is shown in Figure 5 below.
In this paper, we use self-playing games to produce data, and
then use the data of the winning players to train the model for
optimization. Ten processes were used for training, and one training
session was conducted 250 times per game for 40 rounds each with
a learning rate of 0.001. While using JJWorld’s testing software(The
National Competition‘s testing software with official support), the
games were tested for 30 rounds with four players randomly seated.
The unoptimized AI were players with the names TEST 2 and TEST
4, and the optimized AI were TEST 1 and TEST 3. TEST 3 trained
longer than TEST 1, and the final results are shown in Table 4
As can be seen in Table 4, the unoptimized model won fewer
times and scored lower than the optimized model. The optimized
model, on the other hand, wins more times and score higher, while
the longer the model is trained, the better the model is optimized
and the more the model can learn the rules of Mass Mahjong.
5.3 Results of the practical competition
The Mass Mahjong decision system designed in this paper was
entered into the 2021 National University Computer Gaming Com-
petition. The Mass Mahjong agent designed in this paper won the
second prize in the competition. When playing against each other,
some of the results are shown in Table 5 below.
Among them, BISTU-Mahjong1 was the runner-up team last
year, based on the deep learning model, and BISTU-Mahjong2 was
the participating team last year, basing on rules, which won the
5th place last year. The Mass Mahjong decision system based on
transfer learning proposed in this paper performs better than the
above two, indicating that the algorithm proposed in this paper has
a certain effect.
When playing against other players’ AI, the model performed
average, and the reasons for this were analyzed as follows: On the
one hand, there is not enough time for model training; on the other
hand, the models of Eat, Pong and Kong are implemented based
on rules, and they are not well coordinated with the Discard mod-
els. Finally, the Mass Mahjong decision system based on transfer
learning proposed in this paper won the second prize.
6 CONCLUSION
In this paper, in order to solve the problem of differences in the
rules of adjacent Mahjong domains and the simultaneous absence
of data, transfer learning is used to remove features specific to the
ICIAI 2022, March 04–06, 2022, Guangzhou, China Yajun Zheng and Shuqin Li

Table 4: Competition results for AI with different levels of learning

Team Name Total Score Total Number Of Wins Winning Number

TEST 3 67 84 14
TEST 1 -2 42 7
TEST 4 -11 30 6
TEST 2 -54 30 5

Table 5: Comparison Results
Team Name Total Score
DIHU 170
BISTU-Mahjong1 46
BISTU-Mahjong2 -105
Example AI -111
transferred source model, and then the model is migrated to a new
Mahjong, and finally the model is further optimized by building
a self-play system. In this paper, the Bloody Mahjong model was
transferred to the Mass Mahjong model, which eventually won the
second prize in the 2021 National University Computer Gaming
Mass Mahjong Competition. Since some of the models used in this
paper are rule-based models, in the future work, it is possible that
for the Eat, Pong, Kong and Listen models, deep learning models
can be trained to replace the current rule-based models, so that
each model can have a good coordination effect, improve the overall
decision-making ability of the model.
ACKNOWLEDGMENTS
This work is supported by Normal projects of promoting graduated
education program at Beijing Information Science and Technology
University. (NO. 5212010937), by Normal projects of General Science
and Technology research program (NO. KM201911232002), by Con-
struction Project of computer technology specialty (NO.5112011019)
, and by Normal projects of promoting graduated education pro-
gram at Beijing Information Science and Technology University
(NO.5112011041) .
REFERENCES
[1] Silver D, Huang A, Maddison C J, et al. Mastering the game of Go with deep
neural networks and tree search[J]. nature, 2016, 529(7587): 484-489.
[2] Zha D, Xie J, Ma W, et al. DouZero: Mastering DouDizhu with Self-Play Deep
Reinforcement Learning[J]. arXiv preprint arXiv:2106.06135, 2021.
[3] Silver D, Hubert T, Schrittwieser J, et al. Mastering chess and shogi by self-
play with a general reinforcement learning algorithm[J]. arXiv preprint arXiv:
1712.01815, 2017.
[4] Van der Kleij A A J. Monte Carlo tree search and opponent modeling through
player clustering in no-limit Texas hold’em poker[J]. University of Groningen,
The Netherlands, 2010.
[5] Li J, Koyamada S, Ye Q, et al. Suphx: Mastering Mahjong with Deep Reinforcement
Learning[J]. arXiv preprint arXiv:2003.13590, 2020.
[6] Gao S, Li S. Bloody Mahjong playing strategy based on the integration of deep
learning and XGBoost[J]. CAAI Transactions on Intelligence Technology, 2021.
[7] Qingyue Wang. Game of Mahjong [M]. Chengdu: Shurong Chess Publishing
House.2003.
[8] Gao S, Okuya F, Kawahara Y, et al. Supervised Learning of Imperfect Information
Data in the Game of Mahjong via Deep Convolutional Neural Networks[J].
Information Processing Society of Japan, 2018.
[9] Gao S, Okuya F, Kawahara Y, et al. Building a Computer Mahjong Player via
Deep Convolutional Neural Networks[J]. arXiv preprint arXiv:1906.02146, 2019.
[10] Wang M, Yan T, Luo M, et al. A novel deep residual network-based incomplete
information competition strategy for four-players Mahjong games[J]. Multimedia
Tools and Applications, 2019: 1-25.
[11] Huang G, Liu Z, Van Der Maaten L, et al. Densely connected convolutional
networks[C]//Proceedings of the IEEE conference on computer vision and pattern
recognition. 2017: 4700-4708.

233