Big Data Driven

Retention Improvement
A/B Testing Driven Case Study

Sachin Uppal
Marketing Director
Play Games24x7 Pvt. Ltd.

© Play Games24x7 Pvt. Ltd.

New Player Journey on RummyCircle.com

Registration

Checkout

Game-Play Retention

© Play Games24x7 Pvt. Ltd.

Background and Motivation
• After registration players add money to their accounts and
become cash players and play rummy game for cash.
• We measure D1 retention i.e. player returning to site on day
1 after making a deposit on day 0.
• D1 metric is a predictor for future retention rates and Life
time of a player on the site.
• Running subsequent campaigns through emails, SMS and re-
marketing has little effect on bringing back such players.

© Play Games24x7 Pvt. Ltd.

Retention Case Study
• Challenge: A significant proportion of players do not last
beyond first day on the site.

• Hypothesis: If we make a lucrative time-bound offer to

selected players to bring them back on next day, the D1
retention will improve while our offer costs will be
minimum.

• Objective: To build a predictive model that selects the

players who are going to leave (fence sitters) and make a
real-time limited time offer to improve D1 retention.

© Play Games24x7 Pvt. Ltd.

The Proposed Solution – Segment Identification
• Identify which players are likely to return and which players
are potentially one-timers.

• Using this model, we performed an a priori segmentation of

players into Fence-Sitters and Enthusiasts.

© Play Games24x7 Pvt. Ltd.

Experiment and Approach
• We divided the fence sitters into two groups randomly.
• The control group of fence sitters didn't get an offer while
the test group got an offer.
• We let the test run till we got 99% statistical significance.
• We measured the D1 retention as well as subsequent ROI
on the test group.

© Play Games24x7 Pvt. Ltd.

The Experiment - Control Group

© Play Games24x7 Pvt. Ltd.

The Experiment - Test Group

© Play Games24x7 Pvt. Ltd.

The Experiment - Test Group

Offer

Urgency

© Play Games24x7 Pvt. Ltd.

Results
Over a 2-month period, this campaign yielded an ROI of 62%.

Category Day1 Retention

Fence-Sitters: Test 59%

Fence-Sitters: Control 54%

Enthusiasts 67%

• We saw a lift in D1 retention by 9.2% and a 5% in absolute terms.

• In-fact this gave a lift in the LTV of fence sitter players.

© Play Games24x7 Pvt. Ltd.

Learning
• Making offers to the relevant audience in real-time works
great from ROI and LTV perspective.

© Play Games24x7 Pvt. Ltd.