A Product Manager’s Handbook

For Mobile App Optimization

Table of Why Your App Needs Optimization 02

Contents How to Optimize Your Mobile App 05

Define KPIs and Metrics 07

Identify Gaps in the Expected Use and 08

Actual Use

Formulate Hypotheses 09

Test Your Hypotheses 10

Select Your Target Audience 19

Analyze Your Test Results 21

Learn and Repeat: Update Your Product 23

Roadmap

Conclusion 24
Why Your App
Needs Optimization

A Product Manager’s Handbook for Mobile App Optimization 02

So you’ve created a new app and done your best to ensure that there’s a product-
market fit. With right marketing, you can get your app to your ideal users and they’ll MOBILE APP CHURN RATE
start using it, right?

But you’ll soon realize that despite the market fit, getting users to install your app is
difficult. More than 80 percent of apps do not even show up in the top lists in Apple
Store1. Every category has stiff competition that crowds search results. And if you
manage to acquire users, then there’s the bigger challenge of retaining them.
90%

USERS LOST
77%

So how exactly should you build a sustainable mobile app business?

This is precisely where optimization fits in. Testing what works for your users and what
3 DAYS 30 DAYS
doesn’t helps you constantly improve their experience with your app and, therefore,
increases their likelihood to continue using it.
An app loses 77% of its daily active
Whether you are just getting started or already have an app, you must find ways to users within the first 3 days and
90% within the first 30 days.²
make sure that users engage with and keep coming back to your app. This guide should
help you deliver delightful experiences at every step of your user journey.

¹ Dan Rowinski. “Another reason app discovery is broken,” Applause.com, last modified February 15 2015.
² Andrew Chen. “New data shown why lossing 80% of your users is normal and that the best apps do better,” Andrew Chen Blog.

A Product Manager’s Handbook for Mobile App Optimization 03

There are several reasons to optimize your mobile application:

Lower Customer Acquisition Cost

Users are more likely to install an app on recommendation from a friend than from a sponsored post on
Facebook. Great user experience turns your existing users into your advocates. Word of mouth is the best
kind of advertisement. It gets you users at no additional marketing cost and hence, lowers your customer
acquisition cost.

Improved Conversion Rates

Directly asking users why they didn’t complete a transaction isn’t a great idea. They may have dropped off
because of a lack of social proof, insufficient trust in the product, low quality of product images, or
something as simple as wanting to browse other similar products. Whatever the reason may be, it can be
difficult for them to explain it. That’s where optimization comes in handy. By continuously optimizing, we
can tell what drives user behavior and what causes them to complete or abandon a purchase. Data-driven
optimization ensures that every decision you make is based on evidence, and not on gut feeling.

Better Customer Retention

Going back to the question of building a sustainable mobile app business—optimization is useful in the
growth phase of an app, but critical in the maturity phase when your focus shifts from releasing new
features to growing and nurturing the existing user base. Only by continuously optimizing the app can you
communicate your value proposition to users and reiterate it. And only when users understand the value
your app offers, will they use it repetitively.

A Product Manager’s Handbook for Mobile App Optimization 04

How to
Optimize Your
Mobile App
It is easy to prioritize building new features over optimizing the user flow
for your app. However, when users are unable to find the same features
incorporated into the app, they drop off the conversion funnel at various
places. And leaky funnels result in a loss of revenue.

App optimization is not easy, but is manageable through a

defined process.

Before you start, it is important to understand why people

use (or will use) your product. What problem does your app
solve for them?

Does it allow users to accomplish a task, stay connected with other

users, or is it meant to entertain them? Understanding the primary
purpose of your mobile app helps you define its key performance
indicators (KPIs).

A Product Manager’s Handbook for Mobile App Optimization 06

 Define KPIs and Metrics

The first step in the optimization process is to define its KPIs.

VWO integrates with most analytics
Depending on the type of an app, it could mean a user making an
eCommerce purchase or booking a holiday package.
platforms, to enable data capturing and
perform in-depth analysis. These include:
There are usually multiple steps required for a user to complete
the desired action, including searching, shortlisting items,
comparing shortlisted items, and then finally making a purchase.
All such user actions should be recorded on your analytics
platform. For a quick overview of the key metrics, we recommend
creating a dashboard on your analytics platform.

For a quick overview of the key metrics, we recommend creating

a dashboard on your analytics platform.

A Product Manager’s Handbook for Mobile App Optimization 07

 Identify Gaps in the Expected Use and Actual Use

When analyzing your app, we recommend encouraging your team to ask many questions, even the tough ones. You must note these down too,
because these are the key to optimizing your app. Don’t be surprised if you are faced with questions such as:

At which stages of the conversion funnel do users drop off?

How many steps does it take for a user to complete the desired action? Is it optimum?

Why aren’t users clicking a particular button?

Why are the users going back from the cart to search again?

What causes drop-off at the onboarding stage?

What keeps users from signing in?

Is it not obvious that the app requires sharing the location information? Why do users
prefer denying location access?

After identifying all potential areas for optimization, the next step is to sort this list of questions in the order of priority. Distinguish
between optimization areas which will directly impact your KPIs and the ones which will have less to no impact.

A Product Manager’s Handbook for Mobile App Optimization 08

 Formulate Hypotheses HYPOTHESIS

For every optimization idea that you put on your

Based on Data
plan, it is best to formulate a clear hypothesis. It
can look something like this: I expect that The change

Will address The problem

EXAMPLE 1 EXAMPLE 2

Based on the data of users looking at product images,

Based on the data of users coming to the landing page,

I expect that the ability to skip login I expect that adding better quality and bigger images,

Will address the problem of users not adding items to their cart and
Will address the problem of users abandoning the onboarding screen
ultimately buying

A Product Manager’s Handbook for Mobile App Optimization 09

 Test Your Hypotheses
VWO has been instrumental in helping
After you have a list of hypotheses you want to Movoto increase conversion rates on
test, it is time to validate these with A/B testing. our Mobile App and responsive web site.
Tests are relatively simple to set-up and
Begin
Beginwith
withselecting a mobile
selecting app testing
a Mobile App tool that suits
Testing Toolyour the Product Team is able to quickly
requirements. Here are some criteria that should help you understand impact. In a recent mobile
that suits your requirements. Here are some
decide if a third-party SDK can go with your mobile app: app test, we added a new map link and
criteria that should help you decide if a third- call button to a key landing page.
party SDK can go with your mobile app: The results of this test gave us
SIZE confidence to change the main
The SDK has to be lightweight such that it does not CTAs in the mobile app.
impact the size of your app.

RAM USE
As mobile devices have scarce RAM availability,
the SDK should not use a lot of RAM.

PERFORMANCE AND AVAILABILITY

Laura Dansbury
It has to perform well and be available all the time.
VP of Product

A Product Manager’s Handbook for Mobile App Optimization 10

For your reference, here is an analysis of VWO Mobile App Testing Solution:

Impact Analysis: VWO Mobile App A/B Testing

iOS SDK Android SDK

Detailed Analysis Detailed Analysis

SIZE SIZE
Approximately 100 KB Approximately 200 KB

RAM RAM
Uses only about 300 KB of RAM at any Uses only about 100 KB of RAM at
given time any given time

PERFORMANCE AND AVAILABILITY

The VWO SDK is available even when there is no Internet connection. Additionally, the VWO SDKs
are tested thoroughly in multiple rounds to ensure that there are no bugs to impact your app.

If you want to be sure about your choice, you can also measure the effect of the VWO SDK on
different parameters of your app.

A Product Manager’s Handbook for Mobile App Optimization 11

VWO’s variables and code blocks make it easy for developers and product managers to create and run an A/B test. A
developer can define the app variables in VWO, and then A/B tests can be run on those variables without deploying
code.
Control C

Let’s assume that your hypothesis reads something like:

Based on the data of users clicking the pop-up

I expect that showing a different message

will help us increase the number of conversions on this screen

To implement this, sign in to VWO, and then create a variable.

A Product Manager’s Handbook for Mobile App Optimization 12

The corresponding code changes in the app would look like this:

SWIFT OBJECTIVE-C JAVA

VWO.stringFor(key: "in-app-message", [VWO stringForKey:@"in-app-message" VWO.getStringForKey("in-app-message",

defaultValue: "Use our House Agent!") defaultValue:@"Use our House Agent!"]; "Use our House Agent!");

It is always advisable to preview all the

changes before submitting an update to the
Preview
App Store or Google Play Store. It not only
helps you visualize the changes, but also
lets you verify that these can be
You can use VWO to select your device or simulator,
implemented as planned.
and then preview all the changes before updating
them. VWO also supports gesture-based previews.
Read more about the preview mode here.

A Product Manager’s Handbook for Mobile App Optimization 13

 The previews for this test would look like this:

Control C Variation V1

A Product Manager’s Handbook for Mobile App Optimization 14

Before setting the A/B test live, you must define goals for the test, that is, the leading or lagging metric that you’re trying to improve by running
this test.

It can be something as simple as a user moving to the next step, a user making a purchase, or a user skipping to the next step and completing a
purchase action. For this test, the goal is to increase the conversion rate for an in-app purchase, so the goal is named in “inAppPurchase”.

After this, you can proceed to include the following code snippet in the mobile app:

SWIFT OBJECTIVE-C JAVA

VWO.trackConversion [VWO trackConversion: VWO.trackConversion

("inAppPurchase") @"inAppPurchase"]; ("inAppPurchase");

A Product Manager’s Handbook for Mobile App Optimization 15

At the Goals screen during the test creation process, you will be able to identify the optimum time duration for the test to arrive at a result, by
entering the following values in the duration calculator:

Average number of daily visitors

Your current conversion rate for Control

The lift in conversion rate you care about

You also get to exercise maximum control over testing by choosing from 3 different modes of testing:

Quick Learnings Balanced Maximum Certainty

This mode gives you faster results and Ideal for most tests, the Balanced To make smarter decisions by
is great for finding trends quickly. mode allows for better planning of minimizing the potential loss for
your testing pipeline. revenue-critical tests, pick the
Maximum Certainty mode of testing.

A Product Manager’s Handbook for Mobile App Optimization 16

Entering accurate values allows
VWO SmartStats to give you a
good estimate of the duration in
which it can deliver results.

VWO SmartStats

Our Bayesian-powered statistics

engine helps you arrive at
intelligent results to make smarter
business decisions, while reducing
your testing time by half.

A Product Manager’s Handbook for Mobile App Optimization 17

To begin with, we
recommend rolling out
tests to a small percentage
of your app users, say
20%, and gradually include
more users in the test.
This will help you observe
the impact of the control
and variation on the
defined goal. Also, you will
have better control over
the impact of this test on
your overall metrics.

You can choose the size of your test participants from the
campaign settings available in VWO.

A Product Manager’s Handbook for Mobile App Optimization 18

Select Your Target
Audience

VWO also offers a wide

variety of options to
target your experiments.
You can target users of a
particular iOS/Android
version or app version, or
even define custom
variables to choose who
should become a part of
the test.

Read more about targeting in VWO here.

A Product Manager’s Handbook for Mobile App Optimization 19

While deploying your A/B test by using VWO, you can also integrate your VWO account with an analytics platform to monitor the use by
segmenting users, based on whether they see the control or the variation.

Your VWO account can be integrated with numerous analytics platforms. The list of these platforms, along with suggested implementations, is
available here.

Sample integration: Code for integrating VWO with Google Analytics

49 [NSNotificationCenter.defaultCenter addObserverForName:VWOUserStartedTrackingInCampaignNotification object:nil queue:nil usingBlock:^(NSNotification * _Nonnull note) {

50 NSDictionary *campaign = notification.userInfo

51 NSString *event = [NSString stringWithFormat:@"VWO Campaign - %@ - %@", campaign[@"vwo_campaign_name"], campaign[@"vwo_campaign_id"]];

52 NSString *action = [NSString stringWithFormat:@"%@ - %@", campaign[@"vwo_campaign_name"], campaign[@"vwo_campaign_id"]];

53 NSString *label = [NSString stringWithFormat:@"%@ - %@", campaign[@"vwo_variation_name"], campaign[@"vwo_variation_id"]];

54

55 id<GAITracker> tracker = [GAI sharedInstance].defaultTracker;

56 [tracker send:[[GAIDictionaryBuilder

57 createEventWithCategory:category

58 action:action

59 label:label

60 value:nil] build]];

61 }];

A Product Manager’s Handbook for Mobile App Optimization 20

 Analyze Your Test Results

Per the traffic on your app, you may start seeing results within few days. If the traffic is not sufficient, you may want to increase the traffic
percentage to include more users in the experiment.

This is what the result would look like in your VWO account:

Precise Conversion Rate range Potential Loss

SmartStats calculates the precise range of Understand what you stand to lose if you
your true conversion rate. deployed A when in fact B was actually better.

C Use our House Agent V1 In App Message

CHANCE TO BEAT

VA R I AT I O N CONVERSION IMPROVEMENT POTENTIAL USE OUR CONVERSIONS /

C A ALL
R AT E ( R A N G E ) LOSS HOUSE AGENT VISITORS)

2.23%
BASE

C Use our - 9.05% - - 250/11,221

House Agent (1.89% - 2.61% )

In App 11.28%
V1 8~10% - 100% 100% 1,600/14,187
Message (10.61% - 11.98% )

Probability of A beating B Chance to Beat All

Forget significance, SmartStats gives you Get the probability of a
the actual probability of A beating B. variation beating not just
control but other variations
as well.
SmartStats uses potential loss to decide when to end a test. Based on the test results, you’re shown one of the
two statuses:

We have a winner! V1 In App Message has shown to have between 8.0% and 10.0% be<er conversion rate than
all other varia?ons including C Use our House Agent.
Recommended ac?on: Discard all other varia?ons and go with the winner.

Winning Variation: When the difference between the lower limit of the winning variation is at least 1% higher than the
upper limit of the losing variation, SmartStats declares it as the winner.

We have a Smart Decision! V1 In App Message has shown to have between 10.61% and 11.98% be<er
conversion rate than all other varia?ons including C Use our House Agent.
Recommended ac?on: Discard all other varia?ons and go with V1 In App Message

Smart Decision: When the potential loss of deploying a variation is negligible to be within your comfort threshold,
SmartStats recommends it as a smart decision to deploy this variation.

Smart decisions minimize the potential loss

Potential loss is the lift you would have lost if you would have deployed A as the winner, when in fact B was better.
SmartStats uses potential loss to end tests. When the potential loss of a variation drops into the comfort threshold,
SmartStats suggests that it’s a smart decision to deploy the variation. With a smart decision, you stand to improve your
conversion rate and will not lose anything because the potential loss from a false positive is now minimized.

A Product Manager’s Handbook for Mobile App Optimization 22

Learn and Repeat: Update Your Product Roadmap

Based on the results of each new A/B test on your mobile

application, you may:

OR

Implement the hypothesis for all the Discard the hypothesis,

users, if it resulted in significant increase if it did not lead to any improvement.
in conversions.

If you note that the metric you are tracking shows only marginal improvement after a test, it is a good idea to
document your learning and generate new ideas to test. In any case, you should discuss test results with your team
and update your product roadmap regularly. The key is to remember that optimization is a marathon, not a sprint. It
is a continuous process to identify areas of improvement and enhance user experience in a dynamic marketplace.

A Product Manager’s Handbook for Mobile App Optimization 23

 Conclusion

How to Optimize Your Identify Gaps in the Test Your Analyze Your
Mobile App Expected Use and Hypotheses Test Results
Actual Use

Formulate Select Your Learn and Repeat:

Define KPIs and Metrics
Hypotheses Target Audience Update Your Product
Roadmap
 Optimizing your app to consistently deliver delightful user experience is
just as important as introducing new features and products. Great UX
gets you returning users and turns them into advocates for your mobile
app, which attracts new users too.

VWO allows you to follow a structured approach to conversion rate

optimization through process-driven A/B testing.
It starts with defining conversion goals and uses these as the baseline to
locate gaps in the expected and actual use of your app. As the next step,
your team should brainstorm to formulate hypotheses and create a
priority-based testing pipeline to address these gaps. With VWO, it is
simple for product managers and developers to make changes to the
code and set up A/B tests, target these to custom audiences, and
analyze results.

It is essential to learn from each test, adjust hypotheses, and keep

testing. The use of optimization process should become a habit so that
you soon find yourself continuously looking for ideas to optimize your
mobile app with each winning test.

For more details on mobile optimization and

pricing-related information, contact our Sales team.

REQUEST VWO DEMO

A Product Manager’s Handbook for Mobile App Optimization 25

About VWO

The VWO Experience Optimization & Growth Platform is the only connected platform that enables organizations to optimize their entire audience
journey - from click to conversion. You can build stellar digital experiences that convert better by discovering visitor behavior insights, easily
turning these insights to experiments, expanding experiments to multichannel digital properties, and bringing back lost audience through effective
engagement- all via a single connected platform. Embrace the power of the VWO Connected Platform to see your core conversion & growth metrics
go up everyday.

Hypothesis

Age

Visitor Reco
rding

01
Gender

03
02

Location

Test 1

Test 2
TRUSTED BY THOUSANDS OF
LEADING BRANDS

SIGN UP FOR FREE TRIAL

REQUEST A DEMO

+1 844-822-8378 | sales@wingify.com

vwo.com