Fact sheet | How to reach ‘top performer’ status

and improve your Mobile DevOps maturity

What percentage of your releases should be hotfixes? The Mobile DevOps

Assessment (MODAS), the first of its kind, benchmarked 1600+ mobile DevOps teams
last year to find out just that.

MODAS shows you how to improve and mature as a mobile DevOps team by
illustrating, with data, what it takes to be a top performer at four key stages of the
app delivery process: creation, testing, deployment, and monitoring. This fact sheet
focuses on the second phase of the app delivery process: testing.

Creation Testing Deploying Monitoring

Planning and building Testing apps Pushing apps to Measuring health, performance
your apps and pushing fixes the app stores and user feedback

Approximately what share of

your releases are hotfixes?

10%
Top-performing Mobile App teams

25%+
Average-performing teams

Notable findings from the survey:

• 58.9% of teams spend less than 30% of their engineering hours on testing on RC,
while 24.2% devote 30-59% of their hours to testing
• Just 10.4% of teams said they test as many devices as possible with a device farm,
with 31% saying they test the most commonly used devices in their user base
• While finance teams performed status security analyzes more than other groups,
the number is still surprisingly small at 17%
• The most common testing framework is XCTest/XCUITest at 46.9%,
followed by Espresso (28.2%), Appium (23.9%)
 7 top tips to improve your app testing process

1. 4.
Implement Unit testing Use AI and ML in testing to
increase test coverage
It’s the lowest level of testing, but it’s one of the cheapest, not to
mention it’s the easiest way to catch defects early and minimize Become more efficient by incorporating AI and machine learning
code complexity. into mobile testing tools.

Top tip: Top tip:

Check out our article on the different methods of
Read our article on how AI and Machine Learning are
unit testing in mobile.
changing the mobile testing landscape

2. 5.
Implement continuous testing Consider other testing types
with test automation throughout the development process. testing not only focus on the app’s functionality but there are
The benefits of continuous integration? It means your team can different aspects we should consider to increase the app quality
catch bugs and issues before they become critical, reduce the and deliver qualitative apps to the customers such as:
debugging time by figuring out the root cause, and reduce delays
that usually come with manual tests: waiting on QA’s/WC’s. • Localization Testing (snapshot testing)
• Accessibility Testing
• Non-functional testing:
3. • Static analysis
Build scalable, reusable, and • App size measurement/monitoring
maintainable testing frameworks • Security testing (static and dynamic)
• Performance (ex: memory leaks)
With Espresso, XCUITest, Detox, or Appium
(based on the app type).

• Mastering the mobile testing pyramid 6.

• Continuous testing for mobile apps at scale Implement test insights and
with Sauce Labs and Bitrise analytics in order to:
• Getting started with testing Jetpack Compose
• Test and track performance and stability
• The 4 main factors to consider as
part of your test automation strategy • Understand the impact of failure

• 19 automated mobile testing resources • Provide actionable insights

to take you from beginner to expert • Test coverage
• Test execution time
• Defect detection rate

“The easier a tool is to leverage, the easier the developers’ job is. We can
integrate it with different platforms and use it in a variety of ways. It just works.”

- Matt Robinson,
iOS Engineer at Reddit
6.
Consider the following best practices
with test automation
To reduce the execution time and accelerate
the delivery process:
• Run UI tests on cloud devices (to increase the coverage and
tackle the Device andOS Fragmentation challenge)
• Screen sizes
• Connection types
• Power consumption/battery life
• Location-dependence
• OS legacy and new versions
• Run automated tests in parallel (test shards). Sharding your
tests can be a great way to boost the speed of your tests using
hardware in the cloud.
• Implement Test Reports: once different tests suites/types for
mobile apps have been implemented, documentation should
also be created to keep track of how these tests behave.
Be a part of the world’s first mobile-dedicated
DevOps assessment.
Ready to improve your Mobile DevOps processes and deliver with
confidence? Talk to an expert to see what Bitrise can do for you -
trusted by the world’s leading apps.
Talk to an expert
Top tip:
A test report is an important document in software testing
and should always contain the following information:
• Test summary: including the total number of
test cases with the different statuses such as passed,
failed and skipped/ignored.
• Test description: including the test name,
class or package.
• Test duration: including the execution time
for the test case.
• Test artifacts: including screenshots, videos,
and any other logs.
Using Bitrise, once your build has run, you can click the
‘Test Reports’ button and view all your unit, UI, and device
test results collected in a single dashboard with Bitrise Test
Report Add-on. Once you’ve analyzed your build’s test
results, you can easily navigate back to your app through the
top menu bar. More information about how to deploy the
test results can be found here.
Take the survey