Mobile Apps Performance Testing

8/18/2017 Mobile Apps Performance Testing
This is Google's cache of http://qainsights.com/mobile-apps-performance-testing-whitepaper/. It is a snapshot of the page as it appeared on 29 Jul 2017
07:12:27 GMT.
The current page could have changed in the meantime. Learn more
Full version Text-only version View source

Tip: To quickly find your search term on this page, press Ctrl+F or -F (Mac) and use the find bar.
$ 0.00
December 25, 2012
Mobile Apps Performance Testing Whitepaper
The post-PC era is already over. Mobile usage will surpass the PCs in lightning fast mode. By default, all business organizations
started developing mobile apps for their customers. Below is the key statistics about mobile usage from Business Insider and
Nielson 2012.
In 2011, there were 835 million smartphone users, 5.6 billion feature phone users Business Insider
Global internet usage will more than double by 2015 Business Insider
64% of mobile phone time is spent on apps Nielson 2012
13.4% texting, 11.1% browser, 5.5% social networking apps, 5.4% dialer, 5.3% email/IM, 2.3% music/video, 1.1% camera, 55.8%
other apps Nielson 2012
http://webcache.googleusercontent.com/search?q=cache:SAf1zVxSEzQJ:qainsights.com/mobile-apps-performance-testing-whitepaper/+&cd=44&hl=en&ct=clnk&gl=in 1/12
From the above statistics, it is crystal clear that users spending more than 50% of their time on apps. Popular stores like Apple
Apps Store, Google Play store has more than 650,000 and 600,000 apps respectively. Mobile app revenue will be $46 billion by
2016. Mobile Apps is an emerging digital industry today.
It is very much critical to validate the functionality and the performance of the apps before go-to-market. Just pick few of the low
rated apps in the Apple Apps Store or in the Google Play Store or in any other store, the key reason of low ratings is because of
lack in non-functional aspects of the apps.
Validating the non-functional aspects plays a vital role in Quality Assurance. This whitepaper focuses on mobile apps
performance testing and its approach. It stresses more on the native apps aspects.
What are mobile apps (applications)?
Mobile app is software which accomplish specific task. E.g. Calendar, Music, Videos etc. It saves time and effort of the users by
performing simple touch, pinch and zoom, tap, swipe, slide etc.
Mobile Load Testing
Mobile Load Testing helps us to study the behavior of the apps under various loads. This study reveals how the application
handles concurrent users effectively. It is not possible to test the performance using real human users. By simulating virtual users
(VUsers) using performance testing tool, it is simple to reproduce the huge number of concurrent users.
Difference between mobile testing and mobile apps performance testing
Before start mobile load testing, it is important to design the strategy. There are difference between mobile testing and mobile
apps performance testing.
Mobile testing is a process of validating the functional aspects of the mobile device. Mobile apps performance testing is a process
of validating the non-functional aspects of the apps installed in the mobile device.
Types of apps
There are three main types of apps in the mobile device. It is challenging to identify the type of performance testing is required
and the type of mobile application under test. Below are the types of apps for the mobile device.
Native Apps:
Apps which are developed by mobile OS, e.g. apps developed in Objective-C for iPhone, apps developed in Java for Android mobiles
etc. Native apps came first; famous examples of native apps are Address Book, Calendar, Music, Videos, and Games etc.
Web Apps:
Web apps designed in languages like HTML, JavaScript, and PHP etc. in a way that can be accessed via internet connection. E.g.
Facebook can be accessed in mobile using http://m.facebook.com and Gmail can be accessed in http://m.gmail.com
Hybrid Apps:
Hybrid apps designed to access web applications which are embedded in the mobile. E.g. Twitter app, Apples Apps Store, LinkedIn
app etc.
Challenges in Mobile Load Testing
There are few challenges in mobile load testing when compared to traditional load testing. Mobile devices access internet via
various mobile technologies. Following are the key differences:
Users access internet using various mobile bandwidths and network. In US, people use 3G, 4G LTE technologies, in India people
prefers 2G network, because 3G and 4G are emerging networks. Following are the few points play a critical role:
Simulating bandwidth and speed

Slower quality connection has high response time which extends the user sessions, and also utilizes more battery
power.
2G connection will be in Kbps where as 3G and 4G in Mbps
More number of devices
New mobile devices hitting the market every day. iOS, Android, Windows etc. It is important to validate all the aspects of
all the devices.
Users Growth
Exponential growth in mobile users, hence it is important to predict the business growth by forecasting in advance.
Addressing mobile apps load testing
Using traditional tools like HP LoadRunner, IBM Rational Performance Tester, Apache JMeter, it is possible to simulate the mobile
web applications by modifying the user agent string in the request headers.
Below is the snapshot of LinkedIn.com in IE 8 (default user agent string).
Below is the snapshot of LinkedIn.com in iPhone 4S. LoadRunner is automatically redirecting from http://www.linkedin.com to
https://touch.www.linkedin.com
It is because of the modified user agent string in the request headers. Based on the user agent string, server will redirect to the
appropriate URLs.
Mobile website applications can be easily load tested by modifying the user agents. But the important drawback here is we
cannot create realistic real time load. Because users have different devices, different carriers, upload/download speed, different
signal strength and different technologies to access internet via wireless protocol. Following are the important point needs to be
considered while designed the workload model:
1. Signal Strength [Poor, Average, Good, Very Good]

2. Mobile technology [EDGE, 2G, 3G, 4G LTE, WIFI]
3. Bandwidth [Low, Medium, High, Very High]
4. Download Speed
5. Upload Speed
6. Battery Power
7. Geographical Location
Earlier native apps performs purely client side activity i.e. stand alone apps, but modern apps interacts with the server. E.g. Big
Fish Casino game app in Apple store, when you launch this app, it connects with the game server to update your profile, score,
chips and when you play with others, it sends/receives requests/responses.
To validate the client side parameter such as CPU usage, memory utilization, 2D, 3D graphics rendering etc. there are tools
available in the market such as Quadrant, Smart bench, AnTuTu System Benchmark, CF-Bench, Linpack, GL Benchmark etc.,
Using above tools, validation of client side parameters can be validated and analyzed. As I mentioned, modern native apps will
make connections with the server to accomplish the tasks.
Strategy to conduct performance tests on the native apps:
Following strategy helps us to conduct a performance tests on the native apps.
1. Requirements Elicitation and Understanding

2. Objective
3. Effective Workload Model

4. Creating and Conducting realistic load tests
5. Result Analysis and Benchmarking
Crystal clear requirements elicitation and thorough understanding helps testers to validate and verify the functional and non-
functional aspects of the apps.
Defining the objective guides testers to create effective realistic workload model for performance testing. Testers need to identify
the different set of configurations of hardware/software, platforms, and the test objective.
Workload should be based on real-time scenario, not all users will have 4G LTE connection, hence the workload model should
consists of combination of bandwidth settings, carriers, signal strength, geographical location etc.
Following are the unrealistic workload model, which is also testers should keep in mind:
Why above scenarios are unrealistic?
4G is not yet deployed completely in India. Only cities like Bangalore and Kolkata has 4G, other cities yet to deploy. While creating
workload, it is important to know about the geographical locations.
Also, mobile web applications dont need any upgrades or maintenance. Because it use browser to access the contents, whereas
native apps needs to be upgraded whenever the newer version is available. Scenarios should cover upgrades, installation and
uninstallation also.
Modern tools have the capability of all customizing the bandwidth profile, signal strength, upload speed, download speed etc. See
below snapshot of Neotyss NeoLoad Bandwidth profiles picker.
Image Courtesy: NeoLoad
Effective creation and conducting load tests will reveal behavior of the mobile apps. Also, if the concurrent users are heavy, the
probability of mobile apps failures will be more.
Following metrics should be captured and compared with the benchmarks:
Response time
Throughput
Network Utilization
CPU
Memory
Packets/Bytes/Hits sent and received
Constructive analysis of above metrics will helps to fix the issues identified in the mobile apps.
Conclusion:
Mobile market is bright in future, also mobile usage is exponentially increasing among internet users, it is important to design and
deploy mobile apps in the market. Also it is very challenging to address various components of mobile apps such as functionality,
usability, performance issues from native apps perspective. However, by defining and designing effective strategy and workload,
it will be stress-free task to validate and verify the apps.
Testing should start along with the development life cycle, to save the costs. By following above said approach, it will transform
the challenges and produce commendable result.
About the Author

Latest Posts
About NaveenKumar Namachivayam
Hi there! NaveenKumar N is a passionate and experienced Performance Test Analyst based in Chicago, USA with skill sets in
Manual Testing, Test Automation, Performance Testing, and Test Estimation. He loves to learn and experiment new trends and
models in Software Testing. Please stop by at his personal blogs: Excel Blog, DealsBrook, NaveenKumarN.in, and Affiliate Insights
blog and Software Testing Memes.
Perf Calculator free iOS app for performance testers - July 16, 2017
How to performance test AngularJS or ReactJS applications? - June 25, 2017
How to run Apache JMeter tests with Visual Studio Team Services? - June 6, 2017
Google Test Automation Conference GTAC 2017 - June 2, 2017
Whats new in StormRunner 2.5? - June 1, 2017
December 25, 2012 NaveenKumar Namachivayam Testing LoadRunner, Mobile apps, Mobile Testing
2 Comments on Mobile Apps Performance Testing Whitepaper
web load testing tools free

February 14, 2013 at 10:50 pm
Please tell me that youre heading to keep this up! Its so great and so important. I cant wait to read a lot more from you. I just feel like you know so
substantially and know how to make people listen to what you might have to say. This blog is just too cool to become missed. Terrific stuff, genuinely.
Please, PLEASE keep it up!
Reply
Appvance Inc.
April 4, 2013 at 3:39 am
Do you know if the TCP strategy itself, that is number of HTTP requests per persistent connection differs between Mobile and Web? I know the content
will be different, as you describe, when the HTTP headers are manipulated but I wondered if any other behavior changes on either the HTTP Client or
Server. I am thinking connection strategy might change for Mobile.
Reply
Leave a Reply
Your email address will not be published. Required fields are marked *
Comment
Name *
Email *
Website
Post Comment
SUBSCRIBE NOW TO RECEIVE LATEST UPDATES.
QAINSIGHTS NEEDS YOUR FEEDBACK.
Have a suggestion? Click here to provide your feedback.
SPONSORS
SPONSOR SPONSOR

Mobile Apps Performance Testing

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Mobile Apps Performance Testing

Uploaded by

Copyright:

Available Formats

8/18/2017 Mobile Apps Performance Testing

Full version Text-only version View source

December 25, 2012

Mobile Apps Performance Testing Whitepaper

What are mobile apps (applications)?

Mobile Load Testing

Difference between mobile testing and mobile apps performance testing

Challenges in Mobile Load Testing

Simulating bandwidth and speed

Addressing mobile apps load testing

Below is the snapshot of LinkedIn.com in IE 8 (default user agent string).

1. Signal Strength [Poor, Average, Good, Very Good]

Strategy to conduct performance tests on the native apps:

Following strategy helps us to conduct a performance tests on the native apps.

1. Requirements Elicitation and Understanding

3. Effective Workload Model

Why above scenarios are unrealistic?

Image Courtesy: NeoLoad

Following metrics should be captured and compared with the benchmarks:

About the Author

About NaveenKumar Namachivayam

2 Comments on Mobile Apps Performance Testing Whitepaper

web load testing tools free

SUBSCRIBE NOW TO RECEIVE LATEST UPDATES.

QAINSIGHTS NEEDS YOUR FEEDBACK.

Have a suggestion? Click here to provide your feedback.

You might also like