Development of a cross-platform Code

Coverage calculation tool for automated and

interaction based testing using Playwright, Cypress
and Electron

A Colloquium report based on Summer Internship

IPG-MTech

by

Abhishek Garain(2017IMT-002)

ABV INDIAN INSTITUTE OF INFORMATION

TECHNOLOGY AND MANAGEMENT
GWALIOR-474015

2021
1

CANDIDATES DECLARATION
I hereby certify that the work, which is being presented in the report, entitled
Development of a cross-platform Code Coverage calculation tool for automated and
interaction based testing using Playwright, Cypress and Electron, submitted to the
institution is an authentic record of my own work carried out during the period
22February 2021 to 30 June 2021. I have also cited the reference about the
text(s)/figure(s)/table(s) from where they have been taken.

Date: 13th September 2021 Signatures of the Candidate

Date:August 08 , 2021

Employee Name: Abhishek Garain

Employee ID: 300054132

Dear Abhishek Garain,

This is in furtherance to your resignation dated June 30 , 2021, wherein you had requested to be relieved from the services.

Kindly note we have completed all your Full & Final Settlement formalities and you are relieved from the services of “Rakuten India Enterprise Private Limited” with
effect from June 30 , 2021.

We thank you for your association with Rakuten India. We wish you great success in your future endeavours.

Your service record with the company is as follows:

Date of Joining Rakuten India Enterprise Private Limited: February 22 , 2021
Last Working Day at Rakuten India Enterprise Private Limited: June 30 , 2021
Last Designation: Intern

For Rakuten India Enterprise Private Limited

*Nisha Verma

Manager - Human Resources nisha.verma@rakuten.com

*This is a system generated letter and does not require any signature
 3

ABSTRACT
Code coverage becomes an important aspect in real life software development.
It is not only considered as a good practice but also becomes a necessity to generate
and maintain a threshold of code coverage, below which, the team of developers
needs to work on the testcases to make sure that every part of the code is tested
and there is absence of dead code in the project.
The internship project mainly consisted to two tasks, the first one being, building an
automated code coverage calculation tool that can track the code coverage of a
project in real-time or based on automated testcases, the second task being,
integrating the code coverage tool with Rakuten’s existing software testing platform,
Atssa2.0. Keywords: Code Coverage, Testcases, Testsuite, Atssa2.0, Cypress, JSON
 4

ACKNOWLEDGEMENTS
I take this opportunity to express my profound gratitude and deep regards to all my
mentors for their exemplary guidance, monitoring and constant encouragement
throughout the course of this internship. The blessing, help and guidance given by
them time to time shall carry me a long way in the journey of life on which I am
about to embark. I also take this opportunity to express a deep sense of gratitude to
my internship mentors Mr. Madhusudan Ganda(Senior Engineering Manager, GATD
Team, Rakuten India),
Mr. Rakhsit Arora(QA Lead, GATD Team, Rakuten India), Mrs. Sona Ojus(Software
Engineer, GATD Team, Rakuten India) for their cordial support, valuable information
and guidance, which helped me in completing the tasks through various stages.

(Abhishek Garain)
 TABLE OF CONTENTS
ABSTRACT 3
LIST OF TABLES 6
LIST OF FIGURES 7
1 Company Profile 10
1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.2 About the GATD team . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.2.1 About Atssa2.0 . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.3 Mission and Vision . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.3.1 Always Improve, Always Advance . . . . . . . . . . . . . . . . 12
1.3.2 Passionately Professional . . . . . . . . . . . . . . . . . . . . . 12
1.3.3 Hypothesize → Practice → Validate → Shikumika ....... 12
1.3.4 Maximize Customer Satisfaction . . . . . . . . . . . . . . . . . 13
1.3.5 Speed!! Speed!! Speed!! . . . . . . . . . . . . . . . . . . . . . 13
1.4 How Rakuten is transforming India . . . . . . . . . . . . . . . . . . . . 13
2 Problem Statement and Objectives 14
2.1 Problem Statement ............................ 14
2.2 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3 Literature Review 16
3.1 Code Coverage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.2 Importance of Code Coverage for a project . . . . . . . . . . . . . . . . 16
3.2.1 Examples of why code coverage is important . . . . . . . . . . 17
3.3 Types of Code Coverage metrics used in a project . . . . . . . . . . . . 17
3.4 Existing tools and platforms ....................... 18
3.4.1 BrowserStack:. . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.4.2 Cypress: ............................. 18
3.4.3 Zalenium: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.4.4 Disadvantages of existing platforms: . . . . . . . . . . . . . . . 19
5

TABLE OF CONTENTS 6
 3.5 Why Desktop app? ............................ 19
3.6 Testcases ................................. 20
3.7 Testsuites ................................. 20
3.8 Types of testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.8.1 Automation Testing . . . . . . . . . . . . . . . . . . . . . . . . 20
3.8.2 Manual Testing . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.8.3 Unit Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.8.4 Integration Testing . . . . . . . . . . . . . . . . . . . . . . . . 21
3.8.5 Regression Testing . . . . . . . . . . . . . . . . . . . . . . . . 22
3.8.6 Performance Testing . . . . . . . . . . . . . . . . . . . . . . . 22
3.8.7 Load Testing ........................... 22

4 Methodology 23
4.1 Tools and Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
4.1.1 Cypress . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
4.1.2 Instrumentation of code ..................... 24
4.1.3 IstanbulJS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
4.1.4 nyc ................................ 24
4.1.5 Playwright ............................ 24
4.1.6 XtermJS ............................. 25
4.1.7 Jenkins .............................. 25
4.1.8 ElectronJS ............................ 25
4.1.9 ReactJS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
4.1.10 NodeJS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
4.1.11 ngrok . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
4.1.12 Docker .............................. 26
4.2 System Design .............................. 27
4.2.1 Steps to achieve the final code coverage reports . . . . . . . . . 28
4.2.1.1 Instrumenting customer’s code ............ 28
4.2.1.2 Choice of testsuite: Manual testcase execution . . . . 28
4.2.1.3 Choice of testsuite: Automated testcase execution . . 30
4.2.1.4 Generation of code coverage reports ......... 30
4.2.2 Sending the reports back to the cloud platform ......... 33

5 Results and Conclusion 34

5.1 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
5.2 Conclusion ................................ 34

REFERENCES 35

LIST OF TABLES
 1.1 Facts at a glance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

LIST OF FIGURES
1.1 Rakuten’s Logo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.2 Rakuten x FC Barcelona ......................... 11

2.1 Problem Statement ............................ 14

3.1 Knight Capital Group’s stocks went down within minutes . . . . . . . . 17
3.2 The JSON coverage variable . . . . . . . . . . . . . . . . . . . . . . . 19
3.3 Unit Tests in Cypress test suite . . . . . . . . . . . . . . . . . . . . . . 21
3.4 Integration Tests in Cypress test suite . . . . . . . . . . . . . . . . . . . 22
4.1 Cypress’s dashboard ........................... 23
4.2 Playwright . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
4.3 Jenkins pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
4.4 Tunneling localhost:3000 with ngrok . . . . . . . . . . . . . . . . . . . 26
4.5 Flowchart of the system . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.6 The code coverage app . . . . . . . . . . . . . . . . . . . . . . . . . . 28
4.7 Script to instrument code . . . . . . . . . . . . . . . . . . . . . . . . . 28
4.8 The out.json file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
4.9 Command to generate the reports . . . . . . . . . . . . . . . . . . . . . 31
4.10 Code coverage report in HTML format having 50% code coverage . . . 32
4.11 Code coverage report in HTML format having 100% code coverage . . 32
4.12 Code coverage report in text format ................... 33
4.13 Structure of the folder . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
8
 9
LIST OF FIGURES

ABBREVIATIONS

PoC Proof of Concept PM

Product Manager
EM Engineering Manager
KT Knowledge Transfer
LOC Lines of Code
GATD Global AdTech Team
CHAPTER 1

Company Profile

Figure 1.1: Rakuten’s Logo

This chapter includes the profile of Rakuten, Rakuten India and the Global
AdTech Team at Rakuten India of which I was a part of, during my four months
internship.

1.1 Introduction
Rakuten Group, Inc. is a Japanese electronic commerce and online retailing company
based in Tokyo, founded by Hiroshi Mikitani in 1997. Centered around Rakuten
Ichiba, its a business-to-many E-Commerce platform and one of the largest e-
commerce sites in Japan, its businesses include financial services utilizing fintech, as
well as digital content and communication services such as messaging app Viber, e-
book distributor Kobo, and Japan’s fourth mobile carrier Rakuten Mobile. Rakuten
has 18,000+ employees worldwide, operating in 29 countries and regions, and its
revenues totaled US$7.2 billion with operating profits of about US$347.9 million as
of 2016. It is often referred to as "the Amazon of Japan".
Its major investments include Buy.com (now Rakuten.com in the US), Priceminister,
Ikeda, Tradoria, Play.com, Wuaki.tv, Ebates, Viki, The Grommet. The company also
has stakes in Ozon.ru, AHA Life, Lyft, Cabify, Careem, Carousell and Acorns. [1]

10
Founded 7 February 1997
Type Public
E-Commerce, Cellular Networks, TV and Entertainment,
Services
AI, Contextual Ad platform, E-books
CHAPTER 1. COMPANY PROFILE 11
Rakuten Ichiba, Rakuten Networks, Rakuten Viki, Rakuten
Products
Advertising, Rakuten Kobo, Rakuten TV and many more
Headquarters Setagaya, Tokyo, Japan
Indian Office Crimson House Bangalore, Karnataka, India
Table 1.1: Facts at a glance

Figure 1.2: Rakuten x FC Barcelona

1.2 About the GATD team

The Global AdTech team mostly consists of SDEs and QA members working on the
displaying and managing contextual Ads on Rakuten Ichiba, the E-commerce website
of Rakuten. Along with working on the Ads platform, they are also working on some
projects that are yet to be released in the market. One such project is Atssa2.0 that I
was working on during the internship.
CHAPTER 1. COMPANY PROFILE
12

1.2.1 About Atssa2.0

Atssa is codeless software automation testing platform built with keeping Dev and
QA engineers requirement at the core. It enables them to perform functional
testing, performance testing and model-based testing throughout the life cycle of
the product/project. The testing can be done across multiple operating systems and
browsers. Atssa comes fully packed with features and can perform multiple tests
efficiently and effectively on a single-click.
Atssa is a powerful platform for automation because the users can create and
execute hundreds of tests on hundreds of machines simultaneously. The platform
also has auto tag suggestion feature that can help create tests fast and efficiently.
Atssa also makes it easier to manage tests, while also providing feature rich test
reports with screenshots and detailed test data about passing and failing of tests.

1.3 Mission and Vision

Rakuten Group has an unswerving commitment to building an egalitarian society by
empowering individuals and companies to be successful in business and in life. The
Five Principles for Success are designed to keep each of us on track, to ensure we
make continuous progress toward our ultimate goals. [2]

1.3.1 Always Improve, Always Advance

There are only two kinds of people in the world: Best Effort people are satisfied with
the status quo; when they fail, they console themselves by saying "I did my best."
Get Things Done people are absolutely committed to reaching their goals. With
enough determination and effort by being a Get Things Done person, you can
achieve anything.

1.3.2 Passionately Professional

Rakuten is an organization composed of true professionals. Real professionals in any
field whether in sports, the arts, science, business or anything else, only reach the
top because they have the self-discipline to develop and improve themselves
continuously. Only by consistently working many times harder than our competitors
can we consistently win.

1.3.3 Hypothesize → Practice → Validate → Shikumika

Success in business depends on making and executing concrete specific action plans.
CHAPTER 1. COMPANY PROFILE 13
1.3.4 Maximize Customer Satisfaction
Rakuten is first and foremost a service company. We’re proud to serve our
customers and we ceaselessly seek to improve their experience.

1.3.5 Speed!! Speed!! Speed!!

The internet makes the window of opportunity for every small business. Accomplish
in one month what it takes other companies a year to do, because we can only win
by being many times faster than our competitors.

1.4 How Rakuten is transforming India

Rakuten India is the development centre and key technology hub of the Rakuten
Group, Inc. They enable businesses with Rakuten Inc’s depth of knowledge in
multiple streams of technology such as Mobile and Web Development, Web
Analytics, Platform Development, Backend Engineering, Data Science, Machine
Learning, Artificial Intelligence and much more. Their unique 24/7 support center
ensures reliability and sustenance of the Rakuten Ecosystem. With dedicated
centres of excellence for Mobile Application Development, Data Analytics,
Engineering, DevOps and Information Security, they ensure the success of multiple
units of Rakuten Group, Inc. With 1000+ employees and growing, Rakuten India is
housed in Crimson House Bangalore in the heart of the city.
[3]

CHAPTER 2

Problem Statement and Objectives

2.1 Problem Statement

 Figure 2.1: Problem Statement

This chapter deals with the problem statement that acted pivotal for the
internship project and the objectives set forth and accomplished during the
internship period. The internship mainly focused on one problem to solve: There are
no available platforms that can find code coverage based on real-time software
testing.

The project majorly consists of two goals that we had to achieve.

• To create a codeless automated code coverage calculation tool that can

measure code coverage for any particular project based on manual testing or
automation testing.

• To integrate the code coverage tool with the existing Atssa2.0 platform for
preserving the code coverage reports for all the testcases present in the
portal.

14
 15
CHAPTER 2. PROBLEM STATEMENT AND OBJECTIVES

2.2 Objectives
• To develop a method to fulfill the requirements.

• To choose the tech stack to develop the project based on high level research
and PoC.

• To choose the platform on which the tool needs to be developed based on

discussion with PMs and stakeholders.

• To design the high level system architecture of the whole project and get it
approved from senior EMs and leads.

• Prepare documentation and KT sessions for team-mates.

CHAPTER 3

Literature Review
The sole purpose of doing an extemsive literature review of all the available
technologies was needed to create a market viable product that can be shipped
easily to the customers without any hassle. Following are the topics that were
explored in-depth during the course of the internship.

3.1 Code Coverage

Code coverage is a metric that helps us to understand how much of the codebase
was tested and which part of the codebase is not used by any feature. It’s an
important feature to help us assess the quality of the testcases and quality of the
code. [4]

3.2 Importance of Code Coverage for a project

Before building any project based on an important software engineering metric, we
need to understand the importance of that particular metric and why it is widely
used in the industry.
The importance of code coverage for any project are: [5]

• Find out which parts of the codebase is covered by tests and what is not: This
will help up to write testcases for uncovered code.

• Find out code execution paths that are missed.

• High code coverage points to a well-written and testable code.

• This will also enforce a culture of writing unit tests using code coverage rules.

• A high code coverage leads to confidence in code written by developers.

• A high code coverage is crucial to investors to pitch them that we are not
wasting lot of capital in maintaining big piece of code.

16
• A high code coverage matters to some potential customers and brings
confidence in them.
CHAPTER 3. LITERATURE REVIEW 17
• A low code coverage could lead to code refactoring making it more optimal.

• Code coverage reports also signifies whether tests are executed or not.

3.2.1 Examples of why code coverage is important

• A famous quote by Edsger Djikstra, the person behind the famous Djikstra’s
shortest path algorithm says that, Testing shows the presence, not the
absence of bugs, so we can understand that writing unit tests is not enough to
find presence of bugs.

• Knight Capital Group, a HFT firm lost $460m in 45 minutes on 01 August, 2012
due to deployment of a new algorithm. While the algorithm was perfectly
tested to check for the presence of bugs, what they forgot was, removing old
pieces of code, and that interfered with the newly added algorithm and that
brought down the servers during the peak business hours. The RCA report
stated that their mistake was to not have any code coverage calculation tool
that could have reported them the correct usage of the code before
deployment of the algorithm.
[6]

Figure 3.1: Knight Capital Group’s stocks went down within minutes

3.3 Types of Code Coverage metrics used in a project

1. Statement Coverage: Statement Coverage helps to find the coverage related
to the smallest possible units available in the code such as assignments, return
statements, assertions, loops and conditionals.
CHAPTER 3. LITERATURE REVIEW 18

2. Branch Coverage: Branch Coverage evaluates the of logical branches covered

in the code such as, if-else block.

3. Function Coverage: Function Coverage refers to the number of functions

present in the code that were tested.

4. Line Coverage: Line Coverage refers to the number of lines of code evaluated
by the tests. [7]

3.4 Existing tools and platforms

While creating a new platform to achieve a goal and release it for the end-users, we
did a lot of market research to find out the potential opponents and shortcomings of
their platforms that we were going to address in our platform.

3.4.1 BrowserStack:
BrowserStack is an Indian cloud web and mobile testing platform providing
developers the power to test their websites and mobile applications across different
types of browsers in different platform in real-time. BrowserStack is a leading
platform for testcase execution but it doesn’t provide any feature to calculate code
coverage for the testcases. [8]

3.4.2 Cypress:
Cypress is a newly launched front-end testing tool built for the modern web. They
address the key difficulties developers and QA engineers face when testing modern
web applications. They make it possible to set up tests, write tests, run tests, debug
tests. It is often compared to Selenium and other browser automation framework,
however Cypress is both fundamentally and architecturally different. [9]
Cypress provides this library called @cypress/instrument-cra that helps to
instrument code written in ReactJS.

3.4.3 Zalenium:
Zalenium is a flexible and scalable container based Selenium Grid with video
recording, live preview, basic auth & dashboard. Zalenium is a very powerful
platform to execute testcases but at the same time, to generate code coverage
reports, we need to write scripts manually. [10]
CHAPTER 3. LITERATURE REVIEW 19

3.4.4 Disadvantages of existing platforms:

The already available platforms are good enough to execute already available
testcases in a testsuite and find the code coverage of the project after the testcase
execution has completed.
But the only disadvantage of these platforms are, they can’t help to find the code
coverage of a project based on manual testcase execution and find code coverage
based on real-time testcase execution.
To address this particular issue, we had to create our desktop app that can track the
code coverage JSON object in real-time and generate the code coverage reports as
and when the user interacts with the project.

3.5 Why Desktop app?

As mentioned previously, we did an intensive PoC before coming to the point that
we need to use a desktop app to achieve the code coverage reports, some of the
reasons to choose a desktop app as a development platform rather than a website
were:

• The JSON coverage variable resides in the devtools of the browser but normal
browsers can’t provide a programmatic interface to access those variables.
CHAPTER 3. LITERATURE REVIEW 20

Figure 3.2: The JSON coverage variable

• To achieve this feature, we need to trigger a new browser programatically,
such that we get complete access of the devtools and browser variables.

• So, we need a platform to trigger our own browser, and that’s where the
desktop app comes into picture.

• Also, we had to execute some scripts in the user’s syatem, and this couldn’t be
done from a browser as we get access to a virtualized environment in a
browser but we can easily get access to user’s system through a desktop app.

3.6 Testcases
A test case is a set of steps that are performed on a project to determine if it satisfies
software requirements and functions correctly. The purpose of a test case is to
determine if various features within a system are performing as desired and to
confirm that the system satisfies all related standards, guidelines and stakeholder
requirements. The process of writing a test case can also help reveal errors or
defects within the system.

3.7 Testsuites
Testsuites are softwares that are used to run automated testcases, it helps to track
which particular testcases are currently running, the testcases that have already
completed execution and the testcases that are yet to be executed.

3.8 Types of testing

There are various types of software testing methodologies that have different
purpose to solve in the software engineering world. They are:

3.8.1 Automation Testing

The use of software tools to control the execution of tests, compare their outcomes,
and report functions to avoid manual activities is called automation testing.
So, the tests will be written by developers or the QA engineers and some sort of
testsuite will be used to execute the testcases.
CHAPTER 3. LITERATURE REVIEW 21

3.8.2 Manual Testing

Manual testing is a software testing process where, testcases are manually executed
without the need of any automated tool. All testcases are manually executed by the
tester according to the end user’s perspective. It ensures whether the application is
working as mentioned in the requirements document or Jira tickets.
So, the tests will be written by developers or the QA engineers and it will be the
responsibility of the QA engineers or the developers to execute the testcases
manually to check for performance degradation or presence/absence of bugs in the
project.

3.8.3 Unit Testing

Unit Testing is a type of software testing where individual units or nuclear
components of a software are tested. The purpose is to confirm that each unit of the
software code performs as expected. Unit Testing is done during the development of
an application by the developers.

Figure 3.3: Unit Tests in Cypress test suite

3.8.4 Integration Testing

Integration Testing is defined as a type of testing where individual software modules
are integrated logically and tested in one go. A typical software project consists of
multiple software modules, coded by different programmers. The purpose of this
CHAPTER 3. LITERATURE REVIEW 22

level of testing is to find out if features are breaking after individual components are
merged.

Figure 3.4: Integration Tests in Cypress test suite

3.8.5 Regression Testing

Regression Testing is defined as a type of software testing to confirm that a recent
program or code change has not adversely affected existing features.

3.8.6 Performance Testing

Performance Testing is a software testing process to test the speed, response time,
stability, reliability, scalability and resource usage of a software application under
particular workload.
The main purpose of performance testing is to check if the project is complying to
the standards that are mentioned in the documentation, features like accessibility
etc.

3.8.7 Load Testing

Load Testing is a non-functional software testing process in which the performance
of software application is tested under a specific expected load. It determines how
the software application behaves while being accessed by multiple users
simultaneously. The main purpose of performance testing is to check if the project is
able to handle huge amount of traffic when it goes into production.
CHAPTER 4

Methodology

4.1 Tools and Techniques

4.1.1 Cypress
Cypress is a newly launched front-end testing tool built for the modern web. They
address the key difficulties developers and QA engineers face when testing modern
web applications. They make it possible to set up tests, write tests, run tests, debug
tests. It is often compared to Selenium and other browser automation framework,
however Cypress is both fundamentally and architecturally different. [9]
Cypress provides this library called @cypress/instrument-cra that helps to instrument
code written in ReactJS.

Figure 4.1: Cypress’s dashboard

23
4.1.2 Instrumentation of code
To find out which part of code has been executed and which part of code has not
been executed, we have several options, the first one being, use large scale logging
commands like console.log() in JavaScript or using debuggers like binding,pry in Ruby
CHAPTER 4. METHODOLOGY 24

on Rails etc. But these approaches are clearly not feasible when we are working with
large piece of codes having 1000+ LOC. So, to deal with these big codebases, we can
place counters after each line of code, and after execution of a testcase, we can
retrieve the value of these counters and we can easily comment if a particular line of
code is touched or not, also, we can comment about the frequency of execution of
each line of code based on the values of these counters. This task is known as
instrumentation, where the library adds counters to all lines of codes and compares
two JSON objects containing values of these counters before and after execution of a
testcase.

4.1.3 IstanbulJS
IstanbulJS helps to instrument ES5 and ES2015+ JavaScript code with line counters,
so that we can track the quality of our unit-tests exercise the given codebase.
IstanbulJS comes as a NPM package that needs to be installed in any ES5+ projects
and it needs to be run with react-scripts to instrument the codebase.

4.1.4 nyc
nyc is a command-line-client for Istanbul that works well with most JavaScript testing
frameworks: tap, mocha, AVA, etc. In my project, nyc was used to generate code
coverage reports from binary data containing counters of each line of code.

4.1.5 Playwright
Playwright is a NodeJS library to automate Chromium, Firefox and WebKit with a
single API. Playwright is built to enable cross-browser web automation that is
evergreen, capable, reliable and fast. Headless execution is supported for all the
browsers on all platforms.

Figure 4.2: Playwright

4.1.6 XtermJS
XtermJS is a front-end component written in TypeScript that lets applications bring
fully-featured terminals to their users in the browser. XtermJS was used to integrate
a terminal inside the desktop app such that users can execute any command inside
CHAPTER 4. METHODOLOGY 25

the desktop app that will help to ease the lives of users. Also, all the commands that
are run by the desktop app will be executed in these terminals directly and not
behind the scenes such that user has clarity that what all commands are executed in
their system.

4.1.7 Jenkins
Jenkins is the leading open-source automation server, it provides hundreds of
plugins to support building, deploying and automating any project. Jenkins is a self-
contained, open source automation server which can be used to automate all sorts
of tasks related to building, testing, and delivering or deploying software.

Figure 4.3: Jenkins pipeline

4.1.8 ElectronJS
Electron is a free and open-source software framework developed and maintained
by GitHub. It helps to develop desktop GUI applications using web technologies: it
combines the Chromium rendering engine and the NodeJS runtime.
The code coverage desktop app was built on top of ElectronJS, it was chosen on the basis
of the literature review and PoC that our team did in the initial sprint.
4.1.9 ReactJS
React is a free and open-source front-end JavaScript library for building user interfaces
or UI components. It is maintained by Facebook and a community of individual
developers and companies. React can be used as a base in the development of single-
page or mobile applications
CHAPTER 4. METHODOLOGY 26

ReactJS was used to create the UI for the code coverage desktop app. MaterialUI was
the UI library used to design the components.

4.1.10 NodeJS
NodeJS is an open-source, cross-platform, back-end JavaScript runtime environment
that runs on the V8 engine and executes JavaScript code outside a web browser.
NodeJS was used as the runtime to power the desktop app and connect it with the
frontend packages and to render the UI.

4.1.11 ngrok
ngrok provides a real-time web UI that can introspect all HTTP traffic running over
tunnels. It is a cross-platform application that enables developers to expose a local
development server to the internet with minimal effort.
ngrok was used in the project to tunnel the user traffic to execute testcases on a local
version of the project in our cloud based testcase execution tool i.e. Atssa2.0.

Figure 4.4: Tunneling localhost:3000 with ngrok

4.1.12 Docker
Docker is a set of platform as a service products that use OS-level virtualization to deliver
software in packages called containers. Containers are isolated from one another and
bundle their own software, libraries and configuration files; they can communicate with
each other through well-defined channels.
Docker images of Windows and Ubuntu were used to generate an executable file
for the ElectronJS project as I only had access to a MacOS system. Docker was also
used to dockerize the React project into a docker image such that it can run anyone’s
system without much hassle.
CHAPTER 4. METHODOLOGY 27

4.2 System Design

Figure 4.5: Flowchart of the system

Figure 4.6: The code coverage app

4.2.1 Steps to achieve the final code coverage reports

4.2.1.1 Instrumenting customer’s code

The first step before choosing any testsuite is to instrument the user’s code such that,
once the code is hosted and the project is used, JSON coverage variable can be tracked
CHAPTER 4. METHODOLOGY 28

and the reports can be generated. To instrument the code, we are using
cypress/instrumentcra library that is used to spawn the react-scripts process to
instrument the code and host the code at the same time.

Figure 4.7: Script to instrument code

4.2.1.2 Choice of testsuite: Manual testcase execution

In manual testcase execution, the users can easily test a particular scenario by manually
testing the project and finding the code coverage simultaneously.
1. To facilitate this task, we will need to record all the movements of the user in the
browser to find which features are used, and similarly, we can generate the JSON
object for the coverage data.

2. As we know that we cannot extract devtools variables from a normal browser, we

would requrie a browser automation framework like Selenium or Puppeteer or
Playwright. After an intensive PoC, we came to the decision that Playwright can
be used to solve our needs because it’s the modern open source browser
automation framework powered by Microsoft that can easily be integrated with
NodeJS, as the bindings for Selenium were only working seamlessly for Java.

3. Once the choice of browser automation framework had been chosen, we

integrated it with the desktop app.

4. The user needs to enter two things to find the code coverage:

(a) The localhost port where the project is hosted

(b) The folder in which the project is located

5. Once the user fills in all the details and starts the process, the instrumentation of
the code begins.

6. Once the instrumentation process ends, user will be provided with a choice of
browser where they want to test the scenario. They are:
CHAPTER 4. METHODOLOGY 29

(a) Chromium: Chromium is an open-source browser project aiming to make

Internet surfing much more safer, faster, and more stable for the users.

(b) Firefox Nightly: Nightly is an unstable testing and development platform for the
Mozilla Firefox browser.

(c) Webkit: WebKit is the web browser engine that powers all the Internet related
apps of MacOS like Safari, Mail, App Store etc.

All of these browsers serves as an engine that powers Google Chrome, Mozilla
Firefox and Safari browser respectively and is mostly used to develop
softwares that needs programmatic access to the browser internals.

7. The users can perform manual testing in these newly opened browsers and can
manually test their project and automatically the code coverage reports will be
updated.
4.2.1.3 Choice of testsuite: Automated testcase execution

In automated testcase execution, the users can use a testsuite like Atssa2.0 to
execute a testcase in the cloud and after the testcase is executed, the JSON object
can be extracted from the browser and that can be used to calculate the code
coverage reports.

1. Facilitating this task is easier as comparison to the manual testcase execution as

the testcases are being run in the cloud browser automation framework and we
don’t need to track the coverage variable in real-time. The coverage JSON
variable will be extracted from the browser after the testcase has been
executed.

2. To execute the testcases and find the coverage variable, we need a tunneled
URL of user’s localhost URL. To achieve this, we are using ngrok to tunnel the
localhost URL and Atssa2.0 will use this URL to run the testcases.

3. Once the testcases are executed, the browser automation library i.e. Selenium
will extract the coverage object and it will save the response in the MongoDB
database.

4. Once the user wants to explore the results of the executed testcases in Atssa2.0
dashboard, the code coverage variable will be sent as an API response to the
dashboard.

5. The user needs to enter two things to find the code coverage:
CHAPTER 4. METHODOLOGY 30

(a) The user needs to copy the coverage object from the dashboard and paste it in
the desktop app.

(b) The folder that contains the project.

6. Once the user fill in all the details and starts the process, the code coverage
reports are generated.

4.2.1.4 Generation of code coverage reports

Once the JSON coverage data has been extracted from the browser automation framework
and accessible to the desktop app, the data will be written in the file ‘./.nyc_output/out.json‘,
after that, the nyc tool will be used to use the ‘out.json‘ file to generate code coverage
reports in HTML form.

Figure 4.8: The out.json file

CHAPTER 4. METHODOLOGY 31

Figure 4.9: Command to generate the reports

Figure 4.10: Code coverage report in HTML format having 50% code coverage

Figure 4.11: Code coverage report in HTML format having 100% code coverage
CHAPTER 4. METHODOLOGY 32

Figure 4.12: Code coverage report in text format

4.2.2 Sending the reports back to the cloud platform

The reports that are generated in the user’s system can be sent back to the cloud for
future purpose. To solve this feature, we provided a feature to select the folder from
user’s system and the HTML files can be uploaded to Amazon S3 storage, once the
files are stored, it can be retrieved from Atssa2.0 dashboard, this also solves the
purpose of referring to the reports in future to check if the code coverage has
increased or decreased in comparison to current reports. Based on the verdict, we
can comment that the tests are robust enough to check all the features and utils
implemented in a project.

Figure 4.13: Structure of the folder

CHAPTER 5

Results and Conclusion

5.1 Results
• All the proposed objectives were successfully implemented.

• The code coverage desktop app was packaged and shipped to the customers
in the form of executable files for Windows, Linux and MacOS platforms.

• The reports storage mechanism was successfully integrated with the cloud
infrastructure.

• Enterprise users can download the desktop app to manually calculate the code
coverage, else normal users can track the JSON code coverage variable
through the dashboard of Atssa2.0.

5.2 Conclusion
The four months internship at Rakuten was a complete success in terms of
understanding a multi national corporation from a close point of view and in terms
of learning new technologies and experimenting new approaches to build new stuffs
with a tinge of business logic to make a product successful. One of the biggest
takeaway was how big teams manage their work and get work done with regular
meetings, weekly sprints and how agile methodology of making a software works.
Several concepts which were learned theoretically like agile methodology was used
practically during the internship. Several non-technical concepts like growth
mindset, importance of diversity and inclusion, necessity to pursue other passions
etc. were also emphasised by the organization, especially the senior management.
This gave valuable life lessons. I also got the opportunity to learn several new
technologies like ElectronJS, Code Coverage, System Design etc. My manager was
extremely happy with my progress right from the first week and was impressed by
the ramp-up speed and this motivated me further. My

34
CHAPTER 5. RESULTS AND CONCLUSION 35
internship was a great learning experience which was both productive and
enjoyable.
REFERENCES
[1] Rakuten Group Inc. Rakuten group inc. https://global.rakuten.com/corp/.

[2] Rakuten Group Inc. Rakuten shugi | rakuten

group inc.
https://global.rakuten.com/corp/about/philosophy/principle/ethic.

[3] RIEPL. Rakuten india enterprise private limited. https://corp.rakuten.co.in/.

[4] Atlassian. An introduction to code coverage.

https://www.atlassian.com/continuous-delivery/software-testing/code-
coverage.

[5] Rafiullah Hamedy. 10 reasons why code coverage matters, 2020 (accessed
Sept 11, 2021). https://codeburst.io/10-reasons-why-code-coverage-
matters9a6272f224ae.

[6] Bishr Tabbaa. The rise and fall of knight capital — buy high, sell low. rinse and
repeat., 2018 (accessed Sept 11, 2021). https://medium.com/dataseries/the-
riseand-fall-of-knight-capital-buy-high-sell-low-rinse-and-repeat-
ae17fae780f6.

[7] Thomas Hamilton. Code coverage tutorial: Branch, statement, decision, fsm,
2021 (accessed Sept 11, 2021). https://www.guru99.com/code-
coverage.html.

[8] BrowserStack. Browserstack. https://www.browserstack.com/.

[9] Cypress. Cypress. https://www.cypress.io/.

[10] Zalenium. Zalenium. https://opensource.zalando.com/zalenium/.

36