Proc.

of the 2nd International Conference on Electrical, Communication and Computer Engineering (ICECCE)
12-13 June 2020, Istanbul, Turkey

A Comparative Analysis of Quality Page Object and

Screenplay Design Pattern on Web-based
Automation Testing
*Tessy Badriyah
Dini Yuniasri Umi Sa’adah
Politeknik Elektronika Negeri
Politeknik Elektronika Negeri Politeknik Elektronika Negeri
Surabaya, INDONESIA
Surabaya, INDONESIA Surabaya, INDONESIA
tessy@pens.ac.id
dini.yuniasri@gmail.com umi@pens.ac.id
*Corresponding author

Abstract— In software engineering, a design pattern is a
standard solution for solving problems that often occur in the
design, including developing automation testing. Today, page
object and screenplay design patterns are often used by quality
assurance engineers to develop the web-based automation
testing design. The application of the design pattern for
automation testing cannot be arbitrary because selecting the
right design pattern should be able to support the optimization
of automation testing. Therefore, in this study, a comparative
analysis of the two design patterns was proposed using the
QMOOD (Quality Model of Object-Oriented Design) model.
The QMOOD model has the characteristics of four levels and
three mappings. The main assessment of the QMOOD model
shows in the first level, i.e., Functionality, Reusability,
Flexibility, Understandability, Effectiveness, and Extendibility.
While the three mappings contained in QMOOD serve as a
link between levels so that inter-levels on QMOOD can be
integrated. By applying the QMOOD model as a parameter to
measure the quality of the page object design and screenplay
pattern, we know that the two design patterns have their own
superiority. This research can become a reference or reference
source for quality assurance engineers in optimizing
automation testing by knowing the quality values of the page
object design and screenplay design.
Keywords— Automation Testing, Design Pattern, Page
Object, Screenplay, QMOOD
I. INTRODUCTION
Automatic testing to be one of the alternative methods
used by several companies in conducting software testing. It
is related to the testing of automation software, which can
significantly reduce the effort required to carry out the
testing [1]. Besides, automation of software testing can also
increase the number of tests that can be done in a limited
time significantly as well. So, the testing should take many
hours in the running process will take several minutes by
using automation testing [2].
In software engineering, design patterns become a
standard solution to resolve the problems that often occur in
the design of the software, as well as in developing
automation testing. The use of the design pattern that can
speed up the development process and the maintenance is the
reason for the importance of using design patterns on
automation testing. It is because the design pattern is a
template design that is ready to be used to solve problems in
a variety of situations, in which the design template derived
from the design that has been successfully run by expert
designers before. By applying the techniques that have been
tested, the design pattern will become more accessible to a
new developer [3].
So far, quality assurance engineers often use page object
design pattern compared to other design patterns in
developing automation testing. The use of page objects that
make objects on a web-based application into objects in
smaller parts, making the page object is easy to implement
and easy to do maintenance. But nowadays, there is a design
pattern named screenplay design pattern that is no less
popular among engineers in developing quality assurance
testing automation. Implementing some software engineering
principles, i.e., the Single Responsibility Principle and Open-
Closed Principle in the screenplay design pattern, make
quality assurance engineers easier to maintain and document
[4]. However, in determining the design pattern for
automation testing would not be indiscriminate, because the
right design pattern should be able to support the
optimization of the automation of testing, especially on the
web. The primary key of design pattern to determine whether
it can support the optimization automation testing or not can
be known from the quality of the design pattern. Assessment
of Functionality, Reusability, Flexibility, Understandability,
Effectiveness, and extendibility of a design pattern can be the
way to determine the quality of the design pattern.
II. RELATED WORKS
Some researchers have conducted some researches
related to determining a good design pattern to support the
development of automation of testing and other software
development. A research related to the effectiveness of the
page object design pattern to the development of automation
of testing has been examined by Leotta et al. [5]. In this
research, they compared the use of automation web testing
by using a page object design pattern and without using a
pattern page object design based on the time needed to
maintain and the total LOC modified in each test case. Based
on this research, the time required by the web automation
testing without the use of design pattern needs a longer time
when compared with web automation testing page by using

978-1-7281-7116-6/20/$31.00 ©2020 IEEE

Authorized licensed use limited to: Cornell University Library. Downloaded on August 31,2020 at 08:53:20 UTC from IEEE Xplore. Restrictions apply.
page object design pattern. The total LOC that modified on
web automation testing without using a design pattern is also
higher when compared to the automation testing web page
by using page object design pattern.
Ampazoglou et al. presented a methodology for
comparing the three design patterns. The design pattern that
they test is Bridge, Visitor, and Abstract Factory design
pattern, with two alternative solutions using multiple quality
attributes [6]. From this research, we know that the design
pattern can implement the make getting more or less
beneficial than the alternative design. More specifically, the
identification of thresholds can be beneficial for decision
making during system design and refactoring..
While Pradhan, Dwivedi, and Rath conducted a study to
find out the analysis of the influence of the design pattern of
Bridge, Visitor, and Abstract Factory by applying QMOOD
(Quality Model of Object-Oriented Design) [7]. The main
assessment of the design patterns is Functionality,
Reusability, Flexibility, Understandability, Effectiveness,
and Extensibility by using QMOOD. The implementation of
the QMOOD in determining the effect of the design pattern
provides the objective results. It can help developers choose
the specific attributes to optimize the software when using
the design pattern.
III. METHODOLOGY
This study was inspired by Pradhan et al.’s study [7].
This methodology can be used to determine the differences
in the quality of page object and screenplay design patterns
implemented in web-based automation testing. Figure 1
shows the system design in implementing this study.
Fig. 1. Research System Design
Figure 1 shows that this research's flow process starts
from the input until the process of the expected output of this
research. The input used in this study is test cases from a
website. Test cases are the components and the criteria that
need to be tested in a system. The test cases used in this
research were included in a Happy-Flow condition. Happy-
Flow is a component condition tested according to the
procedure or the rules and conducted to get the desired
results.
Authorized licensed use limited to: Cornell University Library. Downloaded on August 31,2020 at 08:53:20 UTC from IEEE Xplore. Restrictions apply.
Then the inputted test cases will be translated into the
main project scenario for the process of automation testing.
The main project's contents depend on the design pattern that
will be used, page object, or screenplay. The design pattern
will be integrated with the reporting framework and
automation framework. In this study, the reporting
framework used is the Serenity BDD framework. Serenity
BDD is an open-source library that aims to help users write
better and structured quickly and easily, making it easier for
users to do the maintenance. As for the automation
framework that is used in this research is Selenium.
Selenium is a series of open-source tools, which can make
web-based automation testing more efficient.
From the automation testing process, the output will be
obtained in the form of a test report. From each test done, it
will automatically generate the results of the trial by the
Reporting Framework. Later, the Reporting Framework will
display the report results of each scenario's achievements and
present them in easy to analyze and understand display.
However, to determine the quality of a design pattern, the
test report's result is not enough. Particular calculations are
also needed to measure each quality of the design pattern by
applying the QMOOD method. That way, researchers will be
facilitated in analyzing the quality values of the design
pattern.
A. QMOOD Calculation
Bansiya and Davis have defined metrics that can be
weighted together to give a value for different software
quality properties that they have set [8]. And based on the
research conducted by Sharma and Dubey [9], QMOOD
considered a comprehensive assessment method that can
cover all aspects of the quality of a software.
QMOOD (Quality Model for Object-Oriented Design) is
a hierarchical model that implements a way to map the code
metrics to a higher level of abstraction [7]. The structural
illustration of QMOOD is shown in Fig. 2.
Fig. 2. QMOOD Structure
From the illustration in Fig. 2, it shows that QMOOD has
four levels (L1 to L4) and three mappings (L12, L23, L34)
used to connect the four levels. At the first level, L1,
represents Design Quality Attributes, which consists of
Functionality, Reusability, Flexibility, Understandability,
Effectiveness, and extendibility. At the second level, L2 of
QMOOD represents Object-Oriented Design Properties that
include abstraction, encapsulation, coupling, cohesion,
complexity, and size of design used to represent the
characteristics of design properties. At the third level, L3
represents the Object-Oriented Design, which supports the
Design Properties. On the fourth level of QMOOD, it
represents the Design Component, which allows classes or
relationships. In this study, we have determined several
Object-Oriented Design Metrics related to Design Properties,
which can be seen in TABLE I.
 TABLE I. OBJECT-ORIENTED METRICS AND DESCRIPTION website. Prestashop is a content management system (CMS)
Design Metrics software used to develop a shopping cart system on e-
No. Name of Metric
Property used commerce [10]. There are six scenario testing carried out in
1. Design Size Size Size of Design this study included in a Happy-Flow state. The following are
six scenario testing that will be carried out as a whole on the
2. Hierarchies NOC Number of Hierarchies Prestashop website.
3. Abstraction ANA Average Number of Ancestors
1) The user or administrator can perform well into the
4. Encapsulation DAM Data Access Metric main website on the frontend and backend.
2) The user can order the product based on
5. Coupling DCC Direct Class Coupling
recommended products that are on the main page.
Cohesion Among Methods of a
6. Cohesion CAM
class
3) The user can order products by inputting the desired
product in the search column.
7. Composition MOA Message of Aggregation
4) The user confirms the order with an existing address.
Measure of Functional
8. Inheritance MFA
Abstraction
5) The user confirms the order by entering a new
address.
9. Polimorphism NOP Number of Polymorphic
6) The administrator can change the status of customer
10. Messaging CIS Class Interface Size bookings on the backend.
11. Complexity WMPC Weighted Method Per Class
And in this trial, we have completed and successfully run
the six testing scenarios by implementing the page object
design pattern and screenplay on the automation testing.
In the first mapping, L12 is a mapping that connects the Furthermore, each matrix contained in the TABLE I will be
first and second levels of QMOOD, as shown in TABLE II. implemented in each design pattern that has been developed
In the second mapping, L23 is used to connect between the before. So the value of each of these metrics can be used to
Design Properties and Design Metrics. And in the third the next level, which is Object Oriented Design Properties.
mapping, L34, are used to inform the design metrics that will This following sections will describe how we get the value of
be applied to the classes and the relationships contained in the relevant metrics.
the class diagram [7].
A. Size of Design
TABLE II. OBJECT-ORIENTED METRICS AND DESCRIPTION In this study, to obtain the value of the size of the design,
we calculate the total number of classes in a design pattern to
Quality get the value of this metric [8].
Definition Name of Metric
Attributes
Reflecs the presence of object-
oriented design characteristics Coupling, Cohesion, B. Number of Hierarchies
Reusability that allow a design to be Messaging, Design This metric is a metric that aims to get the total of the
reapplied to a new problem Size
without significant effort. number of class hierarchy in a design pattern [8].
Encapsulation,
Characteristics that allow the
Flexibility incorporation of changes in a
Coupling, C. Average Number of Ancestors
Composition,
design.
Polymorphism The average number of ancestors is a metric that aims to
Abstraction, measure the abstraction property in a design pattern. To
The properties of the design that
Encapsulation, measure this metric, you can implement the following
Understanda Coupling, Cohesion, calculation [8].
enable it to be easily learned and
bility Polymorphism,
comprehended.
Complexity, Design 1. For all classes, calculate the total anchor at any level.
Size For example, if class A inherits class B which in turn
The responsibilities assigned to Cohesion,
the classes of a design, which are Polymorphism,
inherits class C. From the anchestor class A gets a
Functionality
made available by alsses through Messaging, Design value of 2.
their public interfaces. Size, Hierarchies 2. Calculate the average of the steps above, for example
Refers to the presence and usage Abstraction,
of properties in an existing design Coupling, from the example above, the results obtained are 1
Extendibility
that allow for the incorporation of Inheritance,
new requirements in design. Polymorphism
( )
This refers to a design’s ability to Abstraction, The average of the total ancestor is the main point to get
achieve the desired functionality Encapsulation,
Effectiveness and behaviour using object- Composition, the value of the average number of ancestors.
oriented design concepts and Inheritance,
techniques. Polymorphism D. Data Access Metrics
Source: A Hierarchical Model for Object Oriented The Data Access Metrics aims to calculate how well the
Design Quality Assessment [8] encapsulation in a design pattern. This metric is calculated
by summing the total of private and protected attributes
contained in a class and dividing by the total number of
IV. PROPOSED WORK attributes that exist in all the classes [8]. These metrics are
class-centered and can be combined to cover the entire
In this research, we will implement automation testing on
design by taking the average value of DAM values for all
the front end and backend modules of the Prestashop
classes with at least one attribute in the design.

Authorized licensed use limited to: Cornell University Library. Downloaded on August 31,2020 at 08:53:20 UTC from IEEE Xplore. Restrictions apply.
E. Direct class coupling [8]. This polymorphic behavior is usually characterized by
The direct class coupling matrix estimates how a class implementing overiding and overloading.
can occur coupling. Pressman describes the types of coupling
on his study [11]. And in this study, the researchers add up J. Class Interface Size
the total of the coupling types to be used as the primary Class Interface Size calculated by counting all public
assessment of class direct coupling. TABLE III shows the methods in the class [8].To extract metrics that are centered
total direct class coupling in this study. on the design of metrics centered on this class, researchers
take CIS mean values for all classes.
TABLE III. RESULTS OF DIRECT CLASS COUPLING ASSESSMENT
K. Weighted Method Per Class
Type of Coupling Page Object Screenplay
The number of methods is only calculated as the average
Content Coupling 0 0 of the number of methods in the class [8]. From metrics
Common Coupling 0 0 centered on this class, a metric centered on design can be
obtained by taking the sum of all classes in the design and
Control Coupling 3 1 dividing it by the total class in a design pattern.
Stamp Coupling 0 0 To find out the results of the calculation of each metric
Data Coupling 0 0 above, it can be seen in TABLE IV showing the results of
the metrics obtained per design pattern.
Type use Coupling 11 3
Inclusion or import
0 0 TABLE IV. TOTAL VALUE FOR EACH METRICS
Coupling
External Coupling 0 0 Design Property Page Object Screenplay
Total 14 4 Design Size 21 39

Hierarchies 0 0

Abstraction 0 0
F. Cohesion Among Methods of a class
Cohesion between methods in a class is a metric Encapsulation 0.809524 0.394366
centered on a class that is calculated as follows [8]. Coupling 14 4
1. Calculate the parameters of all types to all methods
Cohesion 0.0454545 0.035714
in class.
Composition 9 0
2. For each method, divide the number of parameter
types for the method with the value from the first Inheritance 0 0
step. Polimorphism 0 9
3. Take the average value calculated for all methods in Messaging 84 102
the class.
Complexity 4 2.41025641
The average of these values is the value of the Cohesion
among methods of class metric.
Each of these metrics will be integrated in Object-
G. Message of Aggregation Oriented Design. From the Object-Oriented Design,
Message of Aggregation is a metric that calculates the calculations will be made for each Quality Attributes. The
number of data declarations that have a type in the class results of each Quality Attributes are shown in V TABLE.
specified by the user [8].
TABLE V. OBJECT-ORIENTED METRICS AND DESCRIPTION
H. Measure of Functional Abstraction
Quality Attributes Page Object Screenplay
Measure of Functional Abstraction is a metric that aims
to measure the inheritance property in a design pattern [8]. Reusability 49.01136363 69.5089285
To measure this metric, you can implement this following Flexibility 1.202381 3.5985915
calculation.
Understandability 1.272142905 7.926541785
1. Count the number of methods that are inherited by a
class. Functionality 23.10999999 19.36
2. Calculate by dividing the results of the steps above Extendibility -7 2.5
with the total method accessed by the class. Effectiveness 1.9619048 1.8788732
From these calculations, it will be found that the value of
this metric will be 0 to 1.

I. Number of Polymorphic
V. OBSERVATION
Number of Polymorphic is calculated as the number of
methods in a class that can express polymorphic behavior Based on TABLE V, it appears that each design pattern
has the advantage that can be highlighted on the quality.

Authorized licensed use limited to: Cornell University Library. Downloaded on August 31,2020 at 08:53:20 UTC from IEEE Xplore. Restrictions apply.
Page object design pattern has better quality than the [8] J. Bansiya and C. G. Davis, “A Hierarchical Model for Object-
Oriented Design Quality Assessment,” IEEE Trans. Softw. Eng., vol.
screenplay on the assessment of functionality and 28, no. 1, pp. 4–17, 2002.
effectiveness. In contrast, the screenplay design pattern has a [9] S. Amit and K. D. Sanjay, “Comparison of Software Quality Metrics
better quality of reusability, flexibility, and extendibility. for Object-Oriented System,” Int. J. Comput. Sci. Manag. Stud., vol.
One of the reasons the page object design pattern, in 12, no. June, pp. 12–24, 2012.
general, has lower ratings than the screenplay design pattern [10] S. S. Jagtap and D. B. Hanchate, “Development of Android Based
is the metric value of coupling. Page object design pattern Mobile App for PrestaShop eCommerce Shopping Cart ( ALC ),” Int.
Res. J. Eng. Technol., vol. 4, no. 7, pp. 2248–2254, 2017.
has a higher metric value of coupling than the screenplay
[11] R. S. Pressman, Software Quality Engineering: A Practitioner’s
design pattern, which is 14: 4. This makes reusability, Approach, Seventh. New York: McGraw-Hill, 2010.
flexibility, understandability, and extendability in the page
object design pattern has a lower value than the screnplay
design pattern, especially on extendability values. And due
to the function of the functionality and effectiveness does
not require an assessment of the coupling metric, making the
page object design pattern has a higher value than the
screenplay design pattern.

VI. CONCLUSION
In this research, a qualitative comparative analysis of
page object and screenplay design patterns has been carried
out implemented on web-based automation testing. Quality is
one of the key design patterns to determine whether it can
support optimization automation testing. By knowing the
results of quality analysis of the page object and screenplay
design pattern to become a benchmark or reference source
for quality assurance engineers in selecting a design pattern
that will be used to optimize their testing automation. This
research applies the QMOOD (Quality Model of Object-
Oriented Design) approach to get the quality value from the
design pattern. The output of QMOOD is the value of
Functionality, Reusability, Flexibility, Understandability,
Effectiveness, and Extensibility of each design pattern. And
from this research, it can be seen that the page object design
pattern has better quality than the screenplay design pattern
in terms of functionality and effectiveness. While the
screenplay design pattern has better quality than the object
page design pattern in terms of reusability, flexibility,
understandability, and extendibility.

REFERENCES

[1] D. M. Rafi, K. R. K. Moses, K. Petersen, and M. V. Mäntylä,

“Benefits and limitations of automated software testing: Systematic
literature review and practitioner survey,” in 2012 7th International
Workshop on Automation of Software Test, AST 2012 - Proceedings,
2012, pp. 36–42.
[2] M. Fewster and D. Graham, Software Test Automation Effective use
of test execution tools. London: Pearson Education Limited, 1999.
[3] E. Gamma, R. Helm, R. Johnson, and J. Vlissides, Design Patterns:
Elements of Reusable Object-Oriented Software, 1st ed. Baarn, 1994.
[4] D. Kovalenko, Selenium Design Patterns and Best Practices.
Birmingham: Packt Publishing, 2014.
[5] A. Stocco, M. Leotta, F. Ricca, and P. Tonella, “Why Creating Web
Page Objects Manually if It Can Be Done Automatically?,” in
Proceedings - 10th International Workshop on Automation of
Software Test, AST 2015, 2015, pp. 70–74.
[6] A. Ampatzoglou, G. Frantzeskou, and I. Stamelos, “A methodology
to assess the impact of design patterns on software quality,” Inf.
Softw. Technol., vol. 54, no. 4, pp. 331–346, 2012.
[7] P. Pradhan, A. K. Dwivedi, and S. K. Rath, “Impact of Design
Patterns on Quantitative Assessment of Quality Parameters,” in
Proceedings - 2015 2nd IEEE International Conference on Advances
in Computing and Communication Engineering, ICACCE 2015, 2015,
pp. 577–582.

Authorized licensed use limited to: Cornell University Library. Downloaded on August 31,2020 at 08:53:20 UTC from IEEE Xplore. Restrictions apply.