‘CODELESS’ TEST

AUTOMATION:
HOW VISUAL PROGRAMMING IS MAKING
IT A REALITY

©2019 Odin Technology Ltd. All rights reserved.

BACKGROUND

The requirement for test automation has never been greater, with many
organisations moving towards Continuous Delivery and DevOps regimes. With
these rapid Build/Deploy cycles, test automation has become critical to success,
removing the manual testing bottleneck is critical to keeping shorter cycle times.

Fundamentally, the challenge of test automation can be described as; how to

“Test automation articulate a test script (the output of the test analysis process to exercise test
cases) in a way that can be interpreted by a machine for execution purposes.
has always faced Essentially – creating machine readable test scripts.

one fundamental
challenge, that of Since their inception traditional automation tools have relied on coding languages
as the means to represent test scripts for execution. But there exists the
knowledge fundamental problem, most organisations rely on human derived test scripts, the
output from test analysts who define and articulate them. The analysts are
capture and experts in their subject matter and in the field of testing, interpreting system
specifications and expertly deriving test cases from that knowledge, scripts to
representation” exercise those cases are typically documented in natural language and stored in a
repository for later execution manually.

It is fair to say most test analysts are not skilled in writing and maintaining code,
the language required of test automation tools, so how can we capture their
output in a form that can be interpreted by machines or alternatively converted
into code for automation tools?

Commonly, skilled automation engineers or SDETs are hired who take test scripts
and rearticulate them in coded form. Rightly or wrongly automation for most
organisations is a translation exercise from human readable form to machine
readable form, essentially re-articulating the output from the test analysis process
in another form. There are clearly efficiencies to be made in this process.
WHAT IS CODELESS TEST AUTOMATION?

In brief, Codeless Test Automation is removing the
translation step from the process described above.
Providing a means of defining test scripts directly in a
form readable by both Humans and machines. Codeless
Test Automation is aimed at solving the fundamental
knowledge capture problem, empowering test analysts,
not conversant with code and software development to
deliver test scripts in a machine-readable form, skipping
the secondary translation step.
The appeal of codeless test automation comes down to
cost. The traditional script-to-code process requires
duplication of effort, articulating the same information in
two forms, one readable by humans, the second translated
into a machine-readable form, code. This requires
additional resources, if both steps could be combined it
lowers the cost considerably. So what options are there?

“learning to code
is only half of the
battle, with coding
comes a plethora
of tooling and
ancillary skillsets”

WHY NOT TRAIN EVERYONE TO CODE?

Give the individuals responsible for deriving the test scripts in the first place the ability to directly articulate them in
machine readable form, code. The barriers to this approach are obvious. We are adding a third skillset to those of
domain expertise and test expertise. Many will have no desire to take that journey or indeed the mindset to do so, it
is often said that the skills of a tester and a developer are very different, the combination of the two is still a rare
commodity. Needless to say, many test analysts will make that journey successfully, hats off to those that do.

There is still the problem that even if test cases can be directly derived and articulated in code, that all stakeholders
in the software development process need to be able to read those test cases. This is acutely obvious in risk
acceptance, audit and compliance processes, those responsible for signing off on test cases and assessing business
risk in going live. It is unlikely that those responsible will want to learn to interpret code in order to asses risk.

It must be said, that learning to code is only half of the battle, with coding comes a plethora of tooling and ancillary
skillsets, use of Integrated Development Environments (IDEs), build orchestration tools, frameworks and third-party
libraries. Setting up a simple Java Selenium test could require knowledge of numerous components; Eclipse, Maven,
TestNG, and Selenium on top of the Java language itself. Extending that to improved reporting with Extent, DB calls
with JDBC and data abstraction with Excel with Apache POIs, quickly the learning curve becomes logarithmic!
ABSTRACTION

Providing an abstraction layer has two benefits, the first that if articulated correctly it should be readable by all
stakeholders in the software development process, secondly it should be machine readable. Abstraction has a price, the
payoff is in flexibility, whilst humans can overcome limitations in interpretation, can an abstraction articulate the nuances
and precision of instruction required by automated execution tools?

Numerous solutions to approach this problem have been seen over the years, and yet we still see automation rates as
low as 15-20% across many organisations. So, let’s look into the history a little and how test automation approaches have
evolved abstraction layers.

CAPTURE-REPLAY

Capture-Replay (Record and Playback) – A process whereby a coded test “we still see
script is created by recording user actions whilst they perform steps
manually on an application, directly skipping the translation step. The automation rates
backbone of test automation tools for decades, the little red button is seen
by many as contributing to the over-expectation placed on automated as low as 15-20%
testing in organisations. Many vendors have pushed the ease of use of
capture-replay tools as a major selling point. Allow your test analysts to across many
instruct the automation tools by demonstration.
organisations”
In reality there are numerous pitfalls in using capture-replay; Reliability of
recorded scripts, often fragile or incorrect code is produced in this fashion.
The code doesn’t incorporate accepted best practices of modularity for
maintenance and parameterisation for varying test data, also requiring
amendment to incorporate validation actions. Maintenance and code
refactoring overheads mean this approach alone rarely scales and projects
will resort to direct coding or other more structured approaches.
KEYWORD FRAMEWORKS

Stemming from the Testing with Action Words
methodology proposed by Hans Buwalda in 2002. The
process of defining reusable parameterised test actions
that can be sequenced together to form larger
processes. Often architected using Excel as a ‘front-end’,
enabling non-technical testers to be core to the
automation process using the Excel interface to
articulate test scripts.
Each action word still requires implementation as code so
the team needs to be comprised of both technical and non-
technical roles, with the workload split. Action words and
Keyword approaches still require coding and maintenance
so additional resource beyond the analyst role is required.

Whilst many traditional automation tools have their own sophisticated action/keyword solutions built in, it is still
very commonplace to see organisations creating and maintaining their own, often equally sophisticated, Keyword
Frameworks in-house. Sadly, in many cases these do not offer the return on investment promised, typically requiring
teams to maintain and support them and are often vanity projects.

“BDD requires
wholesale buy-in
from all business
stakeholders to be
successful”

BEHAVIOURAL DRIVEN DEVELOPMENT

First proposed by Dan North, the use of a ‘ubiquitous language’ for articulating use cases (typically seen implemented
in Gherkin form, Given, When, Then etc.) that all Stakeholders in the software development process can interpret,
and more importantly can be executed directly by BDD frameworks like Cucumber and SpecFlow. Often interpreted
incorrectly as a test automation solution, BDD requires wholesale buy-in from all business stakeholders to be
successful and has not always delivered on its promise when this isn’t the case. Moreover, the ubiquitous language
clauses, similar to Keywords, still require implementation in code form. BDD sits in the Hybrid camp, requiring both
domain expertise and coding skillsets to achieve success.
THE RISE OF CODELESS AUTOMATION TOOLS

Codeless automation has in fact been around for many years but is recently seeing a resurgence. This can be put down in
part to the widespread adoption of Open Source libraries such as Selenium and Appium. The execution ‘engines’
previously the prized IP of traditional tool vendors are open for all to build abstraction layers upon. Improvements in UI
and visualisation technology have allowed for creative solutions to appear outside of the traditional vendor space.

With these Open source libraries the same fundamental script to code translation exercise applies, even more acutely so,
as they do not come with any form of so Let’s explore some typical codeless abstractions:

VISUAL DESIGN TREES TABLE BASED ABSTRACTIONS

Found in popular GUI automation tools such as Probably the most common form of abstraction,
Microfocus Unified Functional Tester (UFT) or IBM test scripts are represented in a series of rows in a
Rational Functional Tester (RFT) test steps are table, with actions or commands making up the
represented in icon form in treeview controls: steps in a form closer to natural languages.

Selenium IDE is a form of a table based abstraction:

Similarly to Visual Design trees these are coupled

Often these are coupled with a form of Target UI
Element management that allows for UI
elements to be maintained independently and
referred to in a non-technical fashion. The
ubiquitous ‘object repository’, but they do make
for easily read and interpreted scripts.
with Element Management for maintenance and
readability.
There are many traditional vendors now
incorporating table-based abstraction
approaches to test design into their tools. A
plethora of solutions for Open Source libraries,
typically Selenium and Appium now exist in both
desktop and cloud applications.
INCREASING COMPLEXITY

Both Table and TreeView abstractions make for compelling solutions for simple examples outlined above. In reality
test scripts are not entirely comprised of simple step by step interactions. It is commonplace to find easily articulated
natural language steps, such as:

• “Check the Account ID in the third column of the user table is a combination of Name and DOB”
• “Enter a date in the policy field 30 days from now”
• “Repeat the previous steps until the shopping basket contains 10 items”

Here the Table and TreeView abstractions break down or become overly complex. Essentially test scripts are simple
algorithms in their own right, just articulated in natural language form. They never become as complex as application
code but still have elements of algorithmic constructs and operators. Repeat, If, Until, Combine, plus etc.

ENTER VISUAL PROGRAMMING

Visual programming Languages (VPLs) are not new, they “let “This really is the problem
users create programs by manipulating program elements
graphically rather than by specifying them textually”, that abstractions need to
“programming with visual expressions, spatial arrangements of
text and graphic symbols”. address head on, the
A quick look at Wikipedia shows they have been around for requirement for simple
many years in different forms, primarily applied in Robotics,
Process Automation, Internet of Things (IoT), Embedded
algorithmic constructs
Systems, Multimedia, Games and Education. occurring in test scripts”
Let’s look at some examples.
FROM FLOW DIAGRAMS TO BLOCKS

There are a number of different visual programming styles primarily based on visual canvases where elements can be
dragged and connected in some fashion on a design canvas. Some common forms are shown below, executable flow
diagrams, from Flowgortihm and Node red, and block coding from Grasp.io:

Flowgorithm – flowgorithm.org IBM NodeRed – nodered.org Grasp.io

The last of these three forms, Block Coding, was the creative
output of Google research and MIT projects into code abstraction
which led to a way to represent simple algorithmic constructs in a
form that is easy to understand, explore and apply.
“The application of Visual
programming in robotics and
Based on the indent structures of text-based code, statements process automation is telling,
and structures are represented in visual form as a series of much like test scripts these
interconnecting blocks. These can be connected in a flexible way
into sequences and with inputs and outputs to articulate an applications do still require
algorithm, an example from a robotics application: algorithmic constructs but
not to the level of complexity
Adoption of block coding has rapidly expanded in robotics, home that typical UI based
automation embedded systems design and IoT development, but is
more commonly known in a colourful form for educating aspiring application code requires”
coders, with the wonderful Code.org and Scratch projects.

We are already seeing vendors adopting visual programming forms for Test Automation applications and it is our
belief that they hit the sweet-spot in providing enough of an abstraction to be easy to adopt quickly, whilst
maintaining the flexibility to cope with algorithmic constructs. Visual Programming is making ‘Codeless’ Test
Automation a reality now…
ABOUT SCRIPTWORKS

Scriptworks is the Industry’s first automated testing

platform based on the principles of block coding. Users
can quickly define test scripts for automation without
the requirement to become familiar with an IDE, set up
a build an execution framework and more importantly
without the need to learn the syntax of a new and
complex coding language.

Best practices of Modularity and Parameterisation are built in.

Importantly Scriptworks does not do away with code altogether, a key
feature in flexibility and portability, code is still present, generated
automatically from the visual abstraction.

A comprehensive set of predefined blocks make up the abstraction

with complete Selenium and Appium functionality, including
common requirements of working with Browsers, Mobile Devices
etc. as well as ancillary functionality for dealing with Excel Data
Sources, APIs and more. Custom blocks can easily be added to the
abstract layer with their own corresponding custom code
generated. It is easy to extend the already comprehensive existing
capabilities.

Scriptworks combines its own comprehensive execution and

reporting frameworks based on Node.JS. This is entirely
portable, callable from CLI, CI/CD tools like Jenkins and
TeamCity and enables high levels of concurrent execution.

A DASH OF ARTIFICIAL INTELLIGENCE MAGIC

Scriptworks adopts a pragmatic approach in the adoption of AI through integration of features from leading AI
providers like Applitools for Visual Testing and Google’s powerful ML based OCR to hit traditional pain points in the
test automation field as well as introducing new solutions of its own to aid in script creation and maintenance.

Brittle element identification and poor resilience to change has led to maintenance difficulties and is a primary
contributor to low automation adoption rates. The introduction of Smart selectors, based on semantic identification
and combined Heuristic techniques can remove this brittleness from the outset.

Scriptworks aims to allow users to create tests without the need to learn and maintain large Selector Libraries and
complex Page Object Models.
 info@scriptworks.io
www.scriptworks.io
@scriptworks_io

©2019 Odin Technology Ltd. All rights reserved.