Summary of Class Purpose and Topics

Our end goal is to be able to create a Framework that is able to test any Web Site, Web Service and
Database, and create Test Suites that would perform Quality Assurance evaluation of specific sites,
databases and web services. Our code will be separated out into the Framework and Test Suites, so that
we can maximize the amount of code that can be reused between test suites. In this class I will
reference an existing Framework and Test Suites that are written using the C# .NET Framework tool
suite, Continuous Integration tool Jenkins and a testing methodology that utilizes Cucumber
specifications.

Student demographic is employees that have gone through initial junior Software Development
Engineer in Test training. Students should have a basic grasp of programming foundation and have been
exposed to some of the tools used, but not their initial setup. Students taking this class should be
interested into how the frameworks and test suites are put together from disparate projects and code
that is used to glue it all. This class is not intended to create Software Architects, or advance employees
to full SDET position. Instead this class aims to familiarize students with modern frameworks, testing
tools and open source projects that are used in the industry. This familiarization should allow for faster
training and transition time once on an active project, reinforce previously learned skills and boost
confidence both for employee and clients in skills learned.

This class will be covering topics and allowing students time to setup their own automation project, in
order to learn topics by coding and configuring relevant software.

A Framework is a set of generalized code that provides the tools to create a Test Suite for a specific
purpose. This separation is needed, because often times while software or web sites have similar
functionality, the way to navigate and do verifications on them can be very different. We do not want to
start from scratch each time we are adding a test suite and we want to save effort, time and money to
speed up the process of creating new test cases. The Framework provides a solid code base that allows
us to implement these savings.

Continuous Integration is a software development practice to methodically test software as it is put

together, instead of testing the whole software package at the end of development. This allows
developers to fix the code piece meal and ensure that code that is already written, is not disturbed by
new changes and functionality. Our selected tool to manage scheduling and setting up configurations for
different environments is Jenkins.

An analogy would be building a boat. Once we have the outer hull, we have tested it for leaks and know
that it is leak free. Once we add benches, we want to test again that the hull remains leak free, ie nails
or drilling for bolts and applying seals was done correctly. Once those previous test cases pass, we want
to add a load equal to that of a person sitting on those benches, we run the previous library of tests and
verify that the boat is leak free or not. This is Continuous Integration testing Quality Assurance. The
alternative is to build a boat completely, put a sailor in and then lower the boat into water and hope it
floats.

Cucumber, is the Behavior-Driven Development methodology specification that allows us to write

human readable test case scenarios and map them to actual code. It is a meeting point where non-
programmers and programmers can meet and speak the same language to define specifications,
acceptance criteria, review and troubleshoot results.

Technology Stack

Windows – operating system and our development environment

C# - programming language

.NET Framework – set of libraries to speed up development

SpecFlow – Cucumber implementation in C#

NUnit – C# Test framework.

Selenium – Open Source project and library to automate web browser testing.

MyBatis – Mapping of Database queries to C# code and data objects.

RestSharp – C# Library for accessing Restful Web Services.

.NET Framework Connected Services Libraries – C# Library and Tool for accessing SOAP web services.

Visual Studio – IDE for developing code.

NuGet - package manager of C# and .NET Framework libraries.

GIT – source code management system.

GIT Meld – tool optimized for resolving GIT code merging conflicts.

Jenkins – Continuous Integration server (and windows executors).

Windows Batch Commands – Control and execute test suites as part of Jenkins jobs.

HTML – build reports to be sent out as HTML email content.

Email – Simple to setup distribution of Test run reports.

Excel – Simple and accessible way of keeping data.

The Framework
Our Framework is shaped by the tools we decide to use. The most mature implementation of Cucumber
in C# is the SpecFlow project. Use of Cucumber Scenarios and SpecFlow mapping of those scenario steps
to actual code takes over how code is arranged and executed.

NuGet – Package Manager

To manager different libraries and software packages, we are using NuGet package manager. NuGet
package manager is integrated into Visual Studio and can also be used as a stand alone .exe application
that we use as part of Jenkins Job build script. It is very straight forward to use, we are able to search for
packages and NuGet will fetch up to date list of packages from online repositories.

NuGet will also help us with dependencies of libraries and packages as the tool lists and installs required
packages as part of installing a selected package.

References for our project, that are similar to a package dependency build file, will also be added and
updated as part of NuGet package management.

Cucumber

Cucumber, is the Behavior-Driven Development methodology specification that allows us to write

human readable test case scenarios and map them to actual code. It is a meeting point where non-
programmers and programmers can meet and speak the same language to define specifications,
acceptance criteria, review and troubleshoot results.

The Specflow project auto generates NUnit test code based on the mapping of Scenario Steps to
programmer defined methods. An interesting side effect of this is that, if Scenario steps are built to be
self-contained, or have a method of sharing data, these steps can be rearranged, reused and combined
as needed.
The top code snippet is from a Cucumber Feature file. It shows steps that are easy to read and
understand what this scenario is supposed to test:

The test setup is the Given: Given that the user is on a web site homepage, while using a device of
model iPhone6.

The action to test is clicking on a header link, named The App.

And verification steps are as follows: Check that the user landed on the App page. And that the user sees
the link to download and install the IOS app. And that the user sees the link to download and install the
Android app.

The next Snippet is of the first step listed in our scenario example. The mapped stepped is of a method
that calls previously written methods that themselves were mapped at some point. This is an example of
reusing and combining steps, to simplify the current, more complex scenario.

The third code snippet is of a Cucumber step mapped to actual code that can run. The language that
each step is written in, is called Gherkin. Gherkin is more of a specification, that encourages users to
write succinct and to the point steps that are one liners.

More documentation on Cucumber can be found at their project page:

https://docs.cucumber.io/guides/overview/

Generally, in testing, the package of NUnit provides the testing framework where tests are defined and
implemented. However, because SpecFlow now controls how code is called for execution, and
autogenerates the code for NUnit tests, it also sets prerequisites on how code is arranged. In our
Framework Overview diagram, SpecFlow is shown at the very top of our structure. The way we interact
with the code to control its execution is purely through SpecFlow’s Cucumber implementation. In our
framework, Cucumber steps can be shared between test suites as well. This allows us to create some
special steps in the framework and test suites, that are then used as background steps to provide setup
for test suite scenarios.

Cucumber and SpecFlow is a core technology for our example framework.

Note on NUnit:

In our example framework, NUnit testing Framework is generally relegated to providing Assertion
methods that are integrated into Cucumber reporting functionality. While we need to be familiar with
NUnit and what it does, SpecFlow autogenerates most of the core functionality that we would use NUnit
for in a QA project.

At the end of Jenkins job runs, we will be able to see Cucumber style reports of tests (Scenarios) that
were run:

Selenium

Selenium is an open source project that provides libraries and packages to control web browser like
software. The project itself only provides interfaces and supporting software to control browsers.
Companies that build and release browsers, as well communities, create browser versions that selenium
remote control can hook into and run the browser in an automated fashion.

A Selenium controlled browser is designed to mimic a live user’s interaction with a web page. By design
we are limited to only interacting with visible elements. More and more websites introduce active code
that reacts to user interaction. While selenium is not design to directly test technology and code that
allows such dynamic interaction, it tests all web page functionality from a stand point that a live user
could verify, by seeing displayed links, text, elements and structural design of pages. We are provided
with tools to scroll, click, type, drag and navigate around visible links, as well as verify all kinds of
elements by any and all applicable selectors that comb the page source code to precisely identify the
right elements.

Currently, Chrome is the most used browser to develop and test web content. Chrome driver and
automated browsers are used in our example project.

The beauty of selenium is that we are able to use and code for interfaces that in turn control browsers.
This means that we can setup configurations and reuse our code to run different browsers. Common
browsers that are used in the industry are Chrome, Internet Explorer, Firefox, Safari and Opera, as well
as their mobile versions and mobile simulated modes.

Another aspect of selenium is to leverage the Selenium Grid. Selenium Grid provides the Selenium Hub
that is a scheduler app that our code can interface with and request browser sessions blindly, without
knowing where these browsers are running. Selenium Grid also provides Selenium Node app, this app
hosts browser drivers and respected system identification and configuration that is supplied to the Hub
as well as the list and number of available browsers on the particular node. Hub and nodes can be actual
machines, ie PCs and laptops or they could be Virtual Machines on actual hardware or in the cloud.

Leveraging Selenium Remote Control drivers and Grid allows for very flexible and simple setups as well
as being able to grow to massive scale grids.

Documentation for Selenium Project can be found at:

https://www.seleniumhq.org/

And Grid can be found at:

https://github.com/SeleniumHQ/selenium/wiki/Grid2

Selenium is a core technology for our example framework.

Jenkins

Jenkins server is an open source project that provides web interface to manage and schedule individual
jobs as well as control pipelines that perform actions based on success or failure of individual jobs.
Multiple jobs can be setup to cover all environments that run the code. Generally, in a CI development
life cycle, there are Development, QA, Staging and Production environments.

Jenkins also provides tools to configure job runs, or we could provide a batch file type script to build
what we need, with parameter substitutions:

GIT

Git is a source code version control software that allows us to quickly backup our work, share it between
programmers, automated executors and Cron job builders. Git allows multiple users to share and merge
their code very quickly. Previous version control systems were unwieldy and required senior
programmers to control and provide code to more junior programmers.

Git allows for lightweight branching, code substitution on the fly, as well as powerful tools to merge and
cherry-pick code from commits. Git allows all users to have full or partial local repositories and
decentralized sharing of the code, where each user’s local repository can act as repository for others.
Command line, GUI and Web GUI tools are available for this software.
Documentation for git is located at:

https://git-scm.com/

Exercise:

1. Install GIT.
2. Initialize a new repository.
3. Create a text document with “Hello World” as its contents.
4. Commit your first commit.
5. Create a new branch.
6. Change contents to “Hello World, you are great!”.
7. Create your second commit.
8. Compare branches.

Putting it all together

We will learn by recreating our example framework on each students’ personal laptop.

Our example framework was developed from a proof of concept with incremental due dates and desired
capabilities. IE: first, selenium was added to the project and a simple script was created to access the
Google search homepage. Then NUnit was added to create a test and leverage Test Integration
functionality of Visual Studio. Then SpecFlow was added and changed how everything was called and
worked. This project had seen drastic changes. We will avoid this and try to keep everything as straight
forward as possible and instead from ground up, we will approach this from top to bottom.

Stage 1: Create our git repository and back it up.

1. Request or create a new repository in the online Git server provided by the company. (Similar to
GitHub).
2. Clone this empty repository using GIT.
3. Create a README.txt file with a brief description of what you will be developing.
4. Commit your first repository commit.
5. Git push your new changes up to online git tool (similar to GitHub). – you have created the
landing page for your project. The readme will be automatically imported and displayed upon
landing on your online location of your git repository.
6. In your local git repository instance Switch branch from ‘master’ to a new branch ‘develop’. –
you are now ready to add new code.

Stage 2: Gathering and function checking our tools:

1. We will start our initial empty first Visual Studio Project in our previously created Git repository.
2. Open NuGet package manager and install SpecFlow and SpecFlow Runners. These will in turn
install packages on which they are dependent, this includes NUnit.
3. Install SpecFlow Visual Studio Plugin – this will integrate SpecFlow autogenerating of classes
with Visual Studio.
4. Create your first Cucumber feature file and add scenario
5. Map the new Cucumber steps to C# code.
 6. Make sure your new scenario is seen in Test Explorer and runs.
7. Commit your work into Git and upload it to online git instance.

Stage 3: Adding Selenium to project.

1. Using NuGet package manager add Selenium RC and Selenium Google Chrome driver to your
project.
2. Create a new Cucumber scenario that will describe step to start the web driver and go google
homepage.
3. Create a new mapping in the code that runs the steps.
4. In new step definition in the C# mapped methods, add code to start the selenium web driver
and navigate to Google Search home page.
5. Add a Cucumber step and it’s mapped C# method that will sleep the thread for 60 seconds, so
that you can visually confirm that the google page loaded in your automated browser instance.
6. Commit and upload your code.

Stage 4: Base pages and page objects.

1. Identify common elements (such as url root).
2. At this step we will be creating the separation between Framework and Test suites

Our framework and Test suites will start looking more and more like the example framework and Test
Suite:

Further examples and exercises are Company IP