Keyword-driven Test Automation Framework Concept, features, architecture

Misha Suboch

Topics Covered

TAF is a program which helps to perform automation testing effectively. Let us see TAF:

Features

Architecture

Test Automation Framework

Workflow

Future

Presentation agenda

1. 2. 3. 4. 5. 6.

Features and Advantages Usage Workflow Architecture Artifacts Implementation in Your Project TAF Core Features

Features and Advantages

Features

1 2 3

Keyword-Driven

Product-independent
Tool-Independent
Business Logic GUI Objects Steps Utilities

Maintainability

Advantages

Keyword-driven Product-independent Compatible with any automation tools on the market Aligned with Continuous Integration Frameworks (single entry point, unified test reports, etc.)

Workflow

Automation with TAF

Initialization
Study Application Mock up Test Scenarios Implement Scenarios Debug /Verify Run Test Scenarios Create New Test Scenarios Tune for New Build Implement New Steps

Development

Usage

Maintainability

Architecture

High-Level Architecture Test Scenario 1 Test Scenario 2 Test Scenario N Test Bug List

TestComplete TAF
Run TAF.exe

TAF Core

Custom Automation tool Another Automation tool

XML HTML Tool Specific Test Reports

10

Detailed Architecture

11

TAF Core, Responsibilities Read Configs Read Test Suite/Unified Test Scenario Read Bug List Load Data Constant variables
Stores Substitute Calculate Expressions

Create Execution List

Run Automation Tool TAF Execution Optional (Cleanup) Command Create Reports

12

TAF.exe Config

13

TAF Core Source Code

TAF Core code is generated using Test-Driven Development approach, it is very well structured and covered by unit tests (more than 300 passed unit tests validates TAF Core quality).
14

Detailed Architecture

15

Communication between TAF Core and Automation Tool TAF

ExecutionList.txt

TAF Core

Automation Tool TAF

TestResultsSummary.txt

1.When scenario is read and parsed, TAF Core generate Execution List and run Automation Tool TAF. 2.After executing all tests, Automation Tool TAF generates Test Results Summary and exits. 3.TAF Core reads Test Results Summary and generates final results.

16

Automation Tool TAF

17

Automation Tool TAF

Tool SilkTest TestComplete Quicktest Pro Watir

Status of TAF support Supported Supported (TAF Core) Supported (TAF Core) Supported (TAF Core)

Test Data

Test Scenarios

Selenium

Supported (TAF Core)

Actions

Scripts

Test Cases
Driver Controller Frames Utilities

Verifies
Screens

18

Controller in TestComplete

19

Step Implementation

20

TestComplete, TAF Detailed Log

21

Artifacts

22

Unified Test Scenario

23

Test Result Report for Scenario

Metrics

Summary, Passed

24

Test Result Report for Scenario

Summary, Failed

Summary, Failed with Bug Link

Summary, Known Issue

25

Implementation in Your Project

26

How to Proceed and Take Advantage of TAF Automation with TestComplete, Watir or QTP: Get Test Complete TAF / Watir TAF / QTP TAF Automation with another tool:

Implement controller

Develop test scenarios

Develop test scenarios

Implement steps

Implement steps

27

TAF Core Features

28

TAF Core Features: Overview

1. Unique keywords in test scenarios allow you to generate unique text and numbers in specified format 2. Ability to run several test scenarios at once, even if written in different automation tools 3. Ability to use the same test scenario for different automation tools, when one tool could not cover all test configurations (browsers/OS). 4.Unified test results (in .xml) across different automation tools/projects/applications provide integration options with external tools (like Cruise Control). 5. Ability to mark failed test cases as known issue and link to related bugs in test results. 6.Keywords Feature: Load/Repeat keywords allow to create data-driven test cases. Application keyword allow you to work in one test scenario (or even in one test case) with different applications.

29

TAF Core Features: Overview

8. Arithmetic expressions calculation and Variables values substitution. Ability to exchange variables values between different test scenarios, automation tools or even when different test scenarios should be run one by one on different servers. 9. Unified test scenario could be easily accessed for review by customer or business analyst (no programming knowledge required) 10. Working with results: Auto-archiving of previous test results. Test results contains information about what workstation/server was used for scripts run, when and which TAF Core version they were generated by. 11. Extensive configuration features: execution (run selected tests/complete scenario), results output, logging, etc.

30

Test Automation Framework

Misha Suboch misha.suboch@gmail.com

31