Automatic test automation

What we do, how far we got, and where do we want to go

Agenda

Our original target

How far did we manage

Fully automatic test of business processes

Fully automatic test of output management

Perspectives

SAP Tools or non SAP tools for testautomation

2
Agenda

Our original target

How far did we manage

Fully automatic test of business processes

Fully automatic test of output management

Perspectives

SAP Tools or non SAP tools for testautomation

3
Targets: Testautomation

Testautomation of the essential business processes in order to confirm

the correct functionality of the software
 to simulate the essential business processes covered by the software
 to apply fully automatic tests of very complex business processes
 to significantly reduce staff required for the manual test
 Shorter lead times, optimizing the use of available testing ressources
 Precise repeatability of test scenarios across multiple releases

Testresult reliabilty
 Increased product stability and quality
 Increase the reliabilty of the test performance
 Increasing the test coverage level

Documentation of test results

 Automatic generation of test results documentation

4
Agenda

Our original target

How far did we manage

Fully automatic test of business processes

Fully automatic test of output management

Perspectives

SAP Tools or non SAP tools for testautomation

5
How far did we manage

Testautomation of the essential business processes in order to confirm

the correct functionality of the software

 to simulate the essential business processes covered by the software

 to apply fully automatic tests of very complex business processes
 to significantly reduce staff required for the manual test
 Shorter lead times, optimizing the use of available testing ressources
 Precise repeatability of test scenarios across multiple releases

Testresult reliabilty
 Increased product stability and quality
 Increase the reliabilty of the test performance
 Increasing the test coverage level

Documentation of test results

Automatic generation of test results documentation

6
How far did we manage
- the essentials

We had to find solutions to these problems:

 Business processes do not pay attention to the input methods of the
software
- Some of these input methods are not supported by eCatt ( IDOC and
batch processing)

 In order to achieve fully automatic regression tests we must find a way to

produce test data dynamically without human interference

 In order to achieve absolute test result reliability, we had to find a way to

remove factors which would reduce the sharpness of the test processes

 The complicated and complex business processes consist of a large

number of steps which need to be performed in a controlled sequence and
to exchange parameters in a controlled and secure manner

7
How far did we manage
input methods
Examples of test processes with various input methods

Dialogue – „normal“ online user transaction – easily supported by eCatt

Batchprocess – Dunning ( occasionally with a user interface to type in the information to
be processed – but the processing itself is batch and asynchronous

Creation and processing of input files

Creation and processing of IDOC files

Automatic database inspection checks

8
How far did we manage
Dynamic supply of test data

Some „trivial“ test data problems

 Which date do I need for this dunning test proces?

 Which date is it in two months?
 How do I know if this person has already been „inserted“ into the system
 How do I know, which date it is in 20 days – also if it needs to be a bank day

9
How far did we manage
test result reliability

How to reach test reliability:

 You need to know everything about the test data

 How old it is
 Which attributes it has
 Each and every attribute
 Each and every connection or link to other data elements

10
How far did we manage
complex business processes

 The means of referenced eCatt scripts does not offer a configurative

flexibility

 The referenced eCatt scripts would not be the right means for the input
methods which can not be processed with eCatt

11
How far did we manage
Requirements - summary

 If you want to control your test process, you must create and process all
required data yourself. You cannot rely on the quality of data already in
the system.

 You need to be able to produce valid test data automatically and supply
this test data also automatically to your test process.

 You need to be able to service different kind of input methods in order

to be able to offer test of entire business processes.

 If you want to provide a fully automatic test service you must focus on
relatively low maintenance.

12
How far did we manage
Solution – head lines

 Our test processes generate and, where required, process their test
data themselves.

 Our test processes support different input methods.

 The steps of our test processes are linked together and exchange
parameters through configurations (rather simple table entries).

 Due to the configuration ability we concentrate our maintenance on

changes in the software products which we test.

13
How far did we manage
Solution – head lines

 We developed additional SAP functions in order to embed the SAP

eCatt standard functions into a solution which would fit our
requirements.
- The addtitionals provide several test configuration functions
- Test process definition
- Test steps parameter exchange
- IDOC configuration
- Setup of test files
- Etc
 Test performing functions
- Start up and scheduling of test processes (not available in the eCatt
environment)
- Utilities to start up the test steps in the correct sequence
 Test result displaying functions
- Various views on the results of the performed tests

14
Agenda

Our original target

How far did we manage

Fully automatic test of business processes

Fully automatic test of output management

Perspectives

SAP Tools or non SAP tools for testautomation

15
Fully automatic test of business processes
Basic example

One testprocess contains several test steps

The steps of the testprocess are linked together through table entries
Each test step produces parameters
These parameters can be picked and processed up by succeeding steps
The test steps are also supplied with test data from the standard eCatt test data
container

16
Fully automatic test of business processes
Testprocesses – Monitoring overview

17
Fully automatic test of business processes
testprocess – test steps

18
Fully automatic test of business processes
testprocess – test steps

19
Fully automatic test of business processes
testprocess – test parameters and values

20
Fully automatic test of business processes
The parameters are picked up by suceeding steps

21
Fully automatic test of business processes
Some examples – screen shots

To minimize maintenance efforts, it is very important, to create the test scripts

as modular as possible. For example, there is the business partner dialogue;
in oscare ®, there are 39 different roles for bp’s and according to most of them,
the dialogue is different, mainly concerning the number of tab strips, which
ones are to process and their order.

Hereinafter two examples are shown by screenshots, at first a school and then
an employer are created.

22
Fully automatic test of business processes
Creation of a school as a business partner
Fully automatic test of business processes
Creation of a school as a business partner
Fully automatic test of business processes
Creation of a school as a business partner
Fully automatic test of business processes
Creation of a school as a business partner
Fully automatic test of business processes
Creation of a school as a business partner
Fully automatic test of business processes
Creation of an employer as a business partner
Fully automatic test of business processes
Creation of an employer as a business partner
Fully automatic test of business processes
Creation of an employer as a business partner
Fully automatic test of business processes
Creation of an employer as a business partner
Fully automatic test of business processes
Creation of an employer as a business partner
Fully automatic test of business processes
Creation of an employer as a business partner
Fully automatic test of business processes
Creation of an employer as a business partner
Fully automatic test of business processes
Creation of an employer as a business partner
Fully automatic test of business processes
Creation of an employer as a business partner
Fully automatic test of business processes
Creation of an employer as a business partner
Fully automatic test of business processes
Some examples – screen shots

To cover all different requirements with one script, it is neccessary, to

encapsulate every tab strip within one single script. All of those scripts are
referred within a so called ‚masterscript‘. The testdatacontainer is used, to
manage the process by different flags, like for example ‚I_DO_PARTNERINFO‘. If
the flag is set, the relevant tab strip is processed, otherwise not.

Moreover, the same tab strips can have different positions, depending on the
role. To handle that fact, a table had to be developed, where the information is
stored, which tab strip can be found at what place (according to the role), what
is it‘s name and what is it‘s ID.

38
Fully automatic test of business processes
Some examples – screen shots

Below all entries for the ‚Partnerinfo‘-tab are shown:

39
Fully automatic test of business processes
Some examples – screen shots

Here the protocol entry, that was originated while choosing the ‚Partnerinfo‘-tab
strip during runtime, when name and id were determined:

40
Agenda

Our original target

How far did we manage

Fully automatic test of business processes

Fully automatic test of output management

Perspectives

SAP Tools or non SAP tools for testautomation

41
Fully automatic test of output management
What was the problem and how it was solved

A precise and reliable test of the correct generation and formation of RDI and
XSF files is rather problematic.

 The requirements describing how these fields supposedly should be

populated were defined and stored in testautomation.
 Functions to examine the correct population of the fields of these output
files were developed.
 Most of the required business processes had already been test
automated.
 Eventually we just had to add the output function and connect them to
the field inspection functions.

42
Fully automatic test of output management
Two kinds of field inspection

Mainly in order to avoid loosing control of the automated output inspection

checks we split the checks up in two categories.

 Appearance checks – does the output field actually occur in the output
stream?
 Content checks – does the output field hold exactly the expected value?

43
 Appearance checks
RDI
Target:
 Do all required fields occurr?
 Did unexpected fields occurr?
Business
How it works:
 We test automate the business process process
 The steps of the business process
populate the data fields, which should
eventually show up in the output streams.
 The OPM field documentation of the data
streams is transferred to a simple R/3
table
 The appearance checks are appended to
the business and print part process

Printing part

OPM
checks

44
 Appearance checks
Examples

Comparison with the OPM documentation in order to check that no unexpected fields
occurred

Selective check in order to inspect if these three fields occurred.

45
 Content Checks
How it works

The content check contains two components

Extraction of the expected field content

 Configuration table entries set up exactly from which teststeps, exactly which parameter
the expected content of this output field holds.
 The function that extracts the expected field content reads these configuration entries
and picks out the parameter values.

Comparison - expected with actual output stream fields

 This function compares the expected field content with the actual stream fields.

46
Content Check

Businesspro
cess

Print part

OPM
checks

47
 Content Checks

Extraction of expected values

48
 Content chekcs

Comparison – expected - actual

49
Agenda

Our original target

How far did we manage

Fully automatic test of business processes

Fully automatic test of output management

Perspectives

SAP Tools or non SAP tools for testautomation

50
Perspectives
Where do we go from here

ABAP ORACLE

JAVA
BSP SAPGUI ORACLE
WebDyn FORMS

JSP CRM2007
pro
WebClient
JSF UI

WD4A Portal

WebDynpro
NWBC

CompositionEnvironment
WebDynpro
Visual
Composer
WD4J Composition Tools Webbrow
ser
AS Java

51
Agenda

Our original target

How far did we manage

Fully automatic test of business processes

Fully automatic test of output management

Perspectives

SAP Tools or non SAP tools for testautomation

52
 SAP or non SAP tools for testautomation

SAP-Ecatt
SAP native free to use tool
Very integrated automation capabilities and interfaces

No serious support of non SAP GUI Dialogues

Tosca
Acquisition and / or licensing costs

„Videocam“-Technology
Films everything also WEB dialogues (understands nothing)
No SAP-Integration
No SAP Interfaces
Very limited (if at all available) test result checks

Compuware und Mercury

High priced and licensing costs

Films everything also WEB dialogues (understands nothing )

No SAP-Integration
No SAP Interfaces

Very limited (if at all available) test result checks

53
 SAP or non SAP tools for testautomation
No non SAP product provides a professional SAP integration

With these products you won‘t be able to conduct

 Regression tests without manual processing of test data - (this is already

the first knock-out)

 A detailed query or inspection of the SAP Database content, in order to

evaluate the result of yout test operation. (this is how it got the second
knock-out)
 supply an exact repeatability without sacrifizing the test reliability

 If you want to try it anyway, it would require extreme ressources,

furthermore the question: "Why don‘t we do the whole test manually
anyway" would have to be answered

54
 eCATT or different SAP tools for testautomation?

SAP eCATT SAP Quick Test Professional by HP

 Supported protocols:  Supported protocols :

 SAP GUI  SAP GUI
 WebDynpro Java  WebDynpro
 WebDynpro ABAP (only selected objects)  Java
 SAP Web Services  Web, SAP-Web (Portal)
 License costs: none  Win32
 .NET
 Terminal (3270, 5250)
 PowerBuilder
 Oracle Forms, Oracle Applications, Siebel,
PeopleSoft
 MS Office, VB, Visual Age
 License costs : depends

55