2016 Intl. Conference on Advances in Computing, Communications and Informatics (ICACCI), Sept.

21-24, 2016, Jaipur, India

Exploratory Test Oracle using Multi-Layer

Perceptron Neural Network
Wellington Makondo

Raghava Nallanthighal

Innocent Mapanga

Prudence Kadebu

Department of Information
Technology

Department of Information
Technology

Department of Computer
Science

Department of Software
Engineering

Harare Institute of Technology

Delhi Technological University

Harare Institute of Technology

Harare Institute of Technology

Harare, Zimbabwe
wmakondo@hit.ac.zw

New Delhi, India

nsraghava@gmail.com

Harare, Zimbabwe
imapanga@hit.ac.zw

Harare, Zimbabwe
pkadebu@hit.ac.zw

AbstractIn the context of exploratory testing (ET), human

knowledge and intelligence is applied as a test oracle. The
exploratory tester designs and executes the tests on fly and
compares the actual output produced by the application under
test with the expected output in the testers mind. The
shortcoming of human oracle is that they are fallible, that is
exploratory testers do not always detect a failure even when a
test case reveals it. Depending on a human tester to evaluate
program behaviour has also some problems such as cost and
correctness. Therefore, in this paper an effort has been made to
explore the feasibility of using a multilayer perceptron neural
network (MLP-NN) as an exploratory test oracle. The MLP-NN
was improved by adding another weight on each connection to
perfectly generate reliable exploratory test oracles for
transformed different data formats.
Keywords: Test Oracle, Software Testing, Multilayer
Perceptron Neural Network, Software under Test.

I. INTRODUCTION
Software testing has always been a crucial phase in the
software development life cycle. The previous studies has
witnessed many software projects failures that result from lack
of proper testing. Software that has been improperly tested can
easily break a companys reputation. A better relationship
between testing and requirements would finally result in
enhanced tests and better-quality software [1]. Consequently,
test methods were established to mitigate this limitation [2].
When the flow of requirements is parallel to the
development phase, the systematic test case based approach is
considered difficult to use owing to the constant requirement
change while designing tests [3]. Hence, an approach such as
ET can be worthwhile for such environments, where the
requirements elicitation has not been completed early in the
software development process.
ET is a software testing approach, at times called ad-hoc
testing [4] which is not dependent on test cases documentation
prior to execution. Today, most researchers defined ET as an
activity where a tester learns, design and execute the tests
simultaneously [4]. To summarize, this means the tester is
discovering the application, studying its functions and running
tests based on his/her knowledge. There is no logical method
followed as in scripted testing that guides the tester the test steps

978-1-5090-2029-4/16/$31.00 @2016 IEEE

to run the tests. The tester himself controls the test designs
while running and studying the application under test. This
supports designing effective test cases while exploring
unrealized parts of the software.
In modern days, this approach has gained attention and
reputation particularly among professional software testers [5].
The ET term was initially coined by Bach and Kaner in 1983
and it was adopted by many professional testers [6]. ET is used
in combination with other testing techniques in industry [6]
which includes, Risk Based Testing, Cross functional testing,
Requirements based testing, Pair wise testing, Acceptance
testing, Checklist based testing, Scripted Testing, Usability
Testing, Security Testing and Boundary Value Analysis.
Exploring means focused wandering, navigating through a
space with a general assignment, but without a pre-defined
route. Exploration encompasses on going studying and
execution of tests [7].
In the context of ET, human knowledge and intelligence is
applied as a test oracle. A test oracle is defined as a technique
used to differentiate an accurate result from an inaccurate result
during testing [8]. The challenge of finding a reliable oracle is
referred to as the oracle problem.
In practice, the identification of failures is left to be decided
by a human tester [9]. Therefore, the oracle problem is very
important in ET and solved using the testers knowledge and
skills. The shortcoming of human oracle is that humans are
fallible, that is exploratory testers do not always detect a failure
even when a test case reveals it. Depending on human testers to
evaluate program behaviour has also some problems such as
cost and correctness. How could this be tackled with some
subroutines to do such work?
The motivation for work done in ET is its exponential
adoption by many software development firms. ET has been
recommended as a worthwhile approach due to numerous
advantages it has [6], [7]. The quality of the software under test
(SUT) is highly dependent on the ability of the tester. The
exploratory tester may not properly understand the
requirements and specifications of the software and
consequently endangers the quality of the product by judging
correct results as wrong or wrong results as correct. Sometimes

1166

2016 Intl. Conference on Advances in Computing, Communications and Informatics (ICACCI), Sept. 21-24, 2016, Jaipur, India

the tester may not be able to conclude whether the result is

correct or not.
Our research goal is to apply artificial neural networks in a
simulated application under test to predict the test oracle of the
executed tests. MLP-NN has not been applied to deal with
different data formats in software development to facilitate
learning and make decisions on test combinations executed.
In our study, we used MLP-NN and evaluated different
parameters that gives good results. Although not explicitly
stated in the studies, partial knowledge of the internal
functionality of the software under test need to be possessed by
the exploratory testers. The expected outputs generated by the
MLP-NN allows the exploratory tester to judge the test results
correctly.
II. LITERATURE SURVEY
The test oracle research literature is comparatively very
small in software testing. A test case comprises of a set of inputs
and their corresponding expected outputs for the SUT.
Automatic generation of test inputs is fairly stress-free, but
generation of expected results is a relatively difficult task.
Testers cannot have confidence to do automated testing without
expected outputs. A test oracle is a trustworthy source of
expected results. It can be a program specification or simply the
developers knowledge that verifies the way a program
operates. A correct oracle would behave equally the same way
with the SUT and absolutely trusted. The SUT would accept all
input specified and correct result would always be produced.
Recent dynamic software testing approaches rely on the
presence of test oracle, i.e., an instrument for proving that the
SUT has correctly performed on a certain execution [10].
A partial test oracle [11] was suggested to prove the
correctness of observed results in XML data. The SUT
considered was a query program which takes XML document
as input acquired from an XML repository of any form and
produces XML data as a result. The oracle automatically
provide a response about the accuracy of the module being
tested with a certain degree of accuracy at feasible cost.
Other test oracle methods exploited artificial intelligence
and pattern recognition techniques. These approaches involve
writing additional program with the similar specifications as
well as using pattern recognition and artificial intelligence
based oracles [12]. In the application of a typical pattern
recognition based oracle, the unit emulates human testers
knowledge and decision making process. The oracle imitates an
accurate version of the module tested and make use of a
distance measure to check the distance between the tested
module output patterns and output patterns of oracles.
The oracle problem exist when either the oracle does exist
or not but very expensive to be used. Metamorphic testing [13]
uses metamorphic relations, properties of the SUT represented
in form of relations amongst the inputs and the outputs of
multiple executions, in order to aid validate the perfection of a
program. This approach was used to lessen the oracle problem
in numerous systems and to improve numerous software
examination and testing methods. The authors witnessed that
identification of an adequate amount of suitable metamorphic

relations for testing by nave testers was feasible with minimum

training. Besides, the approach is cost-effectiveness and could
be enhanced using more diverse metamorphic relations.
Currently machine learning methods [14] have been
employed in the software engineering specifically in software
testing field. However, the algorithms are difficult to discover
the faults in certain applications as it is hard to find the test
oracle to be used for computed outputs verification. A novel
functional testing approach was attempted to verify the test
oracles. The expected execution output for a given application
is produced and verified by the oracle whether the SUT behaved
correctly or not and issues a pass or fail verdict.
Techniques for automatic generation and execution of test
cases were reported to improve coverage of many conditions
without the increase of significant costs. Yet, one problem of
automatically generating and executing test cases is the oracle
problem. A model based oracle generation method [15] for unit
testing belief desire intention agents was presented. A fault
model was developed based on features of the main components
to record the kinds of defects that may be faced, and describe
how to generate automatically a passive, partial oracle from the
agent design models.
Search based test generation method can output enormous
volumes of test inputs automatically. Though, it is tough to
define the test oracle for every test input. A mining approach
[16, 17] was presented to build a decision tree model allowing
generation of test inputs from the Java bytecode. Java bytecode
was converted into Jimple representation. The resulting extracts
predicates derived from the control flow graph of the Jimple
code are used as attributes for organizing training data to form
a decision tree. Faults were injected to assess the performance
of the technique.
III. PROPOSED METHODOLOGY
Our proposed methodology employs MLP-NN to facilitate
learning of executed test cases and hence classify them as
reliable test oracles. The multilayer perceptron neural (MLP)
network model was simulated using MATLAB software where
the network has one hidden layer containing one or more
neurons and one output layer with only one neuron. Ideally, a
MLP network that performs best normally uses selected
parameters from the experiments.
A. Architecture of Proposed Exploratory Test Oracle
The test oracle requires a simulated model of the application
under test that imitates its behavior, and it should produce
accurate expected results automatically for all input
combination. Figure 1 shows the procedure of using the
proposed automated exploratory test oracle.
Test inputs are first passed to the simulate model for
compliance then mapped to numeric data to be handled by
MLP-NN, then the MLP-NN is trained. The trained MLP-NN
becomes usable for generating reliable exploratory test oracles.
Testing follows using the trained MLP-NN generating partial
test oracle. The same inputs are given to the application under
test (AUT) and actual outputs are produced. Based on the actual
outputs and expected outputs, the exploratory tester determine
whether the test passed or not.

1167

2016 Intl. Conference on Advances in Computing, Communications and Informatics (ICACCI), Sept. 21-24, 2016, Jaipur, India

Testers
knowledge or
pre-designed
tests

Training
dataset
Inputs
Simulated
AUT

layer and also output layer is associated with an activation

function. There is one layer placed between the input and output
layers. The design of feed forward network is such that the
signals are fed straight forward throughout the entire network.
In figure 2 below, there is no back-feeding of signals into the
previous layers.

Testing
Phase

Training
Phase

Input

Input

Simulated
AUT

n inputs

AUT

Data
Transformation

Data
Transformation

No. hidden
neurons

No. output
neurons

MLP NN
as a Test
Oracle

Trained
MLP NN
as a Test
Oracle
Expected
output

Actual
output

Human
Comparator

Bias

Figure 2: Multi-layer Perceptron Neural Network

Decision:
Correct
Or
Incorrect

Figure 1: Proposed Exploratory Test Oracle Architecture

The number of inputs of AUT would be equal to the number
of inputs defined in the MLP parameters. The AUT is simulated
by defining the input data types, length (min and max) and
range (min and max) for numerical inputs.
In order to train the MLP-NN, we have created a real test
case training dataset. The tool works by taking a 2 dimensional
array of test cases in CSV file format. The test case data
comprises of different data types, i.e. string, alphanumeric,
integer or other forms. Before using test dataset directly in
training MLP-NN, extra work must be done. It is essential to
map the data into input space of MLP-NN since MLP-NNs can
only learn from numerical values.
All data types were mapped into numeric domain by
converting to ASCII equivalence values. Each character is
converted to its ASCII equivalence, but the ANN cannot handle
all these values once as a single input. So, we summarized the
ASCII codes to be taken as a single input.
B. Feedforward MLP-NN
A MPL-NN consists of layers of nodes in a directed graph,
with each layer connected to the next one. Each node in hidden

This type of network consists of input layer, one hidden

layer and output layer. Input layer is represented by one or more
input nodes in the multilayer architecture. At every moment
there is an activity denoted by xi in each neuron. Neurons are
connected by directed connections that are denoted by w, which
is a weight or strength. When node i is connected with node j,
then the connection weight is denoted as wij.
Inputs or other elements are converted into signals and
correspondingly strengthened by the weights. In the inside of a
neuron, all incoming signals are summarized in equation (1).
The inner sum corresponds to:
= +

(1)

where w0 is bias weight.

The summarized input is taken as an argument by the
activation function and gives the result or output value of the
node. The activating function is denoted with f in equation (2)
and the resulting activity with xj.
= 

(2)

and equation (3) shows a comprehensive formula with

summarized activity.

1168

=  )

(3)

2016 Intl. Conference on Advances in Computing, Communications and Informatics (ICACCI), Sept. 21-24, 2016, Jaipur, India

C. Activation functions
Activation functions decides how the neurons and the
network should process signals. The function determine when
the neuron should be active depending on the given threshold
when reached or not. We applied linear, sigmoid and
hyperbolic tangent activation functions. For training process,
experiments were carried out to evaluate the activation
functions that address the problem better.
A sigmoid activation function uses a sigmoid function and
is defined as follows:


(4)

respect to weights. The output obtained is clearly determined

by the inputs, weights and thus the error too. Suppose we have
n weights, the error will be denoted by:

As the error E is a function of all weights, we need to

consider, how a single weight w change affects the error E. If f
is a differentiable and continuous function, this is the same as
observing the partial derivative of E with respect to , i
in the point w.
Therefore, to find the gradient vector, an error term for
node i is defined as

The hyperbolic tangent activation function is defined as

follows:


(5)



The output neurons are activated and provide the entire

network's result to the outside world, which can be the next
neural network or the human user. Output layer in this
multilayer architecture is represented by one output node.
Network neurons are numbered starting with xi, xj, and xk
indicating the activities in input layer through hidden layer to
output layer. If the desired output for a given input vector x is
dk = (dk1, dk2, dk3) and the observed or actual output is xk = (xk1,
xk2, xk3). Then the measure of error for this is given below
(7)
and in general, if we had m outputs, the measure of error
would be:
(8)

The sum of squared differences between desired and actual

outputs of all the m output nodes are defined. So, to obtain the
overall difference for the whole data set means that we are
summing the output over all input vectors where the number of
inputs is denoted by p.

(6)

(11)

Applying the chain rule, the recursive formula for can

be

written as

A linear activation function is perhaps the least used

activation function. We considered experimenting to check
whether it gives good results or not. The Equation (6) shows the
linear function.

(10)

(9)

Considering the error as a function of the weights in the

network and take a closer look at the partial derivatives with

if node i is a output node


otherwise,

(12)

Where  is the weight connection from node i to j; and

 is zero if there is no direct connection. Then the weight

update  for online, pattern by pattern learning is


(13)

where is the learning rate that affects the convergence

speed and stability of the weights during learning.
Training data is processed till the error rate is reached.
When training completes all the data connections in the network
will be saved in the weight matrix. Different elements in input
vectors have imbalanced contributions for ANN. Uniform
transformation is needed for equalization impaction of all input
variables. The principle of executing a test follows that all
inputs given to the software under test must be valid to be
considered as successful. Otherwise, if the graphical user
interface of software under test have one wrong input, the test
is considered as invalid or failed.
So it is difficult to differentiate summarised ASCII values
that are valid from invalid ones. Therefore, the ANN also fails
to classify such data. We then considered the validation aspect
of software under test that is, the valid inputs must be associated
with weights that differ from weights associated with the
invalid inputs. This has given rise to the modification of MLP
neural network to Two Weighted MLP-NN.
A larger weight is selected when the input is valid in order
to approximate towards the desired class 1 and smaller weight
when input is invalid. This allows the ANN to be able to classify
invalid and valid input combinations. A combination of valid
inputs should be classified as correctly executed, class 1 and

1169

2016 Intl. Conference on Advances in Computing, Communications and Informatics (ICACCI), Sept. 21-24, 2016, Jaipur, India

invalid inputs or combination of valid and invalid inputs should

be classified as wrongly executed in class 0.
Moreover, the decision of the structure of ANN is a more
challenging problem. How many neurons in hidden layer are
required for specific problem requires cautious analysis. This
process could not be implemented entirely automatically. To
find the appropriate parameters, a number of experiments were
carried out varying the parameters.
Also, training set is very important for supervised ANN
learning. Selection of training set from predefined test cases is
another main problem that requires to be addressed carefully.
After attaining the expected MLP-NN, it could be used in
predication for new input data.
D. Testing Phase
After the MLP-NN is properly trained, it becomes feasible
to use it as automated exploratory test oracle. The process of
using the MLP-NN based test oracle is as follows: the logical
component under test is executed using the inputs provided by
the test case, while the trained MLP is given the same inputs.
Observed outputs which must be evaluated are produced by the
SUT and the expected outputs by the MLP-NN. Human
intelligence is employed as a comparator to compare the results,
and any deviation is reported as possible SUT defect. This
procedure may be automated with little human intervention in
preparing the environment and the required dataset.
E. Summary of Exploratory Test Oracle
Exploratory test oracle was summarized in the following steps.
Step 1: Construct neural networks and initialize the weight.
Step 2: Simulate application under test.
Step 3: Load test cases for training.
Step 4: Perform data transformation i.e. map non-numeric data
into numeric.
Step 5: Perform feedforward pass in the MLP
Step 5: Using back propagation algorithm to train neural
networks in the training sets.
Step 7: When the stopping criterion is not satisfied, repeat step
5 and 6 until all inputs of the AUT are fed to the MLP, otherwise
keep the weights and go to step 8.
Step 8: Get testing inputs of AUT and input it to neural
networks. Obtain the approximate output from neural networks.
Determine if the test oracle is a failure or not by the result.
Step 9: If you need to test a different module, go to step 1.
Otherwise, the process finishes.
F. Test cases
A dataset with 160 test cases was created to be used in the
MLP-NN based on AUT specifications. It comprises of invalid
test cases that fall in class 0 and valid ones that fall in class 1 as
shown in Table I. Each data set has four features and its
corresponding class. The AUT used for experimenting is user
interface for adding a user in a system with four input boxes
namely, Username, Password, Employee id and security access
level.
The dataset was divided into 60 for training and 100 for
testing. This datasets contains 50% valid test cases and 50%
invalid test cases.

TABLE I: SAMPLE TEST CASES

IV. EXPERIMENTAL RESULTS

The main objective of this study was to predict the
exploratory test oracle using MLP-NN. In order to choose
optimal parameters that gives good results, experiments were
done differing parameters values and evaluate the results. The
excellent networks, are the ones that give the highest correct
classification percentage and outputs that are close to 0 or 1, if
the test failed or passed respectively. Such a network will be a
chosen MLP network for the system. The network would be
stopped from training once the error threshold value converged
and all training datasets have been applied to the network.
Accuracy of MLP networks was evaluated to determine the best
parameters that suited the network to perform correct
classification of test oracles.
A. Changing Activation functions
Table II shows a summary of the outputs of combinations
of activation functions at hidden layer and at output layer.
TABLE II. MLP PERFORMANCE OF TEST ORACLE CLASSIFICATION FOR
VARIOUS ACTIVATION FUNCTIONS

No.
Code
1
2
3
4
5
6
7
8
9

Network
Code
sig-sig
sig-tanh
sig-pul
pul-pul
Pul-sig
pul-pul
tanh-tanh
tanh-pul
tanh-sig

Class 0
Min

0.09107
0.01454

0.007612
0.06734
0.073464

Class 1
Max

Min

0.1754
0.8583
0.0437
0.9171
Infinite training
Infinite training
Infinite training
Infinite training
0.08501
0.9291
0.09352
0.41424
0.86547

Max

0.8786
0.9395

0.953
0.89706

The results have shown that using sigmoidal activation

function at hidden layer and hyperbolic tangent activation
function at output layer yields the best results. Pure linear
resulted in infinite training execution. It executed finitely when
the network code was tanh-pul but classified all test oracles in
failed class. Therefore, in the subsequent experiments sigmoid
activation function at hidden layer and hyperbolic tangent
activation function at output layer were used.
B. Changing Number of Neurons in Hidden Layer
Few neurons gives desirable outputs but the network learns
very slowly. Many neurons results in outputs with large error
but learns much faster. We therefore choose to use 4 neurons in
hidden layer. Since we are predicting the test oracle, we are
expecting an output from a single pass as passed or failed; that
is why we have one output neuron at output layer.
C. Error Threshold Variations

1170

2016 Intl. Conference on Advances in Computing, Communications and Informatics (ICACCI), Sept. 21-24, 2016, Jaipur, India

Based on the different error threshold values that were

experimented with, a higher error threshold value results in few
iterations executed to reach a shopping criteria and a higher
percentage of true error estimate. The highest true error
estimate 19.272% was obtained when error threshold was 0.1
and the least number of epochs 6 while with the smallest error
threshold 0.001, the MLP-NN performed 50 iterations to reach
the stopping criteria while the true error estimate was 3.0756%.
Therefore, 0.1 error threshold was chosen to be used in the MLP
network.
D. Learning Rate variations
The higher the learning rate (maximum of 1.0) the faster the
network is trained. Though, the network has a higher chance of
being trained to reach a local minimum solution, which is a
point at which the network become stable on a solution that is
not the most optimal global solution. Therefore, 0.3 was chosen
as the learning rate for our MLP network as it provides optimum
performance and has a better chance to stabilize at the most
optimal global solution.
V. CONCLUSION AND FUTURE WORK
The current work has been focused on application of
multilayer perceptron neural network as an exploratory test
oracle to generate expected output of the AUT. AUT with
different data types was used for experimental purposes and
showed that it cannot learn properly from summed ASCII codes
as correct and wrong inputs cannot be realized easily by the
MLP-NN. This is because the value of the correct and wrong
can be similar or they may be more or less the same or fall in
the same range.
The normal MLP-NN with one weight on each connection
failed to classify the transformed data correctly and hence
modified to two weighted MLP-NN. The improved MLP-NN
with two weights on each connection perfectly generated
reliable exploratory test oracles. The exploratory testers tests
the AUT to produce actual outputs while the exploratory test
oracle produce the approximate or partial expected outputs.
Using testers knowledge and interpretation, the expected and
actual outputs are compared and a fault report is produced.
REFERENCES
[1] Fewster, M., Graham, D., 1999. Software test automation.
Addison-Wesley Harlow, Essex, UK.
[2] Everett, G., McLeod, R., 2007. Software testing: testing across
the entire software development life cycle. Wiley-IEEE
Computer Society Pr.

[3] Itkonen, J., Rautiainen, K., 2005. Exploratory testing: a multiple

case study. In: Empirical Software Engineering, 2005. 2005
International Symposium on. p. 10 pp.
[4] Bach, J., 2000. Session-based test management. Quardev
Laboratories.
[5] Itkonen, J., Mantyla, M., Lassenius, C., 2007. Defect detection
efficiency: Test case based vs. exploratory testing. In: Empirical
Software Engineering and Measurement, 2007. ESEM 2007. First
International Symposium on. pp. 6170.
[6] Naseer, A., Zulfiqar, M., 2010. Investigating exploratory testing
in industrial practice, Blekinge Institute of Technology,
COM/School of Computing.
[7] Tinkham, A., Kaner, C., 2003. Exploring Exploratory Testing.
In: STAR East conference, www.testingeducation.org.
[8] A. Abran, J. W. Moore, P. Bourque, R. Dupuis, and L. L. Tripp,
Guide to the Software Engineering Body of Knowledge 2004
Version. Los Alamitos, CA, USA: IEEE Computer Society, 2004.
[9] L. Baresi and M. Young, Test oracles, University of Oregon,
Dept. of Computer and Information Science, Eugene, Oregon,
U.S.A., Tech. Rep. Technical Report CISTR- 01-02, Aug. 2001.
[10] Itkonen, J.; Mantyla, M.V.; Lassenius, C., "The Role of the
Tester's Knowledge in Exploratory Software Testing," Software
Engineering, IEEE Transactions on , vol.39, no.5, pp.707,724,
May 2013
[11] Kim-Park, D.S.; de la Riva, C.; Tuya, J., "A Partial Test Oracle
for XML Query Testing," Testing: Academic and Industrial
Conference - Practice and Research Techniques, 2009. TAIC
PART '09. , vol., no., pp.13,20, 4-6 Sept. 2009.
[12] I. Gondra, Applying machine learning to software faultproneness prediction, J. Syst. Softw., vol. 81, no. 2, pp. 186195,
Feb. 2008.
[13] Huai Liu; Fei-Ching Kuo; Towey, D.; Tsong Yueh Chen, "How
Effectively Does Metamorphic Testing Alleviate the Oracle
Problem?," Software Engineering, IEEE Transactions on , vol.40,
no.1, pp.4,22, Jan. 2014.
[14] Monisha, T.R.; Chamundeswari, A., "Automatic verification of
test oracles in functional testing," Computing, Communications
and Networking Technologies (ICCCNT),2013 Fourth
International Conference on , vol., no., pp.1,4, 4-6 July 2013.
[15] Padgham, L.; Zhiyong Zhang; Thangarajah, J.; Miller, T.,
"Model-Based Test Oracle Generation for Automated Unit
Testing of Agent Systems," Software Engineering, IEEE
Transactions on , vol.39, no.9, pp.1230,1244, Sept. 2013.
[16] Weifeng Xu; Tao Ding; Hanlin Wang; Dianxiang Xu, "Mining
Test Oracles for Test Inputs Generated from Java Bytecode,"
Computer Software and Applications Conference (COMPSAC),
2013 IEEE 37th Annual , vol., no., pp.27,32, 22-26 July 2013.
[17] Weifeng Xu; Hanlin Wang; Tao Ding, "Mining Auto-generated
Test Inputs for Test Oracle," Information Technology: New
Generations (ITNG), 2013 Tenth International Conference on ,
vol., no., pp.89,94, 15-17 April 2013.

1171