Proposed the ANOVA Method for Analysis of Proficiency Test Result

*R. Hegazy, **M. I. Mohamed, *A. Abu-Sinna

*Researcher at national institute of standards, Nis, Egypt

**Professor at national institute of standards, Nis, Egypt

1. Abstract

Analysis of variance "ANOVA", as a trustful statistical method most commonly

applied to the results of the experiment to determine the percent contribution of each
factor, is used in this paper in comparison with Z-Score statistical method. Z-Score,
Zeta Score, and En Number are the most credible statistical methods used in
proficiency testing evaluation. In this paper four different PT examples were
demonstrated, both ANOVA as well as Z-Score were used for the analysis of the
testing. The first example showed a consistency of the results between both of them,
however in the other three examples dissimilarity happened between the two results.
While Z-Score gave the participant laboratory a satisfactory result, ANOVA gave the
same laboratory unsatisfactory result. Final results showed the advantage and
reliability of ANOVA analysis over Z-Score analysis.

2. Introduction
The proficiency test (PT) is a powerful tool to help a laboratory to demonstrate its
competence [1]. Proficiency testing determines the performance of individual
laboratories for specific tests or measurements and is used to monitor laboratories’
continuing performance. Proficiency testing is also called interlaboratory comparison.
As this term implies, proficiency testing compares the measuring results obtained by
different laboratories. In a proficiency test one or more artifacts are sent around
between a numbers of participating laboratories. Each laboratory measures the
artifacts according to a given set of instructions and reports its results to the
administrator. The results reported by each laboratory for a measurand are compared
to the reference value for that measurand [2]. The reference value can be determined
in various ways. The two most common ways are to use a reference laboratory or use
the average of the values reported by the participants.
The following are some of the potential benefits which may be available to
participating laboratories [3, 4]:
- Confirming competent performance
- Identifying testing or measurement problems
- Comparing methods and procedures
- Improving performance
- Educating staff
- Instilling confidence in staff, management and external users of laboratory
services
- Comparing operator capabilities
- Generating reference materials
- Determining method precision and accuracy
- Satisfying regulators and accreditation bodies
- Providing laboratories with additional risk management.

1
Appropriate statistical design of a proficiency testing scheme is essential. Data
analysis method should be chosen in such away to truly describe the variation of the
results between the participating laboratories [5].
The aim of this research is to give a new statistical methodology for comparing the
proficiency testing results. The present work proposed using the ANOVA analysis,
through using the F distribution to compare the results of the proficiency tests. Some
different examples on hardness test are analyzed by Z score and ANOVA.

Following are a presentation for some the most credible statistical methods used in
proficiency testing evaluation.

3. Methodology:

In proficiency testing, usually output data treated using different analysis criteria,
mostly are; Z-Score, Zeta-Score, and Normalized error value En [6, 7]. However, the
aim of this research is to give a new statistical methodology for comparing the
proficiency testing results. The present work proposed using the ANOVA analysis,
through using the F distribution to compare the results of the proficiency tests. Some
different examples on hardness test are analyzed by Z-Score and ANOVA.
Next are the equations used in this comparison with the max allowable limits for
satisfaction results for both types.

3.1 Z-Score "Z":

Z score is calculated from the following equation:
x X
Z= (1)

Where:
x is the participant's result ;
X is the assigned value;
σ is the standard deviation for proficiency assessment [8].
‫׀‬z‫≥׀‬2 indicates “satisfactory” performance
2<‫׀‬z‫ < ׀‬3 indicates “Questionable” performance
‫׀‬z‫≤׀‬3 indicates “Unsatisfactory” performance

3.2 Analysis of Variance ANOVA F-test:

ANOVA F-test is a collection of statistical models used to analyze the differences
between group means and their associated procedures (such as "variation" among and
between groups). In ANOVA setting, the observed variance in a particular variable is
partitioned into components attributable to different sources of variation. In its
simplest form, ANOVA provides a statistical test of whether or not the means of
several groups are equal, ANOVAs are useful in comparing (testing) three or more
means (groups or variables) for statistical significance [9].

Where:
MSR Mean square ratio
MS Mean square
SS Some of squares
I No. of treatments

2
 nT No. of Cases.
According to F Test tables [10], the the minimum Mean Square Ratio (MSR) required
for factor to be significant at 95% and 99% which is the most used confidence levels
in the testing laboratories using 2 σ, and 3σ control limits confidence levels is equal to
5.32 and 11.3 respectively.

4. Examples

The following are some examples to demonstrate the comparison between the
analysis of results of proficiency test using ANNOVA and Z-Score methods. These
examples are different both in variance between Lab 1 and Lab 2 and in variance
among Lab 2 itself. Take into consideration that Lab 1 is always denoting the
reference laboratory, while Lab 2 is always denoting the participant laboratory.

4.1 Example (1):

The results of two hardness laboratories for Rockwell C hardness tests are as follows:

Table (1) Data obtained from Example No. (1)

Lab 1 Lab 2
Reading (1) 62.40 62.00
Reading (2) 63.00 62.20
Reading (3) 62.90 63.00
Reading (4) 62.20 62.20
Reading (5) 62.70 62.10

First, we will use Z-Score for analyzing data of Example (1).

Table (2) Z-Score results for data obtained from Example No. (1)
Lab 1 Lab 2
Average 62.64 62.30
Z- score - 0.87

Secondly, using ANOVA for analyzing data of Example (1).

Table (3) ANOVA results for data obtained from Example No. (1)
DOF MSR
Column 0.289 1 0.289 2.11
Residual 1.092 8 0.136
Total 1.381 9

From Table (2) it is seen that Z-Score showed a result of 0.87 which is less than
allowable limit, While in Table (3) it is seen that the mean square ratio is equal to
2.11 which is below the MSR required for both 95% and 99% confidence levels.

4.2 Example (2):

The following example gives results for hardness Rockwell with more diversity for
the results of laboratory 2 than those given in example (1).

3
 Table (4) Data obtained from Example No. (2)
Lab 1 Lab 2
Reading (1) 62.40 62.00
Reading (2) 63.00 62.20
Reading (3) 62.90 62.10
Reading (4) 62.20 62.20
Reading (5) 62.70 62.10

First, we will use Z-Score for analyzing data of Example (2).

Table (5) Z-Score results for data obtained from Example No. (2)
Lab 1 Lab 2
Average 62.12 62.64
Z- score - 1.45

Secondly, using ANOVA for analyzing data of Example (2).

Table (6) ANOVA results for data obtained from Example No. (2)
DOF MSR
Column 7.397 1 7.397 32.66
Residual 1.812 8 0.226
Total 9.209 9

From Table (5) it is seen that Z-Score showed a result of 1.45 which is less than
allowable limit. However n Table (6), it is seen that the mean square ratio is equal to
32.66 which is higher than the MSR required for 95% confidence levels and lower
than the MSR required for 99% confidence level.

4.3 Example (3):

The following example shows results for lab code 2 which seem to be higher than the
results of lab 1.

Table (7) Data obtained from Example No. (3)

Lab 1 Lab 2
Reading (1) 62.40 65.00
Reading (2) 63.00 65.30
Reading (3) 62.90 65.20
Reading (4) 62.20 65.10
Reading (5) 62.70 65.20

First, we will use Z-Score for analyzing data of Example (3).

Table (8) Z-Score results for data obtained from Example No. (3)
Lab 1 Lab 2
Average 62.28 65.16
Z- score - -1.87

4
Secondly, using ANOVA for analyzing data of Example (2).

Table (9) ANOVA results for data obtained from Example No. (3)
DOF MSR
Column 15.88 1 15.88 252
Residual 0.504 8 0.063
Total 16.38 9

From Table (8) it is seen that Z-Score showed a result of -1.87 which is less than
allowable limit. However n Table (9), it is seen that the mean square ratio is equal to
252 which is much higher than the MSR required for both 95% and 99% confidence
levels.

4.4 Example (4):

The following example shows results for Lab 2 which seem to be much higher than
the results of Lab 1 and much variance between its results.
Table (10) Data obtained from Example No. (4)
Lab 1 Lab 2
Reading (1) 62.40 66.00
Reading (2) 63.00 66.50
Reading (3) 62.90 65.00
Reading (4) 62.20 62.20
Reading (5) 62.70 69.00

First, we will use Z-Score for analyzing data of Example (4).

Table (11) Z-Score results for data obtained from Example No. (4)
Lab 1 Lab 2
Average 62.64 65.74
Z- score - -1.33

Secondly, using ANOVA for analyzing data of Example (4).

Table (12) ANOVA results for data obtained from Example No. (4)
DOF MSR
Column 24.025 1 24.02 7.749
Residual 24.804 8 3.100
Total 48.829 9

From Table (11) it is seen that Z-Score showed a result of -1.33 which is less than
allowable limit. However n Table (12), it is seen that the mean square ratio is equal to
7.749 is higher than the MSR required for 95% confidence levels and lower than the
MSR required for 99% confidence level.

5. Discussion

Four examples shown in above with results designed for the purpose of showing the
difference in judgment between ANOVA F-test and Z-Score. Example (1) shows that

5
the average of result of the Lab 2 is very similar to the average of the results of the
reference lab (Lab 1) and its results with no significant variance. This lab has
satisfactory results using Z-score analysis and using ANOVA analysis with 95%, and
99% confidence level.
Example (2) shows that Lab 2 has small difference in result average and small
variance in the readings, and has unsatisfactory result using ANOVA analysis with 95%
confidence level and questionable with 99% confidence level although it has
satisfactory results using Z-score analysis.
Example (3) shows significant difference between the average of the two results of
Lab1 and Lab 2 with no significant variance of the results of the Lab 2, which leads to
unsatisfactory result using ANOVA analysis with 95%, and 99% confidence level.
However Z-score didn't detect such variance.
Example (4) shows significant difference between the average of the two results of
the Lab 1 and Lab 2 and a significant variance of the results of the lab 2, which leads
to unsatisfactory result using ANOVA analysis with 95% confidence level. Such
variance didn't be detected either by using Z-Score or by using ANOVA analysis
using 99% confidence level.
From the above analysis we noted that by using Z-score analysis, when the variance
in the lab results was increased, the value of standard deviation (σ) was increased,
which leads to decrease Z-score value and the lab has satisfactory results, but the
ANNOVA analysis is very sensitive either to the variation between the average of the
results of the two labs or the variance within the results of the lab.
It is recommended to establish another study for comparing results of ANOVA
analysis with both Zeta-Score and En number analysis in order to having a more
overview outlook.

6. Further scope of research:

(En) and Zeta score, e are calculated from the following equations respectively:
En=(x-X)/√(U_Lab^2+U_ref^2 ) ʓ=(x-X)/√(u_Lab^2+u_av^2 )
Where
U_Lab^ is the expanded uncertainty of the participant’s results.
U_ref^ is the expanded uncertainty of the reference laboratory’s assigned value.
u_Lab^ is the standard uncertainty of the participant’s results.
u_av^ is the standard uncertainty of the assigned value.
It is obvious that the uncertainty is the main factor for making decision on the
performance of the laboratory, which is not a factor for ANOVA calculation, so it
needs further research work to compare the ANOVA analysis with either En or ʓ-
score with addition of special uncertainty analysis for the ANOVA calculation which.

6. References
1. ISO/IEC 17025, "General requirements for the competence of testing and
calibration laboratories", 2005.
2. EA-03/04 European co-operation for accreditation, "Use of proficiency testing
as a tool for accreditation in testing", 2001.
3. R. Hegazy, M. I. Mohamed, A. Abu-Sinna ,"A comparative study of statistical
methods used in analyzing the proficiency testing results of yield stress"
MAPAN June 2010, Volume 25, Issue 2, pp 107-113.

6
4. J. Singh, N. Dilawar Sharma, A. Kumar, and A. K. Bandyopadhyay, "Report
of the proficiency testing in the pneumatic pressure region up to 5 Mpa",
MAPAN September 2014, Volume 29, Issue 3, pp 213-222.
5. Rob G. Visser, "Interpretation of interlaboratory comparison results to
evaluate laboratory proficiency", Accred Qual Assur, 2006.
6. Sanjay Yadav, V K Gupta and A K Bandyopadhyay, "Standardization of
pressure calibration (7-70 MPa) using digital pressure calibrator", MAPAN
January 2010, Volume 69, pp 27-33.
7. S. Basak, S. S. Mukherjee, S. N. Mandal, Rama Das, A. K. Mazumder, J. K.
Mondal, R. Sammaddar, S. Mondal, Dipali Kundu, "Interlaboratory
proficiency testing: Intercomparison in relation to the measurement of alumina,
iron(III) oxide and Titania present in homogenised china clay", MAPAN 2010,
Vol. 25, No. 4, pp 265-272.
8. ISO/IEC 17043, "Conformity assessment-General requirements for
proficiency testing", 2010.
9. Charles Lipson, Statistical Design and Analysis of engineering Experiments,
McGraw-Hill, 2000.
10. APLAC, Calibration interlaboratory comparisons, March 2008.