Computer Programmer Aptitude Battery:

Validation Study

Theodore C. Willoughby

The SRA Computer Programmer Aptitude Battery (i) was developed using
two major hypotheses:

i. The typical general aptitude test was too easy for the
group usually considered for programmer jobs.

2. A test with several ability components would have more

predictive efficiency than a test of only one ability.

The resulting test is currently in use as a selection device. It is

composed of five separately timed parts which take about 90 minutes to ad-
minister. The test is published in reusable test booklets. Examinees mark
their responses on separate self scoring answer sheets. The test can be
easily scored on premises.
Initial validation studies indicated that the battery as a whole had
predictive ability equivalent to that for the various IBM programmer apti-
tude tests.

RESEARCH DESIGN

Students in an undergraduate and a graduate course in Management In-

formation Systems (MIS) at the University of Minnesota completed the battery
of tests. In addition American College Testing Program (ACT) scores were
available for many of the undergraduates. Admission Test for Graduate Study
for Business (ATGSB) scores were available for many of the graduate students.
The criterion was course grade. Completion of programming assignments was
a relatively small portion of the courses. Each course also involved sub-
stantial reading assignments of moderate difficulty in computer data pro-
cessing. Thus the criterion was contaminated. Course grades given were
A, B or C. The student scores on the ACT and ATGSB were generally superior.
The mean ACT total score of the undergraduates was 24.8 which was in the
70th percentile on the Group IV College Freshman Norms. The mean ATGSB
total score of the graduates was 525 which was in the 60th percentile on
the norms. These data indicated that the group was severely restricted in
range. Criterion contamination and restriction in range normally result
in a lowering of the predictive effeciency of the test.

TABLE i: TEST RESULTS FOR GRADUATE STUDENTS

STANDARD PERCENTILE IN
TEST MEAN DEVIATION NORM GROUP
i. CPAB Verbal Meaning
- 22.56 6.87 57
2. CPAB Reasoning
- 16.46 4.35 65
3. CPAB - Letter Series 13.86 4.85 70
4. CPAB - Number Ability 18. i0 4.99 73
5. CPAB - Diagramming 29.76 5.13 69
6. CPAB - Total i00.74 19 .06 78
7. Course Grade 2.98 .69
8. ATGSB - Verbal 29.62 8.75
9. ATGSB - Quantitative 32.85 7.89
i0. ATGSB - Total 524.54 102.44 60
 TEST RESULTS - GRADUATE STUDENTS

Means, standard deviations and percentile norms of each of the vari-

ables are shown in Tables i and 2. The CPAB norms are for experienced

TABLE 2: TEST RESULTS FOR UNDERGRADUATE STUDENTS

STANDARD PERCENTILE IN
TEST MEAN DEVIATION NORM GROUP
i. CPAB - Verbal Meaning 21.19 4.14 49
2. CPAB - Reasoning 16.03 4.05 62
3. CPAB - Letter Series 14.32 3.39 73
4. CPAB - Number Ability 17.65 3.78 69
5. CPAB - Diagramming 29.84 3.92 70
6. CPAB - Total 99.03 12.58 71
7. Course Grade 3.14 .59
8. ACT - English 21.58 3.64 57
9. ACT - Mathematics 27.21 3.68 79
i0. ACT - Social Studies 24.89 3.30 ' • 59
i!. ACT - Natural Sciences 25.05 4.09 63
12. ACT - Composite 24.84 2.34 71

programmers and systems analysts. The ACT norms are those of college fresh-
TABLE 3: INTERCORRELATION MATRIX - GRADUATE STUDENTS
Test 1 2 3 4 5 6 7 8 9
2 .38**
3 .44** .42**
4 .24 .53** .50**
5 .36** .54** .45** .35**
6 .72** .75** .76** .69** .73**
7 .05 -.03 .i0 .08 .ii .08
8 .67** .42* .30 .33 .16 .54* .05
9 .38 .81"* .52** .77** .41" .76** .24 .56**
i0 .62** .67** .44* .60** .30 .72** .15 .91"* .85**

* Significant at .05 level

** Significant at .01 level

TABLE 4: INTERCORRELATION MATRIX - UNDERGRADUATE STUDENTS

Test 1 2 3 4 5 6 7 8 9 i0 ii
2 .36*
3 .05 .17
4 .05 .53** .06
5 .35 .48** .48** .24
6 .58** .79** .51"* .58** .78**
7 .38 .56* .08 .ii .17 .42
8 .04 .ii .59** .18 .38 .38 -.21
9 .09 .43 .21 .69** .52* •59** -.19 .50*
i0 .39 .16 -.28 -.18 .30 .14 .50* -.26 -.15
ii .30 .42 .44 .21 .79** •66** .30 .Ii .41 .42
12 .34 .48* .46* .34 .84** .76** .22 .51" .66** .41 .85**

"7"
!
 men at Ph.D. granting universities. The ATGSB norms were calculated from
the assumption that the tested population had a normal distribution with
a mean of 500 and a standard deviation of i00. These scores provided sub-
stantial evidence of the superiority of the subjects.
The scores of each student on each test were correlated resulting in
the correlation matrix shown in Tables 3 & 4. The correlations within the
CPAB were, with only two exceptions, high and significantly different from
zero at the .01 level. Similarily the quantitative and verbal scores on
the ATGSB were highly correlated. Also, several of the CPAB scores were
correlated at the .01 level w i t h the two ATGSB scales.
Using stepwise regression analysis a regression equation was computed
from the correlation matrix in table 3. The results of the analyses are
shown in table 5. The ATGSB - Quantitative, Number Ability, Diagramming,

TABLE 5: REGRESSION DATA - GRADUATES

Variable Values
Multiple Correlation .58
Variance .34
CPAB -Reasoning Regression Coefficient -.01
- Number Ability Regression Coefficient -.07
- Diagramming Regression Coefficient .03
ATGSB - Verbal Regression Coefficient -.02
- Quantitative Regression Coefficient .12
Intercept 2.19
F VALUE (5,44) 4.51

Reasoning and ATGSB - Verbal tests entered the regression equation in that
order• The multiple R value of .58 was large enough to suggest that the
SRA - CPAB may be considered in a selection validation study for the pro-
grammer job. As was noted above the size of the multiple R was probably
diminished by the criterion measurement problems described and the re-
striction in range of the subjects.

TABLE 6: REGRESSION DATA - UNDERGRADUATES

Variable Value
R - Multiple Correlation Coefficient .85
R2 - Variance .72
CPAB - Reasoning Regression Coefficient .09
- Letter Series Regression Coefficient • 06

- Number Ability Regression Coefficient .04

- Diagramming Regression Coefficient -.04
ACT - Mathematics Regression Coefficient -.08
- Social Studies Regression Coefficient • i0

Intercept i0.3
Value (6,24) 10.2
ACT-Social Studies, Number Ability, Letter Series, and Diagramming tests
entering in that order• The multiple R of .84 is high considering the sample
characteristic previously described.
8
TEST RESULTS - UNDERGRADUATES

The correlations between the parts of the CPAB are not all signifi-
cant. There is more evidence in this sample that the sub-tests are measuring
something different. Three of the six ACT intercorrelations are not signi-
ficant suggesting that these also may be measuring different things. The
correlations between the ACT-Natural Science and the CPAB Diagramming tests
was .79. The correlation between the ACT-Mathematics and the CPAB-Number
ability was .69. Thus, as expected, the ACT and the CPAB sub-tests have
common elements.
The regression results in table 6 found the Reasoning, ACT-Mathematics,

CONCLUSIONS

The results described above give strong indication that quantitative,

reasoning and diagramming abilities were related to the performance of
students in the two introductory MIS courses described. Thus if there is
a need for predicting success in such courses either in academia or in an
intra-company training course there is an excellent possibility that a
validation study will find an appropriate set of tests. Also tests of
these types may be helpful in predicting performance in MIS occupations.

REFERENCES

(i) Palormo, Jean

Computer Programmer Aptitude Battery
Test Manual, 1967
Chicago; Science Research Associates, Inc.