Automatic Generating Test Paper System

Based On Genetic Algorithm

Lirong Xiong
College of Computer Science and Technology
Zhejiang University of Technology
Hangzhou, Zhejiang, 310023, P.R. China
lilybear@zjut.edu.cn
Jianwei Shi
College of Computer Science and Technology
Zhejiang University of Technology
Hangzhou, Zhejiang, 310023, P.R. China
shijianwei2020@163.com
AbstractTraditional test paper generating algorithms have
some shortcomings, such as slow generating speed, low success
probability and low generating quality. The population search
strategy in Genetic Algorithm (GA) provides a very suitable
solution for multi-objective optimization, so applying it to the
issue of test paper auto-generating can achieve good results. In
this paper, we study the issue of test paper generation and
propose the mathematical model and object function. Then some
improvements are made in Simple GA for test paper generation.
Based on the algorithm, a system is designed and implemented.
Keywords-test paper auto-generating; Genetic Algorithm;
constraints; mathematical model
I. INTRODUCTION
The issue of test paper auto-generating is a multi-objective
parameter optimization problem with specific constraints.
Traditional generating algorithms have some shortcomings,
such as slow generating speed, low success probability and low
generating quality. With the characteristics of simpleness,
strong robustness, global search and unfettered by the
restrictive conditions of search space, GA [1] is being paid
more and more attention to. The population search strategy in
GA provides a very suitable solution for multi-objective
optimization, so applying it to the issue of test paper auto-
generating can achieve good results.
Many researchers have done large amount of study in the
field of test paper automatic generating by using GA. QUAN
Hui-yun, WEN Zhong-lin et al. [2, 3, 4] use Simple Genetic
Algorithm (SGA) to realize the test paper auto -generating,
which achieves good results, but the chromosome code is too
long to calculate fitness and SGA easily leads to premature
convergence phenomenon. LU Yihong et al. [5, 6] adopt
grouping real-coded with shorter code to eliminate time-
consuming of encoding and decoding. They also use adaptive
crossover rate and mutation rate, which enhances the search
ability of GA and speeds up the convergence rate. Yang
Luming and Chen Daxin [7] use improved genetic operations
to keep better parent individuals and abandon worse children
individuals, which make the population more excellent and
adaptable.
This paper analyzes the question structure, presents the
mathematical model and objective function of generating test
paper in Section 2. Then GA is applied to the issue of
generating test paper and some improvements in GA are
proposed in Section 3. In Section 4, we realize an automatic
generating test paper system which acquires a good effect of
test paper generation and the system function module is
presented. Section 5 presents the conclusions and future work.
II. PROBLEMS OF TEST PAPER GENERATING
A. Design of Question Structure
The structure of the question library should reflect the
essence of examination questions comprehensively and
objectively. The test paper should be generated intelligently
and easy to involve genetic operations. With the above concern,
we construct the structure of questions as follows: (1) ID; (2)
Type;(3) Difficulty: divided into five levels: easy, easier,
medium, less hard, hard;(4) Cognitive Level: divided into four
levels here: memorizing, understanding, application, synthesis;
(5)Subject;(6)Knowledge Point;(7)Content;(8)Key;(9)Score;
(10)Answer Time;(11) Exposure: Identifying the number of
times of the question being selected for test paper generation.
B. Mathematical Model
(1)
(
(
(
(
(

=
11 . 3 . 2 . 1 .
11 . 2 3 . 2 2 . 2 1 . 2
11 . 1 3 . 1 2 . 1 1 . 1
n n n n
a a a a
a a a a
a a a a
S
A
A A A A A
A
A
When we select a question, we also fix eleven attributes
described above. It is equivalent to an eleven-dimensional
vector ) , , , , (
11 3 2 1
a a a a A , in which component corresponds
attribute i. Generating a test paper with n questions actually
determines a
i
a
11 n order matrix, which is (1). This is a target
matrix using to solve the issue of test paper generating, whose
input parameters should meet constraints as follows:
2010 Second International Workshop on Education Technology and Computer Science
978-0-7695-3987-4/10 $26.00 2010 IEEE
DOI 10.1109/ETCS.2010.250
272
- Total Score Constraint: , is
the score value of question i.

=
=
n
i
i
a TotalScore
1
9 .
9 . i
a
- Total Time Constraint: , is the
estimated answer time of question i.

=
=
n
i
i
a TotalTime
1
10 .
10 . i
a
- Score Distribution Constraint of Question Type:
m j
TypeName a
TypeName a
q a q Type
j i
j i
i
n
i
i i j
,..., 2 , 1 ,
0
1
2 .
2 .
1
9 .
=

=
=
= =

=


.
- Score Distribution Constraint of Cognitive Level:
. 4 , 3 , 2 , 1 ,
0
1
4 .
4 .
1
9 .
=

=
=
= =

=
j
CLName a
CLName a
r a r CL
j i
j i
i
n
i
i i j


- Score Distribution Constraint of Difficulty:
5 ,..., 2 , 1 ,
0
1
3 .
3 .
1
9 .
=

=
=
= =

=
j
DiffName a
DiffName a
s a s Diff
j i
j i
i
n
i
i i j


.
- Score Distribution Constraint of Knowledge Point:
l j
KPName a
KPName a
t a t KP
j i
j i
i
n
i
i i j
,..., 2 , 1 ,
0
1
6 .
6 .
1
9 .
=

=
=
= =

=


.
C. Objective Function
Actually we can only acquire optimal approximation
after finite iterations as the final target test paper. We can make
absolute values between target generating constraints and
actual test paper parameter values, then assign a certain weight
to each result, get the sum of them as the
standard of evaluating the generating test paper. The formula is:
) 6 , 2 , 1 ( A = i w
i
) 6 ,..., 2 , 1 ( ,
6
1
= =

=
k e w f
k
k k
(2)
k
e is the absolute value of the relative deviation of constraint
k, is the corresponding weight, and .
Formula (2) is the objective function of the issue of test paper
generating.
) , 1 0 ( s s
k k
w w 1
6
1
=

= k
k
w
III. TEST PAPER AUTOMATIC GENERATING ALGORITHM
BASED ON IMPROVED GENETIC ALGORITHM
A. Chromosome Coding
This paper adopts grouping real-encoding method. A test
paper is mapped to a chromosome. And the questions of the
same paper are regarded as genes and the ID of the question as
the gene value. The questions of the same type are encoded
together.
B. Generation of Initial Population
After the user has entered all the score distributions, the
Improved GA (IGA) adopted in this paper calculates the
number of questions of each question type. Then it randomly
selects corresponding amount of questions, uses IDs to form
chromosome, making sure that IDs are all different in the same
question type. In this way, IGA generates initial population in
the conditions of meeting score distribution of question type.
C. Design of Fitness Function
This paper adopts a fitness function as following form:
F=1 / (1 + f) (3)
f is presented in formula (2). The deviation is smaller, and the
fitness value is bigger, which shows this individual get more
closely to the goal one.
D. Calculation of Score Distribution of Difficulty
This paper adopts the binomial distribution function of
discrete random variables to establish a mathematical model to
calculate score distribution of difficulty by using expected
average score. The binomial distribution function B(n,p) [8]:
k n k k
n
q p C k P

= ) ( (4)
Among it, k = 0, 1, 2, , n, and n is a positive integer, p > 0, q
> 0, p + q = 1.Set q = ExpectedAverageScore / TotalScore,
and set n = v + 1 (v is the number of difficulty level, so if the
difficulty has 5 levels then set n = 6). Giving these values to
(4) to calculate P(k) (k = 0, 1, 2, , n). Because q is usually
larger than 0.6, P(n) and P(n-1) are very smaller, they can be
added to P(n-2). We can use obtained P(k)(k = 0, 1, 2, , n-2)
as the score proportion of difficulty level of k + 1, then
multiply them by total score, finally get the score distribution
curve of difficulty.
E. Improvements in Genetic Operators
Crossover operator adopts the strategy of section single-
point crossover. At first the IGA randomly pairs the individuals
having operated by the selection operator, carries on single-
point crossover for each question type section i with a certain
crossover probability.
IGA immediately evaluates the fitness values of two new
generating individuals after crossover, compares them with
their parent individuals. If the fitness value of the child is the
same as fitness value of its parents, corresponding individual is
considered as invalid one, which is eliminated; otherwise it is
considered as valid one to be kept. In both condition, the parent
individuals are all kept [7]. Mutation Operator only operates
individuals kept by valid crossover with a certain mutation
probability.
IGA then evaluates and does selection operation to the
total collection of new individuals after mutation and
individuals of current parent population. That is to select N (N
is the population size) individuals to form a new parent
273
population from individuals, whose amount is bigger than N
and less than 2N [7]. The Selection Operator is using Roulette
Wheel Selection. Assume that the population size is n, and the
fitness value of Individual i is , so the probability of selected
Individual i is /

. It can clearly be seen that the

bigger the fitness value of individual is, the bigger its selection
probability is.
i
F
i s
F P
i
=
=
n
i
i
F
1
F. Controlling Parameters
Choosing bigger crossover probability in initial iteration
can enhance the search ability of GA. Choosing smaller one in
later iteration can avoid the damage to excellent genes, and
speed up the convergence rate [9]. Continuously adjustments of
the mutation probability with the evolution can avert
Inbreeding phenomenon. So this paper adopts the adaptive
crossover probability and mutation probability, whose formula
[8] is as follows:

<
>
=
avg
avg
c
f f k
f f f f f k
P
'
2
'
max
'
max 1
,
, / ) (
(5)

<
>
=
avg
avg
m
f f k
f f f f f k
P
,
, / ) (
4
max max 3
(6)
Set 3 . 0 , 9 . 0
4 3 2 1
= = = = k k k k . is the largest
fitness value of the population, is the average fitness
value of the population, is the bigger fitness value of the
two individuals waiting for crossover, is the fitness value of
the mutation individual.
max
f
avg
f
'
f
f
G. Realization of Test Paper Auto-generating Algorithm
Using IGA described in the above sections to realize auto-
generating test paper, we should firstly conform the values of
these parameters: maximum number of iterations of genetic
operations MaxGen, population size PopSize, crossover
probability , mutation probability , then input the user
requirements of generating test paper. The specific process of
this algorithm is as Fig.1.
c
P
m
P
IV. TEST PAPER AUTOMATIC GENERATING SYSTEM
A. System Overview
This System has the structure of Browser/Server, and has
five main functional modules: User Information Management
(UIM), Subject Information Management (SIM), Question
Management (QM), Statistics Query Management (SQM), and
Test Paper Management (TPM).
The functional structure of this system is shown as Fig.2.
UIM includes register, modify, delete of user information. SIM
includes add, modify, delete of information of subject,
knowledge point, major. QIM includes query, add, modify,
delete of questions. SQM includes statistics of question type,
score, and difficulty. TPM includes functions for test paper,
such as generating, editing, statistics, etc. Corresponding
runtime presentation will be shown in section B.
Figure 1. Flowchart of Test Paper Automatic Generating
Figure 2. The Functional Structure of Test Paper Auto-generating System
274
B. Auto-generating Runtime Presentation
Fig.3 shows all the parameters for generating test paper,
mainly including subject name, total value, total time, score
distributions of question type, difficulty, cognitive level and
knowledge point. After submitting the above parameters, the
algorithm executes and generates a test paper in the form
of .doc file. Fig.4 shows the test paper generated by GA using
all the parameters set in Fig.3.
Figure 3. The test paper parameters
Figure 4. A test paper generated according to parameters set in Figure 3
V. CONCLUSION AN FUTURE WORK
This paper analyzes the question structure, presents the
mathematical model and objective function of generating test
paper; applies GA to the issue of generating test paper, realizes
an automatic generating test paper system. The system has
been used for the course examination of Computer Network,
improving the quality of test paper and reducing the burden on
teachers. Practice shows that this system has a good value of
usage.
The process of selecting question in this algorithm is not
able to distinguish similar questions (The contents of questions
are similar or just the same), can not avoid the question
collision (The collision between content and answer, that is
content of question A may be the answer of question B, both
are in the same test paper), so corresponding semantic analysis
module should be designed and added to the system.
Furthermore, the question should be assigned several values of
attributes while entering into the database. The work needs
manual process, which has subjective, inefficient issues. A
mathematical model should be constructed to scientifically and
efficiently determine all the attribute values of the question,
realizing the input of question batch and efficiency.
ACKNOWLEDGEMENT
The paper is supported by Excellent Course Construction
Project (119000315), Computer Network Quality Course
Project (124000715) and Graduate Education Reform Project
(124000117) of Zhejiang University of Technology.
REFERENCES
[1] J.H. Holland. Adaptation in nature and artificial systems[M]. Michigan:
The University of Michigan Press, 1975:10-50.
[2] QUAN Hui-yun, FAN Guo-chuang, ZHAO Ting-lei. Intellegent system
of test paper autogenerating from test bank based on Genetic
Algorithms[J]. Journal of Wuhan University: Natural Science Edition,
1999, 45(5): 758-760.In Chinese
[3] WEN Zhong-lin, CAI Qing-wan, LI Yuan-xiang. Intellegent system of
test paper autogenerating from test pool based on Genetic Algorithms[J].
Journal of Hubei Institute for Nationalities: Natural Science Edition,
2000, 18(3): 53-55. In Chinese
[4] WEI Ping, ZHANG Yuan. Test paper generation with Genetic
Algorithm[J]. Journal of Ningbo University: NSEE, 2002, 15(2): 47-50.
In Chinese
[5] LU Yihong, LIU Hong. Auto-generating examination papers based on
integer coding and adaptive Genetic Algorithm[J]. Computer
Engineering, 2005, 31(23): 232-233. (In Chinese)
[6] ZHANG Ya-jing, YANG Yi, QIU Jing, BAI Chun-yu. Application of
general item bank in auto-generating paper based on GA[J]. Journal of
Yunnan Agricultural University, 2005, 20(5): 714-719. (In Chinese)
[7] Yang Luming, Chen Daxin. Application of an improved Genetic
Algorithm in composing a test paper[J]. Computer and Digital
Engineering, 2004, 35(5): 76-79. (In Chinese)
[8] ZHANG Yue-qin, LI Hui-shu, REN Bo. Maneuver of composing
examination papers from questions in view of Genetic Algorithms[J].
Journal of Inner Mongolia Agricultural University, 2006, 27(3): 102-105.
In Chinese
[9] Yuan Huimei. Adaptive crossover rate and mutation rate in Genetic
Algorithms[J]. Journal of Capital Normal University: Natural Science
Edition, 2000, 21(3): 14-20, 20(1): 60-61. In Chinese
275