JFD: Automatic Java Fuzz Driver Generation
Zhuo Chen
College of Computer
National University of Defense Technology
Changsha Hunan China
chenzhuo9797@126.com
ABSTRACT
Java is widely used in many areas. There have been a lot of works
on improving the security of Java. In the industry, fuzzing is the
most efficient software testing technique to discover real-world
vulnerabilities and improve software security. Recent efforts are
seen to make library fuzzing more automatically and have
performed well in general library fuzzing. However, these tools
cannot solve problems in Java library fuzzing appropriately. In this
paper, we present JFD, an automatic Java fuzz driver generation
system, which can generate fuzz drivers based on consumer
programs, that utilize target library programs. Our approach
consists of three parts: a static-analysis-based method to analyze
call dependencies graphs of target Java library, a value-set-based
method to analyze argument dependencies graphs of target Java
library, and a method to synthesize fuzz drivers in the style of JQF.
We then evaluate JFD on JVM native libraries. JFD can generate
appropriate fuzz drivers.
CCS CONCEPTS
• Software and its engineering → Software creation and
management → Software verification and validation → Software
defect analysis→Software testing and debugging • Security and
privacy→Software and application security→Software security
engineering
KEYWORDS
Java, fuzzing, fuzz driver generation, JQF, software security
1 Introduction
Java is one of the most popular programming languages. It is
widely used in desktop applications, Web applications, distributed
systems, and embedded system applications due to its "write once,
run many places" characteristic. In the TIOBE Index, which tracks
trends in the popularity of programming languages, Java held the
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. Copyrights for
components of this work owned by others than ACM must be honored.
Abstracting with credit is permitted. To copy otherwise, or republish, to
post on servers or to redistribute to lists, requires prior specific permission
and/or a fee. Request permissions from Permissions@acm.org.
EITCE 2021, October 22–24, 2021, Xiamen, China
© 2021 Association for Computing Machinery.
ACM ISBN 978-1-4503-8432-2/21/10…$15.00
https://doi.org/10.1145/3501409.3501564
862
Yongjun Wang†
College of Computer
National University of Defense Technology
Changsha Hunan China
†
Corresponding author wangyongjun@nudt.edu.cn
top spot from April 2015 to April 2020, only to be overtaken by C
in May 2020 to fall to second place.
Fuzzing is one of the most popular software testing techniques,
which can find various weaknesses in the program by generating a
large number of test inputs. Fuzzing was first proposed by Miller
et al.[4], the University of Wisconsin, the USA in 1988, after which
it developed into an effective, fast and practical method for finding
software vulnerabilities.
At present, most studies are mainly focused on general
programs that can be executed directly, but a large number of
library programs are often ignored because they cannot be
executed directly. Compared with general programs, fuzzy testing
of library programs has more problems. Libraries cannot be run as
stand-alone programs, instead, libraries need to be invoked through
another application. Triggering vulnerabilities deep in the library
remains challenging because a specific sequence of API calls is
required to build the necessary state.
The common method to solve the fuzzing of library programs
is to write a fuzz driver manually which can be executed directly
by analyzing the API call dependency of the library under test, and
then use various fuzzer to test. However, due to the diversity of
libraries, this method requires testers to spend a lot of energy and
has a low level of automation and intelligence.
There has been some work into automated fuzz driver
generation. JDriver[1] is an automatic driver class generation
framework for AFL-based fuzzing tools. JDriver can generate
method sequences to mutate the status of the class instances.
However, its analysis of the dependency is simple, resulting in the
generated fuzzing driver classes are not efficient. FUDGE[2] is
Google's recently introduced semi-automatic fuzz driver
generation framework. FUDGE constructs the fuzz driver by
scanning the program's source code for vulnerable function calls
and generates the fuzzy driver by parameter substitution.
FuzzGen[3] is another tool for the automatic generation of fuzz
drivers. By analyzing existing programs of calling libraries,
FuzzGen obtains the dependencies between APIs of the target
library and then generates fuzz drivers according to the
dependencies. FuzzGen can generate fuzz drivers based on
existing programs, but only supports C language, not Java
language.
We design and implement JFD, an automatic Java fuzz driver
generation system. It analyzes dependencies among APIs based on
the analysis of existing programs using the target library. Based on
the dependencies among APIs, JFD generates Java fuzz drivers for
JQF, which is widely used in fuzzing Java programs. Overall, we
make the following contributions:
(1) Based on existing programs that use the target library,
analyzing dependencies among APIs, including calling
dependencies and argument dependencies.
(2) Based on the dependencies, generating Java fuzz drivers
that can efficiently expose vulnerabilities in the target
library for JQF, the most popular Java fuzzer,
automatically.
The remaining paper is organized as below: Section 2
introduces related works. Section 3 describes our approach,
Section 4 depicts the implementation, and Section 5 shows our
evaluation results. We illustrate our thoughts on future work in
Section 6, and conclusions are given in Section 7.
2 Related Work
2.1 Fuzzing
According to the different methods of generating test cases,
fuzzing technology can be divided into fuzzing technology based
on generation and fuzzing technology based on mutation. Fuzzing
technology based on generation generally produces test cases
under the guidance of prior knowledge according to defined rules.
The prior knowledge determines the quality of the generated test
cases. A good generation model can produce more effective test
cases, so as to cover more of the input space. However, the
dependence on prior knowledge limits the application scope of this
kind of fuzzy testing tool. Mutation-based fuzzing is to modify
existing test cases to generate new test cases. This approach is
relatively less dependent on prior knowledge. Due to the locality
of program input, the test cases generated by such tools cover
fewer paths.
According to the understanding degree of the program under
test, fuzzing technology can be divided into black-box fuzzing
technology, white-box fuzzing technology, grey-box fuzzing
technology. The information includes code coverage, data flow
coverage, the program's memory usage, CPU utilization, or any
other information that can guide test case generation.
2.1.1 Black-box Fuzzing. Black-box fuzzing is also known as
black-box random testing. Black-box fuzzing techniques do not
require any information from the target program or input format,
but instead create input by random mutations (such as bit-flipping
byte copy, and byte deletion, etc.) of the seed file of a given format
through some predefined rules. Recently the black-box fuzzing
also uses grammar or enters a specific knowledge to generate half-
black-box fuzzing effective input test because of its effectiveness
in discovering software vulnerabilities and use of simple, very
popular in the software industry. However, the disadvantage of the
black-box fuzzing technique is also obvious. It is difficult to
generate test cases covering a large number of execution paths in
target programs. Because of this blindness, black-box fuzzing
often has low code coverage in practice.
2.1.2 White-box Fuzzing. Based on the internal logic knowledge
of the target program, the white box fuzzing uses a method that can
863
theoretically explore all the execution paths of the target program,
and realizes the comprehensive and fast search of the target
program through dynamic symbol execution and coverage
maximization heuristic search algorithm. Unlike black-box
fuzzing technology, white-box fuzzing technology requires
information from the target program and uses the information
needed to guide the generation of test cases. Specifically, using a
given specific input to start the executable, white-box fuzzing tools
under the input along the path of execution of all the conditional
statements to collect symbol constraints. So, after one execution,
the white-box fuzzing tool combines all the symbolic constraints
together to form a path constraint and then the white-box fuzzing
tool systematically negates one of the constraints and solves the
new path constraint and the new test case directs the program to
run a different execution path. Using a heuristic search algorithm,
the white-box fuzzing tool can find errors as quickly as possible in
the target program theoretically, white-box fuzzing can generate
test cases that cover all program paths. However, in practical, due
to the numerous execution paths in the actual software system and
the low precision of solving constraints in the symbolic execution
process, the code coverage of white-box fuzzing cannot reach
100%. One of the most famous white-box fuzzing tools is
SAGE[5]. SAGE targets large Windows applications and uses
some optimizations to handle a large number of execution paths.
2.1.3 Grey-box Fuzzing. Grey-box fuzzing technology is between
black-box fuzzing technology and white-box fuzzing technology.
The common method of grey-box fuzzing is code implement.
Through this method, the grey-box fuzzing tool can obtain the code
coverage of the target program at run time, and then use this
information to adjust its mutation strategy, to create a more
execution path, or to find vulnerabilities in faster test cases.
Another method of grey-box fuzzing is taint analysis, which can
trace taint data stream by code implement.
AFL[6] is a representative of grey-box fuzzing tools. It aims at
improving coverage and generating test cases based on mutation.
AFL uses branches as the target of coverage. AFL determines
whether the input execution covers the new branch according to
the items in the branch table. AFL defines a lot of modification
operations, including random modification and modification with
specific values. The mutation strategy based on the genetic
algorithm adopted by AFL makes it faster, more stable, and has
higher coverage than the previous fuzzing tools. In practice, AFL
found a number of real vulnerabilities.
2.1.4 Comparison. Generally speaking, the traditional black-box
fuzzing only uses random mutation methods to generate test cases,
which cannot achieve high code coverage. It usually can only find
shallow program vulnerabilities. its advantages are lightweight,
fast, and easy to use. In contrast, white-box fuzzing or grey-box
fuzzing can achieve higher code coverage and thus discover more
hidden program vulnerabilities, but they require more time and
space and are more complex to build.
Grey-box fuzzing is similar to white-box fuzzing in that both
methods use information about the target program to guide the
generation of test cases. But there is an obvious difference: the
grey-box fuzzing only uses some runtime information about the
target program (such as code coverage, tainted data flow, etc.) to
determine the state of the explored path. Because grey-box fuzzing
uses only partial information to guide test case generation, it does
not guarantee that using this information will generate better test
cases to override new paths or trigger specific program
vulnerabilities. In contrast, white-box fuzzing systematically
explores all execution paths using the source code or binary code
of the target program. By using symbolic execution and constraint
solvers, white-box fuzzing ensures that the resulting test cases will
lead the target program to explore new execution paths. Therefore,
white-box fuzzing can help to reduce blindness in fuzzing more
thoroughly. However, compared to grey-box fuzzing, white-box
fuzzing is not very practical in the industry because it is very
expensive in terms of time and resource consumption and faces
many challenges (such as path explosion, memory modeling, and
constraint solving, etc.). To sum up, both methods use the
information of the target program to reduce the blindness of black-
box fuzzing, but to different degrees.
2.2 Java Fuzzing
JFuzz[7] is a mutation method combined with a disintegration test
proposed by Zhu et al. Disintegration test is a test based on
disintegration relationships. JFuzz defines a variety of heuristics
mutation operations. However, JFuzz needs to be defined by users,
and it is difficult for users who are not familiar with the program
under test to find appropriate metamorphosis relationships, which
leads to JFuzz has not been widely used
Kelinci[8] is an agent between AFL and Java programs
implemented by Kersten et al. Kelinci transplanted the implement
module of AFL and implement the code used for statistical branch
coverage into the program under test. After the fuzzing starts,
Kelinci receives test cases with AFL and feeds back branch
coverage and execution results to AFL. In this way, Kelinci
realized fuzzing for Java programs. Because AFL’s test case is
stored in the file, it is relatively easy to test for methods that
process files, but for methods that do not process files, testers need
to read the contents of the file in an appropriate way and construct
variables of the appropriate type for the function under test.
JQF[9] is a coverage-guided Java fuzzing platform
implemented by Rohan Padhye et al. JQF contains AFL, Zest[10],
PerfFuzz[11], and other test backends, while also allowing for
expansion. JQF provides a good foundation for Java fuzzing by
enabling practitioners to use familiar attribute-based testing styles,
using coverage-guided fuzzing for test programs that require
structured input, and enabling researchers to implement new
coverage-guided fuzzing algorithms to drive attribute-based
testing.
2.3 Automatic Fuzz Driver Generation
JDriver is an automatic driver class generation framework for
AFL-based fuzzing tools. It can build driver class for method
whether processing files or not. It analyzes the call/modification
relationship between each field and method to get
call/modification graphs. Then it generates fuzzing driver classes
according to the graphs. JDriver can generate method sequences to
864
mutate the status of the class instances. However, its analysis of
the dependency is simple, resulting in the generated fuzzing driver
classes are not efficient.
FUDGE is Google's recently introduced semi-automatic fuzz
driver generation framework. FUDGE constructs fuzz drivers by
scanning the program's source code for vulnerable function calls
and generates fuzz drivers by parameter substitution. FUDGE
works well in specific scenarios, but it tends to produce too many
fuzz drivers for large projects where invalid results need to be
manually removed.
FuzzGen is another tool for the automatic generation of fuzz
drivers. By analyzing existing programs using target libraries,
FuzzGen obtains the dependencies between APIs of the target
library and then generates fuzz drivers according to the
dependencies. FuzzGen can generate fuzz drivers based on
existing programs, but only supports C language, not Java
language.
3 Approach
3.1 Overview
Figure 1 shows the high-level architecture of the JFD system. First,
JFD extracts all the APIs from library programs. Then, the API call
dependency of the target library is obtained through static analysis
of library consumer programs, which use the target library, and the
parameter dependency in the call relationship is obtained through
value set analysis, so as to obtain the API dependency graph. Then
JFD synthesizes the fuzz driver for the target library from the API
dependence graph.
Figure 1: High-level overview
3.2 API Dependencies Analysis
The key algorithm of the API dependencies analysis is shown in
Algorithm 1. Firstly, the API list is extracted by analyzing target
library programs. Then, JFD analyzes consumer programs utilizing
the target library by static analysis to get the API call dependencies
of the target library. Finally, the value-set analysis of the target
library program is continued, and the dependencies between the
arguments of APIs in the API dependencies graph is analyzed, and
the API dependencies graph is supplemented. For example, the
result of the previous API call is the first parameter of the next API
call, or a parameter is fixed.

Algorithm 1 Dependence analysis
Input: libProg (Library program)
consumerList (consumer program)
Output: apiDepGraph (API Dependence Graph)
1 apiList ← newArrayList()
2 for method in libProg do
3 apiList.add(method)
4 end for
5 apiDepGraph ← newGraph()
6 for consumerProg in consumerList do
7 valueSet ← newArrayList()
8 for stmt in consumerProg do
9 valueSet.update()
10 if stmt ∈ apiList do
11 apiDepGraph.add(stmt)
12 end if
13 apiDepGraph.argDep(valueSet)
14 end for
3.2.1 Call dependencies analysis. An appropriate sequence of API
calls is the basis for generating fuzz drivers. If the sequence of API
calls is not set properly, the fuzzer will consume a lot of energy in
shallow error checking, reducing the efficiency of the generated
fuzz driver.
For each consumer, JFD iterates through its control flow
diagram through static analysis, extracting all APIs of the target
library, resulting in call dependencies graphs. Because the target
library APIs may be scattered across branches of the control flow
graph, it may end up with many call dependencies graphs.
Therefore, the merge of dependencies graphs is important.
To merge the call dependencies graphs, JFD iterates through
each one. Two call dependencies graphs can be merged if and only
if they have at least one node in common. The same node means:
first, the API name is the same; Second, the API takes the same
arguments.
3.2.2 Argument dependencies analysis. Argument dependence is
as important as call dependencies. Fuzz drivers must not only
provide the correct sequence of API but also ensure the accuracy
of parameter passing between APIs. Otherwise, the fuzzer will also
consume a lot of energy in shallow error checking.
In the dependency analysis module, the dependency between
arguments is obtained by value-set analysis. By traversing
sequential statements, different attributes of each parameter can be
obtained based on data flow analysis. The set of possible attributes
is shown in Table 1. For example, the value of a parameter that is
never modified is set to invariant.
865
Table 1: Argument attributes from value-set analysis
Attribute Description
dead Not used
invariant Not modified
predefined A value in a set
random A random value
output As output
dependent Dependent on another argument
3.3 Fuzz Driver Synthesis
The fuzz driver synthesis is based on the output of API
dependencies analysis, API dependencies graph. Based on the API
dependencies graph, JFD sets the API call sequence of the API. In
addition, JFD set arguments according to their different types: for
arguments that can be determined in the API dependencies graph,
arguments are set according to the analysis results; For arguments
that cannot be determined by static analysis, the arguments are set
as target arguments of fuzzing, which will be tested by fuzzer later.
4 Implementation
Our automatic fuzz driver generation approach has been
implemented into JFD with around 10,000 lines of Java code. It
uses the Java optimization framework Soot[12][13][14], which can
support static analysis of Java well, to perform API extraction and
API dependencies analysis. JFD takes library programs and library
consumer programs as input, and produces fuzz driver in the
following steps:
(1) extracting APIs. JFD starts with iterating all of the target
library programs to extract all of the APIs for the
following steps.
(2) analyzing API call dependencies. JFD iterates each
consumer analyzing call dependencies according to the
algorithm described in Section 3.2 and 3.2.1.
(3) analyzing API argument dependencies. JFD iterates each
consumer analyzing argument dependencies according to
the algorithm described in Section 3.2.2.
(4) synthesizing fuzz drivers. Finally, JFD synthesizes fuzz
drivers based on API dependencies graphs in the style of
JQF fuzzer.
5 Evaluation
We first demonstrate our fuzz driver generation with a simple
example in Section 5.1. Then, we work on the JVM native library
javax.xml to evaluate the fuzz driver in Section 5.2.
5.1 Simple Example
We use a simple example to illustrate how our approach works.
Figure 2 shows the source code of the library. The class Foo has
three member methods: Foo (Implicit constructor), foo_init (lines
4-9), and foo_add (lines 11-14). Figure 3 shows the source code of
the library consumer. The class Consumer invokes APIs in the
library Foo in the sequence of Foo (line 8), foo_init (line 9), and
twice foo_add (line 10, 11). Besides, the arguments of the two calls
to foo_add are related. Figure 4 shows the source code of the fuzz
driver generated. It calls APIs according to the consumer's call
sequence and sets the arguments appropriately.

Figure 4: The example fuzz driver

5.2 Evaluating Fuzz Driver Generation

Figure 2: The example library
Figure 5 shows the source code of a fuzz driver generated by JFD
for javax.xml. The fuzz driver initializes, then reads and parses the
XML string from the file, and finally gets the information in the
XML.

Figure 3: The example library consumer

Figure 5: Sample fuzz driver generated by JFD

866

6 Discussion and Future Work
JFD can generate Java fuzz drivers automatically and appropriately.
It still has some opportunities for improvement.
For now, JFD only supports a single library. There are many
scenes that APIs from different libraries interact in Java programs.
Supporting multiple libraries may means the challenge of complex
dependencies analysis. In the future, we will extend JFD to support
multiple library dependencies analysis.
[17] Jun Li, Bodong Zhao and Chao Zhang. “Fuzzing: a survey” Spring
Cybersecurity. 2018
[18] James Fell. “A Review of Fuzzing Tools and Methods”. 2017
[19] Marcel B., Cristian C, et al. “Fuzzing: Challenges and Reflections”. IEEE
Computer Society. 2020
[20] Valentin J. et al. “The Art, Science, and Engineering of Fuzzing: A Survey”
IEEE Transactions on Software Engineering. 2019
[21] Hongliang, Liang, Xiaoxiao, et al. “Fuzzing: State of the Art”. IEEE
Transactions on Reliability, 2018
[22] Lindholm T, Yellin F, Bracha G, et al. The Java® Virtual Machine Specification
[M/OL]. 2014. http://docs.oracle.com/javase/specs/jvms/se8/jvms8.pdf.

7 Conclusions
Research on automatic Java fuzz driver generation technique can
improve the efficiency of Java library fuzzing. In this paper, we
propose a method to generate Java fuzz drivers automatically. Our
approach employs static analysis to analyze APIs call
dependencies graphs and then employs value-set analysis to
analyze APIs argument dependencies graphs. We analyze the API
dependence graph in order to allow fuzz drivers generated to cover
deep paths of library programs. We design a method to synthesize
Java fuzz drivers for JQF. We implement our approach in JFD. We
evaluate JFD on real-world JVM native libraries and prove that
JFD can generate Java fuzz drivers in the style of JQF
appropriately based on existing programs using the target library.

ACKNOWLEDGMENTS
We’d like to express our appreciation to anonymous editors.

REFERENCES
[1] Huang, Z. , & Wang, Y. . (2018). Jdriver: automatic driver class generation for
afl-based java fuzzing tools. Symmetry, 10(10).
[2] Babi D. , Bucur S. , et al. “FUDGE: Fuzz driver generation at scale”. ACM
SIGSOFT Symposium on the Foundation of Software Engineering/ European
Software Engineering Conference, 2019
[3] Ispoglou Kyriakos K, Austin Daniel, Mohan Vishwath, Payer Mathias.
“FuzzGen: Automatic Fuzzer Generation”. USENIX Security. 2020
[4] Miller, B.P.; Fredriksen, L.; So, B. An Empirical Study of the Reliability of
UNIX Utilities. Commun. ACM 1990, 33, 32–44.
[5] P. Godefroid, M. Y. Levin, and D. Molnar, SAGE: Whitebox fuzzing for
security testing[J]. 2012, Queue, vol. 10, no. 1, pp. 20:20–20:27.
[6] M. Zalewski, “american fuzzy lop,” 2017. [Online]. Available:
http://lcamtuf.coredump.cx/afl/
[7] Hong Z . “JFuzz: A Tool for Automated Java Unit Testing Based on Data
Mutation and Metamorphic Testing Methods”. 2015
[8] Kersten Rody, Field Moffett, et al. “POSTER : AFL-based Fuzzing for Java with
Kelinci”. ACM Conference on Computer and Communications Security. 2017
[9] Lemieux, Caroline. “JQF : Coverage-Guided Property-Based Testing in Java”.
International Symposium on Software Testing and Analysis. 2019
[10] Padhye, Rohan Lemieux, et al. “Semantic fuzzing with ZEST”. International
Symposium on Software Testing and Analysis. 2019
[11] Lemieux Caroline, Padhye Rohan, et al. “PerfFuzz: Automatically generating
pathological inputs”. ISSTA 2018 - Proceedings of the 27th ACM SIGSOFT
International Symposium on Software Testing and Analysis. 2018
[12] Vallee-Rai R , Hendren L, Sundaresan V, et al. “Soot—a Java bytecode
optimization framework”. Proceedings of Cascon. 1999.
[13] R Vallée-Rai, Lam P , Verbrugge C , et al. “Soot (poster session): a Java
bytecode optimization and annotation framework”. Addendum to the
Conference on Object-oriented Programming, 2000.
[14] Lam P, Bodden E, Lhoták O, et al. The Soot framework for Java program
analysis: a retrospective [C]. In Cetus Users and Compiler Infastructure
Workshop (CETUS 2011). 2011: 35.
[15] Pengfei Wang, Xu Zhou, et al. “SoK: The Progress, Challenges, and
Perspectives of Directed Greybox Fuzzing”. Arxiv. 2020
[16] “Fuzzing: Hack, Art, and Science”. CACM. 2020

867