GDB Debug Native Part of Java Application (C - C++ Libraries and JDK) by Alexey Pirogov Medium

GDB: Debug native part of java application (C/C++ libraries and JDK) |... https://medium.com/@pirogov.alexey/gdb-debug-native-part-of-java-ap...
Open in app Sign up Sign In
Acceder con Google
Usa tu Cuenta de Google

para acceder a Medium
GDB: Debug native part of java application
No tendrás que recordar más
(C/C++ libraries and JDK)
contraseñas. Accede de forma
rápida, sencilla y segura.
Alexey Pirogov · Follow

Continuar
7 min read · Dec 10, 2018
Listen Share
Why debugging of C/C++ code may be required for Java developers?
1 of 20 27-10-23 11:36
I worked with a few java projects that used native libraries created by another team
from the same organization. Usually, we invoked C++ code from java.
The issue with C++ code invoked from Java is that it is usually not visible from java.
We only see a top-level interface with JNI/JNA but don’t know what is going under
the hood.
As a result, we can’t get a lot of information from java debugger and profiler that we
use daily.
In this post, I’ll describe a gdb debugger that is able to work with the native code. As
an example, we will build a C++ library for Linux (.so-file), invoke from Java and
debug it.
As low-level part of JDK is also written in C++, we will take a look at how to debug
JDK native code also.
GDB
GDB or GNU Debugger is a command-line debugger that comes with most Linux
distributions and supports lots of processors. GDB supports both remote and local
mode.
It is important to note that as of now GDB doesn’t support debugging of Java code
(on https://www.gnu.org/software/gdb/ you won’t find Java in the list of supported
languages). If you want to debug java code from command-line you can try JDB. It
looks similar to GDB but has less functionality.
Having said that GDB doesn’t allow you to debug java-code, it can perfectly debug a
native part (written not in java) of Java application. If it is important for you to debug
both Java and native code “from the same IDE”, you can try to use Netbeans or
Eclipse. Netbeans uses default debugger for Java code and GDB for native code.
However, for IDE users switch between Java and the native code won’t be visible.
Of course, command-line debugging may look weird at first for people familiar with
IDEs. But it has some benefits. One of them is the ability to run on a remote host.
Although remote debug is built-in in java, debugging an application running in
2 of 20 27-10-23 11:36
another part of the globe using IDE remote debuggers could be very sloooooow. GDB
from the other hand runs on the target host. This is especially useful when you need
to evaluate some code during debugging.
Prepare native code

All code is available in my github repo.
To be able to see a native code in the debugger, the code should be compiled in a
special way. Information about method names and variables should be included in
the library or comes as a separate package.
Often the easiest way to include debug info is to add it to the resulted library. Let’s
write a simple java application and a C++ library.
We define an interface of native methods and load the library. The code won’t run,
because the library doesn’t exist right now. We will build it soon.
Method’s names are self-describing: nativePrint — prints constant string to stdout,

nativeSleep — sleeps for ms milliseconds, nativeAllocate — allocates memory for an
array of size n, nativeCrash — crashes application (we will simulate a crash at the end
of the article to check what information is available for investigation).
3 of 20 27-10-23 11:36
1 package jnidemo;
2
3 public class JNIDemoJava {
4
5 static {
6 System.load("/home/alex/src/JNIExperiments/JNIDemo/dist/libJNIDemo.so");
7 }
8
9 private static final int REPETITIONS = 100_000_000;
10
11 private native int nativeCrash();
12 private native int nativePrint();
13 private native int nativeSleep(int ms);
14 private native Double[] nativeAllocate(int n);
15
16 public static void main(String[] args) {
17 JNIDemoJava nativeCall = new JNIDemoJava();
18
19 for (int i = 0; i < REPETITIONS; i++) {
20 nativeCall.nativePrint();
21 nativeCall.nativeSleep(1000);
22 Double[] dArr = nativeCall.nativeAllocate(100);
23 for (Double d : dArr) {
24 System.out.println(System.currentTimeMillis());
25 }
26 }
27 }
28 }
JNIDemoJava.java hosted with ❤ by GitHub view raw
Let’s generate a c++ header file (interface) for methods defined in

JNIDemoJava.java.
# generate c++ header file and put to cpp folder

/usr/lib/jvm/jdk-11.0.1/bin/javac java/jnidemo/JNIDemoJava.java -h
cpp/
4 of 20 27-10-23 11:36
1 /* DO NOT EDIT THIS FILE - it is machine generated */

2 #include <jni.h>
3 /* Header for class jnidemo_JNIDemoJava */
4
5 #ifndef _Included_jnidemo_JNIDemoJava
6 #define _Included_jnidemo_JNIDemoJava
7 #ifdef __cplusplus
8 extern "C" {
9 #endif
10 #undef jnidemo_JNIDemoJava_REPETITIONS
11 #define jnidemo_JNIDemoJava_REPETITIONS 100000000L
12 /*
13 * Class: jnidemo_JNIDemoJava
14 * Method: nativeCrash
15 * Signature: ()I
16 */
17 JNIEXPORT jint JNICALL Java_jnidemo_JNIDemoJava_nativeCrash
18 (JNIEnv *, jobject);
19
20 ...
21
22 /*
23 * Class: jnidemo_JNIDemoJava
24 * Method: nativeAllocate
25 * Signature: (I)[Ljava/lang/Double;
26 */
27 JNIEXPORT jobjectArray JNICALL Java_jnidemo_JNIDemoJava_nativeAllocate
28 (JNIEnv *, jobject, jint);
29
30 #ifdef __cplusplus
31 }
32 #endif
33 #endif
c++ header hosted with ❤ by GitHub view raw
After we got a definition of C++ methods we can implement them:
5 of 20 27-10-23 11:36
1 #include <jni.h>
2 #include <stdio.h>
3 #include <string.h>
4 #include <unistd.h>
5 #include "jnidemo_JNIDemoJava.h"
6
7 JNIEXPORT jint JNICALL Java_jnidemo_JNIDemoJava_nativeCrash(JNIEnv *env, jobject obj) {
8 const char *s = NULL;
9 printf( "%c\n", s[0] );
10 return 0;
11 }
12
13
14 JNIEXPORT jint JNICALL Java_jnidemo_JNIDemoJava_nativePrint(JNIEnv *env, jobject obj) {
15 printf("\nHello World from C!\n");
16 return 0;
17 }
18
19 JNIEXPORT jint JNICALL Java_jnidemo_JNIDemoJava_nativeSleep
20 (JNIEnv *env, jobject obj, jint ms) {
21 usleep(ms);
22 return 0;
23 }
24
25 JNIEXPORT jobjectArray internalNativeAllocate(JNIEnv const *env, jint objNumber) {
26 jclass classDouble = (*env)->FindClass(env, "java/lang/Double");
27 jobjectArray outJNIArray = (*env)->NewObjectArray(env, objNumber, classDouble, NULL
28
29 for (int i = 0; i < objNumber; i++) {
30 jmethodID midDoubleInit = (*env)->GetMethodID(env, classDouble, "<init>", "(D)V"
31 if (NULL == midDoubleInit) return NULL;
32 jobject iDouble = (*env)->NewObject(env, classDouble, midDoubleInit, (double) i
33 (*env)->SetObjectArrayElement(env, outJNIArray, i, iDouble);
34 }
35
36 return outJNIArray;
37 }
38
39 jobjectArray JNICALL Java_jnidemo_JNIDemoJava_nativeAllocate
40 (JNIEnv *env, jobject obj, jint objNumber) {
41 return internalNativeAllocate(env, objNumber);
42 }
6 of 20 27-10-23 11:36
JNIDemo.c hosted with ❤ by GitHub view raw
Our C++ implementation is ready and we can create a shared library:
run “./scripts/buildLib.sh”
1 #!/usr/bin/env bash
2
3 rm -r build/ dist/
4 mkdir -p build dist
5 gcc -shared -c -g -I/usr/lib/jvm/jdk-11.0.1/include \
6 -I/usr/lib/jvm/jdk-11.0.1/include/linux \
7 -fPIC -MMD -MP -MF "build/JNIDemo.o.d" -o build/JNIDemo.o src/cpp/JNIDemo.c
8 gcc -shared -o dist/libJNIDemo.so build/JNIDemo.o -fPIC
buildLib.sh hosted with ❤ by GitHub view raw
One important option is -g. It tells GCC compiler to include debug information in the
library. It increases the size of the library and allows us to see source code during
debugging.
GDB debug
Let’s try to debug our code. You can start gdb and java app together or you can attach
gdb to the running app. Let’s check 2nd option as starting gdb together with the java
app may require modification of startup scripts (modification is quite simple).
Note: I had an issue with default ptrace settings on Ubuntu described here. I fixed it
with next command echo 0 > /proc/sys/kernel/yama/ptrace_scope.
Let’s start debug:
# start java app
# find application PID

jps
# start gdb with application PID

gdb -p 1234
# gdb will pause our application
7 of 20 27-10-23 11:36
# add a breakpoint to our code (all java packages has Java prefix)
(gdb) break Java_jnidemo_JNIDemoJava_nativeAllocate
Breakpoint 1 at 0x7f5e77dfe944: file src/cpp/JNIDemo.c, line 35.
# resume application and wait when it will stop at breakpoint

(gdb) cont
Continuing.
[Switching to Thread 0x7f5ee5ef5700 (LWP 4052)]
Thread 2 "java" hit Breakpoint 1,

Java_jnidemo_JNIDemoJava_nativeAllocate (
env=0x7f5edc013340, obj=0x7f5ee5ef4980, objNumber=100)
at src/cpp/JNIDemo.c:35
35 return internalNativeAllocate(env, objNumber);
# step into our "internalNativeAllocate" function using "s" command

(gdb) s
internalNativeAllocate (env=0x7f5edc013340, objNumber=100)
at src/cpp/JNIDemo.c:20
20 jclass classDouble = (*env)->FindClass(env, "java/lang
/Double");
# after stop at breakpoint we can do different operations (check

# stacktrace, variables, registers, threads, etc.). Let's print
# value of objNumber parameter
(gdb) print objNumber
$6 = 100
# we can try to debug jdk code. In this case we don't have debug
# symbols and won't get a lot of information (but info about
# threads, registers, stacktrace is still available
(gdb) set step-mode on
(gdb) set step-mode onQuit
(gdb) s
24 jmethodID midDoubleInit = (*env)->GetMethodID(env,
classDouble, "<init>", "(D)V");
(gdb) s
0x00007f5ee44fc320 in jni_GetMethodID ()
from /usr/lib/jvm/jdk-11.0.1/lib/server/libjvm.so
#exit
quit
I won’t describe all gdb commands here as list is quite big.
Debug JDK
We just got some experience in debugging native libraries. Our next step is to debug
the native part of JDK (a big part of jdk is written in C++).
8 of 20 27-10-23 11:36
As an example we will debug writeBytes(…) native method that is invoked as a part of

a well-known System.out.println(…) call:
#FileOutputStream.java
...
private native void writeBytes(byte b[], int off, int len, boolean
append)
throws IOException;
...
First, we need to download and compile JDK with extra parameters to preserve
debug symbols.
#clone jdk from mercurial

hg clone http://hg.openjdk.java.net/jdk/jdk
#configure and make build

bash ./configure --with-target-bits=64 --with-debug-level=slowdebug
--disable-warnings-as-errors --with-native-debug-symbols=internal
make clean
make all
Let’s start and debug our app with gdb using jdk from build/linux-x86_64-server-
slowdebug folder. Now we can see the source of writeBytes method
(Java_java_io_FileOutputStream_writeBytes). If we debug our app with usual jdk we
won’t see the source. This also allows us to see variables names.
(gdb) list Java_java_io_FileOutputStream_writeBytes

64 writeSingle(env, this, byte, append, fos_fd);
65 }
66
67 JNIEXPORT void JNICALL
68 Java_java_io_FileOutputStream_writeBytes(JNIEnv *env,
69 jobject this, jbyteArray bytes, jint off, jint len, jboolean
9 of 20 27-10-23 11:36
append) {
70 writeBytes(env, this, bytes, off, len, append, fos_fd);
71 }
72
(gdb)
Automatically start GDB when an application crashes
When java app crashes somewhere in the native code, Linux produces core-dump.
This file contains full memory snapshot with information about threads and
another useful information. We can analyze this file with different tools and gdb is
one of them.
However, in some production system core dumps may be disabled. One of the
reasons is their size. Let’s says that your app uses 80 GB of RAM (Java heap + objects
allocated by native code) then each crash will create an 80 GB core dump file. If you
have some system/script that checks if your app is alive every 5 minutes and restarts
it if it is dead, you can run out of disk space quite fast.
In this case, it is useful to automatically invoke GDB when an application crashes. It

will be much easier to understand which code caused the issue, check variables
state and core dump file isn’t required. To do this we should just add
-XX:OnError=”gdb — %p" option to our app.
To demonstrate this scenario I created the nativeCrash method in our library (it tries
to access memory that wasn’t properly allocated).
1 ...
2
3 JNIEXPORT jint JNICALL Java_jnidemo_JNIDemoJava_nativeCrash(JNIEnv *env, jobject obj) {
4 const char *s = NULL;
5 printf( "%c\n", s[0] );
6 return 0;
7 }
8
9 ...
JNIDemo_nativeCrash.c hosted with ❤ by GitHub view raw
10 of 20 27-10-23 11:36
1 package jnidemo;
2
3 public class JNIDemoJava {
4
5 static {
6 System.load("/home/alex/src/JNIExperiments/JNIDemo/dist/libJNIDemo.so");
7 }
8
9 private native int nativeCrash();
10 private native int nativePrint();
11 private native int nativeSleep(int ms);
12 private native Double[] nativeAllocate(int n);
13
14 public static void main(String[] args) {
15 JNIDemoJava nativeCall = new JNIDemoJava();
16 nativeCall.nativeCrash();
17 }
18 }
JNIDemoJava_nativeCrash.java hosted with ❤ by GitHub view raw
Let’s start this app with -XX:OnError=”gdb — %p” option. Immediately after start
application will crash and we will get gdb in the terminal:
# A fatal error has been detected by the Java Runtime Environment:

#
# SIGSEGV (0xb) at pc=0x00007f7348cba806, pid=10055, tid=10057
#
# JRE version: OpenJDK Runtime Environment (10.0.2+13) (build
10.0.2+13-Ubuntu-1ubuntu0.18.04.4)
# Java VM: OpenJDK 64-Bit Server VM (10.0.2+13-Ubuntu-
1ubuntu0.18.04.4, mixed mode, tiered, compressed oops, g1 gc, linux-
amd64)
# Problematic frame:
# C [libJNIDemo.so+0x806]
Java_jnidemo_JNIDemoJava_nativeCrash+0x1c
#
...
(gdb)
11 of 20 27-10-23 11:36
From the code above, we see that the issue is with

Java_jnidemo_JNIDemoJava_nativeCrash method and it is invoked from thread with id
10057.
To check the source that failed and values assigned to it’s variable we can do next:
find thread gdb-id, switch to this thread, switch to the frame where we expect the
issue. Here how we can do this:
# find a thread that caused an error

(gdb) info threads
Id Target Id Frame
* 1 Thread 0x7f73ad6f2380 (LWP 10055) "java" 0x00007f73ac8aed2d
in __GI___pthread_timedjoin_ex (threadid=140134807701248,
thread_return=0x7ffc2cdb9338,
abstime=0x0, block=<optimized out>) at pthread_join_common.c:89
2 Thread 0x7f73ad6f0700 (LWP 10057) "java" 0x00007f73acfca6c2
in __GI___waitpid (pid=10098, stat_loc=0x7f73ad6eefcc, options=0)
at ../sysdeps/unix/sysv/linux/waitpid.c:30
3 Thread 0x7f73a960e700 (LWP 10058) "GC Thread#0"
0x00007f73ac8b66d6 in futex_abstimed_wait_cancelable (private=0,
abstime=0x0, expected=0,
Programming Java Gdb Jdk Debugging
futex_word=0x7f73a40295a8)
at ../sysdeps/unix/sysv/linux/futex-internal.h:205
...
# switch to this thread

(gdb) thread 2
...
#6 0x00007f73ac0ee228 in signalHandler(int, siginfo_t*, void*) ()
from /usr/lib/jvm/java-11-openjdk-amd64/lib/server/libjvm.so
#7 <signal handler called>
#8 0x00007f7348cba806 in Java_jnidemo_JNIDemoJava_nativeCrash (
env=0x7f73a4012a00, obj=0x7f73ad6ef980) at src/cpp/JNIDemo.c:11
...
# switch to the frame with our code

Follow
(gdb) frame 8
#8 0x00007f7348cba806 in Java_jnidemo_JNIDemoJava_nativeCrash (
Written
11
byprintf(
Alexey"%c\n",
Pirogov
env=0x7f73a4012a00, obj=0x7f73ad6ef980) at src/cpp/JNIDemo.c:11
s[0] );
83 Followers
We found the code that caused the issue. If required we can print values of different
12 of 20 27-10-23 11:36
variables.
P.S. Note
More fromabout segfaults
Alexey Pirogov
When you will start debugging a large application, you may notice that the
execution of gdb suddenly stops. The reason is that gdb automatically stops app
when the segfault occurs. This makes sense for some applications but not for Java
apps. JDK uses different tools that may produce segfaults (speculative memory load,
NullPointerException, etc.). JDK handles SIGSEGV internally, but gdb has no idea
about this. This is why we need to force gdb to ignore them.
(gdb) handle SIGSEGV nostop noprint pass
Alexey Pirogov
Aeron — low latency transport protocol

I attended workshop with Martin Thompson about Aeron messaging. There I got a chance to
clarify a few questions about this protocol. I…
5 min read · Sep 25, 2017
275 1
13 of 20 27-10-23 11:36
Alexey Pirogov
Java on GPU: Pricing options with Monte Carlo simulation

Table of Contents
9 min read · Apr 26, 2019
14 of 20 27-10-23 11:36
Alexey Pirogov
How to price options, FX and other financial contracts with Haskell

Table of Contents
8 min read · Jun 27, 2018
41
Alexey Pirogov
Monitoring of application latency using jHiccup

About JHiccup
7 min read · Jul 15, 2017
11
Recommended from Medium
See all from Alexey Pirogov
15 of 20 27-10-23 11:36
Jorge Gonzalez
Unleashing the Power of Virtual Threads: Turbocharge your Java

Concurrency with Project Loom
Introduction: Java has long been a popular language for building robust and scalable
applications. However, traditional Java threads…
4 min read · Jun 13
3 1
16 of 20 27-10-23 11:36
Chittaranjan Sethi
Introduction to Unit Testing with Google Test (GTest) in C++

In modern software development, unit testing plays a vital role in ensuring the reliability,
stability, and maintainability of code. By…
3 min read · May 22
2 1
Lists
It's never too late or early to start something

15 stories · 177 saves
General Coding Knowledge

Coding & Development

Stories to Help You Grow as a Software Developer

17 of 20 27-10-23 11:36
Vaishnav Manoj in DataX Journal
JSON is incredibly slow: Here’s What’s Faster!

Unlocking the Need for Speed: Optimizing JSON Performance for Lightning-Fast Apps and
Finding Alternatives to it!
16 min read · Sep 28
1.5K 20
18 of 20 27-10-23 11:36
Dmitry Aleksandrov in Helidon
Helidon 4 released!
We are happy and proud to announce that the long-awaited Helidon 4 has been released!
3 min read · 3 days ago
52 1
Naveen
Single-Threaded vs. Multi-Threaded Programs in Java: A Comprehensive

Comparison
Introduction Java is a versatile programming language that supports concurrent programming
through threads. Threads allow programs to…
4 min read · Jul 22
19 of 20 27-10-23 11:36
Alexander Obregon
IntelliJ IDEA Ultimate vs IntelliJ IDEA Community Edition: A

Comprehensive Comparison
Introduction
4 min read · Jun 14
4 1
See more recommendations
20 of 20 27-10-23 11:36

GDB Debug Native Part of Java Application (C - C++ Libraries and JDK) by Alexey Pirogov Medium

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

GDB Debug Native Part of Java Application (C - C++ Libraries and JDK) by Alexey Pirogov Medium

Uploaded by

Copyright:

Available Formats

GDB: Debug native part of java application (C/C++ libraries and JDK) |... https://medium.com/@pirogov.alexey/gdb-debug-native-part-of-java-ap...

Open in app Sign up Sign In

Acceder con Google

Usa tu Cuenta de Google

Alexey Pirogov · Follow

Why debugging of C/C++ code may be required for Java developers?

Prepare native code

Method’s names are self-describing: nativePrint — prints constant string to stdout,

JNIDemoJava.java hosted with ❤ by GitHub view raw

Let’s generate a c++ header file (interface) for methods defined in

# generate c++ header file and put to cpp folder

1 /* DO NOT EDIT THIS FILE - it is machine generated */

c++ header hosted with ❤ by GitHub view raw

After we got a definition of C++ methods we can implement them:

JNIDemo.c hosted with ❤ by GitHub view raw

Our C++ implementation is ready and we can create a shared library:

buildLib.sh hosted with ❤ by GitHub view raw

Let’s start debug:

# start java app

# find application PID

# start gdb with application PID

# gdb will pause our application

# resume application and wait when it will stop at breakpoint

Thread 2 "java" hit Breakpoint 1,

# step into our "internalNativeAllocate" function using "s" command

# after stop at breakpoint we can do different operations (check

I won’t describe all gdb commands here as list is quite big.

As an example we will debug writeBytes(…) native method that is invoked as a part of

#clone jdk from mercurial

#configure and make build

(gdb) list Java_java_io_FileOutputStream_writeBytes

Automatically start GDB when an application crashes

In this case, it is useful to automatically invoke GDB when an application crashes. It

JNIDemo_nativeCrash.c hosted with ❤ by GitHub view raw

JNIDemoJava_nativeCrash.java hosted with ❤ by GitHub view raw

# A fatal error has been detected by the Java Runtime Environment:

From the code above, we see that the issue is with

# find a thread that caused an error

# switch to this thread

# switch to the frame with our code

(gdb) handle SIGSEGV nostop noprint pass

Aeron — low latency transport protocol

5 min read · Sep 25, 2017

Java on GPU: Pricing options with Monte Carlo simulation

9 min read · Apr 26, 2019

How to price options, FX and other financial contracts with Haskell

8 min read · Jun 27, 2018

Monitoring of application latency using jHiccup

7 min read · Jul 15, 2017

See all from Alexey Pirogov

Unleashing the Power of Virtual Threads: Turbocharge your Java

4 min read · Jun 13

Introduction to Unit Testing with Google Test (GTest) in C++

3 min read · May 22

It's never too late or early to start something

General Coding Knowledge

Coding & Development

Stories to Help You Grow as a Software Developer

Vaishnav Manoj in DataX Journal

JSON is incredibly slow: Here’s What’s Faster!

16 min read · Sep 28

Dmitry Aleksandrov in Helidon