Discussion:
[Valgrind-users] Valgrind and JNI java
Alex Vazquez
2015-10-09 07:39:08 UTC
Permalink
Hi all!
I want to use valgrind to detect memory leaks for a library in Java native
interface. For this I made a sample program by pressing a key reserve
memory and another pressing the release and one for exiting the program.

I have 2 problems:
* If I activate the options to detect memory leaks I see too many failures.
* I can not detect which are memory leaks in my library.

Option for valgrind:
valgrind --smc-check=all --trace-children=yes --show-leak-kinds=all
--track-origins=yes -v --leak-check=yes java -Djava.library.path=. App

Is there any way to filter leaks only my library?
Do the choices I am using to search for leaks is right for Java native code?

Thanks!
John Reiser
2015-10-10 14:03:51 UTC
Permalink
I want to use valgrind to detect memory leaks for a library in Java native interface.
For this I made a sample program by pressing a key reserve memory and another
pressing the release and one for exiting the program.
* If I activate the options to detect memory leaks I see too many failures.
Please tell us how many (within a few percent), and post (copy+paste)
two complete example complaints.
* I can not detect which are memory leaks in my library.
Please give an example, and explain why not. Do your routines have names?
Do those names appear in the complaints from memcheck?
valgrind --smc-check=all --trace-children=yes --show-leak-kinds=all --track-origins=yes -v --leak-check=yes java -Djava.library.path=. App
Please state the version of valgrind ("valgrind --version"),
and the hardware architecture, and the operating system and version.
Is there any way to filter leaks only my library?
If you can determine which are in your library, then encode that knowledge
into a filter (using python, perl, sed, awk, ...). However, in general a leak
results from a misunderstanding about some interface or software contract.
Usually there are at least two pieces of software involved in creating a leak:
the allocator and the non-free()r. The leak "belongs" to the combination
of all of the pieces.
Do the choices I am using to search for leaks is right for Java native code?
We cannot say until you post verbatim examples of the output that you get.



------------------------------------------------------------------------------
Alex Vazquez
2015-10-11 17:03:48 UTC
Permalink
Hi John!
I update the code (github): https://github.com/avazquezdev/TestValgrindJava
Post by John Reiser
Please state the version of valgrind ("valgrind --version"),
and the hardware architecture, and the operating system and version.
valgrind-3.10.1
Kubuntu 15.04 (64bits)
Post by John Reiser
Please tell us how many (within a few percent), and post (copy+paste)
two complete example complaints.
I created two files with the outputs whether or no memory leak.
https://github.com/avazquezdev/TestValgrindJava/tree/master/ErrorInfo
Post by John Reiser
Please give an example, and explain why not. Do your routines have names?
Do those names appear in the complaints from memcheck?
When I allocate memory valgrind show the next:
https://github.com/avazquezdev/TestValgrindJava/blob/master/ErrorInfo/Example1.txt
In this text I see nothing to indicate that my library allocates/free
memory.
Post by John Reiser
Post by Alex Vazquez
I want to use valgrind to detect memory leaks for a library in Java
native interface.
Post by Alex Vazquez
For this I made a sample program by pressing a key reserve memory and
another
Post by Alex Vazquez
pressing the release and one for exiting the program.
* If I activate the options to detect memory leaks I see too many
failures.
Please tell us how many (within a few percent), and post (copy+paste)
two complete example complaints.
Post by Alex Vazquez
* I can not detect which are memory leaks in my library.
Please give an example, and explain why not. Do your routines have names?
Do those names appear in the complaints from memcheck?
Post by Alex Vazquez
valgrind --smc-check=all --trace-children=yes --show-leak-kinds=all
--track-origins=yes -v --leak-check=yes java -Djava.library.path=. App
Please state the version of valgrind ("valgrind --version"),
and the hardware architecture, and the operating system and version.
Post by Alex Vazquez
Is there any way to filter leaks only my library?
If you can determine which are in your library, then encode that knowledge
into a filter (using python, perl, sed, awk, ...). However, in general a leak
results from a misunderstanding about some interface or software contract.
the allocator and the non-free()r. The leak "belongs" to the combination
of all of the pieces.
Post by Alex Vazquez
Do the choices I am using to search for leaks is right for Java native
code?
We cannot say until you post verbatim examples of the output that you get.
------------------------------------------------------------------------------
_______________________________________________
Valgrind-users mailing list
https://lists.sourceforge.net/lists/listinfo/valgrind-users
John Reiser
2015-10-12 14:07:08 UTC
Permalink
Post by Alex Vazquez
I update the code (github): https://github.com/avazquezdev/TestValgrindJava
valgrind-3.10.1
Kubuntu 15.04 (64bits)
I created two files with the outputs whether or no memory leak.
https://github.com/avazquezdev/TestValgrindJava/tree/master/ErrorInfo
https://github.com/avazquezdev/TestValgrindJava/blob/master/ErrorInfo/Example1.txt
In this text I see nothing to indicate that my library allocates/free memory.
Looking at https://raw.githubusercontent.com/avazquezdev/TestValgrindJava/master/ErrorInfo/Leak.txt
I see signs of a large disconnect or misunderstanding between valgrind and the installed java.

--10539-- Reading syms from /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java
--10539-- Considering /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java ..
--10539-- .. CRC mismatch (computed 47030d71 wanted 31641b1e)
--10539-- object doesn't have a symbol table

Not finding the symbol table for .../jre/bin/java could mean that valgrind
will miss the connection to some allocators and deallocators, which will cripple
much of memcheck's machinery. Determine why there is no symbol table;
the "CRC mismatch" indicates that the symbol table probably was stripped.
Do not do that.

The next message

--10539-- Reading syms from /lib/x86_64-linux-gnu/libc-2.21.so
--10539-- Considering /lib/x86_64-linux-gnu/libc-2.21.so ..
--10539-- .. CRC mismatch (computed 9362f137 wanted 040e4cfb)
--10539-- Considering /usr/lib/debug/lib/x86_64-linux-gnu/libc-2.21.so ..
--10539-- .. CRC is valid

indicates that /lib/x86_64-linux-gnu/libc-2.21.so also has no symbol table,
but that a substitute was found in /usr/lib/debug. Perhaps you could install
the corresponding jvm/java-7-openjdk-amd64/jre/bin/java (with symbol table)
also beneath /usr/lib/debug ?

In general, if valgrind does not find the symbols, then that is bad.
Don't strip the symbol table; else install the un-stripped version
beneath /usr/lib/debug.


The first complaint regarding executed code:

==10539== Thread 2:
==10539== Conditional jump or move depends on uninitialised value(s)
==10539== at 0x60F6178: Arena::Arena() (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so)
==10539== by 0x677BCD4: Thread::Thread() (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so)
==10539== by 0x6782474: Threads::create_vm(JavaVMInitArgs*, bool*) (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so)
==10539== by 0x646DB7F: JNI_CreateJavaVM (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so)
==10539== by 0x4E3AB8D: ??? (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/jli/libjli.so)
==10539== by 0x561B6A9: start_thread (pthread_create.c:333)
==10539== Uninitialised value was created by a heap allocation
==10539== at 0x4C2BBA0: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==10539== by 0x6642F89: os::malloc(unsigned long, unsigned short, unsigned char*) (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so)
==10539== by 0x60F621F: Arena::operator new(unsigned long, unsigned short) (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so)
==10539== by 0x677BCC4: Thread::Thread() (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so)
==10539== by 0x6782474: Threads::create_vm(JavaVMInitArgs*, bool*) (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so)
==10539== by 0x646DB7F: JNI_CreateJavaVM (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so)
==10539== by 0x4E3AB8D: ??? (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/jli/libjli.so)
==10539== by 0x561B6A9: start_thread (pthread_create.c:333)

might be correct and valid. Or, it might be a cascade from memcheck not being
connected to the java equivalent of calloc(), which initializes to zero.


The next kind of complaint

==10539== Invalid write of size 4
==10539== at 0x855E445: ???
==10539== by 0x854E4E6: ???
==10539== by 0x644AAD3: JavaCalls::call_helper(JavaValue*, methodHandle*, JavaCallArguments*, Thread*) (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so)
==10539== by 0x644A547: JavaCalls::call(JavaValue*, methodHandle, JavaCallArguments*, Thread*) (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so)
==10539== by 0x6409286: instanceKlass::call_class_initializer_impl(instanceKlassHandle, Thread*) (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so)
==10539== by 0x6409318: instanceKlass::call_class_initializer(Thread*) (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so)
==10539== by 0x64094F4: instanceKlass::initialize_impl(instanceKlassHandle, Thread*) (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so)
==10539== by 0x64099A0: instanceKlass::initialize(Thread*) (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so)
==10539== by 0x6409657: instanceKlass::initialize_impl(instanceKlassHandle, Thread*) (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so)
==10539== by 0x64099A0: instanceKlass::initialize(Thread*) (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so)
==10539== by 0x67831A5: Threads::create_vm(JavaVMInitArgs*, bool*) (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so)
==10539== by 0x646DB7F: JNI_CreateJavaVM (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so)
==10539== Address 0x4158630 is on thread 2's stack
==10539== 4096 bytes below stack pointer

indicates that memcheck is not in sync with the java environment.
Either memcheck has not seen (or not recognized) a "change the stack pointer" event,
or java has implicit conventions about threads, stacks, and thread-local storage
that memcheck does not know or does not understand.


The leak check summary

==10539== HEAP SUMMARY:
==10539== in use at exit: 22,902,921 bytes in 1,235 blocks
==10539== total heap usage: 22,721 allocs, 21,486 frees, 44,322,524 bytes allocated
==10539==
==10539== Searching for pointers to 1,235 not-freed blocks
==10539== Checked 3,488,897,480 bytes

indicates that memcheck believes the allocation arenas are around 3.3GB big. That's a lot,
especially for your tiny example program. That's another worrisome sign of a disconnect
between memcheck and java.

The largest leaks

==10539== 5,242,880 bytes in 5 blocks are still reachable in loss record 779 of 779
==10539== at 0x4C2BBA0: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==10539== by 0x6642F89: os::malloc(unsigned long, unsigned short, unsigned char*) (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so)
==10539== by 0x66A1580: ParCompactionManager::initialize(ParMarkBitMap*) (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so)
==10539== by 0x66A8899: PSParallelCompact::post_initialize() (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so)
==10539== by 0x666AD04: ParallelScavengeHeap::post_initialize() (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so)
==10539== by 0x679EC1B: universe_post_init() (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so)
==10539== by 0x63FD939: init_globals() (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so)
==10539== by 0x67825DE: Threads::create_vm(JavaVMInitArgs*, bool*) (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so)
==10539== by 0x646DB7F: JNI_CreateJavaVM (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server/libjvm.so)
==10539== by 0x4E3AB8D: ??? (in /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/jli/libjli.so)
==10539== by 0x561B6A9: start_thread (pthread_create.c:333)

hint that memcheck might not understand the conventions of java GarbageCollector.


In summary: fix the symbol table problems first.


------------------------------------------------------------------------------
Loading...