signal 11 outside teh VM when running native code from a thread

[Gil Carmel]

hi there,

i'm having a strange problem:

i have a method that makes use of some native code (that i compiled --
so i don't think it's a library incompatibility). it works when it's
called from the main thread, but not when it's called from any other
thread.

i am running on a blackdown JVM (1.4.1) on redhat 8.

below is the crash output i get when i run the code from a thread:

any ideas on why running in the main thread vs another thread would
make a difference?

(please CC me at g_carmel AT yahoo DOT com on the response as i don't
check newsgroups often)

gil

------------

An unexpected exception has been detected in native code outside the
VM.
Unexpected Signal : 11 occurred at PC=0x58ECD79F
Function=_ZN3Imf27_GLOBAL__N_ImfHuf.cpphKRmjb17hufUnpackEncTableEPPKciiPy+0x1F
Library=/usr/local/lib/libIlmImf.so.0

Current Java thread:
at com.primordialsoftware.imageio.plugins.openexr.OpenEXR.read(Native
Method)
- locked <0x45588bc0> (a
com.primordialsoftware.imageio.plugins.openexr.OpenEXR)
at com.primordialsoftware.imageio.plugins.openexr.OpenExrImageReader.read(OpenExrImageReader.java:76)
at javax.imageio.ImageReader.read(ImageReader.java:919)
at com.primordialsoftware.ooze.util.ImageUtil.read(ImageUtil.java:71)
- locked <0x542b9230> (a java.lang.Class)
at com.primordialsoftware.ooze.Frame.<init>(Frame.java:32)
at com.primordialsoftware.ooze.ScalableFrameSequence.getOrCreateThumbnail(ScalableFrameSequence.java:55)
at com.primordialsoftware.ooze.format.ImageFilesFrameSequence.addFrames(ImageFilesFrameSequence.java:70)
at com.primordialsoftware.ooze.test.OpenEXRTest3$2.safeRun(OpenEXRTest3.java:126)
at com.primordialsoftware.ooze.util.safe.SafeRunnable.run(SafeRunnable.java:25)
at java.lang.Thread.run(Thread.java:536)

Dynamic libraries:
08048000-0804e000 r-xp 00000000 03:03 1114710
/home/gil/morph/tools/Linux/blackdown/j2sdk1.4.1/bin/java
0804e000-0804f000 rw-p 00005000 03:03 1114710
/home/gil/morph/tools/Linux/blackdown/j2sdk1.4.1/bin/java
40000000-40012000 r-xp 00000000 03:03 933894 /lib/ld-2.2.93.so
40012000-40013000 rw-p 00012000 03:03 933894 /lib/ld-2.2.93.so
40013000-4001d000 r-xp 00000000 03:03 1114855
/home/gil/morph/tools/Linux/blackdown/j2sdk1.4.1/jre/lib/i386/native_threads/libhpi.so
4001d000-4001e000 rw-p 00009000 03:03 1114855
/home/gil/morph/tools/Linux/blackdown/j2sdk1.4.1/jre/lib/i386/native_threads/libhpi.so
4001e000-40021000 r--s 00000000 03:03 754111
/home/gil/morph/tools/Linux/blackdown/j2sdk1.4.1/jre/lib/ext/dnsns.jar
40021000-40022000 r--s 00000000 03:03 279189
/home/gil/morph/lib/oozemedia.jar
40023000-40030000 r-xp 00000000 03:03 753675
/lib/i686/libpthread-0.10.so
40030000-40033000 rw-p 0000d000 03:03 753675
/lib/i686/libpthread-0.10.so
40053000-40055000 r-xp 00000000 03:03 933907 /lib/libdl-2.2.93.so
40055000-40056000 rw-p 00001000 03:03 933907 /lib/libdl-2.2.93.so
40056000-403b2000 r-xp 00000000 03:03 770423
/home/gil/morph/tools/Linux/blackdown/j2sdk1.4.1/jre/lib/i386/client/libjvm.so
403b2000-403cd000 rw-p 0035b000 03:03 770423
/home/gil/morph/tools/Linux/blackdown/j2sdk1.4.1/jre/lib/i386/client/libjvm.so
403de000-403f0000 r-xp 00000000 03:03 933911 /lib/libnsl-2.2.93.so
403f0000-403f1000 rw-p 00012000 03:03 933911 /lib/libnsl-2.2.93.so
403f3000-4048b000 r-xp 00000000 03:03 344111
/usr/lib/libstdc++.so.5.0.1
4048b000-404a0000 rw-p 00098000 03:03 344111
/usr/lib/libstdc++.so.5.0.1
404a5000-404c6000 r-xp 00000000 03:03 753673
/lib/i686/libm-2.2.93.so
404c6000-404c7000 rw-p 00021000 03:03 753673
/lib/i686/libm-2.2.93.so
404c7000-404ce000 r-xp 00000000 03:03 933964
/lib/libgcc_s-3.2-20020903.so.1
404ce000-404cf000 rw-p 00007000 03:03 933964
/lib/libgcc_s-3.2-20020903.so.1
404cf000-404e0000 r-xp 00000000 03:03 1081767
/home/gil/morph/tools/Linux/blackdown/j2sdk1.4.1/jre/lib/i386/libverify.so
404e0000-404e1000 rw-p 00011000 03:03 1081767
/home/gil/morph/tools/Linux/blackdown/j2sdk1.4.1/jre/lib/i386/libverify.so
404e1000-40501000 r-xp 00000000 03:03 1081754
/home/gil/morph/tools/Linux/blackdown/j2sdk1.4.1/jre/lib/i386/libjava.so
40501000-40503000 rw-p 0001f000 03:03 1081754
/home/gil/morph/tools/Linux/blackdown/j2sdk1.4.1/jre/lib/i386/libjava.so
40503000-40515000 r-xp 00000000 03:03 1081768
/home/gil/morph/tools/Linux/blackdown/j2sdk1.4.1/jre/lib/i386/libzip.so
40515000-40517000 rw-p 00011000 03:03 1081768
/home/gil/morph/tools/Linux/blackdown/j2sdk1.4.1/jre/lib/i386/libzip.so
40517000-41be5000 r--s 00000000 03:03 704826
/home/gil/morph/tools/Linux/blackdown/j2sdk1.4.1/jre/lib/rt.jar
41c28000-41c3f000 r--s 00000000 03:03 704827
/home/gil/morph/tools/Linux/blackdown/j2sdk1.4.1/jre/lib/sunrsasign.jar
41c3f000-41cb0000 r--s 00000000 03:03 704819
/home/gil/morph/tools/Linux/blackdown/j2sdk1.4.1/jre/lib/jsse.jar
41cb0000-41cc3000 r--s 00000000 03:03 704817
/home/gil/morph/tools/Linux/blackdown/j2sdk1.4.1/jre/lib/jce.jar
41cc3000-41f7f000 r--s 00000000 03:03 704756
/home/gil/morph/tools/Linux/blackdown/j2sdk1.4.1/jre/lib/charsets.jar
42000000-42126000 r-xp 00000000 03:03 753671
/lib/i686/libc-2.2.93.so
42126000-4212b000 rw-p 00126000 03:03 753671
/lib/i686/libc-2.2.93.so
44157000-44160000 r--s 00000000 03:03 541306
/home/gil/morph/tools/Linux/jai_imageio-1_0-rc/clibwrapper_jiio.jar
5841c000-585db000 r--p 00000000 03:03 458766
/usr/lib/locale/locale-archive
585db000-585e9000 r--s 00000000 03:03 754112
/home/gil/morph/tools/Linux/blackdown/j2sdk1.4.1/jre/lib/ext/ldapsec.jar
585ea000-585f3000 r-xp 00000000 03:03 933927
/lib/libnss_files-2.2.93.so
585f3000-585f4000 rw-p 00008000 03:03 933927
/lib/libnss_files-2.2.93.so
587f8000-58897000 r--s 00000000 03:03 754113
/home/gil/morph/tools/Linux/blackdown/j2sdk1.4.1/jre/lib/ext/localedata.jar
58897000-588b4000 r--s 00000000 03:03 754114
/home/gil/morph/tools/Linux/blackdown/j2sdk1.4.1/jre/lib/ext/sunjce_provider.jar
588b4000-588ff000 r-xp 00000000 03:03 1081747
/home/gil/morph/tools/Linux/blackdown/j2sdk1.4.1/jre/lib/i386/libcmm.so
588ff000-58901000 rw-p 0004a000 03:03 1081747
/home/gil/morph/tools/Linux/blackdown/j2sdk1.4.1/jre/lib/i386/libcmm.so
58901000-5893a000 r-xp 00000000 03:03 1081759
/home/gil/morph/tools/Linux/blackdown/j2sdk1.4.1/jre/lib/i386/libjpeg.so
5893a000-5893b000 rw-p 00038000 03:03 1081759
/home/gil/morph/tools/Linux/blackdown/j2sdk1.4.1/jre/lib/i386/libjpeg.so
5893b000-58945000 r--s 00000000 03:03 803465
/home/gil/morph/tools/Linux/jai-1_1_1_01/lib/mlibwrapper_jai.jar
5897c000-58a3b000 r--s 00000000 03:03 704816
/home/gil/morph/tools/Linux/blackdown/j2sdk1.4.1/jre/lib/javaplugin.jar
58a3b000-58ab6000 r--s 00000000 03:03 279181
/home/gil/morph/lib/icons.jar
58ab6000-58ad2000 r--s 00000000 03:03 279183
/home/gil/morph/lib/jlfgr-1_0.jar
58ad2000-58add000 r--s 00000000 03:03 279185
/home/gil/morph/lib/kunststoff.jar
58add000-58b29000 r--s 00000000 03:03 279186
/home/gil/morph/lib/looks-all.jar
58b29000-58b47000 r--s 00000000 03:03 279187
/home/gil/morph/lib/looks-win.jar
58b47000-58b57000 r--s 00000000 03:03 279188
/home/gil/morph/lib/metouia.jar
58b57000-58b8e000 r--s 00000000 03:03 279190
/home/gil/morph/lib/plastic.jar
58b8e000-58bd5000 r--s 00000000 03:03 279191
/home/gil/morph/lib/vecmath.jar
58bd5000-58c19000 r--s 00000000 03:03 279192
/home/gil/morph/lib/xplookandfeel.jar
58c19000-58c4b000 r--s 00000000 03:03 803462
/home/gil/morph/tools/Linux/jai-1_1_1_01/lib/jai_codec.jar
58c4b000-58da9000 r--s 00000000 03:03 803463
/home/gil/morph/tools/Linux/jai-1_1_1_01/lib/jai_core.jar
58da9000-58e81000 r--s 00000000 03:03 541307
/home/gil/morph/tools/Linux/jai_imageio-1_0-rc/jai_imageio.jar
58e81000-58e8b000 r--s 00000000 03:03 541309
/home/gil/morph/tools/Linux/jai_imageio-1_0-rc/mlibwrapper_jai.jar
58e8b000-58e90000 r--s 00000000 03:03 1622336
/home/gil/morph/work/package/Linux/ooze-0.1/lib/openexr.jar
58e90000-58e97000 r-xp 00000000 03:03 1098254
/home/gil/morph/work/package/Linux/ooze-0.1/bin/libopenexr.so
58e97000-58e98000 rw-p 00007000 03:03 1098254
/home/gil/morph/work/package/Linux/ooze-0.1/bin/libopenexr.so
58e98000-58ed6000 r-xp 00000000 03:03 197296
/usr/local/lib/libIlmImf.so
58ed6000-58ede000 rw-p 0003e000 03:03 197296
/usr/local/lib/libIlmImf.so
58edf000-58ee3000 r-xp 00000000 03:03 295807
/usr/local/lib/libImath.so
58ee3000-58ee4000 rw-p 00004000 03:03 295807
/usr/local/lib/libImath.so
58ee4000-58f26000 r-xp 00000000 03:03 295806
/usr/local/lib/libHalf.so
58f26000-58f27000 rw-p 00042000 03:03 295806
/usr/local/lib/libHalf.so
58f27000-58f3c000 r-xp 00000000 03:03 557892
/usr/local/lib/libIex.so
58f3c000-58f3e000 rw-p 00015000 03:03 557892
/usr/local/lib/libIex.so
58f3e000-58f4a000 r-xp 00000000 03:03 344193
/usr/lib/libz.so.1.1.4
58f4a000-58f4c000 rw-p 0000b000 03:03 344193
/usr/lib/libz.so.1.1.4
58fcd000-58fd3000 r--s 00000000 03:03 737473
/usr/lib/gconv/gconv-modules.cache
58fd3000-58fd4000 r-xp 00000000 03:03 1556539
/usr/X11R6/lib/X11/locale/common/xlcUTF8Load.so.2
58fd4000-58fd5000 rw-p 00000000 03:03 1556539
/usr/X11R6/lib/X11/locale/common/xlcUTF8Load.so.2
58fd9000-591fe000 r-xp 00000000 03:03 1081746
/home/gil/morph/tools/Linux/blackdown/j2sdk1.4.1/jre/lib/i386/libawt.so
591fe000-59213000 rw-p 00224000 03:03 1081746
/home/gil/morph/tools/Linux/blackdown/j2sdk1.4.1/jre/lib/i386/libawt.so
59238000-5929e000 r-xp 00000000 03:03 1081763
/home/gil/morph/tools/Linux/blackdown/j2sdk1.4.1/jre/lib/i386/libmlib_image.so
5929e000-5929f000 rw-p 00065000 03:03 1081763
/home/gil/morph/tools/Linux/blackdown/j2sdk1.4.1/jre/lib/i386/libmlib_image.so
592ae000-592b5000 r-xp 00000000 03:03 442493
/usr/X11R6/lib/libXp.so.6.2
592b5000-592b6000 rw-p 00006000 03:03 442493
/usr/X11R6/lib/libXp.so.6.2
592b6000-59304000 r-xp 00000000 03:03 442501
/usr/X11R6/lib/libXt.so.6.0
59304000-59308000 rw-p 0004d000 03:03 442501
/usr/X11R6/lib/libXt.so.6.0
59308000-59310000 r-xp 00000000 03:03 442471
/usr/X11R6/lib/libSM.so.6.0
59310000-59311000 rw-p 00007000 03:03 442471
/usr/X11R6/lib/libSM.so.6.0
59311000-59325000 r-xp 00000000 03:03 442467
/usr/X11R6/lib/libICE.so.6.3
59325000-59326000 rw-p 00013000 03:03 442467
/usr/X11R6/lib/libICE.so.6.3
59328000-59335000 r-xp 00000000 03:03 442481
/usr/X11R6/lib/libXext.so.6.4
59335000-59336000 rw-p 0000c000 03:03 442481
/usr/X11R6/lib/libXext.so.6.4
59336000-5933a000 r-xp 00000000 03:03 442503
/usr/X11R6/lib/libXtst.so.6.1
5933a000-5933b000 rw-p 00004000 03:03 442503
/usr/X11R6/lib/libXtst.so.6.1
5933b000-59416000 r-xp 00000000 03:03 442473
/usr/X11R6/lib/libX11.so.6.2
59416000-59419000 rw-p 000da000 03:03 442473
/usr/X11R6/lib/libX11.so.6.2
59cb1000-59f88000 r-xp 00000000 03:03 803464
/home/gil/morph/tools/Linux/jai-1_1_1_01/lib/libmlib_jai.so
59f88000-59f8b000 rw-p 002d6000 03:03 803464
/home/gil/morph/tools/Linux/jai-1_1_1_01/lib/libmlib_jai.so
5a3d7000-5a488000 r-xp 00000000 03:03 1081750
/home/gil/morph/tools/Linux/blackdown/j2sdk1.4.1/jre/lib/i386/libfontmanager.so
5a488000-5a49a000 rw-p 000b0000 03:03 1081750
/home/gil/morph/tools/Linux/blackdown/j2sdk1.4.1/jre/lib/i386/libfontmanager.so
5a49a000-5a4b6000 r-xp 00000000 03:03 1556537
/usr/X11R6/lib/X11/locale/common/ximcp.so.2
5a4b6000-5a4b8000 rw-p 0001b000 03:03 1556537
/usr/X11R6/lib/X11/locale/common/ximcp.so.2
5acae000-5acc9000 r-xp 00000000 03:03 1081748
/home/gil/morph/tools/Linux/blackdown/j2sdk1.4.1/jre/lib/i386/libdcpr.so
5acc9000-5acdc000 rw-p 0001b000 03:03 1081748
/home/gil/morph/tools/Linux/blackdown/j2sdk1.4.1/jre/lib/i386/libdcpr.so

Local Time = Fri Jul 11 17:13:41 2003
Elapsed Time = 20
#
# The exception above was detected in native code outside the VM
#
# Java VM: Java HotSpot(TM) Client VM (Blackdown-1.4.1-01 mixed mode)
#

 

[Joseph Millar]

i have a method that makes use of some native code (that i compiled --
so i don't think it's a library incompatibility). it works when it's
called from the main thread, but not when it's called from any other
thread.

My advice is follow the error. It's likely some kind of thread
safety issue in your native code, not the JVM. It's your code
so you have access to the source, so run it in a debugger and
find out precisely where it's crashing. Couple of things to
watch for:

1. JNIEnv pointers are thread specific. DO NOT CACHE ON ONE
THREAD AND USE IT ON ANOTHER.

2. Watch your use of code and libraries that require some
form of initialization. Many times these libraries are
either not thread safe, or must be initialized on each
thread you want to use them on.

3. Watch your use of thread local storage.

Hard to be more specific without seeing the native source code.

--Joe

 

[Gil Carmel]

Thanks for you reply, Joseph. I took your suggestion and looked into
where the crash is happening. I was wondering if you'd mind taking a
quick read through some more info and seeing if anything comes to
mind.

Regarding #1 below, there is no caching of JNIEnv pointers.

Regarding #2 we tried initializing and using the library all in one
thread and nowhere else - it crashes. when we do the exact same thing
from the main thread, it does not crash.

I should point out that there is really no multithreaded use of the
native library itself. It's only the Java code that may access the
native library from its own threads (with only one concurrent access).
This makes me believe that the thread safety of the native code isn't
an issue.

From a couple of printfs i've sprinkled, i've pinpointed the crash as
occurring DURING the call to a particular native C method (from
another native C method). There is no pointer dereferencing or
anything funny going on in the argument list. It simply looks like
the call itself just bombs somehow.

I'm wondering if maybe we are missing some compile/linker flags in the
native library makefile (e.g. to use multithreaded runtime c
libraries, etc). Do you have any experience with compiling something
like this on linux?

I've tried a few random compiler switches I've seen mentioned (like
-D_REENTRANT) all to no avail.

Any info on any of the above points would be greatly appreciated!

Gil

PS Once again please CC: me at (e-mail address removed)

 

[Joseph Millar]

Regarding #1 below, there is no caching of JNIEnv pointers.
Good.

Regarding #2 we tried initializing and using the library all in one
thread and nowhere else - it crashes. when we do the exact same thing
from the main thread, it does not crash.

This is interesting. This points to some sort of thread safety
issue. Is there any other kind of initialization or native
method calls being made from the main thread? You are implying
that your "all in one thread" test did not involve the main
thread. That leads me to believe you are doing some sort of
init work on the main thread but not other threads. Very
hard for me to give you anything else without seeing the code.

I should point out that there is really no multithreaded use of the
native library itself. It's only the Java code that may access the
native library from its own threads (with only one concurrent access).
This makes me believe that the thread safety of the native code isn't
an issue.

A Java thread is just like any other OS thread. On most JVM's
(all the ones from Sun), each Java thread is mapped to one and
only one OS thread. So you do have to be concerned with thread
safety issues if your library is being used from more than one
Java thread. The native code will excute on the same thread
as the calling Java thread. Not sure why you're saying there
is no multithreaded use of the library if multiple threads can
call the routines.

From a couple of printfs i've sprinkled, i've pinpointed the crash as
occurring DURING the call to a particular native C method (from
another native C method). There is no pointer dereferencing or
anything funny going on in the argument list. It simply looks like
the call itself just bombs somehow.

If you haven't already, you really should get this into a
debugger and take a look at the call stack and exact location
of the error, paying special attention to the stack layout.
If the error is happening on the return from the call, you
might find someone is stomping on your stack frame, thus
destroying critical info needed to restore the stack

I'm wondering if maybe we are missing some compile/linker flags in the
native library makefile (e.g. to use multithreaded runtime c
libraries, etc). Do you have any experience with compiling something
like this on linux?

I have, but have never had any problems other than my own
errors. I haven't done it lately, but I could dig up the
flags I used if you need them, but that's not going to be
really helpful here. I suspect the problem is user code,
the thing to do is track it down to the offending line
and examine that puppy real close.

--Joe

 

[Chris Smith]

On 16 Jul 2003 11:49:35 -0700, (e-mail address removed) (Gil Carmel) wrote:

[stuff]

Here's a completely off-the-wall shot. I've seen some problems when
using a VM and native code that have been linked against different
threading libraries. This was mostly a problem in the green vs native
threads universe, but at least Solaris comes with multiple thread
libraries, and Linux certainly can be set up that way. Are you sure you
aren't loading conflicting threading libs with those against which the
VM itself is linked?

--
www.designacourse.com
The Easiest Way to Train Anyone... Anywhere.

Chris Smith - Lead Software Developer/Technical Trainer
MindIQ Corporation