MappedByteBuffer

[Tom Anderson]

I told you it was a gotcha. See the Bug Parade, endless discussion about
it.

Indeed. ISTR the gist of the justification for this is something like 'the
JVM can't tell when the address range is truly not referenced any more, so
it can't safely reuse it'.

Am i right in thinking that on 64-bit machines, it's unlikely to be a
problem in practice?

tom

 

[Alan Gutierrez]

Indeed. ISTR the gist of the justification for this is something like
'the JVM can't tell when the address range is truly not referenced any
more, so it can't safely reuse it'.

Am i right in thinking that on 64-bit machines, it's unlikely to be a
problem in practice?

I'm not sure, but...

For the B+tree I've developed: The B+tree is a paged structure and at
this time, it handles only reference records, or records that are of
fixed with. When it does do variable length records, I'll do blocks
managed by the B+tree storage implementation.

Thus, I'm going to open however many `MappedByteBuffer`s as necessary to
store the B+tree. Each file is a fixed length containing a fixed number
of pages. As pages become free, I'll link into a linked list, but there
will be no closing and reopening of the files.

I curious to see if having many `MappedByteBuffer` instances of a
certain size (~250MB or more) is going to be a problem in practice.

In my case, I'm not trusting the leaking address space. Trying to design
only for what the documentation says it will do.

 

[Paul Cager]

ISTR the gist of the justification for this is something like 'the
JVM can't tell when the address range is truly not referenced any more, so
it can't safely reuse it'.

I had never heard of that gotcha before. Looks like it is described in
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4724038. The
explanation given by Sun sounds a bit odd to me:

"Suppose that a thread operating on behalf of Alice maps a file
into memory and
then unmaps it. A second thread operating on behalf of Bob then maps
some
other file that the underlying operating system happens to assign to
the same
memory address. Now Alice's thread can read, and possibly even
modify, the
contents of Bob's file. Oops."

I take that to mean that Alice could subvert Java's security manager
(she couldn't subvert the operating system's permissions using this
method). Have I got that right? If so then I can't really follow Sun's
logic.

 

[m_hess]

I had never heard of that gotcha before. Looks like it is described inhttp://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4724038. The
explanation given by Sun sounds a bit odd to me:

    "Suppose that a thread operating on behalf of Alice maps a file
into memory and
then unmaps it.  A second thread operating on behalf of Bob then maps
some
other file that the underlying operating system happens to assign to
the same
memory address.  Now Alice's thread can read, and possibly even
modify, the
contents of Bob's file.  Oops."

I take that to mean that Alice could subvert Java's security manager
(she couldn't subvert the operating system's permissions using this
method). Have I got that right? If so then I can't really follow Sun's
logic.

Well, I'm glad that I'm not the only person who is confused by this
explanation.

I stumbled across this a few months back, and too me it sounds fishy.

It is my understanding, that the memory that is used for the
MemoryMappedFiles is part of the java process memory. It is requested
by the JVM additionally to what it already got for the java heap.
Sticking with their example, they say that Alice "unmapped" the file.
The question is what do they mean by this _exactly_. I would expect,
that it means that the JVM process has given up (or free()'d) that
memory area that was used for the mapped file. So why should it be a
problem if some other process now uses this for (here comes the
important part) WHATEVER it wants to do? That's the job of the virtual
memory management of the underlying OS to make sure no two processes
colide on that level.

Bottomline: To me the whole explanation SUN gave does not make sense.
Maybe they left out some important details, I don't know...


My experience was, that the memory will get reclaimed eventually. But
the problem is that you have no control over when this happens, as
there is no close() or similar for the MemoryMappedFile. Another
problem is, that for as long as the memory mapped file is still
"live", the file is still locked.

I created a unittest for our component which used the memory mapped
file, and I had to fiddle with it somewhat, to just be able to delete
the temporary files the tests created. Not very "usable" ... :-(

Closing remark from my side: We removed the MemoryMappedFile approach
from our software. Not because of the handling problems just
discussed, but because this meant giving up virtually all control over
when and how I/O (many small chunks vs. large bursts) was happening.
This broke our necks so we had to change over to stream based I/O.

Michael