STL containers in shared memory

[phil_gg04]

Dear C++ Experts,

Over the last couple of months I have been writing my first program
using shared memory. It has been something of an "in-at-the-deep-end"
experience, to say the least. At present the shared memory contains a
few fixed-size structs, but I really need to be able to store more
complex variable-sized data in there. So the next task is to work out
how to store C++ objects, and if possible STL containers, in this
shared region.

So far I have got the following bits working:

- Creating the shared memory region, and attaching to it from the
various processes. It can of course end up at different addresses in
the different processes, and this is a key problem.

- Basic C malloc()/free() style allocation in the region, using offsets
rather than pointers so that it works wherever it is mapped.

- Implementations of operator new, new[], delete and delete[] that use
these C-style primitives.

I now have two remaining challenges. First, I'd like to write an
offset<T> class that I can use in places where I would normally use T*.
I think that this is possible; I'll need to define operator*, a
conversion from T* to offset<T>, and a few other things. Any
suggestions would be welcome, though I think I would be able to work it
out eventually. My first decision is whether the offsets are relative
to the start of the region, in which case a "per-process global" is
needed to record the region start address, or they're relative to where
they are stored, which avoids the global but makes the implementation
harder.

For me the more difficult challenge is the STL allocators. I have only
the vaguest idea about this. Presumably I can create an "allocator
object" that the containers will use, without too much trouble (though
an example would be good). But can I persuade the containers that
they'd like to store my offsets, instead of real pointers?

I'm hoping that someone in this group will be able to either point me
to some good examples, or maybe convince me that it's impossible!
Which is it to be?

BTW this is for Anyterm: http://chezphil.org/anyterm/ - could be
interesting if you ever need to do remote server admin.

Cheers, Phil.

 

[Uenal Mutlu]

Hmm. Why not put just the pointer(s) into the shared region, and
dynamically alloc the container(s) (or any other object) from the heap?

 

[Gianni Mariani]

I now have two remaining challenges. First, I'd like to write an
offset<T> class that I can use in places where I would normally use T*.

See austria's "relative pointer"

http://austria.sourceforge.net/dox/html/at__relative__pointer_8h-source.html

It's behaviour is compiler implementation dependant but I think it will
work with most compilers.

....

I'm hoping that someone in this group will be able to either point me
to some good examples, or maybe convince me that it's impossible!
Which is it to be?

The problem you will find is that if you want to have multiple shared
regions in your code you need to know what the offset is for each one.
Hence a parameter to shared memory pointer dereferencing is the offset
of the region you're currently referecing. There is a technique you can
use to take this out as a parameter by using the "this" pointer to
figure out which map you're referencing, but performance may become a
significant issue (although there are a number of strategies for
optimization).

Once you have this problem licked, you have the issue of vtables for
objects with virtual methods. Unless you build in some support from the
compiler, you're SOL.

G

 

[phil_gg04]

Hmm. Why not put just the pointer(s) into the shared region, and

dynamically alloc the container(s) (or any other object) from the

heap?

Because then they're not shared.

--Phil.

 

[phil_gg04]

The problem you will find is that if you want to have multiple shared

regions in your code you need to know what the offset is for each

one.

I'm happy with just the one region, I think.

you have the issue of vtables for objects with virtual methods.

And I can live without virtual methods as well. Really I just need a
few set<struct>, map<int,struct>, strings, etc.

So, will it all just work then??

--Phil.

 

[phil_gg04]

how to store C++ objects, and if possible STL containers, in [a]
shared region
...
offset<T> class that I can use in places where I would normally use T*.
...
can I persuade the containers that
they'd like to store my offsets, instead of real pointers?

Well, it's a bit later and I have some progress, but I'm not hopeful
about being able to store the containers. I have an offset<T> class
that can convert implicitly to and from T* in the obvious way, and it
seems to work in my own code. And I've got an allocator class that
uses my shared memory that I can pass to containers.

All that remains is getting the containers to store offsets rather than
pointers. I changed the typedef for "pointer" in the allocator from T*
to offset<T>, but this doesn't work because the containers (e.g. map)
have explicit T* types in their implementations, rather than using the
allocator's pointer type. So my offsets are converted straight back
into real pointers.

My feeling is that this is insurmountable. Will I have to write my own
map class?

(Gianni, can the Austria relative pointer class be used in standard
containers? I've had a look at it but I don't think I understand it
all.)

--Phil.

 

[Uenal Mutlu]

heap?

Because then they're not shared.

But the pointer to the data is shared. For practical cases this gives the same result.

 

[phil_gg04]

Hmm. Why not put just the pointer(s) into the shared region, and

But the pointer to the data is shared. For practical cases this gives

the same result.

No it doesn't. If I've got two processes and a shared memory region,
if I have a pointer in the shared region that points outside the
region, then it points to different things in each process. E.g. if
one process does

shared_pointer = "foo";

and the other process does

printf("%s",shared_pointer);

then it will NOT print 'foo', but instead whatever that process happens
to have at that address.

--Phil.

 

[pjp]

You need an implementation of the Standard C++ library that makes use
of nonstandard pointer types defined in allocators. AFAIK, only
Dinkumware offers such a library. The C++ Standard encourages this sort
of extension but doesn't require it. See Microsoft VC++ and a number of
other compilers that use our library. Or you can license an add-on
version, for a variety of compilers, directly at our web site.

Without such a library, your problem is indeed insurmountable.

P.J. Plauger
Dinkumware, Ltd.
http://www.dinkumware.com

 

[phil_gg04]

You need an implementation of the Standard C++ library that makes use

of nonstandard pointer types defined in allocators. AFAIK, only
Dinkumware offers such a library.

Thanks. Unfortunately this is for Anyterm
(http://chezphil.org/anyterm/) which is distributed as source code
under a GPL license. So it really needs to compile with g++ and its
default libraries, else no-one is going to use it.

(I'm curious to understand what downside, if any, there is to storing
allocator<T>:ointer rather than explicit T* in the container
implementation. Is this just an easy-to-fix issue with the GNU
libraries, or are they doing it for a good reason?)

--Phil.

 

[Uenal Mutlu]

the same result.

No it doesn't. If I've got two processes and a shared memory region,
if I have a pointer in the shared region that points outside the
region, then it points to different things in each process. E.g. if
one process does

shared_pointer = "foo";

and the other process does

printf("%s",shared_pointer);

then it will NOT print 'foo', but instead whatever that process happens
to have at that address.

Maybe this can give you some hints:
Under Windows there are some API functions for global allocation.
It returns a systemwide usable unique memory handle which in each
process must be deferenced to an ordinary pointer.
So in your case it would work too if your OS had similar methods
for allocation like Window's GlobalAlloc(), or if you write a similar allocator
yourself (ie, one which operates with dereferencible handles) and by using
a reserved area of the shared mem as the mem-pool for the allocator.

 

[phil_gg04]

... systemwide usable unique memory handle which in each

process must be deferenced to an ordinary pointer.

Yes. This is exactly what I am doing. But in each process that
pointer's value can be different.
Hence the need for non-standard "relative pointers", and the
difficulties that they pose when you try to use them inside standard
containers.

--Phil.

 

[Marc D Ronell]

phil> No it doesn't. If I've got two processes and a shared
phil> memory region, if I have a pointer in the shared region that
phil> points outside the region, then it points to different
phil> things in each process. E.g. if one process does

phil> shared_pointer = "foo";

phil> and the other process does

phil> printf("%s",shared_pointer);

phil> then it will NOT print 'foo', but instead whatever that
phil> process happens to have at that address.

GCC used to allow objects to be placed or instantiated in shared
memory. These objects could then be shared between different
processes. However, Placement into shared memory does not seem to
work with the current version of the GCC compiler, ver 3.4.3. The
issue seems to be with the vtable implementation. Object pointers to
the vtable object class definitions in one process do not currently
correctly address the class definitions in secondary processes.

If the GCC compiler group can get placement working again, it may be
possible to revive the shared memory allocator. Or there may be
alternative unrelated solutions. There is an existing bug report on
placement and the related concern with shared memory allocators. See
the following URL:

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=21251

Don't know if this helps, but there is strong interest in developing
workable shared memory allocators for C++.

Marc

--

 

[phil_gg04]

Hi Marc,

GCC used to allow objects to be placed or instantiated in shared
memory. These objects could then be shared between different
processes. However, Placement into shared memory does not seem to
work with the current version of the GCC compiler, ver 3.4.3.

Thanks to Google I saw your gcc bug report before I posted here and I
left a message on your sourceforge page yesterday. Are you really
certain that your code did previously work? Only those classes that
have virtual methods need a vtable pointer, and in my case the data is
not much more than structs, so this aspect of the problem isn't an
issue for me. If your code really did work with classes with vtables
between processes with different address maps in older versions of gcc
then that is very mysterious indeed - could it be that you have only
recently tried it with classes with virtual methods?

Did you get standard containers to work with your allocator?

Regards, Phil.

 

[Gianni Mariani]

one.

I'm happy with just the one region, I think.




And I can live without virtual methods as well. Really I just need a
few set<struct>, map<int,struct>, strings, etc.

So, will it all just work then??

Not really. There is no way of overriding pointer behaviour. You'll
have to re-implement the container classes to use Relative pointer (or
somthing like it).

G