Core dump while accessing contents from Memory Mapped File

[nkrisraj]

Hi,

I have a following structure:
typedef struct
{
RateData rdr;
int RateID;
char RateBalance;

} RateInfo;

typedef struct
{
DayRates *dr;
WeekRates *wr;

} RateData;

First I will create the memoefy for the DayRates, WeekRates structures
inside the RateInfo structure variable. After that Iam going to copy
the contents into DayRates, WeekRatesstructures. Once the all the
structure members inside RateInfo structure variable are initialized, I
will add that node to a AVL tree. This way, Iam going to create an AVL
tree by adding RateInfo nodes to it. Once the total tree is contructed,
I will store this tree as file (rateinfo.map) on the disk.

In my next programming component, Iam going to load this file
(rateinfo.map) into memory using mmap() function call. Once I load the
file into memory using mmap(), Iam able to access the RateID and
RateBalance members of the RateInfo. But, Iam not able to access the
DayRates, and WeekRates structure contents. When I try to access the
members of DayRates , and WeekRates structures, my program is showing
segmentation fault and Iam getting the core dump.

Even though I allocated memory for DayRates , and WeekRates structures
inside RateInfo node while creating the tree, why Iam not able to
access contents of DayRates , and WeekRates structures when I mmap()
file into memory?

Any pointers would be a great help.

Thanks,
-Nikhil

 

[Walter Roberson]

I have a following structure:
typedef struct
{
RateData rdr;
int RateID;
char RateBalance;
} RateInfo;

typedef struct
{
DayRates *dr;
WeekRates *wr;
} RateData;

First I will create the memoefy for the DayRates, WeekRates structures
inside the RateInfo structure variable.

Once the total tree is contructed,
I will store this tree as file (rateinfo.map) on the disk.

In my next programming component, Iam going to load this file
(rateinfo.map) into memory using mmap() function call. Once I load the
file into memory using mmap(), Iam able to access the RateID and
RateBalance members of the RateInfo. But, Iam not able to access the
DayRates, and WeekRates structure contents.

It is not clear from what you wrote whether the "next programming
component" is running in the same process that created the stored tree,
and it is not clear as to whether you released the memory for the
tree between the two parts.

When you write out the tree to a file, you are writing the raw binary
pointer values (DayRates *)dr and (WeekRates *)wr. Those pointer values
are to active memory. If you release that memory, either by freeing it
or overwriting it or by exiting the process, then the referrants of
the pointers cease to exist. If you then bring back the binary values
via the mmap() and try to use the stored pointers, you are going to
be pointing to some random thing. You were lucky in a way to get a
segment fault, because that made it clear that you were doing something
wrong; it would have been worse if there hadn't been any fault and
you ended up using the random data.


In general, there is no way to store and restore pointers between
sessions. What people often do instead is to allocate one big
block of memory to start with, then use their own malloc() and free()
routines to ensure that memory related to that purpose is taken from
that block of memory. Then when it is time to write the data out, they
convert each pointer into an offset from the beginning of the data
structure. When they want to load the data structure back in again,
they reverse the process, allocating a block of memory, and converting
each stored offset into a pointer by adding in the address of the
new instance of the memory block.


There is sometimes another option, that is not portable in terms of
the C standard, but possible if you are willing to restrict execution
to a particular OS and architecture. In the variant I am thinking of,
possible in a number of unix-like systems [but not in C in general],
you can use mmap() to allocate new memory into a fixed virtual address.
You would then do your own malloc() and free() as per the above
(or possibly you might find an extension malloc library to do it for you).
When you finished constructing the tree, you would then save the mmap()
block to a file. Then to restore the tree into a different process on
the same kind of system, you would mmap() the file contents into the
exact same virtual address as was used originally. The pointers would
then all become valid because the contents and virtual addresses would
have been restored.

This option does, of course, assume that your operating system supports
virtual memory, and supports mmap() as a way of allocating new memory,
and supports mmap() of a file to a user- specified virtual address, and
assumes that whatever system the file is restored into has the
same representation of all the types (including pointers!) as the
data was created upon...

 

[nkrisraj]

Hi,

Actually, I have two components, I mean two different processes. The
first component creates the tree and writes it to a file. Once it
writes the tree to a file, the first compoent process exits.

The second component, will be launched as a seperate process. It takes
the file created by the first compoent, and does the mmap() it to the
memory. Once I load the
file into memory using mmap(), Iam able to access the RateID and
RateBalance members of the RateInfo. But, Iam not able to access the
DayRates, and WeekRates structure contents. When I try to access the
members of DayRates , and WeekRates structures, my program is showing
segmentation fault and Iam getting the core dump.

Thanks,
-Nikhil

 

[Eric Sosman]

Hi,

Actually, I have two components, I mean two different processes. The
first component creates the tree and writes it to a file. Once it
writes the tree to a file, the first compoent process exits.

The second component, will be launched as a seperate process. It takes
the file created by the first compoent, and does the mmap() it to the
memory. Once I load the
file into memory using mmap(), Iam able to access the RateID and
RateBalance members of the RateInfo. But, Iam not able to access the
DayRates, and WeekRates structure contents. When I try to access the
members of DayRates , and WeekRates structures, my program is showing
segmentation fault and Iam getting the core dump.

Aha! In your initial post you showed a struct
containing pointers. Did the first process, by any
chance, write the pointer values themselves to the
file, rather than the data they pointed to? Those
pointer values have no meaning in the second process,
and the pointed-to data has disappeared along with
everything else that was in the first process' memory.

I'm guessing, of course, since you haven't provided
a lot of detail. But if I've guessed correctly, you'll
need to re-think your file format.

 

[Walter Roberson]

Actually, I have two components, I mean two different processes.

As -I- just wrote the posting to you a few minutes ago, I know what you
are talking about... but other people won't. Please take the time to
quote appropriate context. google groups is giving you the illusion
that the context is easily visible, but most of the regulars here
(especially those who answer questions a lot) are -not- using google
groups. Please read http://cfaj.freeshell.org/google/

The first component creates the tree and writes it to a file. Once it
writes the tree to a file, the first compoent process exits.

The second component, will be launched as a seperate process. It takes
the file created by the first compoent, and does the mmap() it to the
memory. Once I load the
file into memory using mmap(), Iam able to access the RateID and
RateBalance members of the RateInfo. But, Iam not able to access the
DayRates, and WeekRates structure contents.

Are you certain of that? I strongly suspect that you -can- access those
structure contents. The structure contents are *pointers*, and I
believe that if you check, you will find that the -pointers- can be
read and have the same value that you stored in the first process.
I believe you will find that what you cannot access is that which
is pointed *to* . Which, as I explained in the previous posting, is
because the pointer values are essentially random garbage in the
context of the second process. I gave two solutions, one portable and
the other not, in the previous posting.

 

[Walter Roberson]

Aha! In your initial post you showed a struct
containing pointers. Did the first process, by any
chance, write the pointer values themselves to the
file, rather than the data they pointed to?

I'd be surprised if it turned out that he did -not- store the actual
pointer values and then tried to use the (now-broken) pointers as-is
in the second process.

Those
pointer values have no meaning in the second process,
and the pointed-to data has disappeared along with
everything else that was in the first process' memory.

With the exception, of course, of possibilities such as having mmap()'d
back into exactly the same virtual address space. Not generally portable,
but then mmap() is not generally portable.

 

[nkrisraj]

Hi,

In the second process, I can access the pointer addresses exactly, but
not the pointed values. I mean to say that the addresses of pointers,
DayRates* and WeekRates* are exactly proper as I can see in the first
process. But, I can not access the member values of DayRates* and
WeekRates* structures. When I tried to do that I will get the CORE
dump.

Thanks,
-Nikhil

 

[Walter Roberson]

In the second process, I can access the pointer addresses exactly, but
not the pointed values. I mean to say that the addresses of pointers,
DayRates* and WeekRates* are exactly proper as I can see in the first
process. But, I can not access the member values of DayRates* and
WeekRates* structures. When I tried to do that I will get the CORE
dump.

You still aren't quoting context
The URL describing how to do so with googlegroups was
http://cfaj.freeshell.org/google/


Resupplying context:

You are writing actual pointer values to a file, exiting that process,
starting another and reading the pointer values in from the file.
You happen to be using mmap() for that, but that's not what is causing
the problem: the problem is that those pointers are no longer valid
because the stored data is not being loaded back into the exact same
addresses as it was at when you saved the file.


To fix this problem, you can use either of the solutions that I
posted earlier: you can convert the pointers into offsets before you
write them out, and then convert them back from offsets when you
read them back in; OR [if your system supports it] you can
[non-portably] mmap() the contents back to the *exact* same address
location that the memory had originally... provided that you
took care to manage memory allocation into a block yourself in the
first place rather than just using malloc() and free() [which could
use memory all over your address space.]


Hmmm, now that I think about it, I imagine that in some cases
mmap() back to the same virtual location would not work either.
It is allowable for a "pointer" not to include the complete information
needed to locate the target memory. For example, a "pointer" might
internally be a bit-structure that has a segment number reference and
an offset, or might contain an implicit offset relative to a base
register. If you were to write that information out and pull it in to
a different process, then unless -somehow- the segment numbers or
base registers or whatever also happened to be set to the value
they had in the other process, then the pointers will fail in the new
process. If I understand correctly, x86 architectures may use segment +
offset format. Any architecture that allows access to more physical
memory than is implied by the native ABI pointer width could encounter
the same problem.

 

[Keith Thompson]

Hi,

I have a following structure:
typedef struct
{
RateData rdr;
int RateID;
char RateBalance;

} RateInfo;

[snip]

You've posted nearly the same article three times. A lot of people
have been doing this lately. Apparently groups.google.com has been
having some problems in the last few days (even more than their
usual problems; see <http://cfaj.freeshell.org/google/>).

If you post an article through Google and it doesn't show up
immediately, don't assume that you need to try again. You may just
need to wait a while. Posting again won't help; you'll just get two
copies of the article showing up whenever Google gets around to
posting.