Marshalling an array of Structures

[Animesh]

Hi All,

I don't know whethher this is possible or not. This is the result of a
bad design problem.

Here I go; I have a structure like this:

typedef struct _s_index_entry {
char *doc_id;
double s_id;
} S_INDEX_ENTRY;

And I have mappings of a keyword and a corresponding array of
S_INDEX_ENTRYs described above or:

S_INDEX_ENTRY **sie; which is the case. The length of the double
pointer is also not fixed.

I need to save it to a BerkeleyDB or send it to a client TCP socket. So
I need to marshall all the data and do the needful. I have done
marshalling on a structure but am completely at a loss for marshalling
this array of structures. Is it possible to do this through C.

Also is it possible to marshal a quee of structures which seems to be
another way to get my data accross. Save the array of structures as a
queue of structures and do the code, but that also seems similar.

Thanks in Advance,
-Animesh

 

[Alexei A. Frounze]

....

typedef struct _s_index_entry {
char *doc_id;
double s_id;
} S_INDEX_ENTRY; ....
I need to save it to a BerkeleyDB or send it to a client TCP socket. So

I hope you don't send *pointers* in the network packets as on the other PC
they will all be considered broken unless somehow you manage to align memory
allocation on both sending and receiving PCs... Send raw data and indices
into arrays, but not the pointers, pointers are meaningless on the other PC.

Alex

 

[Animesh]

That's what - I cannot send pointers and hence I need to marshall the
entire data which could be packed off as a data stream over TCP.

When I have a single structure to be passed, its OK and something like:
http://mm.ilug-bom.org.in/pipermail/linuxers/Week-of-Mon-20050905/021783.html

I want to do similar stuff for an array of Structures. Any help or
pointers.

Regards,
Animesh

 

[Eric Sosman]

Hi All,

I don't know whethher this is possible or not. This is the result of a
bad design problem.

Here I go; I have a structure like this:

typedef struct _s_index_entry {
char *doc_id;
double s_id;
} S_INDEX_ENTRY;

And I have mappings of a keyword and a corresponding array of
S_INDEX_ENTRYs described above or:

S_INDEX_ENTRY **sie; which is the case. The length of the double
pointer is also not fixed.

I need to save it to a BerkeleyDB or send it to a client TCP socket. So
I need to marshall all the data and do the needful. I have done
marshalling on a structure but am completely at a loss for marshalling
this array of structures. Is it possible to do this through C.

Also is it possible to marshal a quee of structures which seems to be
another way to get my data accross. Save the array of structures as a
queue of structures and do the code, but that also seems similar.

<off-topic>

If you're using some kind of standard marshalling protocol,
the sort of thing that RMI implementations rest on, you'll most
likely find that the problem of encoding a sequence of objects
has already been solved. Check the documentation. Do this
first before looking for "pure C" solutions: C doesn't have any
so you'll need to roll your own, but if you can take advantage
of something that's already been written and debugged you'll
be well ahead of the game.

</off-topic>

You already have some way of encoding "Here comes a
struct foo" or "Here comes an int" in your outgoing data
stream. It might be an explicit encoding involving type
codes and such, or perhaps it's implicit in the organization
of the data stream. In any event, you say you've already
got suitable conventions for encoding one struct instance.

To encode a sequence, you need just a little more
"decoration" beyond what you've already put together. This
would probably take one of these forms:

- A prefix meaning "Here comes a sequence of N struct foo
instances," followed by N encoded structs.

- A prefix meaning "Here comes a sequence of N somethings,"
followed by the N encoded somethings, each introduced by
its own "Here comes a struct foo" or "Here comes a
struct bar" prefix. More verbose than the first method,
but also more flexible.

- A prefix meaning "Here comes a sequence of struct foo
instances," followed by the encoded structs, followed
by a suffix meaning "That was the last one."

- The same verbosity- and flexibility-increasing variation:
"Here comes a sequence of somethings" followed by the
encoded somethings, each with its own prefix, followed
by a suffix meaning "That's all, folks!"

Choose the method (or variation) that fits most comfortably
with the encoding protocol you're already using for single
objects.

 

[Animesh]

Thanks Eric,

- A prefix meaning "Here comes a sequence of N struct foo
instances," followed by N encoded structs.

This is what I ave been doing, though the prefix contains the number of
elements in the array of structures. I need to first complete the
Berkeley DB Addition then I could go ahead with the Network I/O.

For the BDB part there are more issues with byte alignment which I am
currently working on.

Regards,
Animesh