serialization of structure into a raw memory block

[Alfonso Morra]

I have the following data types:

typedef union {
long l ;
double f;
char* s ;
void* p ;
} Value ;

typedef struct {
int id ;
char *label
Value value ;
size ;
}StructA ;


Does anyone know how I can serialize StructA into a raw memory block and
calculate the number of bytes of the memory block?

Someone out there should surely know how to do this - if even they don't
know how to do it, but can point me to a reference (my Google searches
have not come up with anything remotely useful) - I'd be most grateful

mtia

 

[Moonlit]

Hi,

I would do it as follows:

At a serialize data function (in your case taking a class with void* and
writeptr to keep track of the written bytes or char* I usually use an
ostream istream for it).

Serialize( const CMyStream& MyStream )
{
if( MyStream.Write )
{
memcpy( MyStream.BasePtr + MyStream.Written, &l, sizeof l );
MyStream.Written += sizeof( l );
memcpy( MyStream.BasePtr + MyStream.Written, &l, sizeof f );
MyStream.Written += sizeof( f );


}
else
{
// Copy the other way around
}

}

// TODO: You probably don't want another class accessing the data members of
MyStream directly i.e. hide stuff
// Optional: Take care of little endian big endian conversion
// Optional: Take care of different sizes of longs for different
architectures
// Optional overload << operators for every type


I myself usually use ostream and istream which allows me directly to write
to files (ofstream derived) or memory (stringstream derived) or just to the
console ostream itself
--


Regards, Ron AF Greve

http://moonlit.xs4all.nl

 

[John Harrison]

I have the following data types:

typedef union {
long l ;
double f;
char* s ;
void* p ;
} Value ;

typedef struct {
int id ;
char *label
Value value ;
size ;
}StructA ;


Does anyone know how I can serialize StructA into a raw memory block and
calculate the number of bytes of the memory block?

Someone out there should surely know how to do this - if even they don't
know how to do it, but can point me to a reference (my Google searches
have not come up with anything remotely useful) - I'd be most grateful

You cannot serialize pointers. You can only serialize what they are
pointing to. You haven't told us what your pointers are pointing to so
it's hard to answer the question. I can guess what the char* pointers
point to but the void* pointer is worrying me.

john

 

[John Harrison]

Does anyone know how I can serialize StructA into a raw memory block and
calculate the number of bytes of the memory block?

This part of your question is easy though. Use an ostringstream.

ostringstream buf;
buf.write( ... );
buf.write( ... );
buf.write( ... );
buf.write( ... );
....
string block = buf.str();
block.data(); // the raw data
block.size(); // the size of the raw data

 

[benben]

You cannot serialize pointers. You can only serialize what they are

pointing to. You haven't told us what your pointers are pointing to so
it's hard to answer the question. I can guess what the char* pointers
point to but the void* pointer is worrying me.

john

If the char* pointer is not worrying me, it certainly is bothering me,
because you can tell if that is a char array or a sole pointer to a single
char.

Of course void* is heaps worse.

Ben

 

[Moonlit]

Err, remove const of course

--


Regards, Ron AF Greve

http://moonlit.xs4all.nl

 

[Alfonso Morra]

This part of your question is easy though. Use an ostringstream.

ostringstream buf;
buf.write( ... );
buf.write( ... );
buf.write( ... );
buf.write( ... );
...
string block = buf.str();
block.data(); // the raw data
block.size(); // the size of the raw data

 

[Alfonso Morra]

This part of your question is easy though. Use an ostringstream.

ostringstream buf;
buf.write( ... );
buf.write( ... );
buf.write( ... );
buf.write( ... );
...
string block = buf.str();
block.data(); // the raw data
block.size(); // the size of the raw data

Hi John,

This is the most useful response I have had so far (and that includes
responses from the c.lang ng).

The char* are ptrs to C (null terminated) strings. You can ignore the
void* for now. I just want to get an overview of how to go about
streaming a structure with nested structures/pointers. An example using
a struct that contains a char* alone will suffice. For example, how
would I stream this struct:

struct named_pair_ {
int id ;
char* key ;
}

typedef struct {
int id ;
char *label1 ;
char *label2;
struct named_pair_ *np ;
} structA ;

Could you please elaborate with a little bit more code?. I'm not
familiar with ostringstream and references I have looked up for the
write() method, do not map easily to the "code" you provided above.

mtia

 

[Mirek Fidler]

You cannot serialize pointers. You can only serialize what they are

A little bit off-topic: You can, with some effort:

http://upp.sourceforge.net/src$PtePtr$en-us.html
http://upp.sourceforge.net/reference$Ptr.html

Mirek

 

[ma740988]

Here's something I came up with that mirrors what John Harrison is
referring to with respect to ostringstream. The getBuffer function has
since been re-written to use a vector but this might provide some
clues. Of course the experts will comment if I'm way off so ...

#ifndef DATA_STREAM_H
#define DATA_STREAM_H

# include <iostream>
# include <string>
# include <map>
# include <typeinfo>
# include <sstream>


///////////////////////////////////////////////////
// data_stream
///////////////////////////////////////////////////
struct data_stream
{
std::string classId;
std::string buffer;
size_t curPos;

data_stream(std::string clsId)
: classId(clsId), curPos(0)
{}

data_stream(char* buf, size_t size)
: curPos(0)
{
std::string temp(buf, size);
size_t pos = temp.find(':');
if (pos == std::string::npos || pos == 0 || pos == size-1)
return;
classId = temp.substr(0, pos);
buffer = temp.substr(pos+1);
}

bool isValid() { return curPos <= buffer.length(); }
operator bool() { return isValid(); }

size_t getBufferSize() { return isValid()? classId.size() + 1 +
buffer.size() : 0; }

size_t getBuffer(char*& buf, size_t siz)
{
size_t sizNeeded = getBufferSize();
size_t sizClass = classId.size();
if (buf == NULL || siz < sizNeeded)
{
delete buf;
buf = new char[sizNeeded];
}
memcpy(buf, classId.c_str(), sizClass);
buf[sizClass] = ':';
memcpy(&buf[sizClass+1], buffer.c_str(), buffer.size());
return sizNeeded;
}
};

template <class T>
inline data_stream& operator<<(data_stream& ds, const T& t)
{
std:stringstream out;
out<<t;

ds<<out.str();

return ds;
}

template <class T>
inline data_stream& operator>>(data_stream& ds, T& t)
{
if (ds.isValid()) {
std::string s;
ds>>s;
std::istringstream in(s);
in>>t;
}

return ds;
}

inline data_stream& operator<<(data_stream& ds, const std::string& s)
{
if (ds)
{
ds.buffer += s;
ds.buffer += '\0';
ds.curPos = ds.buffer.size();
}
return ds;
}

inline data_stream& operator>>(data_stream& ds, std::string& s)
{
if (ds)
{
s = &ds.buffer[ds.curPos];
ds.curPos += s.size();
if (ds.curPos < ds.buffer.length() &&
ds.buffer[ds.curPos] == '\0')
ds.curPos++; // add terminating zero
}
return ds;
}

#endif


Depending on wethere I'm transmitting or receiving data..... Lets
assume I'll transmit data.

std::string buffer("served_data:"); // test data
buffer+="2";
buffer+='\0';
buffer+="5.0";
buffer+='\0';
buffer+="Terlim bom bom";
buffer+='\0';

data_stream st (&buffer[0], buffer.length());
if (st)
{
//
}