Java Serializing in C/C++?

[James Kanze]

You seem to be agreeing with me. It wouldn't make sense for a
read on a pipe to block until all N requested bytes are
available. Consider e.g. what happens if the final piece of
data is M < N. It has to return M.

In the classical implementation of pipes (under Unix V7, for
example), a read from a pipe blocked until either the requested
number of bytes were read, or all writers had closed the
connection (the file descriptor). That handles the final piece
of data, but it means that you can easily hang in between.

The classical way of handling this, if you couldn't get the
length otherwise, was to read one byte (so you'd block if
nothing was available), then do an fstat to find out how many
more were there, and read them.

 

[coal]

In some cases, almost certainly. *IF* the Linux behavior is
typical for Unix pipes today, *AND* your messages are guaranteed
to be smaller than the maximum size write to a pipe that is
guaranteed to be atomic (no idea what that is today---it was 4K
the one time I looked at it, but that was in the 1980's), then
reading all that is available and processing all the messages in
it is probably the way to go. In other contexts, it probably
depends. I don't think that there is one absolute rule. (TCP
puts the message length in the header, but that's a very special
context.)

I read somewhere the pipe buffer is around 10K today. I also read
that Unix domain sockets on Linux don't have such a limit.
http://preview.tinyurl.com/32qs3p If possible I would prefer
the sockets, but I'm not sure about performance differences
between the two. Anyway, though, I think it make sense to focus
on sockets first.

In the classical implementation of pipes (under Unix V7, for
example), a read from a pipe blocked until either the requested
number of bytes were read, or all writers had closed the
connection (the file descriptor).  That handles the final piece
of data, but it means that you can easily hang in between.

The classical way of handling this, if you couldn't get the
length otherwise, was to read one byte (so you'd block if
nothing was available), then do an fstat to find out how many
more were there, and read them.

According to R. Stevens ioctl() works on pipes too for Unix
implementations. Probably using ioctl is more portable than fstat to
determine how much data is available.

Brian Wood

 

[James Kanze]

According to R. Stevens ioctl() works on pipes too for Unix
implementations. Probably using ioctl is more portable than fstat to
determine how much data is available.

Stevens is probably talking about modern Unix; I don't think
ioctl worked on pipes under V7. On the other hand, fstat has
always worked. (Posix and the Open Unix Standard don't seem to
guarantee either.)