Data via serial port breaks down in multiple packets

[Muffinman]

Hello all,

I'm not sure whether this is the appropriate place for this question,
but I couldn't find a better place. So here it goes:

I have an application that needs to receive some data via a serial port.
The data is received, however, it breaks down in multiple packets. So
sending 18 bytes often results in two packets of 8 bytes and one of 2
(but sometimes also 8,9&1). Setting VMIN allows me to push it up to how
many bytes I want. However, sometimes I need to receive 4 bytes, other
times 25. It is not consistent, so that's no option.

I need to process commands.
- Processing each packet at a time would not work because it can break
down commands.
- I could fill my own buffer, but timing is quite crucial...

1) So, can I solve this? Is there anything I can do with my
configuration (see below)?

2) Why does this happen? The only thing I can think of is that the
sending device skips a opportunity to send data every so bytes...

I hope someone can help.

Thanks in advance, Maarten


################# serial port polling #################
if(poll_fds[0].revents & POLLIN)
{
bytes_read = read(serial_fd, device_status_message,
sizeof(device_status_message));
if(bytes_read <= 0)
continue;
[...]

############# serial port configuraton #################
/* c_cflag
* setting all the variables according to the configuration file
* and some default settings */
serial_new_attr.c_cflag |= (CLOCAL | CREAD);
if(common_data.serial_parity == 0)
serial_new_attr.c_cflag &= ~PARENB;
else
{
serial_new_attr.c_cflag |= PARENB;
/* enable parity check and strip it from output */
serial_new_attr.c_iflag |= (INPCK | ISTRIP);
if(common_data.serial_even == 0)
serial_new_attr.c_cflag |= PARODD;
else
serial_new_attr.c_cflag &= ~PARODD;
}

if(common_data.serial_2stop == 0)
serial_new_attr.c_cflag &= ~CSTOPB;
else
serial_new_attr.c_cflag |= CSTOPB;
serial_new_attr.c_cflag &= ~CSIZE;
serial_new_attr.c_cflag |= common_data.serial_bits;

/* c_lflag
* enable raw input */
serial_new_attr.c_lflag &= ~(ICANON | ECHO | ECHOE | ISIG);

/* Disabling hardware flow control */
serial_new_attr.c_cflag &= ~CRTSCTS;
/* Disabling software flow control */
serial_new_attr.c_iflag &= ~(IXON | IXOFF | IXANY);

/* Disabling any postprocessing */
serial_new_attr.c_oflag &= ~OPOST;

/* Minimum characters to read (non-canonical) */
serial_new_attr.c_cc[VMIN] = 1;
/* Wait indefinitely for input (non-canonical) */
serial_new_attr.c_cc[VTIME] = 3;

cfsetospeed(&serial_new_attr, common_data.serial_speed);
cfsetispeed(&serial_new_attr, common_data.serial_speed);

 

[James Kuyper]

Hello all,

I'm not sure whether this is the appropriate place for this question,
but I couldn't find a better place.

It isn't. The C standard says nothing about serial ports, read(),
cfsetispeed() or cfsetospeed(), nor about any of the macros or
structures you were using. Functions with those names are installed on
my desktop system, and the man pages for those functions describe them
as POSIX functions. If the functions you're using are the same ones,
then the best place to get answers to questions like this is
comp.unix.programmer.

 

[glen herrmannsfeldt]

On 08/09/2013 11:55 AM, Muffinman wrote:

It isn't. The C standard says nothing about serial ports, read(),
cfsetispeed() or cfsetospeed(), nor about any of the macros or
structures you were using.

But C does describe fread(), which you could use with serial ports,
or other data streams.

The Unix/C model for data is as a stream of bytes. If you write
data with a series of fwrite() calls, there is no reason to
expect the number of bytes read be each of a series of fread()
calls to return the same succession of record lengths.

There are systems that do preserve record lengths. Blocks on unix
tapes being one example. (I believe that comes through to fread(),
but right now I am not sure about that.)

If you need to preserve record marks, you might consider:

https://www.ietf.org/rfc/rfc1831.txt

-- glen

 

[James Kuyper]

But C does describe fread(), which you could use with serial ports,
or other data streams.

And it says nothing useful about what happens when you do so, or at
least, about how it's different from what happens when you open an
ordinary file. There's really no useful answer to his question that can
be derived from the C standard, not even if you re-write it to use
fread() rather than read().

And since Unix open() and ioctl() can enable unix features that can
cause read() to behave quite differently from the way it normally works
when invoked by fread() (consider, for example, blocking vs.
non-blocking I/O), it's generally not a good idea to simply assume that
unix code calling read() can simply be replaced by C code calling
fread(). I'm not sure whether that is or could be relevant in this case
- but that's precisely the point. There will be people in
comp.unix.programmer who would be sure.

 

[Joe Pfeiffer]

Hello all,

I'm not sure whether this is the appropriate place for this question,
but I couldn't find a better place. So here it goes:

I have an application that needs to receive some data via a serial port.
The data is received, however, it breaks down in multiple packets. So
sending 18 bytes often results in two packets of 8 bytes and one of 2
(but sometimes also 8,9&1). Setting VMIN allows me to push it up to how
many bytes I want. However, sometimes I need to receive 4 bytes, other
times 25. It is not consistent, so that's no option.

I need to process commands.
- Processing each packet at a time would not work because it can break
down commands.
- I could fill my own buffer, but timing is quite crucial...

1) So, can I solve this? Is there anything I can do with my
configuration (see below)?

2) Why does this happen? The only thing I can think of is that the
sending device skips a opportunity to send data every so bytes...

You're pretty much out of luck. The read() man page says,

On success, the number of bytes read is returned (zero indicates end of
file), and the file position is advanced by this number. It is not an
error if this number is smaller than the number of bytes requested;
this may happen for example because fewer bytes are actually available
right now (maybe because we were close to end-of-file, or because we
are reading from a pipe, or from a terminal),

So... you tell the system call how many bytes you want, it tells you
how many bytes you get. Lots of my code has loops like this:

numread = 0;
while (numread < numwanted) {
thisread = read(fd, &buf[numread], numwanted-numread);
if (thisread < 0) {
perror("read at line xxx");
break;
}
numread += thisread;
}

Asking in a unix-specific newsgroup certainly couldn't hurt.

 

[Michael Angelo Ravera]

Hello all,
I'm not sure whether this is the appropriate place for this question,
but I couldn't find a better place. So here it goes:

I have an application that needs to receive some data via a serial port.
The data is received, however, it breaks down in multiple packets. So
sending 18 bytes often results in two packets of 8 bytes and one of 2
(but sometimes also 8,9&1). Setting VMIN allows me to push it up to how
many bytes I want. However, sometimes I need to receive 4 bytes, other
times 25. It is not consistent, so that's no option.
I need to process commands.

- Processing each packet at a time would not work because it can break
down commands.

- I could fill my own buffer, but timing is quite crucial...



1) So, can I solve this? Is there anything I can do with my

configuration (see below)?



2) Why does this happen? The only thing I can think of is that the

sending device skips a opportunity to send data every so bytes...



I hope someone can help.



Thanks in advance, Maarten





################# serial port polling #################

if(poll_fds[0].revents & POLLIN)

{

bytes_read = read(serial_fd, device_status_message,

sizeof(device_status_message));

if(bytes_read <= 0)

continue;

[...]



############# serial port configuraton #################

/* c_cflag

* setting all the variables according to the configuration file

* and some default settings */

serial_new_attr.c_cflag |= (CLOCAL | CREAD);

if(common_data.serial_parity == 0)

serial_new_attr.c_cflag &= ~PARENB;

else

{

serial_new_attr.c_cflag |= PARENB;

/* enable parity check and strip it from output */

serial_new_attr.c_iflag |= (INPCK | ISTRIP);

if(common_data.serial_even == 0)

serial_new_attr.c_cflag |= PARODD;

else

serial_new_attr.c_cflag &= ~PARODD;

}



if(common_data.serial_2stop == 0)

serial_new_attr.c_cflag &= ~CSTOPB;

else

serial_new_attr.c_cflag |= CSTOPB;

serial_new_attr.c_cflag &= ~CSIZE;

serial_new_attr.c_cflag |= common_data.serial_bits;



/* c_lflag

* enable raw input */

serial_new_attr.c_lflag &= ~(ICANON | ECHO | ECHOE | ISIG);



/* Disabling hardware flow control */

serial_new_attr.c_cflag &= ~CRTSCTS;

/* Disabling software flow control */

serial_new_attr.c_iflag &= ~(IXON | IXOFF | IXANY);



/* Disabling any postprocessing */

serial_new_attr.c_oflag &= ~OPOST;



/* Minimum characters to read (non-canonical) */

serial_new_attr.c_cc[VMIN] = 1;

/* Wait indefinitely for input (non-canonical) */

serial_new_attr.c_cc[VTIME] = 3;



cfsetospeed(&serial_new_attr, common_data.serial_speed);

cfsetispeed(&serial_new_attr, common_data.serial_speed);

This comes up wether you are reading a serial port, a socket, a terminal, or some other device (even text with funny delimiters from a disk file).

If you need to process data, you must fill a buffer until you have what the command requires and be prepared to save the leftovers for the next time.

If you can't tell where a command ends, you are badly stuck. But most things have either a length designator or a delimiter of some kind.

There are two main techniques:
One is to read data (perhaps a byte at a time) until you have a command and then read the amount of data (perhaps a byte at a time) required for that command (dealing with leftovers, if you somehow read too much).

The other is to read as much data as you can get comfortably and to parse through it (and either keeping the buffer filled or dealing with getting more whenever you aren't sure that you have enough).

It doesn't matter whether you use read (), read_ (), _read (), __read (), fread (), pread (), ioread (), Read (), READ (), recv (), recvfrom (), portread (), readport (), or GimmeSomeLovin ().

 

[Muffinman]

1) So, can I solve this? Is there anything I can do with my
configuration (see below)?

2) Why does this happen? The only thing I can think of is that the
sending device skips a opportunity to send data every so bytes...

I get a few explanations why this might happen from here and
comp.unix.programer. However, the end result seems to be: deal with it.
So I guess, that's the advice I'm gonna go with. Not sure how, but I'll
manage.

Thanks all, Maarten

 

[glen herrmannsfeldt]

I get a few explanations why this might happen from here and
comp.unix.programer. However, the end result seems to be: deal with it.
So I guess, that's the advice I'm gonna go with. Not sure how, but I'll
manage.

In the serial case, each byte is pretty much independent, but it also
happens in the case of TCP. Datagram boundaries are preserved for UDP,
but TCP is a stream protocol.

I previously posted the way it is done when protocols switch from UDP
to TCP, such as NFS.

-- glen