Pipe: FILE(16bits) to FILRE(8,16,24bits)

[Dario de Judicibus]

I need to read a file as BINARY by 16 bits units, and write to another file
as BINARY by 8, 16 and 24 bits units. It is pratically a pipe. I'm looking
for a sample of C++ code simple, performing and portable. Possibly elegant
;-)))))) Thank you in advance.

Dario de Judicibus

 

[Karl Heinz Buchegger]

I need to read a file as BINARY by 16 bits units, and write to another file
as BINARY by 8, 16 and 24 bits units. It is pratically a pipe. I'm looking
for a sample of C++ code simple, performing and portable. Possibly elegant
;-)))))) Thank you in advance.

Aehm. You may not have expressed yourself correctly. But reading
a file in 16 bit chunks and writing the read data in 8 bit chunks
should produce the exact same file.
Writing in 24 bit chunks can add some bytes to the end of the file
if the original filesize is not divisable by 24, but otherwise
the created file is again identical to the original.

So what is your problem ?

 

[Dario de Judicibus]

Aehm. You may not have expressed yourself correctly. But reading
a file in 16 bit chunks and writing the read data in 8 bit chunks
should produce the exact same file.
Writing in 24 bit chunks can add some bytes to the end of the file
if the original filesize is not divisable by 24, but otherwise
the created file is again identical to the original.

So what is your problem ?

Well, I was probably too short. What I meant is to build a pipe that reads a
file by 16 bit chunks, do some encoding, and writes back to another file
variable length chunks (8, 16 and 24 bits). Of course there is encoding, but
I know how to do it. What I was looking for is the most performing and
elegant way to do it with the following constraint:

1. it should work on any platform
2. it should work for Little and Big Endian
3. it should be "int-size" independent

There are a lot of ways to read/write files in C++, cin/cout with
rederection, iostreams, file open/close, and many others. Too many. That's
why I ask some guru to show me, poor mortal, the light. ;-)

DdJ

 

[Thomas Matthews]

Well, I was probably too short. What I meant is to build a pipe that reads a
file by 16 bit chunks, do some encoding, and writes back to another file
variable length chunks (8, 16 and 24 bits). Of course there is encoding, but
I know how to do it. What I was looking for is the most performing and
elegant way to do it with the following constraint:

1. it should work on any platform
2. it should work for Little and Big Endian
3. it should be "int-size" independent

There are a lot of ways to read/write files in C++, cin/cout with
rederection, iostreams, file open/close, and many others. Too many. That's
why I ask some guru to show me, poor mortal, the light. ;-)

DdJ

Probably the most portable method is to read data into an input area,
apply endianism, apply encoding, throw into an output area.

The most efficient method is to read a large chunk into an input area,
process the data then throw into an output area. When reading large
chunks, just make them multiples of 16-bits. This method is more
efficient since you are keeping the stream flowing. Many streams
have a "ramp-up" time, which is minimalized for larger chunks of
data. This method allows the process to be split into three
asychronous tasks: input, process, output.

--
Thomas Matthews

C++ newsgroup welcome message:
http://www.slack.net/~shiva/welcome.txt
C++ Faq: http://www.parashift.com/c++-faq-lite
C Faq: http://www.eskimo.com/~scs/c-faq/top.html
alt.comp.lang.learn.c-c++ faq:
http://www.raos.demon.uk/acllc-c++/faq.html
Other sites:
http://www.josuttis.com -- C++ STL Library book

 

[Dario de Judicibus]

Probably the most portable method is to read data into an input area,
apply endianism, apply encoding, throw into an output area.

The most efficient method is to read a large chunk into an input area,
process the data then throw into an output area. When reading large
chunks, just make them multiples of 16-bits. This method is more
efficient since you are keeping the stream flowing. Many streams
have a "ramp-up" time, which is minimalized for larger chunks of
data. This method allows the process to be split into three
asychronous tasks: input, process, output.

Do you have a sample skeleton for building a pipe? Just READ and WRITE. I'll
add encoding myself, of course ;-)

DdJ

 

[Thomas Matthews]

Do you have a sample skeleton for building a pipe? Just READ and WRITE. I'll
add encoding myself, of course ;-)

DdJ

#include <stdio.h>
#include <stdlib.h>

#define INPUT_BUFFER_SIZE 2048 /* 1024 * 16-bits */
#define OUTPUT_BUFFER_SIZE 4096

unsigned char input_buffer[INPUT_BUFFER_SIZE];
unsigned char output_buffer[OUTPUT_BUFFER_SIZE];

int main(void)
{
unsigned char * p_input;
unsigned char * p_input_end;
unsigned char * p_output;
size_t bytes_read;
size_t bytes_written;
size_t bytes_to_write;
bytes_read = fread(input_buffer, 1, INPUT_BUFFER_SIZE, stdin);
p_input_end = input_buffer + bytes_read;
p_output = output_buffer;
p_input = input_buffer;
while (bytes_read)
{
while (p_input != p_input_end)
{
/* The Process_Data function will increment the */
/* p_output pointer. */
Process_Data(p_input, &p_output);
bytes_to_write = &output_buffer[OUTPUT_BUFFER_SIZE]
- p_output;
if (bytes_to_write < 3 /* 24 bits */)
{
bytes_written = fwrite(stdout,
output_buffer,
1,
bytes_to_write);
if (bytes_written != bytes_to_write)
{
return EXIT_FAILURE;
}
p_output = output_buffer;
}
p_input += 2 * sizeof(char); /* 16-bit increment */
}
p_input = input_buffer;
bytes_read = fread(input_buffer, 1,
INPUT_BUFFER_SIZE, stdin);
}
return EXIT_SUCCESS;
}

In the above skeleton, I have not checked to see if the
freads were successful.

The program fills up an input buffer using data from the
standard input stream (i.e. pipe). The data in the
buffer is processed until the end of the buffer is
reached or the number of bytes read. The processing
function updates the output buffer pointer (after writing
the data to the buffer). If the output buffer is near
full, it is written to the standard output stream (possible
pipe).

You should tweak the size of the buffers to find the most
efficient size for your platform.

The above program has not been compiled nor tested. Any
errors are for the reader correct. The program is just
an illustration of a concept.

--
Thomas Matthews

C++ newsgroup welcome message:
http://www.slack.net/~shiva/welcome.txt
C++ Faq: http://www.parashift.com/c++-faq-lite
C Faq: http://www.eskimo.com/~scs/c-faq/top.html
alt.comp.lang.learn.c-c++ faq:
http://www.raos.demon.uk/acllc-c++/faq.html
Other sites:
http://www.josuttis.com -- C++ STL Library book

 

[Dario de Judicibus]

bytes_written = fwrite(stdout,
output_buffer,
1,
bytes_to_write);

Thank you. Just a question: isn't fwrite synopsis

size_t fwrite(const void *ptr, size_t size, size_t n, FILE *stream);

? I thout stdout should go to the end. Is it a typo or there are
compiler-dependent versions of fwrite? I use BCC.

DdJ

 

[Karl Heinz Buchegger]

Thank you. Just a question: isn't fwrite synopsis

size_t fwrite(const void *ptr, size_t size, size_t n, FILE *stream);

? I thout stdout should go to the end. Is it a typo or there are
compiler-dependent versions of fwrite? I use BCC.

Having a newsgroup doesn't free you from looking in your documentation
fwrite() is standard and yes, it's a typo (most regulars reply by typing
their knowledge from head, so prepare for some easy to fix typos).

 

[Dario de Judicibus]

Having a newsgroup doesn't free you from looking in your documentation
fwrite() is standard and yes, it's a typo (most regulars reply by typing
their knowledge from head, so prepare for some easy to fix typos).

No offense was intended. C++ is a great and powerful language, but after
year you often discover with surprise something new, especially when you
work on different platforms. I wished just to be sure. That's all.

DdJ

 

[Thomas Matthews]

Thank you. Just a question: isn't fwrite synopsis

size_t fwrite(const void *ptr, size_t size, size_t n, FILE *stream);

? I thout stdout should go to the end. Is it a typo or there are
compiler-dependent versions of fwrite? I use BCC.

DdJ

Yes, my bad.
One of my issues with the C streams library is that some functions
the stream is first, others it is last.

As I said, I didn't compile it, it was from the top of my head.

--
Thomas Matthews

C++ newsgroup welcome message:
http://www.slack.net/~shiva/welcome.txt
C++ Faq: http://www.parashift.com/c++-faq-lite
C Faq: http://www.eskimo.com/~scs/c-faq/top.html
alt.comp.lang.learn.c-c++ faq:
http://www.raos.demon.uk/acllc-c++/faq.html
Other sites:
http://www.josuttis.com -- C++ STL Library book