writing to file

[ishekar]

Hi,

I have an application where i want to write data to a file, the data is
being sent from an external source. I know the total size of the data and
then i retrieve the data in small segments from the source. This data is
written to file in a loop.

My question.
1. Will it be useful to increase the file size initially then seek to 0 and
start writing to file. whether there will be any performance improvements
because of the allocation of the file earlier and due to less io? or this
doesnt matter because every fwrite writes to disk(OS may schedule writing to
disk).
2. If this is helps performance, do i have an api to initialise the file
with some size and empty strings?

TFI
ishekara.

 

[Rolf Magnus]

Hi,

I have an application where i want to write data to a file, the data
is being sent from an external source. I know the total size of the
data and then i retrieve the data in small segments from the source.
This data is written to file in a loop.

My question.
1. Will it be useful to increase the file size initially then seek to
0 and start writing to file. whether there will be any performance
improvements because of the allocation of the file earlier and due to
less io? or this doesnt matter because every fwrite writes to disk(OS
may schedule writing to disk).

This is very platform specific, so you might want to ask in a newsgroup
about your OS. I think that usually, the performance when writing is
better, since the kernel doesn't need to search for free clusters on
the disk. But of course, creating the file initially will take time,
too. So if you need your hard drive to be as fast as possible so you
can keep up with a high data rate, it's probably a good idea to create
the file initially. If not, you won't probably gain anything or even
make the total run time worse due to the additional "initialization" of
the file.

2. If this is helps performance, do i have an api to initialise the
file with some size and empty strings?

Exept for just writing it yourself (which is actually very simple),
there is none in standard C++.

 

[John Harrison]

Hi,

I have an application where i want to write data to a file, the data is
being sent from an external source. I know the total size of the data and
then i retrieve the data in small segments from the source. This data is
written to file in a loop.

My question.
1. Will it be useful to increase the file size initially then seek to 0 and
start writing to file. whether there will be any performance improvements
because of the allocation of the file earlier and due to less io? or this
doesnt matter because every fwrite writes to disk(OS may schedule writing to
disk).
2. If this is helps performance, do i have an api to initialise the file
with some size and empty strings?

TFI
ishekara.

The answer to this sort of question is always

1) Do it the simplest way first (in this case don't precreate the file).
2) Once the program is working to some profiling to see if the file I/O
speed is a problem or not. If not then you've saved yourself the bother of
writing the more complex code.
3) If it is a problem then try your alternate method, profile that and see
if it is any faster. Bear in mind that the results on one computer may not
be reflected on another computer.

There is no way in standard C++ of creating a file so big, except writing
out the required number of bytes. Your operating system may have a more
efficient way of doing this.

john

 

[Nick Hounsome]

Hi,

I have an application where i want to write data to a file, the data is
being sent from an external source. I know the total size of the data and
then i retrieve the data in small segments from the source. This data is
written to file in a loop.

My question.
1. Will it be useful to increase the file size initially then seek to 0 and
start writing to file. whether there will be any performance improvements
because of the allocation of the file earlier and due to less io? or this
doesnt matter because every fwrite writes to disk(OS may schedule writing to
disk).
2. If this is helps performance, do i have an api to initialise the file
with some size and empty strings?

1. Keep it simple - don't try to optimize until you know there is a problem
2. There is no way in C++
3. In unix like OSes the function you want is truncate or possibly ftruncate
4. On many file systems it will make no difference because although it will
say that the
file is so big it doesn't necessarily allocate the blocks until you
write to them.
5. The fastest way will almost certainly to use truncate and then map the
file into memory with mmap as it
will remove a layer of copying.
6. I think you really should just use ofstream