memory problem

[Sean Bartholomew]

size = myTextFile.tellg();
myTextFile.seekg (0, ios::beg);
char buffer[size+1];
myTextFile.read (buffer, size);
myTextFile.close();
buffer[size] = '\0';

the above code works great and always worked great.
now, i tried to apply it to a 27MB text file instead of the 300K one
it was used to doing and i get a fault exception. i am 99% positive
its the size of the file. which makes sense. but my entire program
depends of the buffer containing the entire text for searching and
extraction and parsing purposes.
is there a way around this where i could search direct from the drive
instead of a buffer?

 

[Alf P. Steinbach]

* Sean Bartholomew:

size = myTextFile.tellg();
myTextFile.seekg (0, ios::beg);
char buffer[size+1];

Note that this uses a _non-standard_ extension from C99. C++ does
not have dynamic size arrays. In C++ just use a std::vector<char>.

If you're using g++ it's possible that explicitly specifying c++
and "pedantic" mode will help you catching those things, or there
might be other compiler options.

myTextFile.read (buffer, size);
myTextFile.close();
buffer[size] = '\0';

the above code works great and always worked great.

What's the declaration of 'size'?

Not that it should cause the described misbehavior, but also, do
you seek to the end before 'tellg', i.e. previous to shown code?

now, i tried to apply it to a 27MB text file instead of the 300K one
it was used to doing and i get a fault exception.

The most probable cause is that you're trying to allocate 27 MiB on
the stack. When you use std::vector, as you should, those 27 MiB will
instead be allocated on the heap, which has much more room. Also, if
allocation fails you'll then get a std::bad_alloc exception (I think).

Less likely: if the stream is in an error state then 'size' ends up as
-1, which could then be reinterpreted as a huge unsigned value. And if
by any chance 'size' is a 'short' it could end up negative anyway. For
that matter, unlikely is it is it _could_ happen with 'size' as 'int',
on an old compiler -- but (1) stack -> heap, and (2) check for error.

Unfortunately the standard streams do not really support using exceptions
to report errors (if you turn it on you get an exception when _encountering_
end-of-file), but you could make a stream wrapper that translates failures
to exceptions, and then use that wrapper instead of a "raw" stream.

 

[Dave Townsend]

Just a guess, but perhaps your stack doesn't allow you to have
27 megs of space ? Perhaps you can allocate buffer with the
free store instead ( like buffer = new char[27meg] )