Mixing size_t and other types

[Navaneeth]

Hello,

I have a variable

long int size;

Value of this variable will be the size of a file which I am trying to
load into memory. Now I need to allocate memory using malloc. If I
write

malloc(size);

compiler warns about "conversion to ‘size_t’ from ‘long int’ may
change the sign of the result". So I added an explicit cast like

malloc((size_t) size);

I am not sure that this is the correct approach. Is this casting OK?
How can I find out the value of size_t?

Any help would be great!

 

[Ian Collins]

Hello,

I have a variable

long int size;

Value of this variable will be the size of a file which I am trying to
load into memory. Now I need to allocate memory using malloc. If I
write

malloc(size);

compiler warns about "conversion to ‘size_t’ from ‘long int’ may
change the sign of the result". So I added an explicit cast like

malloc((size_t) size);

I am not sure that this is the correct approach. Is this casting OK?
How can I find out the value of size_t?

Strangely enough, size_t is commonly used for sizes, so just make size a
size_t and save to agro.

size_t will be an unsigned type.

 

[Navaneeth]

Strangely enough, size_t is commonly used for sizes, so just make size a
size_t and save to agro.

size_t will be an unsigned type.

Thanks. But this variable holds a value that comes from the standard
function "ftell". It'd be something like,

fseek (fp , 0 , SEEK_END);
size = ftell (fp); /* ftell gives me long int. */
rewind (fp);

So I think I won't be able to make it a size_t. Not sure how to go
about this. Any help?

 

[Malcolm McLean]

malloc((size_t) size);

I am not sure that this is the correct approach. Is this casting OK?
How can I find out the value of size_t?

if (size > 0 && size < SIZE_MAX)
ptr = malloc((size_t) size);
else
ptr = 0;
if(ptr == 0)
/* out of memmory */

 

[Malcolm McLean]

Strangely enough, size_t is commonly used for sizes, so just make size a
size_t and save to agro.

It's even more commonly used for counts, and even more commonly still
for indexes.

 

[Nobody]

size_t will be able to hold any required value that ftell can return.

This isn't true.

There are platforms where size_t is only 16 bits wide but ftell() can
return any signed 32-bit value.

There may well be platforms (although I can't name one) where size_t is 32
bits wide but where ftell() can return larger values.

Typically, size_t is based upon the size of the address space, while
the maximum return value from ftell() will either be the maximum size of a
file or the maximum value which a "long" can hold (e.g. most modern 32-bit
platforms allow files larger than 2 GiB but only provide a 32-bit long,
so you have to use functions beyond those provided by the C standard to
operate upon large files).

 

[chrisbazley]

I can suggest you to read the page at this link:

https://www.securecoding.cert.org/confluence/display/seccode/FIO19-C....

I hope you'll find it helpful.

I'm not sure that I'd describe it as 'helpful'. It basically says
"There is no portable method of determining the size of a file."
Great! :-(

Still a very pertinent link though.

 

[Navaneeth]

Thank you all. I understood the problems with ftell(). But my question
was more about mixing the types. Whatever approach I follow, I get a
long as a file size. Now 'malloc' expects 'size_t'. If I pass this
long to it, compiler generates a warning. In my application, we
consider warnings as errors. So I am unable to continue with the
compilation.

My question was, to avoid the warning, can I cast the long to size_t?
Is that safe?

 

[chrisbazley]

Thanks. But this variable holds a value that comes from the standard
function "ftell". It'd be something like,

fseek (fp , 0 , SEEK_END);
size = ftell (fp); /* ftell gives me long int. */
rewind (fp);

So I think I won't be able to make it a size_t. Not sure how to go
about this. Any help?

You mustn't assign the return value of 'ftell' to a variable of type
'size_t' because it may return -1L on error. In that case you'd end up
with a nonsensical file position value, like assigning the return
value of 'fgetc' to a 'char' variable (another common mistake).

 

[Navaneeth]

It looks like precalculating the filesize is not a good idea. A better
approach would be to read each charater and allocate necessary memory
using "realloc". So I don't have all these size_t and other type
mixing issues.

 

[Tim Rentsch]

Thanks. But this variable holds a value that comes from the standard
function "ftell". It'd be something like,

fseek (fp , 0 , SEEK_END);
size = ftell (fp); /* ftell gives me long int. */
rewind (fp);

So I think I won't be able to make it a size_t. Not sure how to go
about this. Any help?

You can make 'size' be type 'unsigned long'. The error return
for 'ftell()' is -1, which can still be tested for in 'size'
by using, eg,

if( size == -1LU ) ...

and also use 'size' in the malloc() call without warnings about
signedness.

 

[Tim Rentsch]

Thanks. But this variable holds a value that comes from the standard
function "ftell". It'd be something like,

fseek (fp , 0 , SEEK_END);
size = ftell (fp); /* ftell gives me long int. */
rewind (fp);

So I think I won't be able to make it a size_t. Not sure how to go
about this. Any help?

You mustn't assign the return value of 'ftell' to a variable of type
'size_t' because it may return -1L on error. In that case you'd end up
with a nonsensical file position value, [snip]

Right in the sense that the type 'size_t' can be too narrow,
but 'unsigned long' should work just fine, since 'ftell()'
returns a plain 'long'. The error indicator value of -1
can be tested for in an 'unsigned long size' using a regular
equality comparison (ie, == or !=).

 

[Eric Sosman]

Thank you all. I understood the problems with ftell(). But my question
was more about mixing the types. Whatever approach I follow, I get a
long as a file size. Now 'malloc' expects 'size_t'. If I pass this
long to it, compiler generates a warning. In my application, we
consider warnings as errors. So I am unable to continue with the
compilation.

My question was, to avoid the warning, can I cast the long to size_t?
Is that safe?

It's safe, provided you know that the `long' value is (1) not
less than zero and (2) not greater than SIZE_MAX (which you can write
as `(size_t)-1' for older compilers). Under those conditions, the
conversion from `long' to `size_t' will preserve the original value.

I'll presume you can already vouch for (1), because you'll only
get a negative value from ftell() to indicate an error -- and if you
encountered an error, you'd have bailed out before trying to allocate
memory for the file. If (2) doesn't hold you're out of luck, because
there is no way to allocate a single chunk of more than SIZE_MAX bytes;
that is, you'll have to treat (2) failure as an error condition, very
much as if the malloc() call returned NULL.

Finally, about warnings: Compilers are allowed to issue as many
or as few warnings as they please (in addition to whatever diagnostics
the Standard actually requires). Many well-regarded compilers will
issue warnings for perfectly valid C code like `if (x = y) exit(0);'
or `defualt: return -1;'. Sometimes you may be able to silence the
warning by changing the code -- but there's a catch! The modification
that silences one compiler may provoke warnings from another compiler
that used to be quiet! Ultimately, then, the "no warnings" policy is
untenable -- you may be able to stick to it if you restrict yourself
to a small set of compilers (and versions), but somewhere there'll be
a compiler that complains about something innocuous and simply won't
shut up about it. "No warnings" is a good goal, as long as it's
understood that the goal can be approached but never fully attained.

 

[Malcolm McLean]

It makes no sense for a size to be negative so size_t is unsigned.

The problem is that intermediate calculations which eventually yield a
size might well produce negatives.

 

[Nobody]

My question was, to avoid the warning, can I cast the long to size_t?
Is that safe?

No; the result may exceed the range of a size_t.

 

[Nobody]

I'm not sure that I'd describe it as 'helpful'. It basically says
"There is no portable method of determining the size of a file."
Great! :-(

Knowing that a particular question has no answer can save you wasting a
lot of time looking for the answer.

It's also worth remembering that there may not even be a *non-portable*
method for determining the size of a file, either. E.g. Unix treats lots
of things as files, and some of them don't even *have* a size.

And also that "size at this exact moment" isn't necessarily the same thing
as "size N microseconds later when I actually read() it". DOS won't truly
be dead until the last ex-DOS programmer finally realises this.

 

[Ian Collins]

Navaneeth ha scritto:

I can suggest you to read the page at this link:

https://www.securecoding.cert.org/c...k()+and+ftell()+to+compute+the+size+of+a+file

I hope you'll find it helpful.

There's also the small matter of files with a size > LONG_MAX.

 

[BGB / cr88192]

Not that small actually

95% of the time though, one knows when they are likely to be dealing with a
large file, and for the rest of the time, they can simply regard files
larger than a certain "reasonable" size to be invalid...

this is like, for example, assuming that the size of a JPEG or COFF object
is < 16MB.
yes, there *could* be a 16MB JPEG or COFF object passed in, but for sake of
convinience one can assume that it doesn't exist, or check the size and
refuse to accept it if it exceeds a certain limit.

it is still a fairly safe bet at the moment that most files are less than
2GB or 4GB, and so it is a reasonable enough working assumption for most
things.

 

[BGB / cr88192]

It looks like precalculating the filesize is not a good idea. A better
approach would be to read each charater and allocate necessary memory
using "realloc". So I don't have all these size_t and other type
mixing issues.

in the cases where the long/size_t issue will actually matter, malloc and
realloc will likely fail as well...

note that expandable buffer sizing is a problem to get right (unless one
knows how to do it well), as doing it poorly will typically fragment memory
and cause poor performance.

my usual strategy though is to always just expand the buffer by 50% (from
its current size) whenever it needs to expand, which typically works well
enough.

 

[Keith Thompson]

It looks like precalculating the filesize is not a good idea. A better
approach would be to read each charater and allocate necessary memory
using "realloc". So I don't have all these size_t and other type
mixing issues.

A reasonable compromise is to use some (possibly unreliable)
method to determine the size of the file, but be prepared to for
the initial estimate to be wrong. If the file is likely to grow,
start by adding a fudge factor to the initial estimate.

You can also use some system-specific method to get the file's
current size, possibly surrounded by a maze of #ifdefs.