Freeing memory - will it be available immediately

[Herbert Rosenau]

Correct, it may. It may _not_, however, fail because the implementation
chose to hand the memory freed back to the OS. The standard forbids
this, meaning that a failure in a subsequent allocation (of the same or
lesser amount of memory) is at best a QoI issue, at worst a bug, whereas
handing the memory back to the OS such that the OS might re-use it is not
merely a QoI issue or a bug, but a non-conforming behaviour.

No, the standard requires only explicity that the block is not blocked
from further allocation. It does not forbid that free gives the block
back to the OS because that would explicity forbid that free is
nothing than only a wrapper to an OS API that does exactly that. So
where in the standard is mentioned that the whole malloc family is not
only wrappers to OS API? Chapter and verse, please.

Not quite the point, though, is it? It knows the memory _will_ be
available, whereas if the memory is handed off to the OS for use outside
the scope of the program, the guarantee is violated.

There is no guarantee in the standard that freed memory is really
available again on next allocation of the same program. There is only
a requirement that the freed block is available for further allocation
- saying that the block is now unblocked. free() unblocks and the next
request allocs it, so further allocation is done.

--
Tschau/Bye
Herbert

Visit http://www.ecomstation.de the home of german eComStation
eComStation 1.2R Deutsch ist da!

 

[Herbert Rosenau]

Yes, malloc() and free() can be parts of a strictly conforming
program. But if a program output depends on whether implementation
gives memory back to the OS, then it's not strictly conforming.
See 1) and 2).

Where in the stadndard is the requirement that the whole malloc family
is not only a number of wrappers around OS APIs?

--
Tschau/Bye
Herbert

Visit http://www.ecomstation.de the home of german eComStation
eComStation 1.2R Deutsch ist da!

 

[Harald van Dijk]

No, the standard requires only explicity that the block is not blocked
from further allocation. It does not forbid that free gives the block
back to the OS

unless giving the block back to the OS does not make it "available for
further allocation".

Standard library functions can only be implemented as wrappers for system
calls if those system calls meet the requirements of the standard library
functions.

 

[Keith Thompson]

Where in the stadndard is the requirement that the whole malloc family
is not only a number of wrappers around OS APIs?

Obviously there's no such requirement; malloc, free, and friends are
simply required to behave in accordance with the standard's
requirements. I don't think anyone has implied otherwise.

Suppose the OS provides a system call sys_alloc(). If sys_alloc()
happens to satisfy the requirements for malloc(), then malloc() can be
a simple wrapper. (Or, for efficiency, malloc() might use various
techniques to avoid the overhead of a system call on each malloc()
call.) But if sys_alloc(), say, doesn't guarantee that the allocated
chunk is properly aligned, then malloc() *can't* be a simple wrapper.

Note that, whether the standard *allows* free() to return storage to
the OS or not, it certainly doesn't *require* it to do so. An
implementation in which free() always hoards allocated memory for the
exclusive use of the current program would be conforming. (Whether it
should is, at least partly, a QoI issue.)

 

[Herbert Rosenau]

Since the very thing whose behaviour is being defined is the C program,
it is the *only* relevant context in which to define the behaviour.

Where in the standard is defined that "make available by further
allocation" does not only means that free has to unblock the memory
now freed? To disallow to give the memory back to the OS there must be
something in the standard that does not explicity allow that. So where
in the standadard is required that this is the case? The standard
requires so many explicity and it woud have done that for free too -
but there is nothing about that.

The standard requires only that the memory shoulb be available by
further allocation - to any who requires it.

Except that this ignores the very reason for the standard, which is *not*
to define behaviours of "any program".

But it requires that the memory is unlocked by making it available for
further allocation, not blocking it anymore from that. That's all. It
would say explicity that it is not allowed to make the memory
available for futher allocation by another program - but is requires
the further allocation absolute.

The standard lets the implementation simply the freedom to extend the
'further allocation' to other programs because it can neither forbid
nor require 'by other program' because it can't know if or if not the
implementation is able to do so.

On the contrary, it is perfectly reasonable, as they have yet to give any
justification for this magical intrusion of the rest of the world into
the operation of free, contrary to the dictates of the standard, and
apparently alone of all the standard library functions.

So long you can't prove by chapter and verse where the standard
forbids what it even requires 'make available for further allocation'
you are definitevely wrong by saying thet the standard forbids 'make
available by further applicatin'. It can not do so without saying that
explicity.

The fact they cannot justify this means they cannot claim their position
*is* justified, therefore they have no argument.

So again, how about *showing* I'm wrong, rather than simply hand-waving?
Show me where, in the standard's definition of free, it allows *any* form
of implementation-defined behaviour or the like which allows free to
return the memory back to the OS in the manner you describe.

It is you who says that 'make available for further allocation' says
'make not available for further allocation'.

You can't. It's not there. What is there is a guarantee that the memory
is made available for further allocation, a guarantee made in defining
the behaviours and expectations of a C program.

The explicit requirement 'make available for further allocation' means
simple 'make available for further application' and not block it from
that.

Do, please, show me the clause which allows the behaviour you describe,
as it's not to be found in the definition of free. Nor anywhere else as
far as I can tell. You've obviously found such a clause, why can't you
quote it?

You cites it: 'make available....' without any when and if, simply you
must do it but not you should do it if but never if this or that or
this or that not then - make it!

--
Tschau/Bye
Herbert

Visit http://www.ecomstation.de the home of german eComStation
eComStation 1.2R Deutsch ist da!

 

[Herbert Rosenau]

unless giving the block back to the OS does not make it "available for
further allocation".

Standard library functions can only be implemented as wrappers for system
calls if those system calls meet the requirements of the standard library
functions.

So, where in the standard is defined that 'make available for further
allocation' does not mean that it means exactly what the wording 'make
available' without any when or if means exactly that and not 'make
availaable but not when the requirtement comes from....'?

So it is the freedom of the implementation to give the memory back to
the OS as it was borrowed from. Yes it does not require that the block
has to make available immediately and not even in the same size and
even not immediately to the next one. It lefts it to free when it will
give it back and lets it even define conditios internally when it
means that it should give it back to the OS. The standard requires
only 'make available for futher allocation nothing mor, nothing less.

--
Tschau/Bye
Herbert

Visit http://www.ecomstation.de the home of german eComStation
eComStation 1.2R Deutsch ist da!

 

[Kelsey Bjarnason]

[snips]

Yes, malloc() and free() can be parts of a strictly conforming program.
But if a program output depends on whether implementation gives memory
back to the OS, then it's not strictly conforming. See 1) and 2).

It does *not* depend on whether the implementation gives memory back to
the OS; it depends on whether the implementation behaves as guaranteed by
the standard. If *that* makes a program non-conforming, there's no such
thing as a strictly conforming program; they *all* depend on the library
behaving as defined by the standard.

 

[Kelsey Bjarnason]

[snips]

No, the standard requires only explicity that the block is not blocked
from further allocation.

No, it says the block is made available for further allocation.

It does not forbid that free gives the block
back to the OS

No, it doesn't; it merely forbids it doing so in a manner which allows
the OS to consume the memory. The memory, by the standard, *must be*
available for further allocation; the mechanics of doing this are an
implementation detail. If the implementation can do this by handing the
memory back to the OS but somehow "tagging" it as "reserved", so be it.

There is no guarantee in the standard that freed memory is really
available again on next allocation of the same program.

No, not necessarily the next allocation. It does, however, mandate that
the following loop actually terminates:

void *ptr = malloc(1000);
/* assuming malloc worked... */
free( ptr );
ptr = NULL;

while ( 1 )
{
ptr = malloc(100);
break;
}

Since the memory *is* made available for further allocation, the above
loop *must* terminate, barring a fairly serious implementation design
flaw in its memory manager. What it absolutely *cannot* fail to do is
loop forever because the implementation gave the memory back to the OS
without some way to guarantee the memory is made available to the program
again.

 

[Kelsey Bjarnason]

[snips]

So, where in the standard is defined that 'make available for further
allocation' does not mean that it means exactly what the wording 'make
available' without any when or if means exactly that and not 'make
availaable but not when the requirtement comes from....'?

The meaning "made available for further allocation" *does* mean exactly
what it says, which is the point. It does not say "might be available
for further allocation, if the OS or another process hasn't stolen it."

The C standard defines the behaviour of C programs, part of which is
defining the behaviour of the C standard library in a manner those
programs can rely on. In defining this, it includes a definition of the
behaviour of free, a C standard library function, which a C program is
_supposed_ to be able to rely upon working as defined - and that
definition says the memory is "made available for further allocation".
It's a _C_ library function guarantee to a _C_ language program. Free
either meets this guarantee or is broken.

 

[Kelsey Bjarnason]

[snips]

Yes, no!. Increment and decrement is well specified by the standard. For
variables are not pointers it is always 1.

[snip irrelevant crud having absolutely nothing to do with the case at
hand, which involved variables of type int]

No, it does this only partially. It does not specify - should a single
block coming back from the program

[snip irrelevant crud which is purely implementation details, having
nothing to do with the guarantees offered by the definition of free]

It only requests explicity that a block that comes in from free should
NOT holded in use anymore but has to be given free for further
allocation for any c|m|realloc from who ever.

No, not "from who ever". From the C program, the very thing the standard
is defining the behaviour of.

You can't have it both ways. Either the standard defines the behaviour
of a C program (and attendant details such as the C standard library) in
which case "who ever" doesn't apply, the *only* candidate for
consideration is the C program in question, or the standard does *not*
define the behaviour of a C program, in which case we may as well scrap
it as it is no use whatsoever in determining how a C program is supposed
to work.

More formally, it defines the C language, rather than the specific
program, but the effect is the same: when writing a program, one is
*supposed* to be able to deduce, by comparing the code to the standard,
the intended behaviour at least of the constructs involved; obviously the
actual data being processed will alter the program flow, but again in a
manner which is supposed to be deterministic, based upon the standard.

If that is true, if the C standard defines the C language, defines the
intended behaviour of the constructs within a C program and thus of the
program itself, then the definition of free, being part of that standard,
defines how free is supposed to work *for the C program*. It does, and
that definition does *not* allow the memory to be reallocated by
"whomever"; it *requires* it to be reallocatable by the C program.

Again, you can't have it both ways; either it defines the behaviour of
the C program, in which case there is *no* leeway given for "whomever",
or it doesn't define the behaviour of the C program, so we'll have to
look elsewhere to see how C programs are intended to behave. What would
you suggest? A VB reference? The ADA standard? Something else?

 

[Kelsey Bjarnason]

[snips]

And this has *what* to do with the price of tea in China?

free() may return memory from the C program to the OS

Only if it can guarantee the OS will somehow tag that memory as
reserved for the C program. Otherwise, it cannot. Read the definition
of free; there is no allowance for handing the memory back to the OS in
the general case, as that would risk making the memory *not* available
for further allocation.

Proof by repetition?

One can only hope that if it is repeated often enough, the folks who seem
to think free is somehow, apparently alone of all the functions in the
entire library, not required to behave as defined, will eventually clue
in that no, sorry, the standard *does* define its behaviour and gives
*no* leeway for alternate interpretations.

I know, it gets tedious, but other than explaining it to them over and
over and over until it finally sinks in, I see no other way to get them
to grasp this rather simple point.

 

[Kelsey Bjarnason]

[snips]

In contrary to your false assumption there is nothing in the standard
that forbids to give back borrowed memory to the OS and request it from
there again when needed.

Actually, the wording of the standard forbids it. If you disagree,
please point out any part of the definition of free which allows
implementation-defined or unspecified behaviour which would allow free to
behave as you describe.

You can't; it's not there. What is there is a black-and-white guarantee,
made by the C standard, about the C standard library, to the C program
and C programmer: the memory *is* made available for further allocation.

You're arguing that the C standard is *not* in fact defining the
behaviour of the C standard library, that we're free to add in any random
additional crud we feel like, whenever we want, as long as it's not
*explicitly* forbidden by the standard.

So, for example, nothing in the standard *explicitly* forbids the
following from printing "Yes!":

int x = 3;

if ( ++x == 179 )
puts( "Yes!" );

There's nothing in the standard which explicitly forbids this is there?
No, there isn't. If you don't believe me, feel free to check and quote
C&V which forbids this.

No, what the standard does is defines the behaviour that *is* supposed to
- even guaranteed to - happen. That *defined* behaviour is what prevents
the above printing "Yes!". Just as the *defined* behaviour is what
prevents free returning the memory to the OS; the definition of free
*does not allow it* unless the implementation can *guarantee* the memory
will be made available to the program again - which, in the general case,
it cannot, meaning in the general case it *must* not return the memory to
the OS.

 

[Kelsey Bjarnason]

[snips]

Fact is that the standard says nothing about program, nothing about an
OS, nothing about real memory, nothing about files, nothing about any
real device. It defines only an abstract mashine and its behavior from
sight of C.

Correct. Part of that definition is how free behaves, in a manner in
which a C program can rely upon. What is that definition? Oh, yes, the
memory is made available for further allocation.


Thank you for making my point so clearly.

 

[Kelsey Bjarnason]

[snips]

Whether or not the interpretation opposing yours is reasonable, as I've
already pointed out elsethread (hoping, futilely apparently, that it
would finally kill this thread), the standard's _authors_ have declared
it as being the acceptable one. Therefore, whether reasonable or not,
that is how this text _must_ be interpreted.

One may assume, then, that when any question arises about the meaning of
the standard, there will always be a standard committee member standing
by to answer such questions?

No, obviously not. All we have to go on is the standard itself. The
standard which mandates the behaviour I describe. Later versions -
C2010, for example - may define the behaviour otherwise, but for C89/90
and C99 as of now, the behaviour is fixed: free *cannot*, in the general
case, release the memory to be made available to the OS or other
processes. There is *nothing* in the text which even *possibly* allows
for this behaviour.

 

[Harald van Dijk]

It does, however, mandate that
the following loop actually terminates:

void *ptr = malloc(1000);
/* assuming malloc worked... */
free( ptr );
ptr = NULL;

while ( 1 )
{
ptr = malloc(100);
break;
}

I think you missed an "if" there.

void *ptr = malloc(1000);
if (ptr) { /* this isn't the one I meant, but let's check anyway */
free(ptr);
ptr = NULL;
while (1)
{
ptr = malloc(100);
if (ptr != NULL) /* this is the one I meant */
break;
}
}

I'm not convinced that this loop must terminate, but I cannot find a
sensible argument that (on a conforming implementation) it might not.

 

[Kelsey Bjarnason]

[snips]

Exactly, it defines the behavior of strictly conforming programs. So
provide an example of a strictly conforming program whose behavior can
be affected by malloc() giving memory back to OS

The example relies upon the implementation behaving as defined by the
standard. You're arguing that a program is not strictly conforming
because an *implementation* does something other than the standard
requires.

By this argument, there is no such thing as a strictly conforming
program, as one can trivially create an implementation which violates the
requirements of the standard on any arbitrary portion of the program's
operation.

It's silly, spurious, specious argument.

 

[Kelsey Bjarnason]

No. You cannot be shown anything while your eyes and mind are closed.

Goodbye.

"Goodbye", as an indicator of a complete and abject failure to make a
point, is an indicator of an open mind and open eyes?

 

[Kelsey Bjarnason]

[snips]

I have been sitting on my hands with this thread, but I can't any
longer. Your basis for this is that the standard only considers one
program, that there is no "OS" so how else can one interpret "being
available"?

No, that's not my basis; check the thread.

You miss the fact that there *is* another entity here, but only by
implication. It seems reasonable to me to interpret "available for
further allocation" as returning the memory to the place it came from.

And nobody said anything about "to the place it came from".

malloc does not create the memory, it "allocates" it.

Yes, and?

This implies a
sort of change of ownership from some other entity (the OS, of course)
to the program.

Assuming there is an OS; there need not be.

An allocation is an assignment, from one entity to
another. Making memory "available for further allocation" could be
taken to mean the memory reverts to the state (of ownership) that it had
before.

it could be, except that the standard forbids it. The defined behaviour
of free is the defined behaviour of a C standard library function as
relied upon by a C program. The definition of that function, upon which
the program relies, says that the memory *is* made available for further
allocation. Not "may be, if the OS hasn't handed it off somewhere else",
and there is *no* clause for such implementation-defined behaviour
anywhere in the definition.

 

[Kelsey Bjarnason]

[snips]

The committee has discussed this issue in the past and the consensus was
that returning the memory to the OS *does* make it available for future
allocation.

Obviously it does; it simply does so in a manner not allowed by the
standard, which defines the behaviour of free in a manner the C program
can rely upon: in this case, that the memory *is* made available for
further allocation.

If the memory is handed back to the OS, then it might never be available
for further allocation to the C program and the guarantee made by the
definition of free is violated.

 

[Gordon Burditt]

The committee has discussed this issue in the past and the consensus was

Obviously it does; it simply does so in a manner not allowed by the
standard, which defines the behaviour of free in a manner the C program
can rely upon: in this case, that the memory *is* made available for
further allocation.

If the memory is handed back to the OS, then it might never be available
for further allocation to the C program and the guarantee made by the
definition of free is violated.

Not true if the OS only hands the memory to programs the OS can
guarantee the OS can get the memory back from (this would include
non-C programs that can be subject to a OOM killer).