malloc family in C++

[Niklas Norrthon]

No rule without exceptions...
Last week I used malloc in C++ for the first time in the 15 years I've
been playing with the language... I had a buffer which I needed to
shrink every once in a while, so I saved lots of CPU cycles switching
from new[] and delete[] to malloc, realloc and free.

Someone replied, suggesting using vector<>::resize(). But resize just
calls erase which causes destructors of contained objects to be called,
but it does not release any memory held by the vector. My contained
objects were unsigned chars, so destructors or not was not an issue.

Then the suggestion came to use the vector<>::swap() trick, to
return any memory not used. At this point the people over next
door had had enough (and who can blame them?).
So now I move over here...

My problem was:
A large array (a megabyte or two) of unsigned char gets populated by
a library function. Then the data in the array gets processed, and
after a chunk is used it is not needed anymore, so there is no need
to hold on to that memory any longer if the chunk not needed is at the
end of the array (which happens about 25% of the time).

My first attemt was to use a vector, and the swap trick to return
unused memory. There were two problems with this approach: While
swapping twice the memory is allocated, and the entire vector had
to be copied every time.

The solution was of course to switch to malloc and realloc instead.

Now it is functioning and fast. The calls to the C memory management
functions take place deep inside a class, so no other program logic
was affected at all. Only difference is a much smaller memory foot
print, and a faster program.

/Niklas Norrthon

 

[Pete Becker]

The solution was of course to switch to malloc and realloc instead.

Of course it was. <g> Don't let language bigots dictate how you write
code. Sometimes C does things better than C++.

 

[E. Mark Ping]

The solution was of course to switch to malloc and realloc instead.

In our company we had a need for something similar, but wanted the
benefits of vector, so we wrote pod_vector<>, which uses
malloc/realloc, has a "shrink_to_fit" member function and doesn't call
constructors or destructors.

 

[Roland Pibinger]

On Wed, 26 Oct 2005 18:15:04 +0000 (UTC),

In our company we had a need for something similar, but wanted the
benefits of vector, so we wrote pod_vector<>, which uses
malloc/realloc, has a "shrink_to_fit" member function and doesn't call
constructors or destructors.

Unfortunately, not even Boost knows a portable way to determine if a
type is a POD or not:
http://www.boost.org/doc/html/boost_typetraits/reference.html#boost_typetraits.is_pod

Best wishes,
Roland Pibinger

 

[Howard Hinnant]

The solution was of course to switch to malloc and realloc instead.

Now it is functioning and fast. The calls to the C memory management
functions take place deep inside a class, so no other program logic
was affected at all. Only difference is a much smaller memory foot
print, and a faster program.

I'm glad it worked for you.

Question: realloc doesn't really guarantee anything. It can silently
refuse to release the extra memory. It can also internally copy to a
smaller block, just like you had with vector. If you had a better
standard C tool that would either do it, or tell you it couldn't, would
you use it? I.e. would you want to know of an impending failure to
shrink the block in place, and therefore have the opportunity to do
something else?

I'm asking because I'm proposing such a tool. And your use case might
be good evidence to support such a tool. Here is the proposal:

http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1085.htm

This proposal adds a few more functions to the malloc/realloc family,
one of which is specifically charged with reducing (or expanding) the
size of a block *in-place* if it can:

int resize_alloc(void* ptr, size_t size_requested,
size_t* size_received);

-Howard

 

[E. Mark Ping]

Unfortunately, not even Boost knows a portable way to determine if a
type is a POD or not

Our portable way is to have developers choose the container after
profiling, understanding that the objects held will be treated as POD.
That works pretty well.