Memory size?

[Joerg Schwerdtfeger]

Hi folks,

how can I determine the total main-memory size and the size of free
memory available in bytes?

I tried to use mallinfo() from malloc.h - resulting some strange values
in Windows (cygwin, gcc 3.3.1) and always 0 in Linux (2.4.22, gcc
3.3.2).

Thanks in advance,
Jörg

 

[osmium]

how can I determine the total main-memory size and the size of free
memory available in bytes?

I tried to use mallinfo() from malloc.h - resulting some strange values
in Windows (cygwin, gcc 3.3.1) and always 0 in Linux (2.4.22, gcc
3.3.2).

Use the API for your OS. Two OSes then two different solutions.

Be sure to find out what free memory *means* too, on a paging machine.

 

[Dan Pop]

how can I determine the total main-memory size and the size of free
memory available in bytes?

What do you need this information for? Do you realise that the
information about the free memory is extremely volatile and, by the time
you get a chance to use it, it may already be incorrect?

I tried to use mallinfo() from malloc.h - resulting some strange values
in Windows (cygwin, gcc 3.3.1) and always 0 in Linux (2.4.22, gcc
3.3.2).

No such function in the standard C library, so whatever results you got
were the correct ones (your program invoked undefined behaviour by
calling a function that was neither defined by it nor part of the standard
C library).

Dan

 

[Mark McIntyre]

Hi folks,

how can I determine the total main-memory size and the size of free
memory available in bytes?

there's no standard way to do that. You may not be permitted to by your OS.
It might not even be a meaningful question, in a multiuser virtual memory
system.

Ask in a group dedicated to your compiler and/or OS.

 

[Joerg Schwerdtfeger]

What do you need this information for? Do you realise that the
information about the free memory is extremely volatile and, by the time
you get a chance to use it, it may already be incorrect?

I need a data structure with exactly 2^i elements, where i is an element
of IN. With two conditions: 1. the structure must fit in the heap, 2.
the number of elements should be the maximum possible amount. I. e. if
1024 MB main memory is present and about 300 MB is in use, and
sizeof(element)=4 byte, the program should allocate memory for 2^27
Elements (512 MB).

Joerg

 

[Mike Wahler]

I need a data structure with exactly 2^i elements,

So define or allocate one. If your implementation cannot
handle the definition, it will (should) emit a diagnostic.

If your implementation's memory allocation function (e.g. 'malloc())'
cannot succeed, it will return NULL.

where i is an element
of IN.

What is 'IN'?

With two conditions: 1. the structure must fit in the heap,

C does not define 'heap'.

2.
the number of elements should be the maximum possible amount.

The C language cannot determine this in advance. All you can
do is try various sizes until it fails (and note that on a
multitasking system, this 'maximum possible amount' can easily
vary widely each time you try). What you're asking about is
in the domain of your operating system, not the C language.

I. e. if
1024 MB main memory is present and about 300 MB is in use,

Those are platform and OS issues, not C issues.

and
sizeof(element)=4 byte, the program should allocate memory for 2^27
Elements (512 MB).

This *calculation* can be easily done with C. *BUT*: the values
of 'main memory size' and 'memory in use' cannot be determined
with standard C. You'll need to use operating system specific
features. Check your documentation, and/or ask about this
in a forum about your platform.

-Mike

 

[Fred L. Kleinschmidt]

I need a data structure with exactly 2^i elements, where i is an element
of IN. With two conditions: 1. the structure must fit in the heap, 2.
the number of elements should be the maximum possible amount. I. e. if
1024 MB main memory is present and about 300 MB is in use, and
sizeof(element)=4 byte, the program should allocate memory for 2^27
Elements (512 MB).

Joerg

So why not just malloc() that amount? if the malloc returns NULL, you do
not have enough memory to solve the task.

 

[Dan Pop]

I need a data structure with exactly 2^i elements, where i is an element
of IN. With two conditions: 1. the structure must fit in the heap, 2.
the number of elements should be the maximum possible amount. I. e. if
1024 MB main memory is present and about 300 MB is in use, and
sizeof(element)=4 byte, the program should allocate memory for 2^27
Elements (512 MB).

Are you sure you understand what virtual memory is and how it works?

Dan

 

[CBFalconer]

how can I determine the total main-memory size and the size of
free memory available in bytes?

I tried to use mallinfo() from malloc.h - resulting some strange
values in Windows (cygwin, gcc 3.3.1) and always 0 in Linux
(2.4.22, gcc 3.3.2).

I can't answer for Linux or Cygwin, but for DJGPP you can use
nmalloc.zip and the malldbg module. mallinfo.h specifies the
interface, and an info source file nmalloc.txh documents it all.
Available at:

<http://cbfalconer.home.att.net/download/nmalloc.zip>

For all I know the source might function under Cygwin or Linux,
compiled with gcc, but that has not been verified. It sticks
quite closely to standard C, but has some deviations. To use it
you will have to link the malloc.o and malldbg modules before the
standard library.

Bear in mind that it will not show main-memory size etc. in these
days of virtual memory. It will show you how much you have used,
and how much you have freed.

 

[Dan Pop]

I can't answer for Linux or Cygwin, but for DJGPP you can use
nmalloc.zip and the malldbg module. mallinfo.h specifies the
interface, and an info source file nmalloc.txh documents it all.
Available at:

<http://cbfalconer.home.att.net/download/nmalloc.zip>

For all I know the source might function under Cygwin or Linux,
compiled with gcc, but that has not been verified. It sticks
quite closely to standard C, but has some deviations. To use it
you will have to link the malloc.o and malldbg modules before the
standard library.

Bear in mind that it will not show main-memory size etc. in these
days of virtual memory. It will show you how much you have used,
and how much you have freed.

I.e. something completely different from what the OP wants to know...

But who cares, as long as it gives you the opportunity to post yet another
piece of self advertising ;-)

Dan

 

[CBFalconer]

So why not just malloc() that amount? if the malloc returns NULL,
you do not have enough memory to solve the task.

Won't work. He can't do what he wants in portable standard C.
The reason being that many systems will happily return a valid
pointer when usage exceeds main memory, and then thrash data in an
out of disk storage when accessed. You detect all this as the
program slows to a crawl and the disk access light gets brighter.

 

[Mike Wahler]

Won't work. He can't do what he wants in portable standard C.
The reason being that many systems will happily return a valid
pointer when usage exceeds main memory, and then thrash data in an
out of disk storage when accessed.

So it depends upon what you mean by 'memory'. Does 'virtual'
(e.g. disk-paged) memory qualify? 'malloc()' doesn't depend
upon how/where the storage comes from, it simply either succeeds
or fails.

You detect all this as the
program slows to a crawl and the disk access light gets brighter.

Yes, there'd likely be a performance issue with 'virtual' memory schemes.

-Mike

 

[Richard Bos]

I need a data structure with exactly 2^i elements, where i is an element
of IN. With two conditions: 1. the structure must fit in the heap, 2.
the number of elements should be the maximum possible amount. I. e. if
1024 MB main memory is present and about 300 MB is in use, and
sizeof(element)=4 byte, the program should allocate memory for 2^27
Elements (512 MB).

So... even disregarding the existence of virtual memory, what should
happen if two of these programs are executed at _exactly_ the same time?

Richard

 

[Harti Brandt]

On Thu, 27 May 2004, Joerg Schwerdtfeger wrote:

JS>Dan Pop wrote:
JS>
JS>>>how can I determine the total main-memory size and the size of free
JS>>>memory available in bytes?
JS>> What do you need this information for? Do you realise that the
JS>> information about the free memory is extremely volatile and, by the
JS>time
JS>> you get a chance to use it, it may already be incorrect?
JS>
JS>I need a data structure with exactly 2^i elements, where i is an element
JS>of IN. With two conditions: 1. the structure must fit in the heap, 2.
JS>the number of elements should be the maximum possible amount. I. e. if
JS>1024 MB main memory is present and about 300 MB is in use, and
JS>sizeof(element)=4 byte, the program should allocate memory for 2^27
JS>Elements (512 MB).

Find out what the pointer size of your machine is (16, 32, 64), say w.
Then do a loop like:

w = w - 1 - log2_of_item_size;
ptr = NULL;
while (w > 0) {
ptr = malloc(1 >> w);
if (ptr != NULL)
break;
w--;
}

This should give you what you want for some definition of 'available'. On
a POSIX system you should be able to use all the memory you got. Be beware
that overcommitting systems like BSD and Linux may signal your program
when they run out of paging space (or just freeze like I oberseved on a
Linux system).

harti

 

[Richard Tobin]

Won't work. He can't do what he wants in portable standard C.
The reason being that many systems will happily return a valid
pointer when usage exceeds main memory, and then thrash data in an
out of disk storage when accessed.

It's worse than that. Many operating systems overcommit memory,
returning a valid pointer from malloc() and then killing the program
when it tries to access too much of it.

-- Richard

 

[Richard Bos]

It's worse than that. Many operating systems overcommit memory,
returning a valid pointer from malloc() and then killing the program
when it tries to access too much of it.

Quite apart from this rendering any C implementation on that platform
unconforming (after all, if malloc() succeeds, the Standard says you own
that memory), is there _any_ good reason for this practice? I'd have
thought simply telling your user that he can't have that much memory is
preferable to pretending that he can, and then unceremoniously dumping
him in it when he tries to use it. I mean, under OSes like that, what's
the use of all our precautions of checking that malloc() returned
succesfully?

Richard

 

[osmium]

Quite apart from this rendering any C implementation on that platform
unconforming (after all, if malloc() succeeds, the Standard says you own
that memory), is there _any_ good reason for this practice? I'd have
thought simply telling your user that he can't have that much memory is
preferable to pretending that he can, and then unceremoniously dumping
him in it when he tries to use it. I mean, under OSes like that, what's
the use of all our precautions of checking that malloc() returned
succesfully?

Until I see the name of the offending OS, I will take this to be an urban
legend.

 

[Harti Brandt]

On Tue, 1 Jun 2004, osmium wrote:

o>Richard Bos writes:
o>
o>> (e-mail address removed) (Richard Tobin) wrote:
o>>
o>> > In article <[email protected]>,
o>> >
o>> > >Won't work. He can't do what he wants in portable standard C.
o>> > >The reason being that many systems will happily return a valid
o>> > >pointer when usage exceeds main memory, and then thrash data in an
o>> > >out of disk storage when accessed.
o>> >
o>> > It's worse than that. Many operating systems overcommit memory,
o>> > returning a valid pointer from malloc() and then killing the program
o>> > when it tries to access too much of it.
o>>
o>> Quite apart from this rendering any C implementation on that platform
o>> unconforming (after all, if malloc() succeeds, the Standard says you own
o>> that memory), is there _any_ good reason for this practice? I'd have
o>> thought simply telling your user that he can't have that much memory is
o>> preferable to pretending that he can, and then unceremoniously dumping
o>> him in it when he tries to use it. I mean, under OSes like that, what's
o>> the use of all our precautions of checking that malloc() returned
o>> succesfully?
o>
o>Until I see the name of the offending OS, I will take this to be an urban
o>legend.

The orginal BSD VM did overcommitting and so do most BSD derivates. Try
to google for 'overcommit site:freebsd.org' for discussions.

harti

 

[CBFalconer]

Quite apart from this rendering any C implementation on that
platform unconforming (after all, if malloc() succeeds, the
Standard says you own that memory), is there _any_ good reason for
this practice? I'd have thought simply telling your user that he
can't have that much memory is preferable to pretending that he
can, and then unceremoniously dumping him in it when he tries to
use it. I mean, under OSes like that, what's the use of all our
precautions of checking that malloc() returned succesfully?

It is a matter of practicality. It allows the use of 'copy on
write' algorithms, for example, and the large economies they
provide. Another example is the program that wants a sparse
array, and implements it by mallocing a monster array. Only the
portions that are actually used are really assigned memory.

Under most normal circumstances the program will never encounter
any difficulties. However such events as filling of disc space
can prevent actually assigning virtual memory. Simply pausing the
program can lead to indefinite postponement and other nasties,
with nobody aware of any problem.

 

[Ross Kendall Axe]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Harti Brandt wrote:
| On Tue, 1 Jun 2004, osmium wrote:
|
| o>Richard Bos writes:
| o>
| o>> (e-mail address removed) (Richard Tobin) wrote:
| o>>
| o>> > In article <[email protected]>,
| o>> >
| o>> > >Won't work. He can't do what he wants in portable standard C.
| o>> > >The reason being that many systems will happily return a valid
| o>> > >pointer when usage exceeds main memory, and then thrash data in an
| o>> > >out of disk storage when accessed.
| o>> >
| o>> > It's worse than that. Many operating systems overcommit memory,
| o>> > returning a valid pointer from malloc() and then killing the program
| o>> > when it tries to access too much of it.
| o>>
| o>> Quite apart from this rendering any C implementation on that platform
| o>> unconforming (after all, if malloc() succeeds, the Standard says
you own
| o>> that memory), is there _any_ good reason for this practice? I'd have
| o>> thought simply telling your user that he can't have that much
memory is
| o>> preferable to pretending that he can, and then unceremoniously dumping
| o>> him in it when he tries to use it. I mean, under OSes like that,
what's
| o>> the use of all our precautions of checking that malloc() returned
| o>> succesfully?
| o>
| o>Until I see the name of the offending OS, I will take this to be an
urban
| o>legend.
|
| The orginal BSD VM did overcommitting and so do most BSD derivates. Try
| to google for 'overcommit site:freebsd.org' for discussions.
|
| harti

Linux as well by default, so this is no urban legend.

Ross
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFAvMwK9bR4xmappRARAtQtAJ45k8iItmlHo2H9kq6dbax1lQfv+gCg47ci
JkDft/IwvaSRNj9DL4SyAUA=
=YnWV
-----END PGP SIGNATURE-----