how to malloc 2G ram? in freebsd, win2k

[Guest]

hi
i try to malloc 2G size mem . so i wirte code like that.
it failt in freebsd 5.3 and win2k

what's bug in my code ? how can i change it ?

thank all
benjiam


///////////////////////////////////////////////
#include <iostream>
using namespace std;

#include <cstdlib>
const int len = 2000;
int main()
{
char* p1[len];
static int size = int (1<<20);
cout << size << endl;
for (int i =0 ; i < len ;i++)
{
p1 = new char [size];
cout << size << endl;
char *tp = p1;
for ( int x =0 ; x< (size-1); x++)
{
//cout << x << endl;
*(tp+x) = char( (x%26)+ 48);

}
*(tp+size-1) =0;
cout << "p [" << i << "]ok" << endl;
}

for (int i =0 ; i < len ;i++)
{
delete [] p1;
}
return 0;
}
////////////////////////////////////////////

 

[Martin Ambuhl]

hi
i try to malloc 2G size mem . so i wirte code like that.

You have no guarantee that you can do so. You need to check in a
newsgroup, mailing list, or tech support for your implementation.

it failt in freebsd 5.3 and win2k
what's bug in my code ? how can i change it ?

We really can't help, partially because you posted to comp.lang.c, where
we use C, and your code is not C. comp.lang.c++ is a different newsgroup
for a different language.

Examples of C++isms are:

#include <iostream> This is not a C header.
using namespace std;

This is a compilation error.

#include <cstdlib> This is not a C header.
cout << size << endl;

'cout' and 'endl' are undeclared identifiers, and '<<' is a
left-shift operator that probably don't do what you want.

p1 = new char [size];

'new' is an undeclared identifier, and the above is a compilation error.

There are more instances, but I don't care to be tiresome.

 

[Walter Roberson]

: i try to malloc 2G size mem . so i wirte code like that.
:it failt in freebsd 5.3 and win2k

:what's bug in my code ? how can i change it ?

And how are you compiling? It is fairly common for there to be a 2 GB
process size limit unless you specifically compile to use 64 bit
pointers. Yes, in theory with 32 bits you -could- get to 4 GB instead
of 2 GB, but it is not uncommon for half of the memory to be reserved
for system pointers.

Even within the 2 GB limit, it is common for there to be address space
layout issues: it is common for your code not to be placed at address 0
in virtual memory, and it is common for there to be an area reserved
for the stack, and it is common for there to be an area reserved for
the heap, and it is common for there to be one or more areas reserved
for DLLs or equivilent. You may have to give special linking options or
run special post-processors on the executable image in order to move
these reserved areas to give yourself the maximum amount of room.

On Unix systems it is also common for there to be resource limits that
prevent you from using gobs of memory unless you override them or get
the systems administrator to override them. See the 'ulimit' man page
to get yourself started, and the getrlimit() functions. The mechanisms
used by the systems administrator to control the limit would very with
unix version; I do not know the details for freebsd.

 

[Guest]

sorry! i am not prepense
i write it in c , and comp it under win2k and freebsd5.3

in freebsd
p [ 294 ] ok
1048576
Killed


///////////////////////////////////////////
#include <stdio.h>
#include <stdlib.h>
const int len = 2000;
int main()
{
char* p1[len];
int i =0;
int x =0;
static int size = 1<<20;
printf( "%d \n", size);
for ( i =0 ; i < len ;i++)
{
p1 = (char*) malloc (size);
printf( "%d \n", size );
char *tp = p1;
for ( x =0 ; x< (size-1); x++)
{
//cout << x << endl;
*(tp+x) = (char)( (x%26)+ 48);

}
*(tp+size-1) =0;
printf( "p [ %d ] ok\n" , i);
}

for ( i =0 ; i < len ;i++)
{
free(p1);
}
return 0;
}



//////////////////////////////////////////

hi
i try to malloc 2G size mem . so i wirte code like that.

You have no guarantee that you can do so. You need to check in a
newsgroup, mailing list, or tech support for your implementation.

it failt in freebsd 5.3 and win2k
what's bug in my code ? how can i change it ?

We really can't help, partially because you posted to comp.lang.c, where
we use C, and your code is not C. comp.lang.c++ is a different newsgroup
for a different language.

Examples of C++isms are:

#include <iostream> This is not a C header.
using namespace std; This is a compilation error.
#include <cstdlib> This is not a C header.
cout << size << endl;

'cout' and 'endl' are undeclared identifiers, and '<<' is a
left-shift operator that probably don't do what you want.

p1 = new char [size];

'new' is an undeclared identifier, and the above is a compilation error.

There are more instances, but I don't care to be tiresome.

 

[CBFalconer]

.... snip ...
#include <iostream>
using namespace std;

comp.lang.c++ is down the hall to the right.

 

[Jonathan Bartlett]

hi
i try to malloc 2G size mem . so i wirte code like that.
it failt in freebsd 5.3 and win2k

what's bug in my code ? how can i change it ?

I don't know about FreeBSD, but on Windows a process cannot be bigger
than 2G, even if you have more RAM than that. Each process is limitted
to at most 2G. So, if you add the space for your code, libraries, and
existing variables, malloc'ing 2G on a Windows box is guaranteed to fail.

I _think_ Linux gives you 3G, but I could be wrong. I don't know if
these limits persist to AMD-64 or not. However, I would try it on a
64-bit PowerPC -- it should work there.

Jon

 

[Raymond Martineau]

sorry! i am not prepense
i write it in c , and comp it under win2k and freebsd5.3

in freebsd
p [ 294 ] ok
1048576
Killed


/////////////////////////////////////////// [...]

p1 = (char*) malloc (size);

This causes undefined behaviour when malloc returns NULL. Check the return
value.

 

[CBFalconer]

.... snip ...

p1 = (char*) malloc (size);

This causes undefined behaviour when malloc returns NULL. Check
the return value.

No it doesn't. NULL is a perfectly valid value for a char*.
However the useless cast can conceal errors, and should be avoided.

However, there may be a later problem when using p.

 

[Christopher Benson-Manica]

No it doesn't. NULL is a perfectly valid value for a char*.
However the useless cast can conceal errors, and should be avoided.

However, there may be a later problem when using p.

I think this is what he was actually referring to, although his
quoting could have been better.

 

[Guest]

#include <stdlib.h>
const int len = 2000;
int main()
{
char* p1[len];
int i =0;
int x =0;
static int size = 1<<20;
printf( "%d \n", size);
for ( i =0 ; i < len ;i++)
{
p1 = (char*) malloc (size);
printf( "%d \n", size );
char *tp = p1;
for ( x =0 ; x< (size-1); x++)
{
//cout << x << endl;
*(tp+x) = (char)( (x%26)+ 48);

}
*(tp+size-1) =0;
printf( "p [ %d ] ok\n" , i);
}

for ( i =0 ; i < len ;i++)
{
free(p1);
}
return 0;
}

 

[Guest]

hi
I _think_ Linux gives you 3G, but I could be wrong. I don't know if
these limits persist to AMD-64 or not. However, I would try it on a
64-bit PowerPC -- it should work there.

u run my code?

#include <stdlib.h>
const int len = 2000;
int main()
{
char* p1[len];
int i =0;
int x =0;
static int size = 1<<20;
printf( "%d \n", size);
for ( i =0 ; i < len ;i++)
{
p1 = (char*) malloc (size);
printf( "%d \n", size );
char *tp = p1;
for ( x =0 ; x< (size-1); x++)
{
//cout << x << endl;
*(tp+x) = (char)( (x%26)+ 48);

}
*(tp+size-1) =0;
printf( "p [ %d ] ok\n" , i);
}

for ( i =0 ; i < len ;i++)
{
free(p1);
}
return 0;
}

 

[Raymond Martineau]

... snip ...

p1 = (char*) malloc (size);

This causes undefined behaviour when malloc returns NULL. Check
the return value.

No it doesn't. NULL is a perfectly valid value for a char*.
However the useless cast can conceal errors, and should be avoided.

Actually, I was referring to the fact that there wasn't any checking on the
return value (which was later blindly used.)

In this case, I didn'tpoint out the line at which it would fail.

 

[Walter Roberson]

:I _think_ Linux gives you 3G, but I could be wrong.

Something I read awhile ago suggested that it depended on the kernel
version and how you compiled.

:u run my code?

After some cleanup and optimization, yes. Compiled as a 32 bit
executable, it runs on one machine until it pretty much reaches the
end of the storage space "underneath" the virtual address that libc
gets placed at. Compiled as a 64 bit executable and run on a different
machine, it run to completion -- the object layout is different for
the 64 bit executables.

 

[Stephen Sprunk]

sorry! i am not prepense
i write it in c , and comp it under win2k and freebsd5.3

in freebsd
p [ 294 ] ok
1048576
Killed

There is no guarantee that a program can allocate arbitrarily large amounts
of memory. malloc() is allowed to return NULL at any time.

<OT>
Virtual memory doesn't mean that you can allocate 2-4GB in each application;
there is almost certainly a limit that your OS or configuration imposes
below that. Apparently your system's limit is 294MB for this program.

for ( i =0 ; i < len ;i++)
{
p1 = (char*) malloc (size);

The cast is unnecessary (and not idiomatic).

printf( "%d \n", size );
char *tp = p1;
for ( x =0 ; x< (size-1); x++)
{
//cout << x << endl;
*(tp+x) = (char)( (x%26)+ 48);

You forgot to check if p1, and thus tp, is NULL; dereferencing that'll
ruin your day.

}
*(tp+size-1) =0;
printf( "p [ %d ] ok\n" , i);
}

S

 

[Jonathan Burd]

hi

<mowed>

Please do not top-post in c.l.c.

Regards,
Jonathan.

 

[Randy Howard]

:I _think_ Linux gives you 3G, but I could be wrong.

Something I read awhile ago suggested that it depended on the kernel
version and how you compiled.

OT: This can be done under Windows also, provided you have less than
4GB of RAM total and do not have /PAE turned on. Google for boot.ini
options.

After some cleanup and optimization, yes. Compiled as a 32 bit
executable, it runs on one machine until it pretty much reaches the
end of the storage space "underneath" the virtual address that libc
gets placed at.

I think that you will discover that for a system running 32-bit
Windows or Linux with 2GB of RAM or more installed, and using default
kernel options (I.e. no 3GB address space hack), that a reasonably
sized command line app will be able to malloc around 1600-1700MB of
RAM prior to it failing. People that claim that malloc() never fails
should try this for themselves.

Compiled as a 64 bit executable and run on a different machine, it
run to completion -- the object layout is different for the 64 bit
executables.

Malloc will also fail (eventually) on Linux/glibc 64-bit
combinations. The point it fails varies. I tried this a while back
on an x86_64 w/64GB (not a typo) of RAM. I forgot where it failed
in a single process (16GB?), but recall that with pthreads (NPTL),
it would fail at just under 8GB/thread, but you could effectively
chew up the entire 64GB by starting 8 threads and having each ask
for 8GB.

I haven't tried this on a Win64 x86_64 system (yet).

 

[aurgathor]

For one thing, you need a systen capable
of allocating over 2 Gigs of memory, and
that could be a bit challenging to an average
PC.

Next, ever heard of a NULL pointer?

BTW, I run it on my sytem with a target changed to
3 Gig, and with some extra diag code put in there,
and it did go to 2.1 Gig.

(Said PC has 2 Gigs of physical memory, BTW)

 

[Guest]

hi
thanks for your help , and i change it again.


code i fixed
///////////////////////////////////////
#include <stdio.h>
#include <stdlib.h>

const int len = 800;
int main()
{
char* p1[len];
int i =0;
int x =0;
char *tp;
static int size = 1<<20;
printf( "%d \n", size);
for ( i =0 ; i < len ;i++)
{
p1 = (char*) malloc (size);

tp = p1;
if (tp == NULL)
continue;
printf( "%d \n", size );

for ( x =0 ; x< (size-1); x++)
{
//cout << x << endl;
*(tp+x) = (char)( (x%26)+ 48);

}
*(tp+size-1) =0;
printf( "p [ %d ] ok\n" , i);

}

for ( i =0 ; i < len ;i++
)
{
if (p1 == NULL)
continue;
free(p1);
printf("i clean %d \n", i);
}
return 0;
}

///////////////////////

in win2k it can run success,
but in linux and freebsd it fail. killed by os

i wish if malloc() fail, it can continue, in win2k it success
in linux and freebsd it fail.

benjiam

... snip ...

p1 = (char*) malloc (size);

This causes undefined behaviour when malloc returns NULL. Check
the return value.

No it doesn't. NULL is a perfectly valid value for a char*.
However the useless cast can conceal errors, and should be avoided.

Actually, I was referring to the fact that there wasn't any checking on the
return value (which was later blindly used.)

In this case, I didn'tpoint out the line at which it would fail.

 

[Walter Roberson]

thanks for your help , and i change it again.

code i fixed
p1 = (char*) malloc (size);
tp = p1;
if (tp == NULL)
continue;

You do not want a 'continue' there, you want a 'break'.
Otherwise it will loop around again, and fail the malloc
again.

The version below is a bit more efficient.

#include <stdio.h>
#include <stdlib.h>
const int len = 2000;
const int size = 1<<20;
int main()
{
char* p1[len];
char p2[size];
int i =0;
int x =0;
for ( x = 0; x < (size-1); x++ ) p2[x] = (char)((x%26) + 48 );
p2[size-1] = 0;

printf( "%d \n", size);
for ( i =0 ; i < len ;i++) {
p1 = malloc (size);
if ( !p1 ) break;
strncpy( p1, p2, size );
printf( "p [ %d ] ok\n" , i);
}

while (--i >= 0) free( p1 );

return 0;
}


I haven't had a chance yet to try it out compiled for 64 bit pointers
on a system with 1.5 Gb physical memory.

 

[Guest]

i suppose
in case of i =90
p1 = (char*) malloc (size);
tp = p1;
if (tp == NULL)
continue;
it fail
loop
i= 91

it fail
loop
i =92
at this time ,the other process free a lot of memory
and this time it will malloc success.(maybe it is wrong idea,resource
cortroled by os)



You do not want a 'continue' there, you want a 'break'.
Otherwise it will loop around again, and fail the malloc
again.

thanks for your help , and i change it again.

code i fixed
p1 = (char*) malloc (size);
tp = p1;
if (tp == NULL)
continue;

You do not want a 'continue' there, you want a 'break'.
Otherwise it will loop around again, and fail the malloc
again.

The version below is a bit more efficient.

#include <stdio.h>
#include <stdlib.h>
const int len = 2000;
const int size = 1<<20;
int main()
{
char* p1[len];
char p2[size];
int i =0;
int x =0;
for ( x = 0; x < (size-1); x++ ) p2[x] = (char)((x%26) + 48 );
p2[size-1] = 0;

printf( "%d \n", size);
for ( i =0 ; i < len ;i++) {
p1 = malloc (size);
if ( !p1 ) break;
strncpy( p1, p2, size );
printf( "p [ %d ] ok\n" , i);
}

while (--i >= 0) free( p1 );

return 0;
}


I haven't had a chance yet to try it out compiled for 64 bit pointers
on a system with 1.5 Gb physical memory.