Cannot allocate memory when using os.spawn for moving files

[Andreas]

Hello there,

I have a problem moving files from my local harddrive to a NFS share
using a Python script.
The script is used to run a model which produces large (~500MB) binary
output files. The model itself is a Fortran program, and I call it from
my script using the line

os.spawnlp(os.P_WAIT, 'time', 'time', OPT.binary)

where OPT.binary is the filename to be run. This works flawlessly.

However, in order not to clutter up my harddrive, I want to move the
generated files to a network location after each model run. To save
time, I use os.P_NOWAIT here:

os.spawnlp(os.P_NOWAIT, 'mv', 'mv', LOCALFILENAME, REMOTEFILENAME)

where LOCALFILENAME is some string like '/home/andreas/model.bin' and
REMOTEFILENAME is some string like '/home/nfs/model-output/'.

Here I have the problem that after about every 10th model run, I get an
error saying

Traceback (most recent call last):
File "../../../../pyiup/b3dctm/run_b3dctm.py", line 281, in <module>
main()
File "../../../../pyiup/b3dctm/run_b3dctm.py", line 71, in main
copy_model_output()
File "../../../../pyiup/b3dctm/run_b3dctm.py", line 162, in
copy_model_output
os.spawnlp(os.P_NOWAIT, 'mv', 'mv', get_base_filename(RUNDATE) +
'.pdg',
os.path.join(OPT.datastoragepath,OPT.modelname,OPT.runname,RUNDATE.strftime('%Y/
%m')))
File "/usr/lib64/python2.5/os.py", line 635, in spawnlp
return spawnvp(mode, file, args)
File "/usr/lib64/python2.5/os.py", line 584, in spawnvp
return _spawnvef(mode, file, args, None, execvp)
File "/usr/lib64/python2.5/os.py", line 530, in _spawnvef
pid = fork()
OSError: [Errno 12] Cannot allocate memory

And my Python script dies.

Is there anyone who can help me with this problem? I'm on Ubuntu 8.04
64bit with Python 2.5. I am willing to accept that the moving of the
files does not always work, but then it would be really helpful if I
could somehow prevent my script from dying and just try the moving again.

Any help is greatly appreciated!

Cheers,

Andreas

 

[Piet van Oostrum]

A> Hello there,
A> I have a problem moving files from my local harddrive to a NFS share using
A> a Python script.
A> The script is used to run a model which produces large (~500MB) binary
A> output files. The model itself is a Fortran program, and I call it from my
A> script using the line

A> os.spawnlp(os.P_WAIT, 'time', 'time', OPT.binary)

A> where OPT.binary is the filename to be run. This works flawlessly.

A> However, in order not to clutter up my harddrive, I want to move the
A> generated files to a network location after each model run. To save time, I
A> use os.P_NOWAIT here:

A> os.spawnlp(os.P_NOWAIT, 'mv', 'mv', LOCALFILENAME, REMOTEFILENAME)

A> where LOCALFILENAME is some string like '/home/andreas/model.bin' and
A> REMOTEFILENAME is some string like '/home/nfs/model-output/'.

A> Here I have the problem that after about every 10th model run, I get an
A> error saying

A> Traceback (most recent call last):
A> File "../../../../pyiup/b3dctm/run_b3dctm.py", line 281, in <module>
A> main()
A> File "../../../../pyiup/b3dctm/run_b3dctm.py", line 71, in main
A> copy_model_output()
A> File "../../../../pyiup/b3dctm/run_b3dctm.py", line 162, in
A> copy_model_output
A> os.spawnlp(os.P_NOWAIT, 'mv', 'mv', get_base_filename(RUNDATE) +
A> .pdg',
A> os.path.join(OPT.datastoragepath,OPT.modelname,OPT.runname,RUNDATE.strftime('%Y/
A> %m')))
A> File "/usr/lib64/python2.5/os.py", line 635, in spawnlp
A> return spawnvp(mode, file, args)
A> File "/usr/lib64/python2.5/os.py", line 584, in spawnvp
A> return _spawnvef(mode, file, args, None, execvp)
A> File "/usr/lib64/python2.5/os.py", line 530, in _spawnvef
A> pid = fork()
A> OSError: [Errno 12] Cannot allocate memory

A> And my Python script dies.

A> Is there anyone who can help me with this problem? I'm on Ubuntu 8.04 64bit
A> with Python 2.5. I am willing to accept that the moving of the files does
A> not always work, but then it would be really helpful if I could somehow
A> prevent my script from dying and just try the moving again.

On my Mac OS X system the fork man page says:

[ENOMEM] There is insufficient swap space for the new process.
This is errno 12.

So apparently you have too many processes running or you have too little
swap space.

Maybe you can have a `top' process running or run `ps' occasionally to
see how many processes there are and how big they are. Moving files from
a local file system to a network file system means copying and maybe
the mv tries to copy the files in very large chunks.

What you could do is to put the filenames in a work queue when your
Fortran program has finished and run a separate thread that moves the
files one by one.