subprocess help

[Влатко Станковиќ]

Hello,
I'm having some sort of 'problem' when using subprocess calls.
This is the code snipet that i am using:

capture_server1 = '''xvfb-run --auto-servernum ... '''
server1_download =
subprocess.Popen(shlex.split(capture_server1),stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)

out_s1, err_s1 = server1_download.communicate()

time.sleep(2)

capture_server2 = '''xvfb-run --auto-servernum .... '''
server2_download = subprocess.Popen(shlex.split(capture_server2),
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)

out_s2, err_s2 = server2_download.communicate()

The problem is the following:
- The program runs in a loop, where subprocess is called
- It runs for X days, sometimes 3 days, sometimes 5 days
- After that i get the following exception:

File "/usr/lib/python2.7/subprocess.py", line 1091, in pipe_cloexec
r, w = os.pipe()
OSError: [Errno 24] Too many open files


How can i reproduce this on a local machine, and how to make sure that i
wont have any errors like this?

P.S. Version 2.7 is used with this program

 

[Steven D'Aprano]

Hello,
I'm having some sort of 'problem' when using subprocess calls. This is
the code snipet that i am using:

capture_server1 = '''xvfb-run --auto-servernum ... '''
server1_download = subprocess.Popen(shlex.split(capture_server1),
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)

out_s1, err_s1 = server1_download.communicate()
time.sleep(2)

What's the difference between the server1 code (shown above) and the
server2 code (not shown, but identical as far as I can tell)?

[...]

The problem is the following:
- The program runs in a loop, where subprocess is called - It runs for X
days, sometimes 3 days, sometimes 5 days - After that i get the
following exception:

File "/usr/lib/python2.7/subprocess.py", line 1091, in pipe_cloexec r, w
= os.pipe()
OSError: [Errno 24] Too many open files

You have to identify what files are remaining open. What does the xvfb-
run process do? What are the rest of the arguments?

My guess is that, depending on the arguments, sometimes xvfb-run leaves
files open even after the process terminates. You should monitor the open
files with lsof which is available on most Linux systems. I don't know
how to do that on other operating systems.