fork seems to make urlopen into a black hole?

[BobAalsma]

A program took much too long to check some texts collected from web pages.
As this could be made parallel easily, I put in fork.
And the result seems to be that the program simply stops in the line with urlopen. Any suggestions?

Relevant part:

try:
print 'urlopen by', kind_nummer, '- before'
extern_adres = urlopen(adres).geturl()
print 'urlopen by', kind_nummer, '- after'
except IOError:
tekst_string = 'URL "' + adres +'" geeft IO Error'
print tekst_string, 'door', kind_nummer
bloklijst_verslag.append(tekst_string + '\n')
tekst_string = '\t**** Deze pagina wordt overgeslagen\n'
bloklijst_verslag.append(tekst_string + '\n')
adres = ''
except:
fout = sys.exc_info()[:2]
tekst_string = 'URL "' + adres + '" geeft fout ' + fout
print tekst_string, 'door', kind_nummer
bloklijst_verslag.append(tekst_string + '\n')
tekst_string = '\t**** Deze pagina wordt overgeslagen\n'
bloklijst_verslag.append(tekst_string + '\n')
adres = ''
else:
print 'urlopen by', kind_nummer, '- else'
if extern_adres != adres:
tekst_string = '\t**** redirect naar ' + extern_adres
bloklijst_verslag.append(tekst_string + '\n')

From this block, the only response seen is the first print statement ("url open by .... - before") and then nothing else seems to happen in that child.
This happens for all children: the expected number of children is reached and they all reach (only) this point.

 

[Chris Angelico]

A program took much too long to check some texts collected from web pages.
As this could be made parallel easily, I put in fork.

Rather than using the low-level fork() function, you may find it
easier to manage things if you use the multiprocessing module. I don't
know if it'll help in this specific case, but if nothing else, it'll
be easier to see what's going on.

ChrisA