backticks with threads or forks hang program

[Warren]

Greetings

The function of the script below is to execute scripts (perl) at
regular intervals. As I have no idea how long these external scripts
will take to excute I decided to use threads and execute each script
from its own thread. This appeared to work until the number of threads
increased. The greater the number of threads the more lightly the
program will hang. As the output of the external scripts is required
the backticks are used.

I then tried fork very quickly (a very simple test implementation is
in the execute script but commented out, remove the # and add # to the
two lines with $Thread in them to test). This produced the same
problem.

I have also tried system() which does not appear to have the problem,
but without the output it is of no use.

How to reproduce:
The first program is the 'executer' and the second is the script
executed by the 'executer'. In order to reproduce make 13 files of the
'executed' script(containing the TargetFunction) called
Script0.script
Script1.script
.....
Script13.script

Then run the 'executer'. You may have to run it a few times for the
problem to show. If the executer runs for about 15 seconds then ctrl+c
it and re-run. At some point the program should (does for me) stop at
the backtick statement.

Any help in resolving the issue would be greatly appreciated.

Code for executer:
###########################################################3
#!/usr/bin/perl -w
use strict;
use threads;
use threads::shared;
#use IO::Socket;
use Sys::Hostname;
my @PassiveIDs=();

sub ExecuteThread
{
(my $ListID)=@_;
#sleep($ListID);

my $TimeOfStart;
while (1)
{
$TimeOfStart=time();
my $Command="perl Script".$ListID.".script ";
print "Command: ".$Command."\n";
print "Start Cmd $ListID\n";
my $Retval=`$Command`;
print "End Cmd $ListID\n";
print "Result: ".$Retval."\n";

my $SleepTime=(5)-(time()-$TimeOfStart);
if ($SleepTime<0)
{
$SleepTime=2;
}
sleep($SleepTime);
}
}
sub StartTCPClient
{
@PassiveIDs=(0,1,2,3,4,5,6,7,8,9,10,11,12,13);

print "Processing result\n";
my $IDCount=0;
foreach my $ID (@PassiveIDs)
{
$ID=~ s/[\r\n\0\000]//g;

print "Doing PassiveID -".$ID."-\n";

# my $newpid=fork();
# if (! defined $newpid) { #hosed
# die "fork() error: $!\n";
# }
# elsif ($newpid == 0)
# {
# #child
# ExecuteThread ($IDCount);
# }

my $Thread = threads->create("ExecuteThread",$IDCount );
$Thread->detach();

++$IDCount;
}
}

StartTCPClient ();

while (1)
{
sleep(10);
}
########################################################################

Executed script: This script is executed by the above program, need to
make 13 copies with the file names specified above (ie Script0.script,
Script1.script .... Script13.script).

#########################################################################
sub TargetFunction
{
}


my ($a,$b,$c,$d);
($a,$b,$c,$d)=@_;
print TargetFunction($a,$b,$c,$d);
##########################################################################

Op-system: win2000 server
perl: This is perl, v5.8.4 built for MSWin32-x86-multi-thread

Regards

 

[Anno Siegel]

Greetings

The function of the script below is to execute scripts (perl) at
regular intervals. As I have no idea how long these external scripts
will take to excute I decided to use threads and execute each script
from its own thread. This appeared to work until the number of threads
increased. The greater the number of threads the more lightly the
program will hang. As the output of the external scripts is required
the backticks are used.

Don't use threads, use background processes.

Specifically, for every external command, open a filehandle to read its
output from. See "perldoc -f open" and look for "-|". That the commands
are Perl scripts doesn't make a difference here.

You will now have to find out which processes have finished -- a problem
you tried to sweep under a carpet of threads before. There are several
possibilities. Making the filehandles non-blocking and polling them every
so many seconds is one solution. Others would involve select() (q.v.) in
one way or another. For a more general discussion, see perlipc.

Anno