perl threads

[kath]

Hi,
Background:
I have task of calculating dependencies of project(s) of a scenario.
Dependency calculation is done using an external command. Since this
dependency calculation time depends on how big is the project is, it
is difficult to guess how much dependency calculation of a scenario(a
set of projects) takes. To overcome this i am using 'threads' module
to create thread and ask each thread to do the dependency calculation.

I have many scenarios currently and i calculate dependency foreach
scenario. I create one thread for each project in the scenario and
wait for all threads to finish, to take on next scenario. Because i
know 'Perl ithreads are not lightweight!', i am using carefully, in
the sense, I'm not using any shared variables between threads. Each
thread will do dependency calculation, which will produces an output
in a file separately. Later i parse those files to get dependency list
for each project.

Problem:
Sometimes* the script waits forever or the script just hangs. That is
waiting for threads. And cant continue with other scenario.

#this way i create threads
#foreach scenario() ...{
map {my $th = threads->create(\&worker, $_)} @$proj_arr; #proj_arr has
projects of a scenario
# ...}

# This is how i wait for all threads to finish its job
print "waiting for threads to finish";
map {my $k = $_->join} threads->list;

Is there a way i can overcome this? Or my perception about cause for
the problem is right? Because from the log i see, only 'waiting for
threads to finish' at last and script will continue. This is happens
only sometimes. Most of the times the script executes fine.

Atleast i want to be informed about the 'script is not responding or
hanged', because the script is scheduled there will be not be any clue
unless and otherwise, myself has to go and check.

Hope i could explain the problem clearly. I am using 'perl, v5.8.8
built for MSWin32-x86-multi-thread'


Thanks in advance,
katharnakh.

 

[zentara]

know 'Perl ithreads are not lightweight!', i am using carefully, in
the sense, I'm not using any shared variables between threads. Each
thread will do dependency calculation, which will produces an output
in a file separately. Later i parse those files to get dependency list
for each project.

First I don't use win32, I use linux, but the advice should be the same
in this case.
If you don't share variables between threads, and you are writing
results to a file, to be processed later, you don't need threads.
Forking an independent process is better. (I realize on win32 it's all
threads, but Perl imposes it's own weight when using threads, so you
are better off with independent processes.

See the part on Win32:rocess in
http://perlmonks.org?node_id=500663

Problem:
Sometimes* the script waits forever or the script just hangs. That is
waiting for threads. And cant continue with other scenario.

Threads must reach the end of their code blocks, or return, in order
for them to be joined. Maybe try detached threads, or use a shared
variable to signal them to return.

#this way i create threads
#foreach scenario() ...{
map {my $th = threads->create(\&worker, $_)} @$proj_arr; #proj_arr has
projects of a scenario
# ...}

# This is how i wait for all threads to finish its job
print "waiting for threads to finish";
map {my $k = $_->join} threads->list;

Is there a way i can overcome this? Or my perception about cause for

Somewhere in your worker code block, you must listen for a shared
variable, telling the thread to immediately return. Later threads
versions have kill signals you can send to threads, but I don't know if
they work on win32. A thread will not end, just by telling it to join.

zentara

 

[kath]

Hi,
First, thanks for your time.

Somewhere in your worker code block, you must listen for a shared
variable, telling the thread to immediately return. Later threads
versions have kill signals you can send to threads, but I don't know if
they work on win32.

Will work on this.

A thread will not end, just by telling it to join.

I dint know this. May be this was my wrong understanding.

Thanks again,
katharnakh.