fork question

[bruno]

I have a script that goes over an alphabetical list of symbols.
To speed things up I fork.
In each thread I load a list of symbols. When the last symbol is done,
I get a GPF: "Attempt to free unreferenced scalar"

system win2000, activeperl 5.8
The script runs OK when not forking.


Basically it looks like this:


sub loadTables {
open (DATEI, "< $infile") or die "Kann Datei $infile nicht öffnen";
my @symbols = <DATEI>;
close (DATEI);

foreach (@symbols) { <-- thats the line where the GPF occurs
......
}
}

Thanks, Bruno

 

[Eric J. Roode]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

(e-mail address removed) (bruno) wrote in @posting.google.com:

I have a script that goes over an alphabetical list of symbols.
To speed things up I fork.

Fork slows programs down more often than it speeds them up.

I can't address your specific GPF -- I don't know what's causing it --
but perhaps you're overloading the system's memory capacity. Why do you
read the entire file into an array and loop over it? Is it a large file?

sub loadTables {
open (DATEI, "< $infile") or die "Kann Datei $infile nicht öffnen";
my @symbols = <DATEI>;
close (DATEI);

foreach (@symbols) { <-- thats the line where the GPF occurs
......
}
}

Why not simply:

open (DATEI, ....);
while (<DATEI>) {
....
}

?

- --
Eric
$_ = reverse sort $ /. r , qw p ekca lre uJ reh
ts p , map $ _. $ " , qw e p h tona e and print

-----BEGIN PGP SIGNATURE-----
Version: PGPfreeware 7.0.3 for non-commercial use <http://www.pgp.com>

iQA/AwUBP711CWPeouIeTNHoEQIY4wCfQJUyc3fiH218pfegHBR+Gu0z2rwAoNGt
o3XQl93b32Mpgiu2TAiiHM6y
=Uswx
-----END PGP SIGNATURE-----

 

[bruno]

Fork slows programs down more often than it speeds them up.

I know, but for each symbol read from the file the program is getting
an url from the internet, thats where most of the time goes.
The program runs a couple of hours, so in this case forking 20 times
speeds things up by almost a factor 20.

but perhaps you're overloading the system's memory capacity.Why do you
read the entire file into an array and loop over it? Is it a large file?

only 40 k, so memory issues should not be a problem

open (DATEI, ....);
while (<DATEI>) {

Does not change much.
The answer to your first question, is, because I just discovered perl
as a perfect tool for my purposes.

Thanks, Bruno.

 

[bruno]

I just found out, by trial and error, that if I remove the
"substr"-function, the program runs fine. Is it possible, that
"substr" is not thread-safe?

thanks, bruno.

 

[Tassilo v. Parseval]

Also sprach bruno:

I just found out, by trial and error, that if I remove the
"substr"-function, the program runs fine. Is it possible, that
"substr" is not thread-safe?

Doubt it, but it's not relevant in your case anyway. fork() creates
processes and not threads. Unlike threads, processes do not share
anything (save for file-descriptors) and therefore thread-safeness or
not is a non-issue for this case.

Tassilo

 

[Ben Morrow]

Also sprach bruno:


Doubt it, but it's not relevant in your case anyway. fork() creates
processes and not threads. Unlike threads, processes do not share
anything (save for file-descriptors) and therefore thread-safeness or
not is a non-issue for this case.

The OP is on Win2k... fork creates ithreads.
Of course, if substr is not ithread-safe this is a bug in perl.

Ben

 

[Anno Siegel]

The OP is on Win2k... fork creates ithreads.
Of course, if substr is not ithread-safe this is a bug in perl.

Not that I've heard anything specific, but the lvalue-ness of substr
could conceivably lead to problems. After doing "$part = \ substr(
$str, 10, 3)" changes in $$part propagate to $str and change it indirectly.
It must be a bitch to propagate access restrictions on $str to $$part.

Idly speculating,

Anno

 

[Eric J. Roode]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

(e-mail address removed) (bruno) wrote in

I know, but for each symbol read from the file the program is getting
an url from the internet, thats where most of the time goes.
The program runs a couple of hours, so in this case forking 20 times
speeds things up by almost a factor 20.

Okay, I can see that.

only 40 k, so memory issues should not be a problem

Agreed.

(I'd still recommend reading one line at a time; it's a better practice
in general).

- --
Eric
$_ = reverse sort $ /. r , qw p ekca lre uJ reh
ts p , map $ _. $ " , qw e p h tona e and print

-----BEGIN PGP SIGNATURE-----
Version: PGPfreeware 7.0.3 for non-commercial use <http://www.pgp.com>

iQA/AwUBP74EaWPeouIeTNHoEQKajgCeNdkR5JSEoQmCNEUutz8bD1uwHIcAn1AV
ecbbDYxTziO51lqvnNoEWM/x
=Cx0d
-----END PGP SIGNATURE-----

 

[Tassilo v. Parseval]

Also sprach Ben Morrow:

(e-mail address removed)-aachen.de wrote:

The OP is on Win2k... fork creates ithreads.
Of course, if substr is not ithread-safe this is a bug in perl.

Ah, you are right. I missed the bits about the platform used.

Tassilo

 

[Ben Liddicott]

Hi Bruno,

I have found fork() to be unreliable on ActivePerl, in the same way as you. Can I suggest you use system(1, $0, @some_args), which creates an entirely fresh process on Win32, unless there is too much stuff to reasonably fit into @some_args?

See system in perlport for more on this.

Since you suggest you are simply dividing up the work, I'd suggest this would be OK. You can have each write intermediate results to individual files, then collect the information together after they all finish.

 

[bruno]

I have found fork() to be unreliable on ActivePerl, in the same way as

you. Can I suggest you use system(1, $0, @some args)...

Thanks, I'll try that,
Bruno.