Problem with Multi- threaded Server

[xhoster]

On 10 Jul 2006, (e-mail address removed) wrote:



OK, let me rephrase. The way you stated it, that's a terrible
question Most people wouldn't ask that from anyone in a business
environment, let alone their boss.

I think it goes without saying that you have to translate things into
BusinessSpeak, or Mongolian, or whatever the local language is, when you
relay things to others. That said, I'd much rather have a subordinate
correctly insult my requirements when they have a better alternative,
rather than blithely waste weeks of time trying implement my
(mis)conception.

Or at least any subordinate whom I had to micromanage so much that I wasn't
interested in spirited push back from them, I certainly wouldn't want them
going to usenet, rather than to me, to get advice. But I don't see why I
would want a report like that anyway.

Also I wasn't looking to discuss the particulars of Perl threads and
why they may sometimes be better; my list was just suggestions. The
point was that threads are not *always* a bad idea, and when you don't
know the full story (which I'm sure you don't, unless you work with
the OP) it's more helpful to offer gentle suggestions than to be
forceful.

I'm sure I don't know the full story, but I have to work with what I have,
and what I have indicated no need for threads. The gentler suggestions
didn't seem to work. Either the boss is an idiot or the boss has good
reason to use threads but won't shared them with the OP and hence with us.

Do some benchmarks if you like. I'm well-aware that fork() is better
in many ways, but I doubt you'll get better memory performance with
it. If you find otherwise, I'm all ears.

perl -le 'use threads; my @x=1..100_000; foreach (1..10000) { async {
sleep}->detach(); warn $_ unless $_%100; }'

The above started swapping itself to death between 400 and 500 (and it took
precious long even to get to 400).

perl -le 'use threads; my @x=1..100_000; foreach (1..10000) {fork or sleep;
warn $_ unless $_%100; }'

This one rapidly ran up to 4500, at which point I killed it to prevent
running out of processes.

I am not talking about the threaded web server the OP wanted. These
are places where threads are *usually* better than fork(), and
my point was that you can't *always* say threads are worse or better
than forking.

Right, but I wasn't saying that forks were always better, I was addressing
the case at hand, in which we can't even benchmark it to see because the
threading method seg faults. If you want to discuss generalities, I will
say that forks are almost always better than *Perl* threads when you don't
need extensive intercommunication.

Do some benchmarks if you like. I'm sure it takes fewer forks than
threads to run out of resources, all other things being equal.

perl -le 'use threads; my @x=1..10; foreach (1..10000) { async {
sleep}->detach; warn $_ unless $_%100; }'

This seg-faulted at 1200, and took annoying long to get there.

perl -le 'use threads; my @x=1..10; foreach (1..10000) { fork or sleep;
warn $_;}'

This ran out of processes at 6100, very quickly.

This is perl, v5.8.0 built for i386-linux-thread-multi

Xho

 

[Ben Morrow]

Under WinXP SP2, went aaaaalll the way up to 115 (quickly ;-).

Hmmm ... Just stopped creating processes at 64. The program was still
running though. Had to use CTRL-C.

This is hardly a sensible benchmark on Win32, as fork is implemented
with ithreads anyway... it's interesting that they seem to be more
resource-hungry than using threads directly, though...

Under cygwin, it went up to 256.

Do you know if cygwin provides its own fork emulation, and if perl uses
it or ithreads?

Ben

 

[Ted Zlatanov]

perl -le 'use threads; my @x=1..100_000; foreach (1..10000) { async {
sleep}->detach(); warn $_ unless $_%100; }'

The above started swapping itself to death between 400 and 500 (and it took
precious long even to get to 400).

perl -le 'use threads; my @x=1..100_000; foreach (1..10000) {fork or sleep;
warn $_ unless $_%100; }'

This one rapidly ran up to 4500, at which point I killed it to prevent
running out of processes. ....
perl -le 'use threads; my @x=1..10; foreach (1..10000) { async {
sleep}->detach; warn $_ unless $_%100; }'

This seg-faulted at 1200, and took annoying long to get there.

perl -le 'use threads; my @x=1..10; foreach (1..10000) { fork or sleep;
warn $_;}'

This ran out of processes at 6100, very quickly.

This is perl, v5.8.0 built for i386-linux-thread-multi

Thanks for doing these. I'm actually very surprised - I never
expected Perl 5's threads to be this bad. I hope Perl 6 improves on
this (I know it's supposed to). I guess not enough people use
threads, so this area is just not emphasized by the developers.

OP: you should show these to your boss as definite proof that
threading in Perl is probably worse than fork().

Ted