limitations of forking on windows

[John Bokma]

Tassilo v. Parseval wrote:

This race condition. Using an example of the entire process
of creating a semaphore, write to file, close, unlink,
takes a total of fifty milliseconds, which is slow, how
many concurrent processes must be running to win this
race condition, on the average?

Since I have seen this problem in so called webcounters, with only a few
hundred visitors a day, it happens.

I've run some statistical probabilities on this.
What results do you obtain?

That depends on how probability is distributed. If I post a link to a
Usenet group, and 20 people visit it, it's more likely to happen.

Logically, it only takes one incident to mess
up a file, making an assumption a majority of

Yup, and one can be enough.

As you know, in the past, I was the first Perl programmer
to introduce a safe reliable file lock mechanism for Win32

Using the clipboard and Internet Explorer :-D.

 

[John Bokma]

"Ready, set.... GO!"

How often did you encounter this in a real world situation? Reality is
that all people hear GO at a different time, and because of the reaction
time differences, as you skillfully elaborated on, the probability of
two people doing GO at the same time has a P(same time) > 0.

I have seen enough web counters losing count and files corrupted in a
web application set up, to know that human click behaviour doesn't
follow the conditions of your race.

Of those ten people, how many will invoke the
same program, within fifty milliseconds of
others invoking the same program?

Probability indicates, zero.

Read more on probability.

Reaction times amongst people vary greatly.
Human reaction depends on myriad factors,
such as previous experience, skill, age,
cognitive processing speed, relative
intelligence; infinite factors.

The factors are not infinite.

 

[Malcolm Dew-Jones]

Anno Siegel ([email protected]) wrote:
: > Also sprach Purl Gurl:

: [...]

: > > Another problem is if your program crashes before
: > > the unlink of lockfile, you are stuck. No access
: > > can be had from that point forward in time.
: >
: > That is the problem of most locking mechanisms, except when using
: > flock() which isn't readily available everywhere. A solution to those
: > stale lockfiles is to write the PID of the creator into the lockfile. If
: > a lockfile exists, the current process can read the PID and check
: > whether the process referenced by it is still alive.
: >
: > This approach however has its own problems, in particular it requires to
: > lock the lockfile itself.

: Another problem is pid re-use. To be sure that a process with a
: certain pid is still the same process, it is necessary to check that
: the system wasn't rebooted in the meantime. Storing the boot time
: (if available) along with the pid is a possible solution.

The usual solution is to put lock files in a directory that gets cleaned
up at boot time (i.e. deleted).

 

[Sherm Pendley]

If I post a link to a
Usenet group, and 20 people visit it, it's more likely to happen.

If you post a link to a Usenet group, and 20 people visit it, they're
stalking and harassing you and your family.

sherm--

 

[Anno Siegel]

Anno Siegel ([email protected]) wrote:
comp.lang.perl.misc:

: Another problem is pid re-use. To be sure that a process with a
: certain pid is still the same process, it is necessary to check that
: the system wasn't rebooted in the meantime. Storing the boot time
: (if available) along with the pid is a possible solution.

The usual solution is to put lock files in a directory that gets cleaned
up at boot time (i.e. deleted).

....often /var/run for the sysadmin. Joe User would need a directory
that gets such service. /tmp may or may not be one.

Anno

 

[John Bokma]

If you post a link to a Usenet group, and 20 people visit it, they're
stalking and harassing you and your family.

No.. that is if only 2 people visit it. If 20 people visit it, it's a
masterplan of a criminal organisation, masquerading as Usenet regulars,
backed by some big websites, who's members are mentally disturbed and
interested in pictures of the butt of at least one member of their
organisation.

There was something with proxies, but I forgot.

 

[Bryan Castillo]

(MORE attributes stripped by Castillo)

Topic is semaphore file locking.

No, the topic is "limitations of forking on windows".

That module is used for child threading control.

It can be used for child threading control. Events, Mutexes, and Semaphores
in Win32 may be created with or without a name. When creating them with a name
they may be used between different processes.

Try the code below out. Run it once in a window. You will see that it will
get to the point where it is wating for user input. Now run the script in
a 2nd window and it will be waiting to acquire the lock. Go back to the first
window and hit Enter. Then the 2nd script will acquire the lock and wait for
the user to hit Enter.


use strict;
use warnings;
use Win32::Semaphore;

my $sem_name = shift(@ARGV) ||
die "Usage: $0 <semaphore name>";

print "Getting semaphore [$sem_name]\n";

my $sem = Win32::Semaphore->new(1, 1, $sem_name) ||
die "Couldn't create semaphore [$sem_name]\n";

print "Waiting to acquire lock [$sem_name]\n";
$sem->wait();

print "Have lock [$sem_name]\n";

print "Press [Enter] to release lock.\n";
<STDIN>;

$sem->release();

 

[John Bokma]

(attributes stripped by Castillo)


That is a nice module, works well but has
one very fatal flaw.

Do you know what is that fatal flaw?

You seeing it for the first time?

 

[Bryan Castillo]

(attributes stripped by Castillo)


That is a nice module, works well but has
one very fatal flaw.

Do you know what is that fatal flaw?

I don't have a clue. Will you please post what the flaw is?

 

[Uri Guttman]

JB> But I am curious at the ~100% reliable method, maybe it can be decided
JB> if it's actually unreliable or can be made reliable.

/bin/rm -rf ~moronzilla is a very reliable method.

it doesn't understand why 99% reliable is no better than 1% reliable. it
will never understand that as it has never had to earn a living worrying
about such important details.

uri

 

[Dale Henderson]

UG> it doesn't understand why 99% reliable is no better than 1%
UG> reliable. it will never understand that as it has never had to
UG> earn a living worrying about such important details.


This entire thread reminds of an incident when I worked for an
"Interactive Ad Agency". We were collecting survey results for
Subaru of America and some bone-head (me) forgot to lock the
storable files where the data was stored. The result was a
frequently corrupted file and an angry A.E.

Luckily we were able to restore much of the data from backups.

I learned a valuable lesson though: Always lock your files!

 

[Peter J. Acklam]

First twenty years of my life, (...)

You missed "worrying about such important details".
He didn't just say you "never had to earn a living".

Peter

 

[Thomas Kratz]

Bryan Castillo wrote:




Of course. However, I first had to play the testosterone
macho challenge game so you boys will feel comfortable.

Win32 Semaphore is a very nice module. I like it and have
played with it many times, for as many different tasks.
I do like the module.

However, like flock, like fcntl, like lock files, like
many other methods, including my Win32 xcopy hack, there
is a fatal flaw, except this flaw in this module is
really devastating.

Locks can become stale. Semaphore lock files might not
be unlinked. My xcopy, might not remove the read only
attribute. No major problem save for file access denial.

Win32 Semaphore, the module. Do this. Invoke an instance
of your program which runs the module. Invoke another
instance of your program. Return to your original number
one invoked program. Crash it. This is, kill the process
without having ever released the object semaphore lock.

Now your second instance of your program is a zombie.

[more of the same snipped]

Just for the record:

As everyone who can read the module's docs can see, this is exactly why
the wait() method of Win32::Semaphore has an optional timeout parameter
that prevents blocking on the semaphore. So all you have to do is write a
loop around the wait call and calling it quits if a certain time has
passed without getting hold of the semaphore.

Brian was providing an example, not a foolproof implementation.

Thomas

 

[Bryan Castillo]

I will expand on this a bit before one of the boys
here jumps in and starts trolling.

Most easy way to think of this, is a lock object is
not destroyed as should be, when a crash happens.
It is, in essence, memory resident.

After you have played with this, and hopefully your
Win32 system will create lots of zombies for you,
open the module and study how it works.

Let us know what happens on your machine.

My system behaved the way you daid it would. Now that I think about
it, SysV semaphores are worse. I seem to remember SysV semaphores
still exist
after all processes that had one open, terminate. I doubt its the
perl
module that creates this behavior. I'd wager that is just the nature
of OS level semaphores in Windows. I'll go ahead and write a program
in C to
see if I can come up with the same behavior without using perl.

Anyway, I modified the previous example I posted to try and work
around such a situation.

use strict;
use warnings;
use Win32::Semaphore;

foreach my $sig (qw/INT TERM/) {
$SIG{INT} = sub { die "signal: ". $_ };
}

my $sem_name = shift(@ARGV) ||
die "Usage: $0 <semaphore name>";

print "Getting semaphore [$sem_name]\n";

my $sem = Win32::Semaphore->new(1, 1, $sem_name) ||
die "Couldn't create semaphore [$sem_name]\n";

print "Waiting to acquire lock [$sem_name]\n";
$sem->wait();

print "Have lock [$sem_name]\n";

print "Press [Enter] to release lock.\n";
<STDIN>;

END {
print "release semaphore\n";
$sem->release();
}

 

[Tassilo v. Parseval]

Also sprach Purl Gurl:

You have created a classic race condition. Doing so defeats
the very purpose of locking.

How's waiting for a particular ressource and giving up when it doesn't
become available after a given time a race condition?

"Bryan"

Tassylo

 

[Peter J. Acklam]

Consider this,

Resource one writes data while semaphore locking.
Resource two needs to write new data and is waiting.
Resource three needs to write new data and is waiting.

Resource two times out.
Resource one finishes.
Resource three writes new data.

Your resource two data has vanished.

That is just an example of the scenario Tassilo described. You
haven't explained how this is a race condition. I don't know all
the details related to race conditions, so I'm interested in
hearing your explanation.

Peter

 

[Peter J. Acklam]

No, it is not.


Yes, I have.

AFAIK, a race condition is when two or more resources are
accessing the same resources at the same time. That didn't
happen in the example you gave. In that case, the other processes
were just waiting and eventually timed out.

http://childhoodreading.com/Arthur_Rackham/Tortoise_and_the_Hare.html

So that's your reference. Well, it was a good thing I bought
Steven's "Advanced programming in the UNIX environment" today.
Someone here suggested it. As far as race conditions go, I
believe it to be more authoritative than what you're reading.

Peter

 

[Peter J. Acklam]

A race condition exists anytime a "problem" is created
by time, literally time. In my example, data is lost
because it cannot be written before time runs out.

I don't think that is called a "race condition".

Ah, that is such a wonderful fable! Great moral lesson as well.

I agree on that.

Peter

 

[Tassilo v. Parseval]

Also sprach Purl Gurl:

Consider this,

Resource one writes data while semaphore locking.
Resource two needs to write new data and is waiting.
Resource three needs to write new data and is waiting.

Resource two times out.
Resource one finishes.
Resource three writes new data.

Your resource two data has vanished.

Yes, so it has. But that's not a race condition. In the context of this
thread, a race condition is a state in which two parties think that they
have exclusive access to a resource although they might in fact be
sharing it. Two processes writing to the same file would be such a
condition, eventually leading to an inconsistent state of this file.

In your scenario however, data of one source is simply missing. The
difference between that and the above is crucial: With a race condition,
you have the data of several sources indistinguishably mangled into each
other, such as one process partly overwriting data of the other process.

Another thing worthy mentioning is that a race condition, once
happening, cannot be caught. Every involved party honestly believes
things are going right. The case you describe however is trappable.
Process two could for instance create a log file in which it clearly
states that it wasn't able to proceed because it timed out.

The only thing I can grant you is that avoiding races in a portable way
is tricky. The conclusion you drew from that (namely to use a method
that predictibly wont work on any platform) is of course wrong. Every
platform offers at least one way to prevent races and ensure integrity
of data. A portable program therefore choses the method suitable for the
platform it is running on. On UNIXish systems one might be preferring
flock() because it is a convenient and simple mechanism. On windows one
might be using separate lock files or maybe semaphores. If one approach
turns out to have major shortcomings, like producing stale locks, one
would have to work around those as well, often making a significant
rearrangement in the program necessary.

Tassilo

 

[John Bokma]

Purl Gurl wrote:

[ ramblings ]

I contribute these changes to all of you, personally.

Now you. Stop talking about abuse, your family, hack attempts on your
site, and other noise.