flock or IPC semaphore for atomic appends

[jtd]

Hi all,

I'm running linux 2.4.20. I have 50-100 processes appending to a
single file. Each process appends a block of binary data between 1
byte to 100KB in size, at the rate of at least 1 per second. Using
flock appears to ensure that the appends are atomic, but it does use
up a lot of CPU time compared to not locking at all.

My questions:
1) Is my assumption that flock will ensure that appends are atomic
correct?
2) Why is flock processor intensive? Does flock write something to
disk when locking/unlocking?
3) Would IPC semaphores be a better idea since it is an in-memory
structure?

open(F, ">> append.txt");
binmode(F);
while(1) {
flock(F, LOCK_EX);
print F (large block of binary data)
flock(F, LOCK_UN);
sleep(rand(1));
}
close(F);

Thanks for any suggestions,
jtd

 

[A. Sinan Unur]

(e-mail address removed) (jtd) wrote in

2) Why is flock processor intensive? Does flock write something to
disk when locking/unlocking?

From perldoc -f flock:

Two potentially non-obvious but traditional "flock" semantics
are that it waits indefinitely until the lock is granted, and
that its locks merely advisory.

I haven't looked at the source, but I think it is this waiting indefinitely
part (polling?) that's causing CPU usage to go up.

Sinan.

 

[Anno Siegel]

(e-mail address removed) (jtd) wrote in


From perldoc -f flock:

Two potentially non-obvious but traditional "flock" semantics
are that it waits indefinitely until the lock is granted, and
that its locks merely advisory.

I haven't looked at the source, but I think it is this waiting indefinitely
part (polling?) that's causing CPU usage to go up.

No. Typically, file locking is event driven and no polling goes on.
Even when a lock is released implicitly (on close), the system knows
about it and can activate the new lock owner.

Anno

 

[A. Sinan Unur]

(e-mail address removed)-berlin.de (Anno Siegel) wrote in

No. Typically, file locking is event driven and no polling goes on.
Even when a lock is released implicitly (on close), the system knows
about it and can activate the new lock owner.

OK. Thanks for the correction.

Sinan.