system wide mutex

[Asaf Las]

Hi

Which one is most recommended to use for mutex alike locking to
achieve atomic access to single resource:

- fcntl.lockf
- os.open() with O_SHLOCK and O_EXLOCK
- https://pypi.python.org/pypi/lockfile/0.9.1
- https://pypi.python.org/pypi/zc.lockfile/1.1.0
- any other ?

Thanks

/Asaf

 

[Skip Montanaro]

Which one is most recommended to use for mutex alike locking to

achieve atomic access to single resource:

- fcntl.lockf
- os.open() with O_SHLOCK and O_EXLOCK
- https://pypi.python.org/pypi/lockfile/0.9.1
- https://pypi.python.org/pypi/zc.lockfile/1.1.0
- any other ?

As the author of lockfile, I can tell you it only implements advisory
locking. All programs needing to access the locked resources must
cooperate. It also has bugs which have been reported which I have yet
to spend any time fixing.

Beyond that, your question isn't really detailed enough to answer
completely. You don't identify what sort of systems you need this to
work on (Windows, Mac, various flavors of Unix?), whether programs
written in other languages will be involved (you did say "system
wide"), and whether you need to use it in the face of network file
systems like NFS or Samba.

Skip

 

[Asaf Las]

Forget to mentioned - CentOS 6.5 Python v3.3.

 

[Asaf Las]

As the author of lockfile, I can tell you it only implements advisory
locking. All programs needing to access the locked resources must
cooperate. It also has bugs which have been reported which I have yet
to spend any time fixing.
Beyond that, your question isn't really detailed enough to answer
completely. You don't identify what sort of systems you need this to
work on (Windows, Mac, various flavors of Unix?), whether programs
written in other languages will be involved (you did say "system
wide"), and whether you need to use it in the face of network file
systems like NFS or Samba.
Skip

Hi Yes i realized that missed info. The lock is needed
for 2 independently spawned processes to update set of files and non
non thread safe db

/Asaf

 

[Roy Smith]

Which one is most recommended to use for mutex alike locking to
achieve atomic access to single resource:

- fcntl.lockf
- os.open() with O_SHLOCK and O_EXLOCK
- https://pypi.python.org/pypi/lockfile/0.9.1
- https://pypi.python.org/pypi/zc.lockfile/1.1.0
- any other ?

As the author of lockfile, I can tell you it only implements advisory
locking. All programs needing to access the locked resources must
cooperate.[/QUOTE]

This is true of all mutexes, no?

We needed a mutex that worked across multiple machines, so we ended up
implementing a mutex in memcache. We based ours on
http://tinyurl.com/lybx2kl, but if you google for "memcache mutex
python", you'll find a number of implementations.

 

[Marko Rauhamaa]

Which one is most recommended to use for mutex alike locking to
achieve atomic access to single resource:

- fcntl.lockf

I recommend fcntl.flock:

========================================================================
#!/usr/bin/python3

import sys, fcntl, time

with open("test.lock", "w+") as lock:
fcntl.flock(lock.fileno(), fcntl.LOCK_EX)
# begin critical
sys.stdout.write("hello\n")
time.sleep(3)
sys.stdout.write("world\n")
# end critical
========================================================================


Marko

 

[Asaf Las]

Hi

Thanks for replies. It would be good to have blocking implementation.
I have to check fcntl if it works in blocking mdoe on CentOS.

Meanwhile there is Posix Semaphore made for Python:
http://semanchuk.com/philip/posix_ipc/

will try it as well.

/Asaf

 

[Chris Angelico]

This is true of all mutexes, no?

Not quite all; mandatory locks actually prevent other access. But they
require support from whatever makes the resource available - most
commonly, the file system - where cooperative locking doesn't. I've
written a number of cooperative locking systems in the past (eg to
protect a network resource, when the mounted file system didn't offer
the sort of locking I needed), and they're usually the way to go when
there's anything complicated going on.

What, all mutexes?
Yes, all mutexes!
What, all?
Well.... nearly all!

-- with apologies to WS Gilbert, HMS Pinafore, and the Pirates of Penzance

ChrisA

 

[Grant Edwards]

Hi

Which one is most recommended to use for mutex alike locking to
achieve atomic access to single resource:

- fcntl.lockf
- os.open() with O_SHLOCK and O_EXLOCK
- https://pypi.python.org/pypi/lockfile/0.9.1
- https://pypi.python.org/pypi/zc.lockfile/1.1.0
- any other ?

Posix "pthread" mutexes shared memory.

 

[Miki Tebeka]

IIRC creating a directory is atomic in most environments.