locking a process

[Paul Archer]

I am in the process of developing a few scripts that:
1) pull images off a CF card on my laptop
2) move the images to a server
3) rename the images on the server according to the EXIF info


I figure that for 2 & 3, the easiest way to make sure they work is to have
them run in a cron job once a minute (or five). (Wake up, check to see if
there's any work, either do it or quit.)

The problem is that I need to make sure that if the script starts while
another copy of the script is running, it quits. I can work out how to
create lock files, stick in PIDs to make sure the script is really there,
etc. But it seems like someone must have tackled this problem already
(IIRC, I ran across a Perl module that back in my misspent youth).

Does anyone know of a gem (or just some canonical code) that handles this
easily?

Paul

PS I've also thought about using 'daemonize', but I'd still have to worry
about locking.

 

[Eleanor McHugh]

I am in the process of developing a few scripts that:
1) pull images off a CF card on my laptop
2) move the images to a server
3) rename the images on the server according to the EXIF info


I figure that for 2 & 3, the easiest way to make sure they work is
to have them run in a cron job once a minute (or five). (Wake up,
check to see if there's any work, either do it or quit.)

The problem is that I need to make sure that if the script starts
while another copy of the script is running, it quits. I can work
out how to create lock files, stick in PIDs to make sure the script
is really there, etc. But it seems like someone must have tackled
this problem already (IIRC, I ran across a Perl module that back in
my misspent youth).

Does anyone know of a gem (or just some canonical code) that handles
this easily?

Paul

PS I've also thought about using 'daemonize', but I'd still have to
worry about locking.

This is not for the faint-hearted, but if you grab the Ruby Plumber's
Guide from the link in my sig you'll find an example of using POSIX
semaphores with syscall for process synchronisation on slide 41. This
is a work in progress so you'll need to look up some syscall numbers
for your platform, however if you're not in a hurry I'm hoping to have
a ruby/dl version in the next version (for Rails Underground in July)
which will avoid this limitation.


Ellie

Eleanor McHugh
Games With Brains
http://slides.games-with-brains.net

 

[Daniel DeLorme]

I am in the process of developing a few scripts that:
1) pull images off a CF card on my laptop
2) move the images to a server
3) rename the images on the server according to the EXIF info


I figure that for 2 & 3, the easiest way to make sure they work is to
have them run in a cron job once a minute (or five). (Wake up, check to
see if there's any work, either do it or quit.)

The problem is that I need to make sure that if the script starts while
another copy of the script is running, it quits. I can work out how to
create lock files, stick in PIDs to make sure the script is really
there, etc. But it seems like someone must have tackled this problem
already (IIRC, I ran across a Perl module that back in my misspent youth).

What I do for this is simply rename the process:

abort("already running") if `pgrep -f my-wonderful-process` != ""
$0 = "my-wonderful-process"

 

[Brian Candler]

If you're running under a Unix-like system, then you can do something as
simple as this:

File.open("/tmp/mylock","w") do |f|
raise "Already running" unless f.flock(File::LOCK_EX | File::LOCK_NB)
... rest of code
end

Might even be able to combine the open and lock using IO.sysopen, I
haven't dug into that. Only works if all processes are on the same
machine of course.

However I don't think it's necessary for your job here:

1) pull images off a CF card on my laptop
2) move the images to a server
3) rename the images on the server according to the EXIF info

All your script needs to do is to 'grab' each file by renaming it using
File.rename, e.g. File.rename(fn, "#{fn}.working"). If this is
successful then it can be sure it has the file. If it fails, then it
means some other program grabbed the file first, so it can skip. However
if your script is aborted midway through it may leave some files in this
grabbed state, so they would need renaming back again manually.

Another option is to forget cron, and just start the program in the
background; it processes whatever files it can, sleeps for 300 seconds,
and then loops around. This will consume RAM on your machine while the
process sleeps though.

 

[Paul Archer]

What I do for this is simply rename the process:

abort("already running") if `pgrep -f my-wonderful-process` != ""
$0 = "my-wonderful-process"

Cool, thanks. I like this. Of course, it's still possible to have a race
condition, but it should be pretty unlikely in my case.

Paul

 

[Paul Archer]

I am in the process of developing a few scripts that:
1) pull images off a CF card on my laptop
2) move the images to a server
3) rename the images on the server according to the EXIF info


I figure that for 2 & 3, the easiest way to make sure they work is to have
them run in a cron job once a minute (or five). (Wake up, check to see if
there's any work, either do it or quit.)

The problem is that I need to make sure that if the script starts while
another copy of the script is running, it quits. I can work out how to create
lock files, stick in PIDs to make sure the script is really there, etc. But
it seems like someone must have tackled this problem already (IIRC, I ran
across a Perl module that back in my misspent youth).

Does anyone know of a gem (or just some canonical code) that handles this
easily?

Paul

PS I've also thought about using 'daemonize', but I'd still have to worry
about locking.

Just a followup: I've taken a closer look at 'daemons', and it actually
will only allow one instance of the script to run at once. That combined
with a cron job to make sure the script is still running, and I'm good to
go on this front.

Paul