win32 How to make sure a file is completely written?

[justind]

Hello,

I'm using http://code.activestate.com/recipes/156178/ to watch a
folder in windows. It's working perfectly, but sometimes when I try to
open the file immediately after receiving the event, it's not ready to
be opened--if I try to open it with PIL I get "IOError: cannot
identify image file" and if I try it with a text file, it's empty.
This doesn't happen all the time, just occasionally. I think the
problem is that the file isn't completely written because if I make
the script sleep for a second, it works every time. But that doesn't
seem very elegant or robust.

What's the proper way to make sure the file is ready to be read?

I'm just passing the file names from the above recipe to a function.
Something like

def handler(files):
for file in files:
im = Image.open(file)

Thanks

 

[ma]

You have to wait until IO is ready. In Unix, we accomplish this with
fcntl and the default signal SIGIO, I am not sure how you would do
this in Windows.

 

[Tim Golden]

Hello,

I'm using http://code.activestate.com/recipes/156178/ to watch a
folder in windows.

Wow, that takes me back. There's a bit more info (and a different
technique) here if you're interested:

http://timgolden.me.uk/python/win32_how_do_i/watch_directory_for_changes.html

But to come back your question...

It's working perfectly, but sometimes when I try to
open the file immediately after receiving the event, it's not ready to
be opened--if I try to open it with PIL I get "IOError: cannot
identify image file" and if I try it with a text file, it's empty.
This doesn't happen all the time, just occasionally. I think the
problem is that the file isn't completely written because if I make
the script sleep for a second, it works every time. But that doesn't
seem very elegant or robust.

What's the proper way to make sure the file is ready to be read?

I don't believe there's an easy answer to this question. From the
filesystem's point of view some process creates a file. So it
tells you that a file has been created. The filesystem has no
knowledge of when a file is "complete". It could -- altho' it
doesn't -- fire an event when a handle on that file is closed,
but even that wouldn't tell you much: you'd have to be able
to distinguish between the handle which is behind the data
you're looking for and a handle from a virus scanner which is
checking the new file for malware.


To add difficulty, when Windows initiates a file copy (as opposed
to creating a new file) it immediately creates the file at its
full size and then copies data in. Which makes some sense as it
should avoid some element of fragmentation etc. But it removes
one of your options: noting the size of an original file and
comparing it with the size of a copy.

So I think you're out of luck and you're going to have to
put a try-except loop in place.

TJG

 

[justind]

Wow, that takes me back. There's a bit more info (and a different
technique) here if you're interested:

 http://timgolden.me.uk/python/win32_how_do_i/watch_directory_for_chan....

But to come back your question...



I don't believe there's an easy answer to this question. From the
filesystem's point of view some process creates a file. So it
tells you that a file has been created. The filesystem has no
knowledge of when a file is "complete". It could -- altho' it
doesn't -- fire an event when a handle on that file is closed,
but even that wouldn't tell you much: you'd have to be able
to distinguish between the handle which is behind the data
you're looking for and a handle from a virus scanner which is
checking the new file for malware.

To add difficulty, when Windows initiates a file copy (as opposed
to creating a new file) it immediately creates the file at its
full size and then copies data in. Which makes some sense as it
should avoid some element of fragmentation etc. But it removes
one of your options: noting the size of an original file and
comparing it with the size of a copy.

So I think you're out of luck and you're going to have to
put a try-except loop in place.

TJG

Thanks Guys.