Why Can't I Delete a File I Created with Win XP?

[W. eWatson]

I'm trying to store analytic data in a folder called Analysis. If the
user doesn't have the folder, I make one for him, and then write a txt
file into it. In this case a histogram of values, x and frequency.
However, it appears that I made a mistake somewhere and cannot delete it
using the Win XP delete. Here's the code that determines if Analysis
exists, and whether to create it.

print "Path for Histogram", self.current_path
s = self.current_path
s = "Analysis"
s = os.path.join("Analysis",s)
print "s joined ",s <------------- debug print
if not os.path.exists("Analysis"):
os.mkdir("Analysis")
f = file( s, "wb")
if not f:
self.LogError( "Histogram file creation error 1" )
return False

In writing the code to handle all of this, I forgot to write the txt
folder to the file, nor did I close it. It was someone else's code, and
I got caught up in the creation of Analysis. Here's what XP tells me
when I try to delete the folder.

Very funny. I just tried it to get the content of the msg, and XP
deleted it. Let me recreate it and try again. OK, here it is:

Error Deleting Folder or File.
Cannot delete Analysis: It is being used by another person or program.
Close programs that might be using it and try again.

There is no file created, just the folders Analysis\Analysis. One too
many. The second Analysis shows as an icon for a file of size 0KB.

I printed with the debug print above:
Path for Histogram Events\v20070206_055012.06.dat
s joined Analysis\Analysis
Analysis\Analysis should only be Analysis.

 

[John Machin]

         s = self.current_path s referred to something ...
         s = "Analysis"

but now s refers to "Analysis" ... at best, there is redundant &
confusing code; at worst, the source of your problem.

         s = os.path.join("Analysis",s)

and now s refers to r"Analysis\Analysis" (on your platform)

         print "s joined ",s    <------------- debug print
[snip]

There is no file created, just the folders Analysis\Analysis. One too
many. The second Analysis shows as an icon for a file of size 0KB.

I printed with the debug print above:
   Path for Histogram Events\v20070206_055012.06.dat
   s joined  
Analysis\Analysis should only be Analysis.

Huh?? s = os.path.join("fubar", "fubar") should produce r"fubar
\fubar" (as documented) ... If you don't want s to refer to r"Analysis
\Analysis", then quite simply don't do that!

 

[W. eWatson]

The program code is not mine, but I wanted to modify it to produce an
Analysis folder when the user wants histogram file, basically, a txt
file to appear in Analysis.

Elsewhere in the program this is done for another type of data that is
directed to an Events folder. I figured I could copy the code from there
and use it here. Here's other code for Events.
# He's building a time stamp for the file name.
# I'll use this with a slight modification to the name in
# my code
millisecs = int((event_time - int(event_time))*100)
s = "v%4d%02d%02d_%02d%02d%02d.%02d.dat" % (
t.tm_year, t.tm_mon, t.tm_mday,
t.tm_hour, t.tm_min, t.tm_sec, millisecs )
#OK, he's got the file name assembled
s = os.path.join("Events",s)
if not os.path.exists("Events"):
os.mkdir("Events")
f = file( s, "wb" )
if not f:
self.LogError( "File creation error 1" )
return False

I caused the redundancy by just changing the meaning of s with a new
statement below it. I should have commented the first one out. Yeah, I
probably screwed up here and should have and,for purposes of debugging,
just used another file name like s="ADATAFILE2009_mmdd.txt", which does
not exist at this point of the coding stage. So I should have produced
Analysis\ADATAFILE2009_mmdd.txt. If I had done that then I would have
ended up with an empty file in the Analysis folder. However, even at
that, why can't I delete this empty file called Analysis?

When I said I was able to get rid of the "file", it's probably because I
had rebooted at some time after I puzzled over the problem, and now OS
somehow released it's grip, that is, whatever process was trying to use
the "file" too.

but now s refers to "Analysis" ... at best, there is redundant &
confusing code; at worst, the source of your problem.

and now s refers to r"Analysis\Analysis" (on your platform)

print "s joined ",s <------------- debug print
[snip]

There is no file created, just the folders Analysis\Analysis. One too
many. The second Analysis shows as an icon for a file of size 0KB.

I printed with the debug print above:
Path for Histogram Events\v20070206_055012.06.dat
s joined
Analysis\Analysis should only be Analysis.

Huh?? s = os.path.join("fubar", "fubar") should produce r"fubar
\fubar" (as documented) ... If you don't want s to refer to r"Analysis
\Analysis", then quite simply don't do that!

 

[John Machin]

[snip]

         f = file( s, "wb" )
         if not f:
             self.LogError( "File creation error 1" )
             return False

Either you are shadowing the built-in function file() or you haven't
tested this code ... file() aka open() returns a file object (which
cannot have a "false" value) or it raises an exception; the
self.LogError() call can never be executed.

I caused the redundancy by just changing the meaning of s with a new
statement below it. I should have commented the first one out. Yeah, I
probably screwed up here and should have and,for purposes of debugging,
just used another file name like s="ADATAFILE2009_mmdd.txt", which does
not exist at this point of the coding stage. So I should have produced
Analysis\ADATAFILE2009_mmdd.txt. If I had done that then I would have
ended up with an empty file in the Analysis folder.

Ever heard the phrase "need to know"?

However, even at
that, why can't I delete this empty file called Analysis?

Are you trying to delete the file from another command window while
Python is paused at the interactive prompt? In any case describe how
you are trying to delete the file. Fire up another command window, cd
to whatever is the current working directory for your script, do a dir
command, and copy/paste the RELEVANT parts (including all directory
names). Please don't top-post, and before you hit the send button,
please delete any text that is more appropriate to your memoirs than a
problem description.

 

[J]

Are you trying to delete the file from another command window while
Python is paused at the interactive prompt? In any case describe how
you are trying to delete the file. Fire up another command window, cd
to whatever is the current working directory for your script, do a dir
command, and copy/paste the RELEVANT parts (including all directory
names). Please don't top-post, and before you hit the send button,
please delete any text that is more appropriate to your memoirs than a
problem description.

I'm no expert in Python, so YMMV and all that, but sounds like, as was
mentioned before, that the file isn't being properly closed. I was
under the impression that the interpreter would close any open
filehandles as part of garbage collection as the script ends and the
interpreter closes out... but it's good practice to explicitly close
them in any case instead of just assuming that the interpreter or os
will do it for you... applicable to any language, not just python.

But that being said, this brings to mind a question about this... in
*nix, when I can't do something like delete a file or directory, or
unmount a filesystem and cant find the cause, I can do an lsof and
grep for references to the file/directory in question, then work from
there to close anything that still has a lock.

Most all of the development work I've ever done (other than C++ and
VB.Net) has been done on Linux systems. And honestly, most of my C++
stuff was written/compiled on Linux systems too... so I'm not that
experienced with developing applications on windows. Does Windows (XP
or later) have an lsof or lsof type tool to show who/what has a lock
on a file or folder? I've not found anything like that so far, but as
I said, I'm not much of a windows user anyway.

I'm at the point where I can actually write useful code in Python, and
not just hack my way through to a solution, so this is somethign that
would be useful for me to know. I'm about to start work on writing a
test tool in Python 2.6 which is going to have to work with a few
filehandles, both for read and write actions, as well as socket
connections eventually. So being able to find who has a lock on a
given file or directory if the program dies unexpectedly would be
useful.

Cheers
Jeff

 

[W. eWatson]

Google tells me that the program Process Explorer from SysInternals
will provide most of the functionality of lsof... so I'll look into
that and see if it will work. Certainly will help once I start
opening and writing output files with my python scripts on Windows
systems...

What I'm trying to do is really simple. In the Win XP NG, I have two
suggestions to get rid of the Analysis folder and the empty file in it.
One is to use a program like you suggested, and the other is to delete
it from DOS. I just tried cmd prompt, but was not successful. My DOS
skills are long gone, so I have no idea if there is something I
overlooked there. I bored down to Analysis and the into it. DIR showed
an unnamed empty file, so I tried DEL *. I seriously doubt it was removed.

Well, I'm going to reboot sometime later this evening, and knock it out
as I described I was able to do once before after a reboot. Thne I'm
going to fix the Python program and write a file correctly.

 

[W. eWatson]

The original program and code are not mine. I have no idea if that
specific piece of code has ever been tested. Generally the program works
quite well, and when needed creates the Events folder without any
difficulty. That folder is used heavily by writing new data files to it
thousands of times over a month.

I'm trying to do a very simple thing. I go to the Analysis folder, and
try to use Win XP Pro to delete the empty and unnamed file in it. One
just does a right-click on the empty file, and then uses Delete. It
won't let me delete it. If I back up to the folder level, it won't let
me delete the folder, since it's not empty. See the post I made to J
moments ago.

 

[J]

What I'm trying to do is really simple. In the Win XP NG, I have two
suggestions to get rid of the Analysis folder and the empty file in it. One
is to use a program like you suggested, and the other is to delete it from
DOS. I just tried cmd prompt, but was not successful. My DOS skills are long
gone, so I have no idea if there is something I overlooked there. I bored
down to Analysis and the into it. DIR showed an unnamed empty file, so I
tried DEL *. I seriously doubt it was removed.

Well, I'm going to reboot sometime later this evening, and knock it out as I
described I was able to do once before after a reboot. Thne I'm going to fix
the Python program and write a file correctly.

And those are your only options, really. From what I've been able to
see, there is no native ability in Linux to actually see who has a
lock on a file that's been opened. And I completely understand your
frustration.

I've not used the process explorer program, so I really can't say how
well it works for this kind of thing. But every post I found
searching for this indicates that it will at least tell you what still
has a lock on the file and maybe then you can figure out what needs to
be closed from there.

Rebooting should always work in cases like this. Rebooting the system
should clear all file locks and is a last resort for a persistent
stale file lock. So yeah, by rebooting, you'll always be able to
release the lock on that file and then delete it once the system is
back and running.

However, that's not something you'd want to do on a production system
except as a last resort. At least, that's not something that I'd want
to do.

So anyway, since you said the code is not yours, does the code
actually close the files anywhere? I'm assuming that it does at some
point, but if not, that really is something that needs to be added in.
As I said in my last post, I am just a novice in the Python stuff,
but I've written enough code in my life to know that you never assume
that a file will be closed properly by the system once the program is
finished running. I'm not saying that YOU specifically are doing
this, but just making the suggestion as this is the kind of problem
that can happen.

Of course, a program that's dying for some reason is a different story...

 

[Tom Pacheco]

I'm trying to store analytic data in a folder called Analysis. If the
user doesn't have the folder, I make one for him, and then write a txt
file into it. In this case a histogram of values, x and frequency.
However, it appears that I made a mistake somewhere and cannot delete
it using the Win XP delete. Here's the code that determines if
Analysis exists, and whether to create it.

print "Path for Histogram", self.current_path
s = self.current_path
s = "Analysis"
s = os.path.join("Analysis",s)
print "s joined ",s <------------- debug print
if not os.path.exists("Analysis"):
os.mkdir("Analysis")
f = file( s, "wb")
if not f:
self.LogError( "Histogram file creation error 1" )
return False

In writing the code to handle all of this, I forgot to write the txt
folder to the file, nor did I close it. It was someone else's code,
and I got caught up in the creation of Analysis. Here's what XP tells
me when I try to delete the folder.

Very funny. I just tried it to get the content of the msg, and XP
deleted it. Let me recreate it and try again. OK, here it is:

Error Deleting Folder or File.
Cannot delete Analysis: It is being used by another person or program.
Close programs that might be using it and try again.

There is no file created, just the folders Analysis\Analysis. One too
many. The second Analysis shows as an icon for a file of size 0KB.

I printed with the debug print above:
Path for Histogram Events\v20070206_055012.06.dat
s joined Analysis\Analysis
Analysis\Analysis should only be Analysis.

noticed your assigning the same variable twice
-------------------------------
s = self.current_path
s = "Analysis"
-------------------------------
if you are expecting
Analysis\Events\v20070206_055012.06.dat

delete the second line

as others have said if you can not delete the file if must still be open.

why are you talking about deleting a file but only show code for opening it?

 

[W. eWatson]

And those are your only options, really. From what I've been able to
see, there is no native ability in Linux to actually see who has a
lock on a file that's been opened. And I completely understand your
frustration.

I've not used the process explorer program, so I really can't say how
well it works for this kind of thing. But every post I found
searching for this indicates that it will at least tell you what still
has a lock on the file and maybe then you can figure out what needs to
be closed from there.

Rebooting should always work in cases like this. Rebooting the system
should clear all file locks and is a last resort for a persistent
stale file lock. So yeah, by rebooting, you'll always be able to
release the lock on that file and then delete it once the system is
back and running.

However, that's not something you'd want to do on a production system
except as a last resort. At least, that's not something that I'd want
to do.

So anyway, since you said the code is not yours, does the code
actually close the files anywhere? I'm assuming that it does at some
point, but if not, that really is something that needs to be added in.
As I said in my last post, I am just a novice in the Python stuff,
but I've written enough code in my life to know that you never assume
that a file will be closed properly by the system once the program is
finished running. I'm not saying that YOU specifically are doing
this, but just making the suggestion as this is the kind of problem
that can happen.

Of course, a program that's dying for some reason is a different story...

Well, I'm pretty well done now. I fixed the program so it would write
the file and close it. It did this successfully. I'll worry about
getting the oddball file deleted later.

 

[J]

J wrote:

Crud.. that should have said no native ability in Windows... not Linux...

Well, I'm pretty well done now. I fixed the program so it would write the
file and close it. It did this successfully. I'll worry about getting the
oddball file deleted later.

Cool. Glad you got it worked out... good luck!

 

[Hans Mulder]

But that being said, this brings to mind a question about this... in
*nix, when I can't do something like delete a file or directory, or
unmount a filesystem and cant find the cause, I can do an lsof and
grep for references to the file/directory in question, then work from
there to close anything that still has a lock.

It's been a while since a last had this problem. I found a program
named "handle.exe" somewhere on microsoft.com. It spits out a listing
showing which process has which files open. You can kill the offending
processes, and then you can delete your file.

I suppose Google can help you find this program.


Hope this helps,

-- HansM

 

[Dave Angel]

It's been a while since a last had this problem. I found a program
named "handle.exe" somewhere on microsoft.com. It spits out a listing
showing which process has which files open. You can kill the offending
processes, and then you can delete your file.

I suppose Google can help you find this program.


Hope this helps,

-- HansM

Handle.exe and "Process Explorer" (mentioned by J) are both from
Sysinternals, which was acquired by Microsoft, and to be found on their
site.

http://technet.microsoft.com/en-us/sysinternals/default.aspx

http://technet.microsoft.com/en-us/sysinternals/bb896655.aspx for handle.exe
http://technet.microsoft.com/en-us/sysinternals/bb896653.aspx for process explorer

But as long as I'm in the thread, I'll state that I suspect the problem to be simply that the OP is running his script inside some IDE, and it hasn't really ended. So the first thing I'd try is to exit the IDE, and see whether the file is then released.

DaveA