Anonymous file closing

[Sergey Krushinsky]

Hello all,

If I need to read a string from a file and use syntax like:
text = open(filename, 'r').read()
....
is the file object ever closed?

With best regards,
Sergey

 

[Duncan Booth]

Hello all,

If I need to read a string from a file and use syntax like:
text = open(filename, 'r').read()
...
is the file object ever closed?

With best regards,
Sergey

Yes it is closed, but it isn't well defined as to exactly when it is
closed. You can safely assume that the file will be closed sometime between
the read returning and your program exiting.

If you aren't worried about portability, and you are using the C
implementation of Python then you may find it is closed immediately but it
is poor practice to depend on it.

 

[Peter Hansen]

If I need to read a string from a file and use syntax like:
text = open(filename, 'r').read()
...
is the file object ever closed?

Duncan's response says it all, but here's the solution
if you don't like the uncertainty inherent in the above:

f = open(filename, 'r')
try:
text = f.read()
finally:
f.close()

That will ensure that the file is properly closed under
all versions of Python, all platforms...

For small scripts, I do what you did above. For anything
larger I take the more explicit approach.

-Peter

 

[Sergey Krushinsky]

Duncan's response says it all, but here's the solution
if you don't like the uncertainty inherent in the above:

f = open(filename, 'r')
try:
text = f.read()
finally:
f.close()

But the uncertainty remains in case of an anonymous file, doesn't it?

Sergey

 

[Sergey Krushinsky]

Yes it is closed, but it isn't well defined as to exactly when it is
closed. You can safely assume that the file will be closed sometime between
the read returning and your program exiting.

Is it also true when file object exists inside a function? Like:

def a():
text = open(filename, 'r').read()
...
....

It would be natural if the file object's memory were freed after a()
completion.

Sergey

 

[Terry Reedy]

Is it also true when file object exists inside a function? Like:

Short answer: yes.

Longer answer: the file object, like all Python objects, does not exist
inside anything except an implementation-defined-and-private 'dataspace'.
If there were a local name binding to the object, which there is not in the
example below, that binding, but not the object, would exist in the
function's local namespace during a particular invocation.

def a():
text = open(filename, 'r').read()
...

It would be natural if the file object's memory were freed after a()
completion.

'closing' the file, whatever that means for a particular system, and
freeing the object's memory are different operations, which have to occur
in that order.

In languages in which function locals are allocated on the stack just below
with the function's call frame, then deleting everything is trivial -- just
point the stack pointer above the expired call frame -- and therefore
natural. For items on the heap, or in an abstract dataspace, it is not
natural at all, since they carry no information about when and where they
were created. Moreover, if the function passes a refence to an object to a
surrounding context by writing/modify a non-local variable or by the return
statement, then freeing the object would be a mistake. One of the traps in
C is the possibility to return a pointer to a local variable just as it is
'naturally' and automatically freed by the return. CPython's reference
counting allows it to simulate C's 'natural' cleanup without deleting items
that should not be (because of other references). But this is
intentionally not a requirement for all Python interpreters.

Terry J. Reedy

 

[Jeff]

Yes it is closed, but it isn't well defined as to exactly when it is
closed. You can safely assume that the file will be closed sometime between
the read returning and your program exiting.

If you aren't worried about portability, and you are using the C
implementation of Python then you may find it is closed immediately but it
is poor practice to depend on it.

This is true, but I recommend:

1. Only use the C implementation of Python, a.k.a. The Implementation.
2. Depend on the file being closed immediately in this case. If the
authors of the C implementation ever changed this behavior, they would
break *lots* of existing python code, and they would also break one of
the nicest example of how Python just works.

 

[Peter Hansen]

But the uncertainty remains in case of an anonymous file, doesn't it?

What's an "anonymous file"? The term means nothing to me...

 

[Sergey Krushinsky]

What's an "anonymous file"? The term means nothing to me...

The file object created in: ... text = open(filename, 'r').read() ...
has no name, and cannot be accessed elsewhere. In ...f = open(... the
situation is different. It's like in two phrases: 'I was told that...'
and 'Peter told me that...".

Sergey

 

[Paul Rubin]

What's an "anonymous file"? The term means nothing to me...

I think it means the file descriptor created in a statement like

text = open(filename, 'r').read()

It's hard to clean up such a descriptor in a finally clause.

 

[Duncan Booth]

(e-mail address removed) (Jeff) wrote in

....

This is true, but I recommend:

1. Only use the C implementation of Python, a.k.a. The Implementation.
2. Depend on the file being closed immediately in this case. If the
authors of the C implementation ever changed this behavior, they would
break *lots* of existing python code, and they would also break one of
the nicest example of how Python just works.

Any existing Python code which depends on the file always being closed here
is wrong today. If the read method raises an exception the file will not be
closed until the exception traceback object has been cleared and that could
be a long time.

 

[Donn Cave]

Quoth Duncan Booth <[email protected]>:
| (e-mail address removed) (Jeff) wrote in
|
| ...
|>> > text = open(filename, 'r').read()
| ...
|>> If you aren't worried about portability, and you are using the C
|>> implementation of Python then you may find it is closed immediately
|>> but it is poor practice to depend on it.
|>
|> This is true, but I recommend:
|>
|> 1. Only use the C implementation of Python, a.k.a. The Implementation.
|> 2. Depend on the file being closed immediately in this case. If the
|> authors of the C implementation ever changed this behavior, they would
|> break *lots* of existing python code, and they would also break one of
|> the nicest example of how Python just works.
|
| Any existing Python code which depends on the file always being closed here
| is wrong today. If the read method raises an exception the file will not be
| closed until the exception traceback object has been cleared and that could
| be a long time.

Well, we may be getting into a semantic tight spot here. What does
`depend on' mean? I think a reasonable interpretation is that normal
files may be opened and read as shown above without concern about using
up file descriptors, holding files open and preventing them from being
effectively deleted, etc.

That is, in fact, true. In an ordinary Python application, it would be
silly to bundle that kind of thing up in try/finally blocks, and bog the
code down in clutter over something that really just works.

If some particular case comes up where the application should `depend on'
a particular close in a more critical sense, that's different, but that's
not the picture I get from the example. The only bad practice issue is
Java Python.

Donn Cave, (e-mail address removed)

 

[Sergey Krushinsky]

Well, we may be getting into a semantic tight spot here. What does
`depend on' mean? I think a reasonable interpretation is that normal
files may be opened and read as shown above without concern about using
up file descriptors, holding files open and preventing them from being
effectively deleted, etc.

I think, I can figure an example of such 'dependance'. This code:
open(filename, 'w').write('spam ' + open(filename, 'r').read())
does not save 'spam' plus the old file contents, as I expected firstly.
The result is just 'spam'. At the other hand, using file handles we can
make it work as expected.

Sergey

 

[Peter Hansen]

But the uncertainty remains in case of an anonymous file, doesn't it?

Uh, yeah... of course! Duncan posted code with uncertainty,
and I showed what one must change to make it certain (or less
uncertain, after reading Tim Peters' comment). How that would
change the original case with the anonymous file in any way
I just can't imagine. (Clearly I'm missing something, or you
are...)

-Peter

 

[Donn Cave]

I think, I can figure an example of such 'dependance'. This code:
open(filename, 'w').write('spam ' + open(filename, 'r').read())
does not save 'spam' plus the old file contents, as I expected firstly.
The result is just 'spam'. At the other hand, using file handles we can
make it work as expected.

True - I mean, I can see there's a problem here - but it really
is a different sort of problem.

When you combine these operations, you have an order of execution
problem that truncates your file before finalization becomes an
issue: the leftmost open happens first.

Donn Cave, (e-mail address removed)