Base class for file-like objects? (a.k.a "Stream" in Java)

[Guest]

Hello,

(sorry to begin with Java in a Python list ;-)
in Java, when I want to pass input to a function, I pass
"InputStream", which is a base class of any input stream.

In Python, I found that "file" objects exist. While specifying
argument types in Python is not possible as in Java, it is possible to
check whether an object is an instance of some class and that's what I
need - I need to check if an argument is a "file"-like object, and if
yes, behave accordingly, if not, treat the argument as string with
URL.

But I am afraid there is no such a base class - I tried the following:
False
....

Is there some base class to "file"-like (or "stream"-like) objects in
Python? And if not, is it at least planned for Python 3.0?

Thanks for any suggestions,
Boris Dušek

P.S.: The code should finally look in esence something like this:

if isinstance(f, file):
pass
elif isinstance(f, string):
f = urllib.urlopen(f)
else:
raise "..."
process_stream(f)

 

[Gabriel Genellina]

in Java, when I want to pass input to a function, I pass
"InputStream", which is a base class of any input stream.

In Python, I found that "file" objects exist. While specifying
argument types in Python is not possible as in Java, it is possible to
check whether an object is an instance of some class and that's what I
need - I need to check if an argument is a "file"-like object, and if
yes, behave accordingly, if not, treat the argument as string with
URL.

No, it's not what you need, it's what you *think* you need

P.S.: The code should finally look in esence something like this:

Posting this is much better that saying what you think you need.

if isinstance(f, file):
pass
elif isinstance(f, string):
f = urllib.urlopen(f)
else:
raise "..."
process_stream(f)

I can imagine that process_stream is something like this:

def process_stream(f):
...
data = f.read()
...

or similar. Then, you dont need a file object: you need something with a
read() method. So, this is what you should check in your code above.

if hasattr(f, "read"):
pass
elif isinstance(f, basestring):
f = urllib.urlopen(f)
else:
raise TypeError, "Expecting either a readable file-like object or an URL"
process_stream(f)

Or perhaps:

if isinstance(f, basestring):
f = urllib.urlopen(f)
process_stream(f)

and just let the exception happen below at f.read - which explains itself
rather well.

 

[Asun Friere]

In Python, I found that "file" objects exist. While specifying
argument types in Python is not possible as in Java, it is possible to
check whether an object is an instance of some class and that's what I
need - I need to check if an argument is a "file"-like object, and if
yes, behave accordingly, if not, treat the argument as string with
URL.

But I am afraid there is no such a base class - I tried the following:

Yup you are right. Look.[<type 'file'>, <type 'object'>]

"File-like object" refers the the behaviour of the object, not it's
inheritance. Python is an equal opportunity employer, we don't ask
and object what race it is, we simply ask "can you do the job?" This
is known colloquially as "duck-typing." In other words Python has
'polymorphism by interface,' in contradistinction to Java's
'polymorphism by implementation.' Until you understand this you will
be trying to write Java in Python, and you will not have much joy.

P.S.: The code should finally look in esence something like this:

if isinstance(f, file):
pass
elif isinstance(f, string):
f = urllib.urlopen(f)
else:
raise "..."
process_stream(f)

Because of duck-typing, you should almost never use the isintance()
(or type(), hasattr(), or any others that don't immediately come to
mind ...) in actual code. It (usually) breaks Python's polymorphism!
If you find these methods popping up in your code it strongly
indicates that you should be using a try/except statement, if not a
complete change of your code's logic. Search for LBYL (look before
you leap) vs EAFP (easier to ask forgiveness than permission) for a
full explanation.

I'm not sure why you would ever be sending a file object to urlopen
(or is the test isinstance(f, file) supposed to test for an already
opened url?), but you final code should actually look more like
something along these lines:

try :
f = urlopen(f)
except AttributeError :
pass

This is not as elegant as it could be, as it will pass not only on
open files (or urls), but on any type that lacks the .strip method
(ie. it doesn't account for your else condition). You'd probably have
to catch an exception later when you try to use what should be an open
url, or rewrite your code not to get in this position in the first
place. But however you refactor your code, it is advisable to
concentrate on what an object can do (and catch an exception where it
fails to perform), rather on the type of the object.

 

[tleeuwenburg]

In Python, I found that "file" objects exist. While specifying
argument types in Python is not possible as in Java, it is possible to
check whether an object is an instance of some class and that's what I
need - I need to check if an argument is a "file"-like object, and if
yes, behave accordingly, if not, treat the argument as string with
URL.

But I am afraid there is no such a base class - I tried the following:

Yup you are right. Look.>>> type(g).mro()

[<type 'instance'>, <type 'object'>]>>> type(f).mro()

[<type 'file'>, <type 'object'>]

"File-like object" refers the the behaviour of the object, not it's
inheritance. Python is an equal opportunity employer, we don't ask
and object what race it is, we simply ask "can you do the job?" This
is known colloquially as "duck-typing." In other words Python has
'polymorphism by interface,' in contradistinction to Java's
'polymorphism by implementation.' Until you understand this you will
be trying to write Java in Python, and you will not have much joy.

P.S.: The code should finally look in esence something like this:

if isinstance(f, file):
pass
elif isinstance(f, string):
f = urllib.urlopen(f)
else:
raise "..."
process_stream(f)

Because of duck-typing, you should almost never use the isintance()
(or type(), hasattr(), or any others that don't immediately come to
mind ...) in actual code. It (usually) breaks Python's polymorphism!
If you find these methods popping up in your code it strongly
indicates that you should be using a try/except statement, if not a
complete change of your code's logic. Search for LBYL (look before
you leap) vs EAFP (easier to ask forgiveness than permission) for a
full explanation.

I'm not sure why you would ever be sending a file object to urlopen
(or is the test isinstance(f, file) supposed to test for an already
opened url?), but you final code should actually look more like
something along these lines:

try :
f = urlopen(f)
except AttributeError :
pass

This is not as elegant as it could be, as it will pass not only on
open files (or urls), but on any type that lacks the .strip method
(ie. it doesn't account for your else condition). You'd probably have
to catch an exception later when you try to use what should be an open
url, or rewrite your code not to get in this position in the first
place. But however you refactor your code, it is advisable to
concentrate on what an object can do (and catch an exception where it
fails to perform), rather on the type of the object.

I agree that using try statements is often, perhaps usually, a good
idea. That said...

One good reason to avoid using a try: except: methodology is if the
actions taken in the try block could potentially modify data
structures or the underlying data during processing, or if the action
to be tried is expensive.

Being able to check before taking an action that it will be safe can
be important.

In these cases, you can use isinstance. Python 3k, I believe, is
thinking of being able to implement Abstract Base Classes for this
very reason.

As things stand, however, you will not be able to catch all cases with
a simple isinstance call. Try except should work in almost all cases.
If you want to do your own duck-typing, you can use dir(object) to get
its methods and check whether what you need is in there.

Anyway, good luck with it.

 

[George Sakkis]

Hello,

(sorry to begin with Java in a Python list ;-)
in Java, when I want to pass input to a function, I pass
"InputStream", which is a base class of any input stream.

In Python, I found that "file" objects exist. While specifying
argument types in Python is not possible as in Java, it is possible to
check whether an object is an instance of some class and that's what I
need - I need to check if an argument is a "file"-like object, and if
yes, behave accordingly, if not, treat the argument as string with
URL.

But I am afraid there is no such a base class - I tried the following:


False
...

Is there some base class to "file"-like (or "stream"-like) objects in
Python? And if not, is it at least planned for Python 3.0?

Thanks for any suggestions,
Boris Dušek

P.S.: The code should finally look in esence something like this:

if isinstance(f, file):
pass
elif isinstance(f, string):
f = urllib.urlopen(f)
else:
raise "..."
process_stream(f)

Other replies show you how to tackle this in Python 2.x. Python 3K
will come closer to Java by formalizing the concept of abstract base
classes [1] and will most likely include a fine-grained hierarchy of
stream-like base classes [2].

George


[1] http://www.python.org/dev/peps/pep-3119/
[2] http://www.python.org/dev/peps/pep-3116/