PEP 8 exegetics: conditional imports?

[kj]

Conditional imports make sense to me, as in the following example:

def foobar(filename):
if os.path.splitext(filename)[1] == '.gz':
import gzip
f = gzip.open(filename)
else:
f = file(filename)
# etc.

And yet, quoth PEP 8:

- Imports are always put at the top of the file, just after any module
comments and docstrings, and before module globals and constants.

....which seems to condemn conditional imports unequivocally.

Then again, the PEP 8 scriptures do not explicitly mention conditional
imports at all, as far as I can tell, which leaves open the
possibility that they are still righteous.

In fact, venerable passages in the Python standard library source
code, if properly interpreted, can be seen to carry out conditional
imports, such as this fragment recovered from random.py:

if a is None:
try:
a = long(_hexlify(_urandom(16)), 16)
except NotImplementedError:
import time
a = long(time.time() * 256) # use fractional seconds

Or even more clearly, this one from test/pystone.py:

if __name__ == '__main__':
import sys



I seek the wisdom of the elders. Is there a consensus on the matter
of conditional imports? Are they righteous? Or are they the way
of the wicked?

kynn

 

[alex23]

Conditional imports make sense to me, as in the following example[...]
And yet, quoth PEP 8:

    - Imports are always put at the top of the file, just after any module
      comments and docstrings, and before module globals and constants.

...which seems to condemn conditional imports unequivocally.

Really? It doesn't mention conditionals at all, it simply says that
imports should occur before globals and constants, not before
conditions. If you want to put conditions around your imports, then
that's the place to do it.

I seek the wisdom of the elders.  Is there a consensus on the matter
of conditional imports?  Are they righteous?  Or are they the way
of the wicked?

Bear in mind that PEP 8 primarily applies to submissions to the
standard library, in order to provide a standard that aids in
understanding them. If a module is used throughout a body of code,
it's helpful to list these modules at the top of the code, for
clarity. However, if a module is only required under exceptional
conditions, you'll often see it imported at the point which it's
required: as the import occurs near the code, this mitigates the
initial requirement somewhat, and reduces the startup time of the
code.

The style guide also states:
But most importantly: know when to be inconsistent -- sometimes
the style
guide just doesn't apply. When in doubt, use your best judgment.
Look
at other examples and decide what looks best. And don't hesitate
to ask!

Checks around imports are often used to provide cross-version
compatibility. Embedded imports are often used when the code relying
on them is barely used. These are all very common uses.

pystone is a good example. Nothing within the modular code of pystone
requires sys, it's only imported if the module is executed, primarily
for error reporting & argument handling.

 

[Albert Hopkins]

Conditional imports make sense to me, as in the following example:

def foobar(filename):
if os.path.splitext(filename)[1] == '.gz':
import gzip
f = gzip.open(filename)
else:
f = file(filename)
# etc.

And yet, quoth PEP 8:

- Imports are always put at the top of the file, just after any module
comments and docstrings, and before module globals and constants.

...which seems to condemn conditional imports unequivocally.

Then again, the PEP 8 scriptures do not explicitly mention conditional
imports at all, as far as I can tell, which leaves open the
possibility that they are still righteous.

In fact, venerable passages in the Python standard library source
code, if properly interpreted, can be seen to carry out conditional
imports, such as this fragment recovered from random.py:

if a is None:
try:
a = long(_hexlify(_urandom(16)), 16)
except NotImplementedError:
import time
a = long(time.time() * 256) # use fractional seconds

Or even more clearly, this one from test/pystone.py:

if __name__ == '__main__':
import sys

I can't speak for others... but generally you can "tell" when an import
belongs at the top of a module and when you need to make exceptions. I
would say that, as a general rule, they go up top (easier to identify
what external dependencies a module have). There are, of course,
exceptions:

* Exceptions and other not-often-executed blocks
* Inside test functions (where the imported modules are only used
by tests
* Inside main() functions where the imported modules are only used
if the module is run as a script
* When startup-time is important. Often it's necessary to have you
module up and running in an instant and import "expensive"
modules as-needed.
* "Dynamic" import such as plugins, etc.

Of course, like everything else in PEP 8, it's meant as a "guide" and
not as an "order". YMMV.

-a

 

[Albert Hopkins]

Conditional imports make sense to me, as in the following example:

def foobar(filename):
if os.path.splitext(filename)[1] == '.gz':
import gzip
f = gzip.open(filename)
else:
f = file(filename)
# etc.

I should add that in your example I would probably still put the import
at the top, e.g.:

import gzip
[...]

def foobar(filename):
if os.path.splitext(filename)[1] == '.gz':
f = gzip.open(filename)
else:
f = open(filename)

Reason being is that if later on I decide I want to write another
function inside my module that does the same thing I don't have to do
the same conditional import.

Even better, if this is something you find yourself doing often you can
create your own "smart" open and put it in a library:

# file anyfile

import __builtin__
import gzip

def open(filename, ...):
if filename.endswith('.gz'):
f = gzip.open(filename, ...)
else:
f = __builtin__.open(f, ...)

return f

Then:

-a

 

[Peter Otten]

Conditional imports make sense to me, as in the following example:

def foobar(filename):
if os.path.splitext(filename)[1] == '.gz':
import gzip
f = gzip.open(filename)
else:
f = file(filename)
# etc.

And yet, quoth PEP 8:

- Imports are always put at the top of the file, just after any module
comments and docstrings, and before module globals and constants.

...which seems to condemn conditional imports unequivocally.

Then again, the PEP 8 scriptures do not explicitly mention conditional
imports at all, as far as I can tell, which leaves open the
possibility that they are still righteous.

In fact, venerable passages in the Python standard library source
code, if properly interpreted, can be seen to carry out conditional
imports, such as this fragment recovered from random.py:

if a is None:
try:
a = long(_hexlify(_urandom(16)), 16)
except NotImplementedError:
import time
a = long(time.time() * 256) # use fractional seconds

Or even more clearly, this one from test/pystone.py:

if __name__ == '__main__':
import sys



I seek the wisdom of the elders. Is there a consensus on the matter
of conditional imports? Are they righteous? Or are they the way
of the wicked?

If you want to take a rational approach measure speed and memory footprint
of your program both with the conditional and unconditional imports. Then go
with the conditional imports only if you can demonstrate a noticeable
benefit.

This criterion is unlikely to be met for the examples you give above. time
is a built-in module, and gzip a thin wrapper around zlib which is also
built-in.

Peter

 

[alex23]

This criterion is unlikely to be met for the examples you give above. time
is a built-in module, and gzip a thin wrapper around zlib which is also
built-in.

This is something I was _utterly_ unaware of. Is there a list of what
modules are built-in readily available?

The only reference[1] I could find via Google doesn't list zlib at
all.

[1]: http://www.vldb.informatik.hu-berlin.de/Themen/manuals/python/python-texinfo/built-in_modules.html

 

[Dave Angel]

Conditional imports make sense to me, as in the following example:

def foobar(filename):
if os.path.splitext(filename)[1] == '.gz':
import gzip
f = gzip.open(filename)
else:
f = file(filename)
# etc.

And yet, quoth PEP 8:

- Imports are always put at the top of the file, just after any module
comments and docstrings, and before module globals and constants.

...which seems to condemn conditional imports unequivocally.

Then again, the PEP 8 scriptures do not explicitly mention conditional
imports at all, as far as I can tell, which leaves open the
possibility that they are still righteous.

In fact, venerable passages in the Python standard library source
code, if properly interpreted, can be seen to carry out conditional
imports, such as this fragment recovered from random.py:

if a is None:
try:
a = long(_hexlify(_urandom(16)), 16)
except NotImplementedError:
import time
a = long(time.time() * 256) # use fractional seconds

Or even more clearly, this one from test/pystone.py:

if __name__ == '__main__':
import sys

I can't speak for others... but generally you can "tell" when an import
belongs at the top of a module and when you need to make exceptions. I
would say that, as a general rule, they go up top (easier to identify
what external dependencies a module have). There are, of course,
exceptions:

* Exceptions and other not-often-executed blocks
* Inside test functions (where the imported modules are only used
by tests
* Inside main() functions where the imported modules are only used
if the module is run as a script
* When startup-time is important. Often it's necessary to have you
module up and running in an instant and import "expensive"
modules as-needed.
* "Dynamic" import such as plugins, etc.

Of course, like everything else in PEP 8, it's meant as a "guide" and
not as an "order". YMMV.

-a

Excellent points. I'd add a couple more, one pro, and two con:

You need conditionals when you're doing things that are
environment-specific. For example, you might use Win32 stuff when on
Windows, and something else when on Linux. So you might have a pair of
modules which encapsulate the non-portable aspects, and conditionally
import one of those.

You usually want any imports to be complete before starting
multithreading. As somebody else has recently pointed out, there's an
import-lock that's active during imports,

If two modules do mutual imports (not a good idea, but it is sometimes
practically unavoidable), the earlier in the file that the import
happens, the less likely for a problem to crop up. So try to put them
at the top.

 

[Peter Otten]

This is something I was _utterly_ unaware of. Is there a list of what
modules are built-in readily available?

('__builtin__', '__main__', '_ast', '_bisect', '_codecs', '_collections',
'_functools', '_locale', '_random', '_socket', '_sre', '_struct',
'_symtable', '_warnings', '_weakref', 'array', 'binascii', 'cPickle',
'cStringIO', 'errno', 'exceptions', 'fcntl', 'gc', 'grp', 'imp',
'itertools', 'marshal', 'math', 'operator', 'posix', 'pwd', 'select',
'signal', 'spwd', 'strop', 'sys', 'syslog', 'thread', 'time', 'unicodedata',
'xxsubtype', 'zipimport', 'zlib')

Peter

 

[Steven D'Aprano]

I seek the wisdom of the elders. Is there a consensus on the matter of
conditional imports? Are they righteous? Or are they the way of the
wicked?

Being an elder doesn't grant you wisdom. It just might mean you've been
stupid and ignorant for a long time.

For what it's worth, you can take my conditional imports when you've
pried them from my cold, dead hands.