PyWhich

[Billy Mays]

Hey c.l.p.,

I wrote a little python script that finds the file that a python module
came from. Does anyone see anything wrong with this script?


#!/usr/bin/python

import sys
if __name__ == '__main__':
if len(sys.argv) > 1:
try:
m = __import__(sys.argv[1])
sys.stdout.write(m.__file__ + '\n')
sys.stdout.flush()
sys.exit(0)
except ImportError:
sys.stderr.write("No such module '%s'\n" % sys.argv[1])
sys.stderr.flush()
sys.exit(1)
else:
sys.stderr.write("Usage: pywhich <module>\n")
sys.stderr.flush()
sys.exit(0)

 

[Chris Rebert]

Hey c.l.p.,

I wrote a little python script that finds the file that a python module came
from.  Does anyone see anything wrong with this script?


#!/usr/bin/python

import sys
if __name__ == '__main__':
   if len(sys.argv) > 1:
       try:
           m = __import__(sys.argv[1])
           sys.stdout.write(m.__file__ + '\n')
           sys.stdout.flush()
           sys.exit(0)
       except ImportError:
           sys.stderr.write("No such module'%s'\n" % sys.argv[1])
           sys.stderr.flush()
           sys.exit(1)
   else:
       sys.stderr.write("Usage: pywhich <module>\n")
       sys.stderr.flush()
       sys.exit(0)

Nothing wrong per se, but the flush()es seem unnecessary, and why do
stdout.write() when you can just print()?

Cheers,
Chris

 

[Tim Chase]

Hey c.l.p.,

I wrote a little python script that finds the file that a python module
came from. Does anyone see anything wrong with this script?

#!/usr/bin/python

import sys
if __name__ == '__main__':
if len(sys.argv)> 1:
try:
m = __import__(sys.argv[1])
sys.stdout.write(m.__file__ + '\n')

For a quick script in a controlled environment, not bad. In a
hostile environment, I'd be nervous about running arbitrary
module code triggered by the import. Even if non-malicious, some
imports (like PyCrypto) may have some initialization lag which
would be nice to avoid. I think I'd make use of imp.find_module
to write it something like this (untested)

from sys import argv, stderr
import imp
type_map = {
imp.PY_SOURCE: "Source file",
imp.PY_COMPILED: "Compiled code object",
imp.C_EXTENSION: "Dynamically loadabld shared library",
imp.PKG_DIRECTORY: "Package directory",
imp.C_BUILTIN: "Built-in",
imp.PY_FROZEN: "Frozen module",
}
if __name__ == '__main__':
if len(argv) > 1:
for modname in argv[1:]:
try:
fp, pth, desc = imp.find_module(modname)
(suffix, mode, type_) = desc
if fp is not None: fp.close()
print("%s\t[%s]" % (
pth,
type_map.get(type_, "UNKNOWN")
))
except ImportError:
stderr.write("No such module '%s'\n" % modname)
else:
stderr.write("Usage: pywhich <module> [<module>...]\n")

I don't know a good way to tap into other import hooks (such as
the zipfile import) to augment that type_map dictionary.

-tkc

 

[Steven D'Aprano]

Hey c.l.p.,

I wrote a little python script that finds the file that a python module
came from. Does anyone see anything wrong with this script?

Yes -- the most screamingly obvious question has to be, why are you writing
directly to sys.stdout instead of just using print?

#!/usr/bin/python

I believe the recommended, platform independent hash-bang line is

#!/usr/bin/which python

import sys
if __name__ == '__main__':
if len(sys.argv) > 1:
try:
m = __import__(sys.argv[1])

The major risk here is whether or not you trust the module enough to import
it and run arbitrary code. The alternative would be a lot more work: you
would have to duplicate the logic of the import statement, search the
PYTHONPATH, look for packages, inside zip files, etc.

sys.stdout.write(m.__file__ + '\n')

Built-in modules don't have a __file__ attribute. The most obvious example:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute '__file__'

Also, __file__ is not always entirely honest. In Python 3, module.__file__
may report a .py file when it is actually loaded from a .pyc file, even if
the .py file doesn't exist. So if you care about the distinction
between .py, .pyc, .pyo etc. files, looking at __file__ alone will be
inadequate.

except ImportError:
sys.stderr.write("No such module '%s'\n" % sys.argv[1])

That's not a given -- it may be that the module exists, but importing fails
for some other reason. I would recommend not catching ImportError at all,
and just let the standard Python error handling print the traceback.
Especially for a tool aimed at programmers (who else would be interested in
PyWhich?), hiding useful diagnostic errors and replacing them with a
generic, and potentially wrong, message, is very bad.

 

[Tim Chase]

I believe the recommended, platform independent hash-bang line is

#!/usr/bin/which python

I think you mean

#!/usr/bin/env python

-tkc

 

[Chris Angelico]

Especially for a tool aimed at programmers (who else would be interested in
PyWhich?)

The use that first springs to my mind is debugging import paths etc.
If you have multiple pythons installed and aren't sure that they're
finding the right modules, you could fire up PyWhich on an innocuous
module like math or sys, and see if it's loading it from the right
path. People doing this might not necessarily be programmers, they
might be sysadmins; but you're right that it's most likely this will
be used by competent Python programmers.

ChrisA

 

[Steven D'Aprano]

I think you mean

#!/usr/bin/env python

Doh! I *always* conflate env and which. Thank you for the correction.

 

[Tim Golden]

Doh! I *always* conflate env and which. Thank you for the correction.

And there I was thinking you were making a sly and ironic point
about using a utility to find something on the path in order to
run a utility which finds something on the path...

TJG

 

[John Gordon]

Doh! I *always* conflate env and which. Thank you for the correction.

Way to say "conflate"!

 

[Web Dreamer]

Billy Mays a écrit ce jeudi 4 août 2011 14:43 dans

Hey c.l.p.,

I wrote a little python script that finds the file that a python module
came from. Does anyone see anything wrong with this script?


#!/usr/bin/python

import sys
if __name__ == '__main__':
if len(sys.argv) > 1:
try:
m = __import__(sys.argv[1])
sys.stdout.write(m.__file__ + '\n')
sys.stdout.flush()
sys.exit(0)
except ImportError:
sys.stderr.write("No such module '%s'\n" % sys.argv[1])
sys.stderr.flush()
sys.exit(1)
else:
sys.stderr.write("Usage: pywhich <module>\n")
sys.stderr.flush()
sys.exit(0)

It does not work for all modules:

$ python pywhich.py os
/usr/lib/python2.6/os.pyc

$ python pywhich.py sys
Traceback (most recent call last):
File "pywhich.py", line 8, in <module>
sys.stdout.write(m.__file__ + '\n')
AttributeError: 'module' object has no attribute '__file__'

Some modules do not have the __file__ attribute
(usually modules in C)

 

[Billy Mays]

The use that first springs to my mind is debugging import paths etc.
If you have multiple pythons installed and aren't sure that they're
finding the right modules, you could fire up PyWhich on an innocuous
module like math or sys, and see if it's loading it from the right
path. People doing this might not necessarily be programmers, they
might be sysadmins; but you're right that it's most likely this will
be used by competent Python programmers.

ChrisA

I am trying to do debugging. I have had some trouble with multiple
python installs with virtualenv, and I was trying to see where given
modules came from. I knew about the code execution, but I couldn't
think of a clean way to just find out the location rather than load it.

The reason I used stdout was because I was going to be using it in a
tool chain where the stdout might need to be formatted for another
program to read in. Thats also why I was catching ImportError since a
later version of this script might need to do something special with it.

This is also useful to see if python is really using the module you
think it is.

 

[Steven D'Aprano]

The reason I used stdout was because I was going to be using it in a
tool chain where the stdout might need to be formatted for another
program to read in.

print writes to sys.stdout unless you tell it different.
'spam\n'

Syntax for printing elsewhere is ugly as sin in Python 2, but it works:

'spam\nham\n'




Thats also why I was catching ImportError since a
later version of this script might need to do something special with it.

Well you better also catch SyntaxError, because a later version of your
script might need to do something special with it too

Also RuntimeError, ValueError, TypeError... *wink*