Looking for duplicate modules

[Roy Smith]

We got burned yesterday by a scenario which has burned us before. We had multiple copies of a module in sys.path. One (the one we wanted) was in our deployed code tree, the other was in /usr/local/lib/python/ or some such. It was a particularly confusing situation because we *knew* we had uninstalled the copy in /usr/local/lib. What we didn't know was that puppet was set up to check to make sure it was there and reinstalled it automagically if it ever disappeared!

What often adds to the confusion in cases like this is that if you use a package manager (such as apt-get) to manage your module deployments, when you uninstall, the .py files get removed, but the .pyc's stay behind.

I'm working on a tool which scans all the directories in sys.path and finds any modules which appear multiple times in the path. It'll also call out any .pyc's it finds without matching py's.

I know those are not strictly errors, but both of them are, for lack of a better term, "deployment smells" and something to be warned about. Before I invest too much effort into building it, does such a tool already exist?

We're slowly moving more of our deployment to using virtualenv (with --no-site-packages), but we're not fully there yet. And even when we get there, there will still be opportunities to make these sorts of mistakes.

 

[rusi]

We got burned yesterday by a scenario which has burned us before.  We had multiple copies of a module in sys.path.  One (the one we wanted) was in our deployed code tree, the other was in /usr/local/lib/python/ or some such.  It was a particularly confusing situation because we *knew* we haduninstalled the copy in /usr/local/lib.  What we didn't know was that puppet was set up to check to make sure it was there and reinstalled it automagically if it ever disappeared!

What often adds to the confusion in cases like this is that if you use a package manager (such as apt-get) to manage your module deployments, when you uninstall, the .py files get removed, but the .pyc's stay behind.

I'm working on a tool which scans all the directories in sys.path and finds any modules which appear multiple times in the path.  It'll also call out any .pyc's it finds without matching py's.

I know those are not strictly errors, but both of them are, for lack of abetter term, "deployment smells" and something to be warned about.  Before I invest too much effort into building it, does such a tool already exist?

Not remotely an answer.
emacs has a list-load-path-shadows for the same/similar issue in
elisp.
As a language moves to being a system (programming) language, it seems
to me necessary that such a functionality is built into the language
and gets called as part of standard installation procedure.

 

[Steven D'Aprano]

I'm working on a tool which scans all the directories in sys.path and
finds any modules which appear multiple times in the path. It'll also
call out any .pyc's it finds without matching py's.

This sounds like a useful tool. Please consider releasing it under an
appropriate free software licence (e.g. GPL, MIT, or similar compatible
licence).