Function for the path of the script?

[Peter Cacioppi]

Am I the only one who finds this function super useful?

def _code_file() :
return os.path.abspath(inspect.getsourcefile(_code_file))


I've got one in every script. It's the only one I have to copy around. For my workflow ... so handy.

I've got os.path.dirname aliased to dn, so its dn(_code_file()) that I find myself reaching for fairly often...

 

[Skip Montanaro]

Strange. I almost never want to know the location of the file that's
executing, and when I do, I just reference the __file__ module level
attribute.

Skip

 

[Gary Herron]

Am I the only one who finds this function super useful?

def _code_file() :
return os.path.abspath(inspect.getsourcefile(_code_file))


I've got one in every script. It's the only one I have to copy around. For my workflow ... so handy.

I've got os.path.dirname aliased to dn, so its dn(_code_file()) that I find myself reaching for fairly often...

Huh? In what kind of a workflow are you running a python file without
knowing *what* file you are runnung?

Or am I just misinterpreting what this code does?

Confused but curious,
Gary Herron

 

[Chris Angelico]

Huh? In what kind of a workflow are you running a python file without
knowing *what* file you are runnung?

It's very common to want to know what directory you're in - it's a
good way to find data files.

ChrisA

 

[Steven D'Aprano]

Am I the only one who finds this function super useful?

def _code_file() :
return os.path.abspath(inspect.getsourcefile(_code_file))


I've got one in every script. It's the only one I have to copy around.
For my workflow ... so handy.

I've got os.path.dirname aliased to dn, so its dn(_code_file()) that I
find myself reaching for fairly often...

I'm afraid I don't find it useful. I won't say that I *never* need to
check the location of a module, but it's certainly very rare. I'm curious
-- what are you doing that you copy this function into every script?

Also, as given, the above function fails if the source code is not
available (say, you provide only the .pyc file without any corresponding
source). A better way is to inspect the module object's __file__
attribute:

py> import math
py> math.__file__
'/usr/local/lib/python2.7/lib-dynload/math.so'

(Built-in modules which don't exist as a separate file, like sys, have no
__file__ attribute, so be prepared to catch the exception.)


Inside the module itself, the easiest way to get your own path is to
check the global variable __file__.

 

[rurpy]

That's a naive way to do it (though it's often good enough, for a
program only used on one system).

For programs intending to be used across many systems, the data files
often shouldn't be placed in the same directory as the program.

On systems conforming to the Filesystem Hierarchy Standard, it's
forbidden: programs go in a platform-specific location
<URL:http://refspecs.linuxfoundation.org/FHS_2.3/fhs-2.3.html#USRLIBLIBRARIESFORPROGRAMMINGANDPA>,
while platform-independent data files go in a separate location
<URL:http://refspecs.linuxfoundation.org/FHS_2.3/fhs-2.3.html#USRSHAREARCHITECTUREINDEPENDENTDATA>.

What about Python applications installed in /opt?
On my systems, unless a third party app is packaged and
distributed by the OS package manager, it goes in /opt
and AFAIK, it is blessed practice to keep all the app
files in the same subdirectory tree there.

 

[Chris Angelico]

That's a naive way to do it (though it's often good enough, for a
program only used on one system).

I never said it was right for a properly-installed program, just that
it's common - which it definitely is, and this not a Python thing,
it's across all languages. As you say, often good enough for simple
things.

Your main questions definitely still stand though, and I don't know
any answer to them.

ChrisA

 

[Gregory Ewing]

On systems conforming to the Filesystem Hierarchy Standard, it's
forbidden: programs go in a platform-specific location
<URL:http://refspecs.linuxfoundation.org/FHS_2.3/fhs-2.3.html#USRLIBLIBRARIESFORPROGRAMMINGANDPA>,
while platform-independent data files go in a separate location
<URL:http://refspecs.linuxfoundation.org/FHS_2.3/fhs-2.3.html#USRSHAREARCHITECTUREINDEPENDENTDATA>.

But Python code itself is platform-independent, so it
should count as data for the purposes of the FHS,
shouldn't it?

 

[Roy Smith]

I've used ‘os.path.dirname(os.path.abspath(__file__))’ to find the
directory containing the current file, in the past. But that was before
Python's ‘unittest’ module could discover where the test code lives.


What workflow requires you to know the filename of the module, within
the module?

We use:

config_dir = os.path.abspath(os.path.dirname(__file__))
songza_dir, _ = os.path.split(config_dir)
top_dir, _ = os.path.split(songza_dir)
assert os.path.exists(top_dir + "/.hg")
return top_dir

in our config files. This gives us the absolute path to the top of our
source tree (I don't remember why we do the double os.path.split()
instead of the more obvious "../.."). We know the file where this code
lives is two levels down from the top of the source tree.

Once we've got the absolute path to the top of the tree, that's used in
all sorts of places to locate data files, and to generate configurations
for other applications (nginx, etc).

 

[Ian Kelly]

But Python code itself is platform-independent, so it
should count as data for the purposes of the FHS,
shouldn't it?

I don't see why Python files should be treated any differently than
other non-binary executables, e.g. shell scripts.

 

[Chris Angelico]

It is an unfortunate artefact of Unix history that “binary” has an
established connotation of “executable”, encompassing even executable
text files.

That's a lot broader than Unix - people talk about "binaries" meaning
executables in Windows and OS/2 too. Unix is, if anything, _less_
inclined that way - the executable segment is called "text", which
always struck me as a bit odd.

So the separation I'm drawing attention to in the FHS has nothing to do
with whether the files are text files, and everything to do with whether
they're executable programs and code libraries.

Yup. Unix does a fairly good job of blurring the line between
"executables that can be loaded and jumped to" and "scripts that get
loaded by an interpreter". I actually have a few scripts that take
several levels of interpreter, something like:

foo.pike
#!/usr/local/bin/pike

bar.pike:
#!/.../foo.pike --parameter

fum.pike:
#!/.../bar.pike --otherparameter

Unix will happily execute ./fum.pike as "/usr/local/bin/pike
/.../foo.pike --parameter /.../bar.pike --otherparameter ./fum.pike".
There's a limit on the number of interpreters (to prevent loops), but
I haven't hit it

There is one important place, though, where scripts are called data
files, and that's licensing. The GPL, for instance, does NOT cover
your scripts, even if it covers the interpreter, because *to the
language interpreter*, your scripts are just data files. But that's
more of a legal distinction than a filesystem hierarchical one.

ChrisA

 

[Mark Lawrence]

That's a lot broader than Unix - people talk about "binaries" meaning
executables in Windows and OS/2 too. Unix is, if anything, _less_
inclined that way - the executable segment is called "text", which
always struck me as a bit odd.


Yup. Unix does a fairly good job of blurring the line between
"executables that can be loaded and jumped to" and "scripts that get
loaded by an interpreter". I actually have a few scripts that take
several levels of interpreter, something like:

foo.pike
#!/usr/local/bin/pike

bar.pike:
#!/.../foo.pike --parameter

fum.pike:
#!/.../bar.pike --otherparameter

Unix will happily execute ./fum.pike as "/usr/local/bin/pike
/.../foo.pike --parameter /.../bar.pike --otherparameter ./fum.pike".
There's a limit on the number of interpreters (to prevent loops), but
I haven't hit it

There is one important place, though, where scripts are called data
files, and that's licensing. The GPL, for instance, does NOT cover
your scripts, even if it covers the interpreter, because *to the
language interpreter*, your scripts are just data files. But that's
more of a legal distinction than a filesystem hierarchical one.

ChrisA

Quoting from another thread

"What is the difference between "script" code (like Javascript and
visual) made for the screen (where such magic values are utilized) and
compiled source (made for the machine)?"

This obviously impacts on the discussion above, so how does Unix,
Windows and other operating systems distinguish these with respect to
binary, executable, code library or whatever?

 

[Chris Angelico]

"What is the difference between "script" code (like Javascript and visual)
made for the screen (where such magic values are utilized) and compiled
source (made for the machine)?"

This obviously impacts on the discussion above, so how does Unix, Windows
and other operating systems distinguish these with respect to binary,
executable, code library or whatever?

Suppose you pull up a shell - for argument's sake, let's say it's
Debian GNU/Linux and you're running bash. You get a prompt that ends
with a dollar sign, and you type "ls". What's going to get executed?

* An alias? You might have an internal function defined in your
..bashrc, or maybe a wrapper that adds parameters to your command.

* A bash internal command? The shell might directly interpret what you
specified. (I don't think ls is like that, but time is, on my
systems.)

* An external binary? On my systems, /bin/ls is an executable binary,
compiled and ready to run.

* A script? Another alternative to the shell alias, you could have
/usr/local/bin/ls that does something different, then maybe drops
through to /bin/ls. If it starts with "#!/usr/bin/python", it'll get
dropped through to Python for execution.

Chances are you wouldn't know the difference, as a human executing the
commands. And you shouldn't need to care, except in really weird
circumstances (maybe you broke your Python install and need to type
"/bin/ls" to figure out what's going on).

Most programs, trying to execute code, won't care about the difference
between binaries and scripts, though of course exec*() won't parse
bash aliases or internals. But if you need to distinguish for whatever
reason, the easiest way is to look at the magic numbers, which can be
done with the 'file' command:

rosuav@sikorsky:~$ file `which ls`
/bin/ls: ELF 64-bit LSB executable, x86-64, version 1 (SYSV),
dynamically linked (uses shared libs), for GNU/Linux 2.6.26,
BuildID[sha1]=0x55f1e005df252708d4c456dcc2c7dccea1006553, stripped

rosuav@sikorsky:~$ file `which zcat`
/bin/zcat: Bourne-Again shell script, ASCII text executable

Executables happily together, regardless of type.

ChrisA

 

[Roy Smith]

If it starts with "#!/usr/bin/python", it'll get
dropped through to Python for execution.

Even better (for most purposes), use "#!/usr/bin/env python". What that
does is (slight handwave here) search your PATH to find the same version
of Python you would get if you typed "python" at a shell prompt.

If you've only got a single version of python installed, it doesn't
matter. But it'll matter a lot if you have multiple versions (or even
multiple installations of the same version, but with different sets of
installed modules).

 

[Chris Angelico]

Even better (for most purposes), use "#!/usr/bin/env python". What that
does is (slight handwave here) search your PATH to find the same version
of Python you would get if you typed "python" at a shell prompt.

Yeah, I'm aware of that... but I dodged a bit of complexity by
hard-coding the path The shebang you quote drops it through to env,
which waves its hands vigorously and says "Abracadabra", before
dropping it through to the Python that you forgot you ran 'make
install' instead of 'make altinstall' on.

ChrisA

 

[Grant Edwards]

On Sun, Oct 27, 2013 at 1:52 PM, Gary Herron

I've been writing Python programs for 10+ years. Never have I felt a
need to know the location of the file that's running. I do sometimes
want to know the _name_ of the file (for multipurpose programs), but I
don't ever remember caring where that file was.

It's very common to want to know what directory you're in - it's a
good way to find data files.

From a Unix point of view, that's also very wrong. Data files don't
belong in the same directory as the executable.

 

[Grant Edwards]

What workflow requires you to know the filename of the module, within
the module?

If you have a utility that can be used to do several related things,
one way to tell that utility which you want to do is with command line
arguments. For example your utility checks sys.argv[1] for a command
or option flag. Another way is to give the file multiple names, and
check sys.argv[0] to see what name you've been invoked under. The
latter approach not unusual in the Unix world, and has been carried to
a rather astounding extent by the "busybox" project:

http://www.busybox.net/about.html

It's all the command-line utilities you need for a small Unix system
in a single binary: one executable with dozens and dozens of links
(hard or symbolic).

 

[Chris Angelico]

From a Unix point of view, that's also very wrong. Data files don't
belong in the same directory as the executable.

For installed programs, maybe; but we run a lot of scripts out of
source control, like a sanity checker that gets run immediately before
a commit. Some of them need data files that are also source-managed,
so we have them in the same directory. Locating those files is usually
easiest done by tailing onto the script's path.

ChrisA

 

[Peter Cacioppi]

Ben Finney asked

"What workflow requires you to know the filename of the module, within
the module? "

So what I have is a project utility script in my scripts directory. I have a distinct file structure that holds my .py source, my test.py unit tests, and the file based data associated with the unit tests.

Each test.py is in a dedicated directory.

My utility script can be easily tweaked to do a variety of useful things, one of which is leave the interactive session with a variable that points toa testing directory whose unit test failed. It relies on each test.py having a same named function that knows its directory.

It sounds like there is a more pythonic way to do what I am doing, but alsothat I am not completely out to lunch.

Sounds about par for my python code at this point. As my code used to be almost exclusively out to lunch, I think I am improving (thanks guys!).

 

[rurpy]

In the case of applications installed in /opt, data files
quite often go in a subdirectory of the executable (or a
subdirectory of the executable's parent dir) and getting
the executable's dir is the first step to find the data
files.

This is not only *not wrong* but the FHS prohibits locating
such files outside of the /opt subdirectory tree, with some
minor exceptions.

This was pointed out before but since you said you ignore
posts from GG you probably missed it, and will probably miss
this one too, and thus continue to post bad information.

This is a small but illustrative example of why such broad-
brush filtering is a bad idea. But it is your choice.