#!/usr/bin/python or #!/usr/bin/env python?

[Tobias Brox]

[Erik Max Francis]

The point is, they're all part of the same tactic -- the particulars of
sh. Special casing each one is a task without an end. People will come
up with variants that will do the right thing but foil `file`,
intentionally or unintentionally -- just as we've seen in this thread.

I have no clue about Tcl, but it was said some postings ago that "all
Tcl-scripts are recognized as sh". If this is true, I think it ought
to be fixed. Searching for one of the very first lines to begin with
#, end with \, and followed by "exec tcl" doesn't sound like
rocket-science to me, though I've neither studied magic(4) or Tcl
scripting conventions.

I do understand that it would still be many ways to start a Tcl-script
without getting recognized by the above logic, as well as ways to
trick file into reporting a real shell script to be a Tcl-script.
IMHO a 90% hit rate is far better than 0% hit rate.

The right way to approach this with `file` is to acknowledge that such
tricks are inherently sh-specific and leave it identified as a sh file.
Because that is, of course, exactly what it is.

A shell script containing some inline tcl is a shell script, though
when the only shell-command is "start up tcl" and the rest of the file
is tcl-code, I really don't think it can be defined as a "shell
script" anymore. Particularly not if almost all tcl-scripts are
started that way.

 

[Sybren Stuvel]

ZeD enlightened us with:

$ cat test.py
#!/usr/bin/env python

print "Hello, world"
$ file test.py
file.py: a python script text executable

following what you said, test.py is a /usr/bin/env script, not a
python one.

It's still a Python one, since /usr/bin/env won't be executing the
script at all.

Sybren

 

[msoulier]

I understand the difference, but I'm just curious if anyone has any
strong feelings toward using one over the other?

I personally prefer being explicit over implicit, but then, that is in
the Zen of Python.

I work on machines with multiple interpreters installed. I find picking
the first one in the path to be very annoying. That means that it's
going to work differently for different people based on their PATH.

Mike

 

[Christophe]

Stephan Kuhagen a écrit :

Don't yell at me for bringing in another language, but I really like the
trick, Tcl does:


This works by the somewhat weird feature of Tcl, that allows comments to be
continued in the next line with "\" at the end of the comment-line. It
looks unfamiliar, but has several advantages, I think. First it's really
VERY unlikely, that there is no /bin/sh (while I found systems with
different places for env), and you can add some other features at or before
the actual call of the interpreter, i.e. finding the right or preferred
version... - This way I coded a complete software-installer, that runs
either as a Tcl/Tk-Script with GUI, or as bash-script, when no Tcl is
available. - I really would like to have something like that for python,
but I did not find a way to achieve that, yet.

Regards
Stephan

What about that one ( windows .bat file ) :

@echo off
rem = """-*-Python-*- script
rem -------------------- DOS section --------------------
rem You could set PYTHONPATH or TK environment variables here
python -x %~f0 %*
goto exit

"""
# -------------------- Python section --------------------
import sys
print sys.argv


DosExitLabel = """
:exit
rem """


I'm sure it can be updated to work on Unix too

 

[Aahz]

I understand the difference, but I'm just curious if anyone has any
strong feelings toward using one over the other? I was reading that a
disadvantage to the more general usage (i.e. env) is that it finds the
first python on the path, and that might not be the proper one to use. I
don't know if that's a real issue most of the time, but it's at least
something to consider.

The main argument against the env method is that I've seen a fair number
of sysadmins claim that it's less secure. I'm not competent to judget
that claim myself, but I prefer to play safe and stay away from env.
--
Aahz ([email protected]) <*> http://www.pythoncraft.com/

"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are, by
definition, not smart enough to debug it." --Brian W. Kernighan

 

[Erik Max Francis]

but...

$ cat test.py
#!/usr/bin/env python

print "Hello, world"
$ file test.py
file.py: a python script text executable

following what you said, test.py is a /usr/bin/env script, not a python one.

That's rather a silly interpretation of what I said, I think. After all::

% cat > test.gargle
#!/usr/bin/env gargle

asdfasflsafhklsfhklsdfhsdf
max@oxygen:~/tmp% file test.gargle
test.gargle: a gargle script text executable

`file` knows how /usr/bin/env is used because it's not only a standard
Unix idiom, it's the primary purpose for which it exists. That's not
the case with endless variants of /bin/sh scripts whose first statements
involve running another interpreter in some clever way that's not
detected by _that_ interpreter's processing, which is what the
previously discussed tricks amount to.

 

[Erik Max Francis]

A shell script containing some inline tcl is a shell script, though
when the only shell-command is "start up tcl" and the rest of the file
is tcl-code, I really don't think it can be defined as a "shell
script" anymore. Particularly not if almost all tcl-scripts are
started that way.

The problem is that there are endless ways to do that, and figuring out
all the cases makes `file` an sh interpreter, not the magic number
detector it's supposed to be.

 

[olsongt]

The main argument against the env method is that I've seen a fair number
of sysadmins claim that it's less secure. I'm not competent to judget
that claim myself, but I prefer to play safe and stay away from env.

Basically, someone could inject an arbirtrary script called 'python'
into your path that does whatever (rm -fr /) under your user context
when you run the script. But the same thing would happen if you run
'python test.py' instead of '/usr/local/bin/python test.py' to run a
script that doesn't have a she-bang or hasn't been flagged as
executable. Some admins will use a fully-qualified path for every
command to guard against this; I think that can be overkill.

 

[Erik Max Francis]

Basically, someone could inject an arbirtrary script called 'python'
into your path that does whatever (rm -fr /) under your user context
when you run the script. But the same thing would happen if you run
'python test.py' instead of '/usr/local/bin/python test.py' to run a
script that doesn't have a she-bang or hasn't been flagged as
executable. Some admins will use a fully-qualified path for every
command to guard against this; I think that can be overkill.

The primary guard for this is not having publicly-writable things in
your PATH. In other words, this is the argument for not putting things
like /tmp or . (because you might cd to somewhere publicly writable like
/tmp) in your PATH, not really for avoiding /usr/bin/env in hash bangs.

 

[olsongt]

The primary guard for this is not having publicly-writable things in
your PATH. In other words, this is the argument for not putting things
like /tmp or . (because you might cd to somewhere publicly writable like
/tmp) in your PATH, not really for avoiding /usr/bin/env in hash bangs.

Yeah, I agree 100%. I guess the best way to put it is that if random
people are creating scripts called 'python' in your path, you've got
bigger problems than /usr/bin/env calling the wrong program or a plain
old 'python test.py' doing the same.

 

[Stephan Kuhagen]

The problem is that there are endless ways to do that, and figuring out
all the cases makes `file` an sh interpreter, not the magic number
detector it's supposed to be.

It makes it into a pattern-matcher, not an interpreter. But that it is
already.

But right, there are endless ways to do that, but only one or a very small
subset is common. The way I cited is the way mentioned in the
Tclsh-manpage, so it can be (not must, but can) as the standard-header of a
Tcl-Script on Unix-like systems.

Even if the construct sometimes differs a little bit, "file" should be able
to identify, since the manpage of "file" says, that it is not a pure
magic-number-detector. The first part explains how the magic-number-thing
works and then what is done, when that fails:

"If a file does not match any of the entries in the magic file, it is
examined to see if it seems to be a text file.
[...]
Once file has determined the character set used in a text-type file, it will
attempt to determine in what language the file is written. The language
tests look for particular strings (cf names.h) that can appear anywhere in
the first few blocks of a file. For example, the keyword .br indicates
that the file is most likely a troff(1) input file, just as the keyword
struct indicates a C program. These tests are less reliable than the
previous two groups, so they are performed last."

This is not the most reliable way, as the man-page says, but it should work:
if in the first some blocks you can find a statement with a continued
comment-line and the exec-trick, "file" can at least guess, that it is a
Tcl-file. So this would be a valid method to detect that files type,
because a troff-file is a troff file, even when there is no .br in the
first few blocks, but "file" tries to identify it anyway. "file" is not
meant to be to ignore troff files, just because they are sometimes hard to
detect. The same applies to Tcl-files, I think. Not perfectly reliable, but
worth a try, since it is, according to the Tclsh-manpage, the common
header-pattern for a Tcl-script.

So "file" can not be perfect and reliable in every case, but it should try
to take a good guess. If you do not care about the Tcl-headers (and why
should you, this is comp.lang.python... ;-), you are right with your
reasoning. But if you accept, that file can not be perfect anyway and want
it to be as good as possible, then it is some kind of bug or missing
feature in "file" that it recognizes (or tries to) some morphing file
formats but not another (which is fairly wide spread, even if Tcl is not a
modern buzz-word-language these days).

Stephan