Proposal: add sys to __builtins__

[Michael Hoffman]

What would people think about adding sys to __builtins__ so that
"import sys" is no longer necessary? This is something I must add to
every script I write that's not a one-liner since they have this idiom
at the bottom:

if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))

Additionally, the necessity of "import sys" makes some one-liners a
little more unwieldy than they should be--it is surely the module I am
missing the most in one-liners. For example, with this proposal, this
inelegant one-liner:

$ python -c "import sys; print ''.join(sorted(sys.stdin.readlines()))"

could be replaced by:

$ python -c "print ''.join(sorted(sys.stdin.readlines()))"

Since sys is surely the most commonly used module (it is imported in 108
of 188 Python 2.4 stdlib modules on my system, and certainly more than
any other module), I would hope few people would be affected by a
namespace collision.

Other languages (e.g. C#) always make their system namespace available
without needing a special import.

In short, given the wide use of sys, its unambiguous nature, and the
fact that it really is built-in already, although not exposed as such, I
think we would be better off if sys were always allowed even without an
import statement.

 

[Rick Wotnaz]

What would people think about adding sys to __builtins__ so that
"import sys" is no longer necessary? This is something I must
add to every script I write that's not a one-liner since they
have this idiom at the bottom:

if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))

Additionally, the necessity of "import sys" makes some
one-liners a little more unwieldy than they should be--it is
surely the module I am missing the most in one-liners. For
example, with this proposal, this inelegant one-liner:

$ python -c "import sys; print
''.join(sorted(sys.stdin.readlines()))"

could be replaced by:

$ python -c "print ''.join(sorted(sys.stdin.readlines()))"

Since sys is surely the most commonly used module (it is
imported in 108 of 188 Python 2.4 stdlib modules on my system,
and certainly more than any other module), I would hope few
people would be affected by a namespace collision.

Other languages (e.g. C#) always make their system namespace
available without needing a special import.

In short, given the wide use of sys, its unambiguous nature, and
the fact that it really is built-in already, although not
exposed as such, I think we would be better off if sys were
always allowed even without an import statement.

+1 here. As far as I'm concerned, both os and sys could be special-
cased that way. That said, I would guess the likelihood of that
happening is 0.

 

[Steve Holden]

What would people think about adding sys to __builtins__ so that
"import sys" is no longer necessary? This is something I must
add to every script I write that's not a one-liner since they
have this idiom at the bottom:

if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))

Additionally, the necessity of "import sys" makes some
one-liners a little more unwieldy than they should be--it is
surely the module I am missing the most in one-liners. For
example, with this proposal, this inelegant one-liner:

$ python -c "import sys; print
''.join(sorted(sys.stdin.readlines()))"

could be replaced by:

$ python -c "print ''.join(sorted(sys.stdin.readlines()))"

Since sys is surely the most commonly used module (it is
imported in 108 of 188 Python 2.4 stdlib modules on my system,
and certainly more than any other module), I would hope few
people would be affected by a namespace collision.

Other languages (e.g. C#) always make their system namespace
available without needing a special import.

In short, given the wide use of sys, its unambiguous nature, and
the fact that it really is built-in already, although not
exposed as such, I think we would be better off if sys were
always allowed even without an import statement.

+1 here. As far as I'm concerned, both os and sys could be special-
cased that way. That said, I would guess the likelihood of that
happening is 0.

I wonder if it would be worth special-casing the AttributeError
exception handling at the outermost lexical scope to try and import a
module with the troublesome name and then retrying the attribute access
if the import succeeded (possibly even from a path limited to the
standard locations).

That way none of the standard library modules would need to be imported
before use.

I can see that this would create problems as well as solving some, but
it would be nice not to have to import the standard library modules. I
don't personally find it a hardship, but some people do.

though-i-could-just-be-raving-ly y'rs - steve

 

[MrJbQ7]

I wonder if it would be worth special-casing the AttributeError [snip]

What is it that Tim Peters said? "Special cases aren't special
enough..."

Besides, a better way is to use your ~/.pythonrc file for customizing
according to your needs.

A simple:

echo "import sys, os" >> ~./pythonrc

will do the job.

Thanks,
John Benediktsson

 

[Michael Hoffman]

I wonder if it would be worth special-casing the AttributeError [snip]

What is it that Tim Peters said? "Special cases aren't special
enough..."

That suggestion is a little too much magic for me.

Besides, a better way is to use your ~/.pythonrc file for customizing
according to your needs.

A simple:

echo "import sys, os" >> ~./pythonrc

will do the job.

Until someone else tries to use your script or module.

 

[tiissa]

Michael Hoffman a écrit :

Until someone else tries to use your script or module.

A developper should not be too lazy to add one small line in a complete
script/module.
Besides your entire justification to this proposal was based on shell
one-liners, not script or modules.

 

[Michael Hoffman]

A developper should not be too lazy to add one small line in a complete
script/module.

To the contrary, I agree with Larry Wall that laziness is one of the
cardinal virtues of a programmer. Although my personal desire to be lazy
is not at issue here--I always start new scripts and modules from a
template that includes import sys.

Would you argue that the language is superior because half of its
modules must have "import sys" at the beginning, although sys is already
imported? The only difference is that the namespace is not exposed by
default. I suggest that the default be changed.

It would simplify introductions to the language as well. In the current
tutorial it is necessary to use "import sys" before it has time to
explain modules and importing.

Besides your entire justification to this proposal was based on shell
one-liners, not script or modules.

Sorry, that's incorrect; please go back and read the original post again.

 

[Paul Watson]

What would people think about adding sys to __builtins__ so that
"import sys" is no longer necessary? This is something I must
add to every script I write that's not a one-liner since they
have this idiom at the bottom:

if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))

Additionally, the necessity of "import sys" makes some
one-liners a little more unwieldy than they should be--it is
surely the module I am missing the most in one-liners. For
example, with this proposal, this inelegant one-liner:

$ python -c "import sys; print
''.join(sorted(sys.stdin.readlines()))"
could be replaced by:

$ python -c "print ''.join(sorted(sys.stdin.readlines()))"

Since sys is surely the most commonly used module (it is
imported in 108 of 188 Python 2.4 stdlib modules on my system,
and certainly more than any other module), I would hope few
people would be affected by a namespace collision.

Other languages (e.g. C#) always make their system namespace
available without needing a special import.

In short, given the wide use of sys, its unambiguous nature, and
the fact that it really is built-in already, although not
exposed as such, I think we would be better off if sys were
always allowed even without an import statement.

+1 here. As far as I'm concerned, both os and sys could be special-
cased that way. That said, I would guess the likelihood of that
happening is 0.

I wonder if it would be worth special-casing the AttributeError
exception handling at the outermost lexical scope to try and import a
module with the troublesome name and then retrying the attribute access
if the import succeeded (possibly even from a path limited to the
standard locations).

That way none of the standard library modules would need to be imported
before use.

I can see that this would create problems as well as solving some, but
it would be nice not to have to import the standard library modules. I
don't personally find it a hardship, but some people do.

though-i-could-just-be-raving-ly y'rs - steve

This sounds pretty interesting. How about a switch to invoke this
handling for the one-liner crowd and those who wish to use it?

Somehow, I never heard any C programmers suggest that the default
processing not include the need for:

#include <stdio.h>

 

[tiissa]

To the contrary, I agree with Larry Wall that laziness is one of the
cardinal virtues of a programmer.

There's lazy and too lazy.
You don't want to be too lazy to even get out of bed to code in Python.
Of course, with Perl, that's entirely another mattress^Wmatter.

Would you argue that the language is superior because half of its
modules must have "import sys" at the beginning

I wouldn't dare arguing about superiority.

I was just stating your proposal didn't really solve anything. A good
editor/template and .pythonrc already save you the typing of 'import
sys' in scripts for the former and shell command for the latter.

Sorry, that's incorrect

Alright, that was a bit of an overstatement.
I should have said your proposal is perceptibly useful in those shell
one-liners. The distribution of script and modules is another matter.


As for my opinion, you've already guessed I don't perceive 'import sys'
as an issue. Therefore, the special case of an implicit import of sys
does not appeal to me.

 

[Xiao Jianfeng]

This sounds pretty interesting. How about a switch to invoke this
handling for the one-liner crowd and those who wish to use it?

Somehow, I never heard any C programmers suggest that the default
processing not include the need for:

#include <stdio.h>

I think it is because that we cannot modify the C language, but
python is a language that's still evolving.

 

[Colin J. Williams]

What would people think about adding sys to __builtins__ so that
"import sys" is no longer necessary? This is something I must
add to every script I write that's not a one-liner since they
have this idiom at the bottom:

if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))

Additionally, the necessity of "import sys" makes some
one-liners a little more unwieldy than they should be--it is
surely the module I am missing the most in one-liners. For
example, with this proposal, this inelegant one-liner:

$ python -c "import sys; print
''.join(sorted(sys.stdin.readlines()))"

could be replaced by:

$ python -c "print ''.join(sorted(sys.stdin.readlines()))"

Since sys is surely the most commonly used module (it is
imported in 108 of 188 Python 2.4 stdlib modules on my system,
and certainly more than any other module), I would hope few
people would be affected by a namespace collision.

Other languages (e.g. C#) always make their system namespace
available without needing a special import.

In short, given the wide use of sys, its unambiguous nature, and
the fact that it really is built-in already, although not
exposed as such, I think we would be better off if sys were
always allowed even without an import statement.

+1 here. As far as I'm concerned, both os and sys could be special-
cased that way. That said, I would guess the likelihood of that
happening is 0.

+1 for both.

Colin W.

 

[Terry Reedy]

+1 for both.

Some people might prefer that math be special cased. Or some other module.
A neutral criterion is needed.

Terry J. Reedy

 

[Rick Wotnaz]

Some people might prefer that math be special cased. Or some
other module. A neutral criterion is needed.

Actually, I don't think it's beyond reason to suggest that any
module included with the standard distribution be special-cased in
this way. That is, a reference to xxx.func(), without a previous
import of xxx *could* be resolvable automatically, at least for
those modules that can be found in a standard location. Whether
it's a good idea to do so is another question.

Offhand, I am not sure why it would be an insanely poor idea. It
would mean some funky namespace building and possibly redundant
imports, I guess. I'll certainly defer to just about anybody else's
opinion as to the difficulty and advisability, but I believe it
would be possible to do.

Note that I am not saying that a reference to, say, 'argv' should
provoke an automatic import. I don't mean that some automatic
search for a matching function name should be done through some
undefined module chain. I'm talking only about qualified
references, like os.path or sys.stderr.

As it is, a NameError is generated if the proper import has not
been done. At the point of that error, the module name is known.
For the programmer to fix the situation, an import statement has to
be added to the code and then the code must be rerun. I don't see
why it would be impossible to make that happen automatically,
provided the module to be imported was recognized (through some
undefined magic). I'd think the module would have to be known,
because trying to import a nonexistent module -- "import ssy", say
(in the event of a typo for "sys") -- would fail, so there would
have to be a way to avoid looping on the "ssy.func()" line.

Is it worth adding that kind of complexity to execution of a Python
program? Probably not.

Is it even a good idea to try to make it happen automatically?
Possibly not. For one thing, it wouldn't always be the right thing
to do. A developer would most likely want to set up the imports
properly, and to know when they were not correctly set up instead
of having Python fix things quietly. There's a reason why Explicit
is better than Implicit.

But *could* it be done? I'd think so.

 

[Sybren Stuvel]

Rick Wotnaz enlightened us with:

That is, a reference to xxx.func(), without a previous import of xxx
*could* be resolvable automatically, at least for those modules that
can be found in a standard location.

-1 on that one. If I want to use a module, I'll write an import
statement for it. Those import statements make it very easy to get an
overview of the modules in use.

Automatically importing modules has the counter-effect of not
generating errors when they should be. Someone could have a class
stored in a variable 'os' and call a function on it. If this person
forgets to assign a value to 'os', an error message should be given
instead of importing the 'os' module.

Another issue is speed. If every reference in the form xxx.yyy has to
trigger an import when xxx isn't known, it will probably slow down
programs quite badly.

A programming language should not be ambiguous. The choice between
importing a module and calling a function should not depend on the
availability of a (local) variable.

I don't see why it would be impossible to make that happen
automatically, provided the module to be imported was recognized
(through some undefined magic).

The question is not if it's possible or not - in principle, everything
is possible. The question is if it is desirable.

A developer would most likely want to set up the imports properly,
and to know when they were not correctly set up instead of having
Python fix things quietly.
Yep.

But *could* it be done? I'd think so.

Sure.

Sybren

 

[Michael J. Fromberger]

What would people think about adding sys to __builtins__ so that
"import sys" is no longer necessary? This is something I must
add to every script I write that's not a one-liner since they
have this idiom at the bottom:

if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))

[...]

In short, given the wide use of sys, its unambiguous nature, and
the fact that it really is built-in already, although not
exposed as such, I think we would be better off if sys were
always allowed even without an import statement.

+1 here. As far as I'm concerned, both os and sys could be special-
cased that way. That said, I would guess the likelihood of that
happening is 0.

While I'm mildly uncomfortable with the precedent that would be set by
including the contents of "sys" as built-ins, I must confess my
objections are primarily aesthetic: I don't want to see the built-in
namespace any more cluttered than is necessary -- or at least, any more
than it already is.

But "os" is another matter -- the "os" module contains things which
might well not be present in an embedded Python environment (e.g., file
and directory structure navigation, stat). Do you really want to force
everyone to deal with that? Is it so much more work to add "import os"
to those Python programs that require it?

Of course, you might counter "why should we force everybody else to type
`import os' just in case somebody wants to imbed Python?" But then, why
don't we just include the whole standard library in __builtins__? Or,
since that would be too much, maybe we survey the user community and
include the top fifteen most included modules! Where do you draw the
line? Do you really want to hard-code user opinions into the language?

Right now, we have a nice, simple yet effective mechanism for
controlling the contents of our namespaces. I don't think this would be
a worthwhile change. -1.

-M

 

[Rick Wotnaz]

What would people think about adding sys to __builtins__ so
that "import sys" is no longer necessary? This is something I
must add to every script I write that's not a one-liner since
they have this idiom at the bottom:

if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))

[...]

In short, given the wide use of sys, its unambiguous nature,
and the fact that it really is built-in already, although not
exposed as such, I think we would be better off if sys were
always allowed even without an import statement.

+1 here. As far as I'm concerned, both os and sys could be
special- cased that way. That said, I would guess the
likelihood of that happening is 0.

While I'm mildly uncomfortable with the precedent that would be
set by including the contents of "sys" as built-ins, I must
confess my objections are primarily aesthetic: I don't want to
see the built-in namespace any more cluttered than is necessary
-- or at least, any more than it already is.

But "os" is another matter -- the "os" module contains things
which might well not be present in an embedded Python
environment (e.g., file and directory structure navigation,
stat). Do you really want to force everyone to deal with that?
Is it so much more work to add "import os" to those Python
programs that require it?

Of course, you might counter "why should we force everybody else
to type `import os' just in case somebody wants to imbed
Python?" But then, why don't we just include the whole standard
library in __builtins__? Or, since that would be too much,
maybe we survey the user community and include the top fifteen
most included modules! Where do you draw the line? Do you
really want to hard-code user opinions into the language?

Right now, we have a nice, simple yet effective mechanism for
controlling the contents of our namespaces. I don't think this
would be a worthwhile change. -1.

You're right that there is no necessity for such a change. I was
not actually talking about importing *any* module in every case,
but rather about importing, say, 'sys' when, for example, sys.argv
appeared in the code and no import had been specified. In another
post I go into a little more detail about what I meant, but in any
case I did not and do not think it's necessary. I have no problem
keying in the import statement and the current system works fine.
It could be argued that it would be convenient in the case of quick
utility code not to have to import well-known modules explicitly,
and it could be argued that 'sys' in particular contains much that
is so seemingly fundamental that it could be built in. I'd not
argue that it should be split out if it were already built in; it
seems to be borderline standard as it is. I suppose it's a C
mindset talking, there.

When I'm cobbling together a Q&D script, my routine often involves
running it once and then going back in and inserting the single
import line that I forgot. It's a minor annoyance, because it seems
clear to me that the needed information is actually available to
Python when it kicks out its NameError. But it *is* minor, and I am
not seriously proposing a change to Python in this regard.

 

[Christopher Subich]

> While I'm mildly uncomfortable with the precedent that would be set

by including the contents of "sys" as built-ins, I must confess my
objections are primarily aesthetic: I don't want to see the built-in
namespace any more cluttered than is necessary -- or at least, any more
than it already is.


I agree with this sentiment, and I'll also additionally say that 'import
sys' doesn't seem to be needed when writing sufficiently high-level
code. My python mud client (forever in development, but the
structure-code is mostly done) uses TKinter, Twisted, and glue code for
just about everything.

In currently 1,080 lines of Python code (reported by wc -l, so it
includes a few blank lines) in 9 files, I needed "import sys" once. [1]

After I import sys, I use it exactly once -- I check the platform so I
can use the higher resolution time.clock on win32 [time.time on win32
(win2k) seems to have a resolution of 10ms, while on a 'nix I tested
with time.time has at least ms resolution]. I'll probably use sys again
somewhere to build an automagic version/platform string, but uses for it
seem to be very limited.

I also have 0 imports of 'os', and the only immediately useful case that
comes to mind is implementation of a #dir scripting command -- providing
a minimal shell functionality, and this is certainly not a core
component of the program.

In my opinion, using 'sys' and 'os' are extreme examples of "low-level"
Python programming. This sort of thing is probably very useful for
writing actual scripts that replace the sort of work done by shell
scripts, but as programs get more complicated I think they'd be used
(proportionally) less and less.

I'm -0.9 on sys (really don't like the idea but it wouldn't be awful to
see it included in __builtins__, provided it's namespaced appropriately)
and -1 on os.

[1] Actually, it's in there three times, but they're all in the same
file -- I'd just left a legacy 'import sys' in a couple local scopes and
forgot to remove them.

 

[Michael J. Fromberger]

You're right that there is no necessity for such a change. I was
not actually talking about importing *any* module in every case,
but rather about importing, say, 'sys' when, for example, sys.argv
appeared in the code and no import had been specified.

I think I must have missed that post; I will go back and look at it.
However, while I'm here, how would your proposal deal with code like
this:

import foobar

# ... some while later ...
def f( ... ):
...
global foobar, sys
sys = foobar
...

# ... some while even later ...
f( ... )
sys.wallaby("Fear and loathing!")

In particular, we have no import of sys, but the name "sys" is
meaningful as a local alias for a different module. I'm not saying you
couldn't deal with this, but it rules out some of the more obvious ways
of detecting and automatically handling this kind of substitution.

Naturally, you might well ask, "why would you do such a fool thing?" To
this I can only respond: "Never underestimate the ingenuity of fools."

-M

 

[Patrick Maupin]

A programming language should not be ambiguous. The choice
between importing a module and calling a function should not
depend on the availability of a (local) variable.

Yeah, this behavior would be as ambiguous as if we had a system-defined
search-path for modules, where you might get one module or another
depending on the path order. Oh, wait -- we have one of those.

The question is not if it's possible or not - in principle, everything
is possible. The question is if it is desirable.

Exactly. So it pays to try to understand what the payoff might be, and
I always try to be open-minded about that (although I sometimes fail
miserably in this goal). In the case of sys.path, the payoff is that a
Python application has a small chance of running on a completely
different system. In the case of automagic importation, the only
payoff I have seen discussed is that some people would be happier with
a little less typing.

The thing I always find baffling is that people who find it hard to
type "import sys" seem to find it quite easy to write eight paragraphs
explaining why it's bad to have to type "import sys."

Regards,
Pat

 

[Rick Wotnaz]

I think I must have missed that post; I will go back and look at
it. However, while I'm here, how would your proposal deal with
code like this:

import foobar

# ... some while later ...
def f( ... ):
...
global foobar, sys
sys = foobar
...

# ... some while even later ...
f( ... )
sys.wallaby("Fear and loathing!")

In particular, we have no import of sys, but the name "sys" is
meaningful as a local alias for a different module. I'm not
saying you couldn't deal with this, but it rules out some of the
more obvious ways of detecting and automatically handling this
kind of substitution.

Naturally, you might well ask, "why would you do such a fool
thing?" To this I can only respond: "Never underestimate the
ingenuity of fools."

I don't know that this would cause any particular problem with the
[not exactly-]proposed method. The automagic lookup would not be
triggered until a NameError occurred, which would not happen in this
case. As you say, why would anyone -- at least anyone who wanted to
rely on sys.xxx being automatically resolved -- do such a thing? Even
under the current scheme, occluding 'sys' would prevent correct
interpretation of sys.argv. An error is an error in either case.

Now, if 'wallaby' is not part of the foobar namespace, the automagic
system would kick in an incorrectly import sys, and on retry would
still not find 'wallaby' in the namespace. Much merriment would
ensue, I'm sure. At that point, I'd want such a system to have a
nervous breakdown and allow the debugging to begin. Whether I hand-
entered an import statement or not wouldn't change that.