Blog "about python 3"

[Mark Lawrence]

http://alexgaynor.net/2013/dec/30/about-python-3/ may be of interest to
some of you.

 

[Steven D'Aprano]

http://alexgaynor.net/2013/dec/30/about-python-3/ may be of interest to
some of you.

I don't know whether to thank you for the link, or shout at you for
sending eyeballs to look at such a pile of steaming bullshit.

I'd like to know where Alex gets the idea that the transition of Python 2
to 3 was supposed to be a five year plan. As far as I know, it was a ten
year plan, and we're well ahead of expectations of where we would be at
this point of time. People *are* using Python 3, the major Linux distros
are planning to move to Python 3, the "Python Wall Of Shame" stopped
being a wall of shame a long time ago (I think it was a year ago? or at
least six months ago). Alex's article is, basically, FUD.

More comments will have to follow later.

 

[Mark Lawrence]

I don't know whether to thank you for the link, or shout at you for
sending eyeballs to look at such a pile of steaming bullshit.

I'd like to know where Alex gets the idea that the transition of Python 2
to 3 was supposed to be a five year plan. As far as I know, it was a ten
year plan, and we're well ahead of expectations of where we would be at
this point of time. People *are* using Python 3, the major Linux distros
are planning to move to Python 3, the "Python Wall Of Shame" stopped
being a wall of shame a long time ago (I think it was a year ago? or at
least six months ago). Alex's article is, basically, FUD.

More comments will have to follow later.

http://nuitka.net/posts/re-about-python-3.html is a response.

 

[Ethan Furman]

http://nuitka.net/posts/re-about-python-3.html is a response.

Wow -- another steaming pile! Mark, are you going for a record?

 

[Chris Angelico]

Wow -- another steaming pile! Mark, are you going for a record?

Does this steam?

http://rosuav.blogspot.com/2013/12/about-python-3-response.html

ChrisA

 

[Ethan Furman]

Indeed. Every post that disagrees with my opinion and understanding of
the situation is complete BS and a conspiracy to spread fear,
uncertainty, and doubt. Henceforth I will explain few to no specific
disagreements, nor will I give anyone the benefit of the doubt,
because that would be silly.

Couldn't of said it better myself! Well, except for the "my opinion" part -- obviously it's not my opinion, but
reality!

 

[Mark Lawrence]

Wow -- another steaming pile! Mark, are you going for a record?

Merely pointing out the existence of these little gems in order to find
out people's feelings about them. You never know, we might even end up
with a thread whereby the discussion is Python, the whole Python and
nothing but the Python.

 

[Chris Angelico]

You never know, we might even end up with a thread whereby the discussion is
Python, the whole Python and nothing but the Python.

What, on python-list??! [1] That would be a silly idea. We should
avoid such theories with all vigor.

ChrisA

[1] In C, that would be interpreted as "What, on python-list|" and
would confuse everyone.

 

[Mark Lawrence]

Does this steam?

http://rosuav.blogspot.com/2013/12/about-python-3-response.html

ChrisA

I'd have said restrained.

 

[Mark Lawrence]

Wow -- another steaming pile! Mark, are you going for a record?

I wasn't, but I am now
http://blog.startifact.com/posts/alex-gaynor-on-python-3.html. "The
Python core developers somewhat gleefully slammed the door shut on
Python 2.8 back in 2011, though.", which refers to PEP 404 which I
mentioned a month or so ago.

 

[Steven D'Aprano]

http://alexgaynor.net/2013/dec/30/about-python-3/ may be of interest to
some of you.

[...]
I'd like to know where Alex gets the idea that the transition of Python 2
to 3 was supposed to be a five year plan. As far as I know, it was a ten
year plan,

I haven't been able to find anything in writing from Guido or the core
developers stating that the transition period was expected to be ten years,
although I haven't looked very hard. I strongly recall it being discussed,
so unless you want to trawl the python-dev mailing list, you'll just have
to take my word on it *wink*

PEP 3000 makes it clear that Guido van Rossum expected the transition period
to be longer than usual:

I expect that there will be parallel Python 2.x and 3.x releases
for some time; the Python 2.x releases will continue for a longer
time than the traditional 2.x.y bugfix releases. Typically, we
stop releasing bugfix versions for 2.x once version 2.(x+1) has
been released. But I expect there to be at least one or two new 2.x
releases even after 3.0 (final) has been released, probably well
into 3.1 or 3.2.
[end quote]

http://www.python.org/dev/peps/pep-3000/

A five year transition period, as suggested by Alex Gaynor, simply makes no
sense. Normal support for a single release is four or five years, e.g.
Python 2.4 and 2.5 release schedules:

* Python 2.4 alpha-1 was released on July 9 2004; the final security
update was December 19 2008;

* Python 2.5 alpha-1 was released on April 5 2006; the final security
update was May 26 2011.

(Dates may be approximate, especially the alpha dates. I'm taking them from
PEP 320 and 356.)

Now in fairness, Guido's comment about "well into 3.1 or 3.2" turned out to
be rather naive in retrospect. 3.4 alpha has been released, and support for
2.7 is expected to continue for *at least* two more years:

http://www.python.org/dev/peps/pep-0373/#maintenance-releases

which means that 2.7 probably won't become unmaintained until 3.5 is out. In
hindsight, this is probably a good thing. The early releases of 3.x made a
few mistakes, and it's best to skip them and go straight to 3.3 or better,
for example:

- 3.0 was buggy enough that support for it was dropped almost immediately;

- built-in function callable() is missing from 3.1;

- 3.1 and 3.2 both have exception chaining, but there's no way to
suppress the chained exceptions until 3.3;

- 3.1 and 3.2 don't allow u'' strings for compatibility with 2.x.


The 2.8 un-release schedule goes into more detail about the transition, and
why there won't be an official 2.8 blessed by the core developers:

http://www.python.org/dev/peps/pep-0404/

(Python is open source -- nothing is stopping people from forking the
language or picking up support for 2.7. I wonder how many Python3 naysayers
volunteer to support 2.x once the core devs drop it?)

As of June this year, over 75% of the top fifty projects hosted on PyPI
supported Python 3:

http://py3ksupport.appspot.com/

and the Python Wall Of Shame turned majority green, becoming the Wall Of
Superpowers, at least six months ago. (Back in June, I noted that it had
changed colour some time ago.) Alex's claim that "almost no code is written
for Python 3" is, well, I'll be kind and describe it as counter-factual.

Alex points out that the companies he is speaking to have no plans to
migrate to Python 3. Well, duh. In my experience, most companies don't even
begin planning to migrate until six months *after* support has ended for
the systems they rely on. (Perhaps a tiny exaggeration, but not much.)

I won't speak for the Windows or Mac world, but in the Linux world, Python 3
usage depends on the main Linux distros. Yes, ArchLinux has been Python 3
for years now, but ArchLinux is bleeding edge. Fedora is likely to be the
first mainstream distro to move to Python 3:

https://fedoraproject.org/wiki/Changes/Python_3_as_Default

Once Fedora moves, I expect Ubuntu will follow. Debian, Centos and RedHat
will probably be more conservative, but they *will* follow, at their own
pace. What are the alternatives? They're not going to drop Python, nor are
they going to take over support of 2.x forever. (RedHat/Centos are still
supporting Python 2.4 and possibly even 2.3, at least in name, but I
haven't seen any security updates come through Centos for a long time.)
Once the system Python is Python 3, the default, no-brainer choice for most
Python coding will be Python 3.

Alex says:

Why aren't people using Python 3?

First, I think it's because of a lack of urgency. Many years ago,
before I knew how to program, the decision to have Python 3
releases live in parallel to Python 2 releases was made. In
retrospect this was a mistake, it resulted in a complete lack
of urgency for the community to move, and the lack of urgency
has given way to lethargy.
[end quote]


That's only a mistake if you think that migrating to Python 3 needs to be
*urgent*. But it isn't. It's actually a *good thing* if people and
companies can take their sweet time migrating to Python 3. Let's jump back
five years ago. Had Guido announced that Python 2.6 would be followed by
Python 3.0, with little in the way of a transition, people would have been
faced with three unpalatable choices:

1) dig your heels in and stay with Python 2.6 without upstream support or
security updates;

2) migrate to an essentially untested, backwards-incompatible version of
Python which is likely to be buggy (as indeed Python 3.0 turned out
to be);

3) or migrate to some other language.


None of these are good choices, but the difference between 2 and 3 would
have been small: with none of the big frameworks and libraries like numpy
or Zope migrated to Python 3, moving to a completely new language would
have seemed like a reasonable choice. Choose between getting stuck half-way
through a transition because only some of the libraries you need have been
ported, versus move to a new language where you know (or at least *think*
you know) the risks?

Rushing the transition like Alex suggests would have killed Python.

Instead, companies will move when they're ready. Yes, there will be a
deadline, probably in 2017 or 2018, but in the meantime early adopters get
to file back the sharp corners and Python 3.x gets many years to iron out
the bugs. Back when Python 3 was just started, nobody thought that it would
be plausible to have a single code base handle 2.x and 3.x. It turns out
that for many projects, that's actually easier than trying to migrate code
with 2to3. That's the sort of thing which early adopters discovered.
Slow-coaches will find it easier later on because we've had so many years
for the transition.

Alex thinks that the reason people haven't migrated to 3.x is because of a
lack of urgency. And his solution to this lack of urgency is to *reduce*
the level of urgency by another 4 or 5 years by adding a version 2.8. And
no doubt when that fails to work, he'll suggest giving them another 4 or 5
years with version 2.9.

"Hurry up, we're waiting! If you don't hurry up, we'll wait longer!"

I don't think Alex has thought his plan through, unless his plan is actually
to kill Python 3.

 

[Steven D'Aprano]

http://blog.startifact.com/posts/alex-gaynor-on-python-3.html.

I quote:

"...perhaps a brave group of volunteers will stand up and fork Python 2, and
take the incremental steps forward. This will have to remain just an idle
suggestion, as I'm not volunteering myself."

I expect that as excuses for not migrating get fewer, and the deadline for
Python 2.7 end-of-life starts to loom closer, more and more haters^W
Concerned People will whine about the lack of version 2.8 and ask for
*somebody else* to fork Python.

I find it, hmmm, interesting, that so many of these Concerned People who say
that they're worried about splitting the Python community[1] end up
suggesting that we *split the community* into those who have moved forward
to Python 3 and those who won't.





[1] As if the community is a single amorphous group. It is not. It is made
up of web developers using Zope or Django, and scientists using scipy, and
linguists using NLTK, and system administrators using nothing but the
stdlib, and school kids learning how to program, and professionals who know
seventeen different programming languages, and EVE Online gamers using
Stackless, and Java guys using Jython, and many more besides, most of whom
are sure that their little tiny part of the Python ecosystem is
representative of everyone else when in fact they hardly talk at all.

 

[Antoine Pitrou]

I expect that as excuses for not migrating get fewer, and the deadline for
Python 2.7 end-of-life starts to loom closer, more and more haters^W
Concerned People will whine about the lack of version 2.8 and ask for
*somebody else* to fork Python.

I find it, hmmm, interesting, that so many of these Concerned People who say
that they're worried about splitting the Python community[1] end up
suggesting that we *split the community* into those who have moved forward
to Python 3 and those who won't.

Indeed. This would be extremely destructive (not to mention alienating the
people doing *actual* maintenance and enhancements on Python-and-its-stdlib,
of which at least 95% are committed to the original plan for 3.x to slowly
supercede 2.x).

Regards

Antoine.

 

[Roy Smith]

I expect that as excuses for not migrating get fewer, and the deadline for
Python 2.7 end-of-life starts to loom closer, more and more haters^W
Concerned People will whine about the lack of version 2.8 and ask for
*somebody else* to fork Python.

I find it, hmmm, interesting, that so many of these Concerned People who say
that they're worried about splitting the Python community[1] end up
suggesting that we *split the community* into those who have moved forward
to Python 3 and those who won't.

Indeed. This would be extremely destructive (not to mention alienating the
people doing *actual* maintenance and enhancements on Python-and-its-stdlib,
of which at least 95% are committed to the original plan for 3.x to slowly
supercede 2.x).

Regards

Antoine.

I'm using 2.7 in production. I realize that at some point we'll need to
upgrade to 3.x. We'll keep putting that off as long as the "effort +
dependencies + risk" metric exceeds the "perceived added value" metric.

I can't imagine the migration will happen in 2014. Maybe not even in
2015. Beyond that, my crystal ball only shows darkness. But, in any
case, going with a fork of 2.x has zero appeal. Given the choice
between effort + risk to move forward vs. effort + risk to move
sideways, I'll move forward every time.

To be honest, the "perceived added value" in 3.x is pretty low for us.
What we're running now works. Switching to 3.x isn't going to increase
our monthly average users, or our retention rate, or decrease our COGS,
or increase our revenue. There's no killer features we need. In
summary, the decision to migrate will be driven more by risk aversion,
when the risk of staying on an obsolete, unsupported platform, exceeds
the risk of moving to a new one. Or, there will be some third-party
module that we must have which is no longer supported on 2.x.

If I were starting a new project today, I would probably start it in 3.x.

 

[Mark Lawrence]

I expect that as excuses for not migrating get fewer, and the deadline for
Python 2.7 end-of-life starts to loom closer, more and more haters^W
Concerned People will whine about the lack of version 2.8 and ask for
*somebody else* to fork Python.

I find it, hmmm, interesting, that so many of these Concerned People who say
that they're worried about splitting the Python community[1] end up
suggesting that we *split the community* into those who have moved forward
to Python 3 and those who won't.

Indeed. This would be extremely destructive (not to mention alienating the
people doing *actual* maintenance and enhancements on Python-and-its-stdlib,
of which at least 95% are committed to the original plan for 3.x to slowly
supercede 2.x).

Regards

Antoine.

I'm using 2.7 in production. I realize that at some point we'll need to
upgrade to 3.x. We'll keep putting that off as long as the "effort +
dependencies + risk" metric exceeds the "perceived added value" metric.

Do you use any of the features that were backported from 3.x to 2.7, or
could you have stayed with 2.6 or an even older version?

 

[Robin Becker]

I'm using 2.7 in production. I realize that at some point we'll need to
upgrade to 3.x. We'll keep putting that off as long as the "effort +
dependencies + risk" metric exceeds the "perceived added value" metric.

We too are using python 2.4 - 2.7 in production. Different clients migrate at
different speeds.

To be honest, the "perceived added value" in 3.x is pretty low for us.
What we're running now works. Switching to 3.x isn't going to increase
our monthly average users, or our retention rate, or decrease our COGS,
or increase our revenue. There's no killer features we need. In
summary, the decision to migrate will be driven more by risk aversion,
when the risk of staying on an obsolete, unsupported platform, exceeds
the risk of moving to a new one. Or, there will be some third-party
module that we must have which is no longer supported on 2.x.

+1

If I were starting a new project today, I would probably start it in 3.x.

+1

I just spent a large amount of effort porting reportlab to a version which works
with both python2.7 and python3.3. I have a large number of functions etc which
handle the conversions that differ between the two pythons.

For fairly sensible reasons we changed the internal default to use unicode
rather than bytes. After doing all that and making the tests compatible etc etc
I have a version which runs in both and passes all its tests. However, for
whatever reason the python 3.3 version runs slower

2.7 Ran 223 tests in 66.578s

3.3 Ran 223 tests in 75.703s

I know some of these tests are fairly variable, but even for simple things like
paragraph parsing 3.3 seems to be slower. Since both use unicode internally it
can't be that can it, or is python 2.7's unicode faster?

So far the superiority of 3.3 escapes me, but I'm tasked with enjoying this
process so I'm sure there must be some new 'feature' that will help. Perhaps
'yield from' or 'raise from None' or .......

In any case I think we will be maintaining python 2.x code for at least another
5 years; the version gap is then a real hindrance.

 

[Terry Reedy]

I just spent a large amount of effort porting reportlab to a version
which works with both python2.7 and python3.3. I have a large number of
functions etc which handle the conversions that differ between the two
pythons.

I am imagine that this was not fun.

[For those who do not know, reportlab produces pdf documents.]

For fairly sensible reasons we changed the internal default to use
unicode rather than bytes.

Do you mean 'from __future__ import unicode_literals'?

Am I correct in thinking that this change increases the capabilities of
reportlab? For instance, easily producing an article with abstracts in
English, Arabic, Russian, and Chinese?

After doing all that and making the tests
compatible etc etc I have a version which runs in both and passes all
its tests. However, for whatever reason the python 3.3 version runs slower.

Python 3 is slower in some things, like integer arithmetic with small ints.

2.7 Ran 223 tests in 66.578s

3.3 Ran 223 tests in 75.703s

I know some of these tests are fairly variable, but even for simple
things like paragraph parsing 3.3 seems to be slower. Since both use
unicode internally it can't be that can it, or is python 2.7's unicode
faster?

The new unicode implementation in 3.3 is faster for some operations and
slower for others. It is definitely more space efficient, especially
compared to a wide build system. It is definitely less buggy, especially
compared to a narrow build system.

Do your tests use any astral (non-BMP) chars? If so, do they pass on
narrow 2.7 builds (like on Windows)?

So far the superiority of 3.3 escapes me,

For one thing, indexing and slicing just works on all machines for all
unicode strings. Code for 2.7 and 3.3 either a) does not index or slice,
b) does not work for all text on 2.7 narrow builds, or c) has extra
conditional code only for 2.7.

 

[Antoine Pitrou]

Hi,

For fairly sensible reasons we changed the internal default to use unicode
rather than bytes. After doing all that and making the tests compatible etc etc
I have a version which runs in both and passes all its tests. However, for
whatever reason the python 3.3 version runs slower

2.7 Ran 223 tests in 66.578s

3.3 Ran 223 tests in 75.703s

Running a test suite is a completely broken benchmarking methodology.
You should isolate workloads you are interested in and write a benchmark
simulating them.

Regards

Antoine.

 

[Steven D'Aprano]

For fairly sensible reasons we changed the internal default to use unicode
rather than bytes. After doing all that and making the tests compatible
etc etc I have a version which runs in both and passes all its tests.
However, for whatever reason the python 3.3 version runs slower

"For whatever reason" is right, unfortunately there's no real way to tell
from the limited information you give what that might be.

Are you comparing a 2.7 "wide" or "narrow" build? Do your tests use any
so-called "astral characters" (characters in the Supplementary Multilingual
Planes, i.e. characters with ord() > 0xFFFF)?

If I remember correctly, some early alpha(?) versions of Python 3.3
consistently ran Unicode operations a small but measurable amount slower
than 3.2 or 2.7. That especially effected Windows. But I understand that
this was sped up in the release version of 3.3.

There are some operations with Unicode strings in 3.3 which unavoidably are
slower. If you happen to hit a combination of such operations (mostly to do
with creating lots of new strings and then throwing them away without doing
much work) your code may turn out to be a bit slower. But that's a pretty
artificial set of code.

Generally, test code doesn't make good benchmarks. Tests only get run once,
in arbitrary order, it spends a lot of time setting up and tearing down
test instances, there are all sorts of confounding factors. This plays
merry hell with modern hardware optimizations. In addition, it's quite
possible that you're seeing some other slow down (the unittest module?) and
misinterpreting it as related to string handling. But without seeing your
entire code base and all the tests, who can say for sure?

2.7 Ran 223 tests in 66.578s

3.3 Ran 223 tests in 75.703s

I know some of these tests are fairly variable, but even for simple things
like paragraph parsing 3.3 seems to be slower. Since both use unicode
internally it can't be that can it, or is python 2.7's unicode faster?

Faster in some circumstances, slower in others. If your application
bottleneck is the availability of RAM for strings, 3.3 will potentially be
faster since it can use anything up to 1/4 of the memory for strings. If
your application doesn't use much memory, or if it uses lots of strings
which get created then thrown away.

So far the superiority of 3.3 escapes me,

Yeah I know, I resisted migrating from 1.5 to 2.x for years. When I finally
migrated to 2.3, at first I couldn't see any benefit either. New style
classes? Super? Properties? Unified ints and longs? Big deal. Especially
since I was still writing 1.5 compatible code and couldn't really take
advantage of the new features.

When I eventually gave up on supporting versions pre-2.3, it was a load off
my shoulders. Now I can't wait to stop supporting 2.4 and 2.5, which will
make things even easier. And when I can ignore everything below 3.3 will be
a truly happy day.

but I'm tasked with enjoying
this process so I'm sure there must be some new 'feature' that will help.
Perhaps 'yield from' or 'raise from None' or .......

No, you have this completely backwards. New features don't help you support
old versions of Python that lack those new features. New features are an
incentive to drop support for old versions.

In any case I think we will be maintaining python 2.x code for at least
another 5 years; the version gap is then a real hindrance.

Five years sounds about right.

 

[Terry Reedy]

"For whatever reason" is right, unfortunately there's no real way to tell
from the limited information you give what that might be.

Are you comparing a 2.7 "wide" or "narrow" build? Do your tests use any
so-called "astral characters" (characters in the Supplementary Multilingual
Planes, i.e. characters with ord() > 0xFFFF)?

If I remember correctly, some early alpha(?) versions of Python 3.3
consistently ran Unicode operations a small but measurable amount slower
than 3.2 or 2.7. That especially effected Windows. But I understand that
this was sped up in the release version of 3.3.

There was more speedup in 3.3.2 and possibly even more in 3.3.3, so OP
should run the latter.