Why learn Python ??

M

Michele Simionato

Eiffel: Design by contract. The book to read is Meyer's "Object-
Oriented Software Construction". Full of interesting stuff, if you can
stand his self-importance ;-)

I couldn't!
 
P

Peter Hansen

Donn said:
Python has the advantage of a clean start, but
from a software engineering standpoint it really seems like a
hacker language.

Ohh.... nice controversial statement. :)

I would strongly disagree with the above statement for a number
of reasons, including (random thoughts):

- packages
- interactive interpreter
- no static typing
- high productivity
- ease of testing
- flexibility
- extremely broad library support
- low learning curve

and probably a dozen others. I think this has all been discussed before,
though, so I won't belabour the point. Basically, as a software engineer
(nominally a "Systems Design Engineer") I find Python to be hands down
*the* best language I've encountered for use in serious work. I can't
actually think of another which excels on so many fronts.

-Peter
 
J

John J. Lee

Donn Cave said:
Quoth Paul Rubin <http://[email protected]>:
...
| The one attempt I know of to write a substantial Python application
| competitive with similar applications written in C was the Grail web
| browser, and AFAIK that was a complete failure. However, for small to
| medium sized projects which don't have to run fast, Python is great. [...]
But at any rate, I thought we were talking about huge projects
involving many programmers, and I don't think Grail was of that
sort at all.

Right. Wasn't it a research project? In any case, hardly a
large-scale programming effort along the lines of Mozilla!

I don't know what kind of programmer resources were
allocated to it,

No obvious attribution other than "CNRI" in the tarball, but I think
mostly Guido. :)

but I bet it was actually a smashing success in
those terms. As for its size, I'd have to look, I have the code
around somewhere, but I'd guess it was smaller than Mailman or Zope.
Small enough that one programmer could become pretty thoroughly
acquainted with the whole thing.

john[0]$ find /usr/local/src/python/grail-0.6 -name '*.py' | xargs /home/john/bin/pycount.py
lines code doc comment blank file
[...]
34808 25255 2801 2356 4396 total

pycount reports 108756 code lines for Zope 2.6.3 (watch out: for some
reason pycount spits out more than one total line with Zope, which you
then have to add up -- maybe I'm doing something wrong there...).
Given typical reported ratios of Python to C++ lines of code, that's a
lot of code (not to mention all the Zope Python code outside of that
package -- plone, for example; there's a fair amount of C code there,
too, of course). Alex Martelli claimed a C++/Python lines of code per
function point ratio of 5 or 6 in a post in this group -- if you
believe that's a fair rule of thumb, then on that basis you'd expect
550-650 kLoC for a mythical Zope-in-C++ . In reality, since Zope
takes advantage of Python's features, you'd expect to have to write
many more lines of C++ than that, following Greenspun's tenth law.

Who can be sure what the biggest Python code is, though? See Paul
Graham's article on his web store-builder (implemented in Lisp) for
one motivation for keeping it a secret (a motivation of the order of
several tens of millions of dollars, IIRC ;-).

http://www.paulgraham.com/avg.html


(pycount reports 21730 code lines for mailman 2.1.4, BTW)


John
 
D

Donn Cave

Peter Hansen said:
Ohh.... nice controversial statement. :)

I would strongly disagree with the above statement for a number
of reasons, including (random thoughts):

- packages
- interactive interpreter
- no static typing
- high productivity
- ease of testing
- flexibility
- extremely broad library support
- low learning curve

and probably a dozen others. I think this has all been discussed before,
though, so I won't belabour the point. Basically, as a software engineer
(nominally a "Systems Design Engineer") I find Python to be hands down
*the* best language I've encountered for use in serious work. I can't
actually think of another which excels on so many fronts.

But your list describes the perfect hacker language, at
least on some counts. I'm nominally a Software Engineer
myself, but I have no formal CS education and have never
worked on a large project, so I feel right at home with
the hacker culture here. But help me out here, suppose
we're going to go in and develop one of those massive
projects - I don't know, say the IRS needs a new computer
system - what on that list would we care about?

Maybe vague promises like high productivity and ease of
testing would be attractive, but anyone can say that for
some ideal context. Library support is probably not even
on the charts. I would think a major consideration at
the outset of a project like this would be whether the
code that all these teams is writing will eventually come
together and work like it's supposed to, and from that
point of view it's tempting to look for some support for
architectural consistency, like "interfaces" and that sort
of thing.

Donn Cave, (e-mail address removed)
 
C

Cameron Laird

Ohh.... nice controversial statement. :)

I would strongly disagree with the above statement for a number
of reasons, including (random thoughts):

- packages
- interactive interpreter
- no static typing
- high productivity
- ease of testing
- flexibility
- extremely broad library support
- low learning curve

and probably a dozen others. I think this has all been discussed before,
though, so I won't belabour the point. Basically, as a software engineer
(nominally a "Systems Design Engineer") I find Python to be hands down
*the* best language I've encountered for use in serious work. I can't
actually think of another which excels on so many fronts.
.
.
.
From a Python standpoint, "software engineering" has made a
nice start, but it needs to catch up with hacker insights.
 
J

John J. Lee

Paul Rubin said:
It makes some feeble ones but is somewhat hobbled by its origins as a
scripting or "glue" language.

Are you referring to *anything* other than the lack of static type
checking and execution speed?

Also, Python is really still in its
infancy. Maybe after PyPy is completed, someone can try writing a
serious compiler for it, and the issues brought out by writing the
compiler can drive the design of a successor language, Python II or
whatever.
[...]

I'm sure you're right that things will advance. Not sure it would be
best to make a "Python II", though -- Python seems close to a local
maximum.


John
 
J

John J. Lee

Derek said:
Point taken. I once worked on a project where we implemented a
production system in C++ and then implemented it again in Python for
QA purposes. It took about 150k lines of C++ code and 10k lines of
Python. Python took less code because so many modules are bundled
with the language, but the C++ system ran many times faster. It's all
about picking the right tool for the job.

Well, that's a banal truism obscuring the fact that, for the great
majority of projects, optimisation does not require implementation of
the *entire* system in a language like C++. The sum of C++ and Python
is greater than the parts.


John
 
J

John J. Lee

I couldn't!

:))

I admit I have nowhere near finished it either, but more due to my
slowness and the book's vastness than the manner of discourse.

The irritation he inspires *is* productive, as Michael Hudson says.


John
 
J

John J. Lee

David M. Cook said:
You could casually pick up enough Python to be useful in a week or so of
evenings.

Perl is not much harder,

It *seems* so when you first learn it...
but has a lot of "gotchas".

.... and they never end! Ilya Zakharevich (a "Perl God"):

http://www.google.com/[email protected]

[...]
Let me reiterate my current opinion on the topic: the reason for my
puzzlement (and your trouble) is the difference between Perl as a
scripting language and Perl as a programming language. Perl is
absolutely fine as a scripting language. Perl is pretty unusable as a
programming language.
[...]

I'm not sure I'd go quite as far as that, but it's not a million miles
from the truth. Why saddle yourself with the costs of Perl when
Python demonstrates there are no significant associated benefits?
(I'm talking about "programming" not "scripting" here, using Ilya's
language, though I'd argue Python is just as good for scripting as
Perl)


John
 
J

John J. Lee

Donn Cave said:
code that all these teams is writing will eventually come
together and work like it's supposed to, and from that
point of view it's tempting to look for some support for
architectural consistency, like "interfaces" and that sort
of thing.

Not sure about "that sort of thing" <wink>, but more explicit
interfaces do seem like a real issue here. It's interesting to note
that Zope, a notable large Python project, has moved (and is moving)
in this direction.


John
 
J

John J. Lee

Forgot to add: what happened to all those Python interface proposals?

At least one was PEPed, wasn't it?

I suppose it's waiting for Guido-bandwidth.


John
 
P

Paul Rubin

Right. Wasn't it a research project? In any case, hardly a
large-scale programming effort along the lines of Mozilla!

Grail wasn't huge but I think any serious browser qualifies as
substantial. If you're saying it didn't really fail but actually
wasn't even seriously attempted, ok, that's even fewer substantial
programs that have even been tried in Python.
 
P

Peter Hansen

Donn said:
But your list describes the perfect hacker language, at
least on some counts. I'm nominally a Software Engineer
myself, but I have no formal CS education and have never
worked on a large project, so I feel right at home with
the hacker culture here. But help me out here, suppose
we're going to go in and develop one of those massive
projects - I don't know, say the IRS needs a new computer
system - what on that list would we care about?

Never having done a new system for the IRS (yet? ;-) I can't
really say, but I do some pretty hard core stuff and Python
fits the bill for me better than past attempts...
Maybe vague promises like high productivity and ease of
testing would be attractive, but anyone can say that for
some ideal context.

Perhaps, but it's also pretty easy to quantify, at least in the
non-scientific fashion which I've attempted to use for that.
Library support is probably not even on the charts.

Very few systems cannot benefit from good library support.
Presumably the IRS requires networking, some user interface
capabilities, probably a bunch of database interfaces, really
sophisticated testing, and these days maybe even some standardization
in the area of, say, XML or something like that.
I would think a major consideration at
the outset of a project like this would be whether the
code that all these teams is writing will eventually come
together and work like it's supposed to, and from that
point of view it's tempting to look for some support for
architectural consistency, like "interfaces" and that sort
of thing.

I agree, though I personally find it much more effective to use
good tests for this kind of thing. Interfaces don't cut it when
there's not a simple caller-callee relationship between things. I
have lots of components which are separated by various other kinds of
interfaces, such as a particular file format, or XML, or some custom
network protocol. Need tests to check both sides of those equations.

-Peter
 
D

Derek

John J. Lee said:
[...]
Point taken. I once worked on a project where we
implemented a production system in C++ and then
implemented it again in Python for QA purposes. It took
about 150k lines of C++ code and 10k lines of Python.
Python took less code because so many modules are bundled
with the language, but the C++ system ran many times
faster. It's all about picking the right tool for the
job.

Well, that's a banal truism obscuring the fact that, for the
great majority of projects, optimisation does not require
implementation of the *entire* system in a language like
C++. The sum of C++ and Python is greater than the parts.

Python and C++ can also be a bigger mess than sum of either part.
Take your pick.
 
P

Paul Rubin

Peter Hansen said:
Presumably the IRS requires networking, some user interface
capabilities, probably a bunch of database interfaces, really
sophisticated testing, and these days maybe even some standardization
in the area of, say, XML or something like that.

I think if there's one application that needs decimal arithmetic...
 
J

Jp Calderone

John J. Lee said:
[...]
Point taken. I once worked on a project where we
implemented a production system in C++ and then
implemented it again in Python for QA purposes. It took
about 150k lines of C++ code and 10k lines of Python.
Python took less code because so many modules are bundled
with the language, but the C++ system ran many times
faster. It's all about picking the right tool for the
job.

Well, that's a banal truism obscuring the fact that, for the
great majority of projects, optimisation does not require
implementation of the *entire* system in a language like
C++. The sum of C++ and Python is greater than the parts.

Python and C++ can also be a bigger mess than sum of either part.
Take your pick.

Look at how many modules in the stdlib are not implemented in Python.
Look at all the builtin types. It *can* and *does* work, and with very
little mess, if you know what you're doing.

Jp
 
R

Rainer Deyke

Jp said:
Look at how many modules in the stdlib are not implemented in
Python. Look at all the builtin types. It *can* and *does* work, and
with very little mess, if you know what you're doing.

They're also not implemented in C++. They're implemented in C, a very
different language. And they *are* a mess compared to C code that doesn't
have to interface with Python.
 
F

Francis Avila

John J. Lee wrote in message said:
Forgot to add: what happened to all those Python interface proposals?

At least one was PEPed, wasn't it?

I suppose it's waiting for Guido-bandwidth.

I think the general opinion of Guido was, yes, we need something like this,
but it's important enough to require his full and undivided attention, and
this must be given later, when a major revision comes up. I also think he
wasn't fully satisfied with all the options.

I forget where I read this. Maybe the pep-parade, or maybe one of the peps
themselves?
 
E

EP

Is it true that the original Google spider was written in Python?

I came across a paper on the web some time back that I saved and read just
last night:


The Anatomy of a Large-Scale Hypertextual Web Search Engine

Sergey Brin and Lawrence Page
{sergey, page}@cs.stanford.edu
Computer Science Department, Stanford University, Stanford, CA 94305

A neat read, but I'm not sure of the authenticity of the paper: I could be
gullible. It would appear to be a paper written some years back on the
genesis of the Google search engine.

[excerpt]
Running a web crawler is a challenging task. There are tricky performance
and reliability issues and even more importantly, there are social issues.
Crawling is the most fragile application since it involves interacting with
hundreds of thousands of web servers and various name servers which are all
beyond the control of the system.
In order to scale to hundreds of millions of web pages, Google has a fast
distributed crawling system. A single URLserver serves lists of URLs to a
number of crawlers (we typically ran about 3). Both the URLserver and the
crawlers are implemented in Python. Each crawler keeps roughly 300
connections open at once. This is necessary to retrieve web pages at a fast
enough pace. At peak speeds, the system can crawl over 100 web pages per
second using four crawlers. This amounts to roughly 600K per second of
data. A major performance stress is DNS lookup. Each crawler maintains a
its own DNS cache so it does not need to do a DNS lookup before crawling
each document. Each of the hundreds of connections can be in a number of
different states: looking up DNS, connecting to host, sending request, and
receiving response. These factors make the crawler a complex component of
the system. It uses asynchronous IO to manage events, and a number of
queues to move page fetches from state to state.
[/excerpt]

It would seem like the poster boy example for using Python in some
respects, if true.


Eric, Intrigued

"but at least I didn't top post"
 
F

Francis Avila

EP wrote in message ...
Is it true that the original Google spider was written in Python?

I don't know *what* Google uses Python for, but it uses Python.
It would seem like the poster boy example for using Python in some
respects, if true.

Um, nothing new about this:

http://python.org

Read the top center of the front page....
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
474,175
Messages
2,570,946
Members
47,497
Latest member
PilarLumpk

Latest Threads

Top