Python syntax in Lisp and Scheme

[Michael Sperber]

james> are there examples where these little beasties are used in production?

If you mean Oleg's little beasties, then the answer is yes, and in
mission-critical, million-dollar-stake applications at that.

 

[james anderson]

james> are there examples where these little beasties are used in production?

If you mean Oleg's little beasties, then the answer is yes, and in
mission-critical, million-dollar-stake applications at that.

is one at liberty to give anything more than a rhetorical answer?

i would be interested to observe how they express themselves in the large.

 

[Alex Martelli]

Matthias wrote:
...

In the context of programming languages I find studies from Lutz
Prechtel <http://www.ipd.uka.de/~prechelt/Biblio/> or Erann Gat's

Yes, Lutz has been quoted several times on this thread -- and, of course,
his studies have been impugned just as many times as they've been
quoted, anytime somebody did not like any of their implications.

truth". But scientifically examining the act of producing software
should be possible (within limits) if one tries and has enough
funding. ;-)

"shud" is a 4-letter word;-). As this huge thread makes abundantly
clear, social and political considerations, not technical ones, dominate
most discussions of this ilk. It's just like with, say, recreational drugs:
a study appears to show ecstasy can damage the brain, prohibitionists
jump on it with glee and proclaim it the most crucial scientific result of
all times; months later the authors shamefacedly retract the study,
after they and many others had uselessly tried to reproduce its findings, as
they discovered their drug samples had been mis-labeled so they had in
fact been studying a _different_ substance by mistake -- and the
prohibitionists poo-poo the study's retraction as changing nothing of any
importance whatsoever. Who'll try to reproduce the findings of such long
and expensive studies of "the act of producing software" -- and will they
make any real difference, or just be used as argument fodder who people who
already know what they _want_ to believe? Remember the famous
Microsoft-financed benchmarks of Linux vs NT, for example...?-)

I may feel a bit pessimistic at this point, but after the huge amount of
time devoted to this thread and the tiny ROI, I think that's justified!-)


Alex

 

[Alex Martelli]

xreadlines()? What kind of naming convention is that:

An obsolete one (to go with 'xrange'). Since about 3 years, the
correct Python spelling is just "for line in file("input.txt"):" .


I think that using methods for such things is not a particularly good idea.

A generator that takes a sequence (typically an iterator) of strings and
returns as the items the single bytes or words is more general:

def eachbyte(seq):
for s in seq:
for c in s:
yield c

def eachword(seq):
for s in seq:
for w in s.split():
yield w

and now you can loop "for b in eachbyte(file("input.txt")):" etc -- AND you
have also gained the ability to loop per-byte or per-word on any other
sequence of strings. Actually eachbyte is much more general than its
name suggests -- feed it e.g. a list of files, and it will return the lines
of each file -- one after the other -- as a single sequence.

OTOH, eachbyte is NOT particularly good for arbitrary binary files -- if
there happen to be no \n bytes at convenient point it may suck in much
more memory than needed. Besides, typical need on arbitrary binary
files is to loop on block of N bytes for some N -- N==1 is a rather special
case. So one might prefer:

def eachblock(afile, N):
while 1:
block = afile.read(N)
if not block: break
yield block

or variations thereon.


Alex

 

[Alex Martelli]

Pascal Costanza wrote:
...

I definitely agree. Computer science is more a sociological science than
a natural science IMHO.

Amen, hallelujah. So, since I've decided to limit my participation in this
thread to c.l.python, would you kindly set right the guys (such as your
namesake) who (on c.l.lisp with copy to my mailbox but not to here) are
currently attacking me because, and I quote,
"""
Software is a department of mathematics.
"""
....?


Alex

 

[Alex Martelli]

Lutz Prechelt has done a number (at least two that I know of) of such
studies. I did one too: http://www.flownet.com/gat/lisp-java.pdf

Yes, Lutz's studies have been quoted repeatedly on this thread -- and
dissed (by Lispers) for not being good enough (task too simple thus
programs too short, etc etc).


Alex

 

[Dave Benjamin]

I think that using methods for such things is not a particularly good idea.

A generator that takes a sequence (typically an iterator) of strings and
returns as the items the single bytes or words is more general:

def eachbyte(seq):
for s in seq:
for c in s:
yield c

def eachword(seq):
for s in seq:
for w in s.split():
yield w

and now you can loop "for b in eachbyte(file("input.txt")):" etc -- AND you
have also gained the ability to loop per-byte or per-word on any other
sequence of strings. Actually eachbyte is much more general than its
name suggests -- feed it e.g. a list of files, and it will return the lines
of each file -- one after the other -- as a single sequence.

eachbyte is in fact so general, I'd be tempted to give it the name
"iflatten", though I can never decide whether a shallow flatten or a
recursive flatten is worthy of the name "flatten". Here's another way to
loop through words lazily, this time using itertools:

import string
from itertools import imap

def iflatten(seq):
for subseq in seq:
for item in subseq:
yield item

for word in iflatten(imap(string.split, file('input.txt'))):
print word

 

[Alex Martelli]

Daniel Berlin wrote:
...

Yes, but not windows.

Aw:-(. Oh well, so much for the hope of easy extensibility of Python
on Windows by non-MS compilers:-(.

It depends. I've built it with both.

I'm sure you can build Python with different compilers, but I was wondering
about the widely distributed pre-built version, the one Apple includes as
part of OS/X.

At least on Mac, Apple's gcc -fast is better than any other compiler
around, according to recent benchmarks.

Unsurprising to me, but i'm a gcc hacker, so i might be biased a bit.

Most, if not all, optimizations that commercial compilers implement are
or are being implemented in gcc for 3.5/3.6.

Great news. But I doubt that Python on Windows can simply drop the
use of MSVC++, anyway:-(.


Alex

 

[Alex Martelli]

Joe Marshall wrote:
...

...
A single web browser?

I far prefer to have on my cellphone one that's specialized for its small
screen and puny cpu/memory, and on more powerful computers, more
powerful browsers. Don't you?

A single editor?

I prefer to have one for text files, a separate one for video files, and
so on. Editing text and editing video have SO little in common...

A single operating system?

On my cellphone all the way to the datacenter? MS may dream about
that (even though all the "windows" versions they try to sell may have
in common only the name or little more), but it's definitely not my
preference -- I think the OS's for cellphones and those for powerful
servers have too little in common to be "a single OS" sensibly.

A single character encoding scheme?

Now that would be nice, IF one could get away with it. Unfortunately,
throwing away all of the gigabytes of text recorded as other than UCS-4
is definitely impractical, so we're doomed to live with several.

A single analogy?

Perish the thought -- the more the merrier.


Alex

 

[prunesquallor]

Joe Marshall wrote:
...

I far prefer to have on my cellphone one that's specialized for its small
screen and puny cpu/memory, and on more powerful computers, more
powerful browsers. Don't you?

Cellphone? Why on earth would I want one of those?

On my cellphone all the way to the datacenter?

A cellphone with an OS?! The phone I use is implemented
with *wires*.

 

[Alexander Schmolck]

[narrowing down to c.l.l and c.l.p]

Only if your cut boundaries are at the same lexical level. If you cut
across boundaries, it is no longer clear what should happen at the paste.

The only unclarity would arise from dedents of more than one level on the
corner of your cut region, I think.

Also, it is frequently the case that you need to `tweak' the code after
you paste it.

Sure, but I don't generally feel that tweaking python code in emacs is
particularly inconvenient (it is not always as convenient as tweaking lisp
code, which is partly due to the fact that essentially there isn't any proper
implementation of the sexp commands for non-lisp modes; OTOH some operations
are more convenient).

You misunderstand me. In a python block, two expressions are
associated with each other if they are the same distance from the left
edge. This is isomorphic to having a nametag identifying the scope
of the line. Lines are associated with each other iff they have the
same nametag. Change one, and all must change.

Hardly a problem with a decent editor (and surely less of a problem even with
the proverbial notepad than getting the closing paren in the right spot and
reindenting everything).

because the grouping information is replicated on each line, I have to
fix this information in the six different places it is encoded:

Why, for example, not just first insert the new if statement and then run an
editor command that indents all the following lines up to the same indent
level one further indent level? To me this wouldn't seem to me have
intrisically more room for screw-ups than inserting a '(WHEN' and an
additional ')'.

The fact that the information is replicated, and that there is nothing
but programmer discipline keeping it consistent is a source of errors.

Sure there is. Your editor and immediate visual feedback (no need to remember
to reindent after making the semantic changes).

That is because the parenthesis *only* encode the grouping information,
they do not do double duty and encode what they are grouping.
The key here is to realize that the words `DEFUN' and the `IF' themselves
look very different.

Well, I'm evidently still not getting it, because my reply would be "and so do
'def ...:' and 'if ...:' in python" (and you also can't tell whether the 8
spaces on the left margin come from a 'def' and and an enclosed 'if' or vice
versa).

Sure. Here are five parens ))))) How much whitespace is there here:

10 spaces (which BTW I counted in emacs in just the same way that I'd count a
similar number of parens) -- but what has counting random trailing whitespace
got to do with anything? Apart from the fact that only leading whitespace is
significant in python, I've personally never felt any need to count it -- have
you? I have, however felt the need to count parens from time to time (in
python as well).

Still, I'm sure you're familiar with the following quote (with which I most
heartily agree):

"[P]rograms must be written for people to read, and only incidentally for
machines to execute."

People can't "read" '))))))))'.

Funny, the people you just quoted would disagree with you about parenthesis.
I expect that they would disagree with you about whitespace as well.

Why, do they know someone with a paren-counting brain-implant? If so every
lisper should get one -- no more carpal tunnel syndrome from frantic C-M-\
pressing or alphabet wastage due to scheme-style 'aliasing' of '[...]' to
'(...)'.

Seriously, I really fail to see see what the source of disagreement with the
above statement could be.

If I gave you a piece of lisp code jotted down on paper that (as those
hypothetical examples usually are) for some reason was of vital importance and
that on typing it in revealed a mismatch between the indentation and the
parenthesation in a key section of the code -- which interpretation would you
hedge your bets on in the absence of other indicators; the one suggested by
the indentation or the 9 trailing parens?

Anyway, readability is an overarching but not the only important factor in the
design of language's syntax (you mentioned the ergonomics of editing and there
are of course others), and as I said I think sexps are in fact a rather good
syntax for an expression oriented language.

'as

 

[Doug Tolton]

Can't speak for others, but it certainly would be a mistake to apply it
to another HLL.




OK, another Pythonista just told me GVR had greater ambitions. Just
tellin ya what I hear.





No, I got that, but I just wrote it kinda convoluted. And that
self-selection thing is just silly, until people over here:

http://alu.cliki.net/Kenny's RtLS Top-Ten

...come back in a month and update their responses to say "Drat! That
language is every bit as great as I thought it was, but that syntax is
driving me nuts. I'm outtahere!"

Won't happen, btw. Hell, Tolton loved Lisp even before he picked up some
editing tips.

You know, I just remembered a relevant experience I had, only with a
very early release of Dylan during the search I conducted which led to
Common Lisp, aka The Promised Land.

I actually made a bug report to the Dylan team: "hey, when I hit tab the
cursor jumps way the hell out here, just inside the IF. I mean, that's
pretty fucking cool if you meant that to happen, but what's going on?"






Sheesh, who hasn't been exposed to basic? From my generation, that is.
But no matter, the point is anyone can handled parens if they try for
more than an hour.




lessee:

symbols? no
sexprs? no
code as data as code? no

sorry, charlie.




Boy, you sure can read a lot into a casually chosen cliche. But can we
clear up once and for all whether these genius scientists are or are not
as good a programmer as you? I thought I heard Python being recommended
as better for non-professional programmers.

Mind you, to my horror my carefully trained goalie turned out not to
scale at all into game play (my fault) so i am back to square one with
two days to go, so maybe I am not following all this as well as I should.

I will admit getting those editing key strokes was pretty dang nice.

Although I have been doing my coding in Emacs all along.

 

[Pascal Costanza]

Pascal Costanza wrote:
...



Amen, hallelujah. So, since I've decided to limit my participation in this
thread to c.l.python, would you kindly set right the guys (such as your
namesake) who (on c.l.lisp with copy to my mailbox but not to here) are
currently attacking me because, and I quote,
"""
Software is a department of mathematics.
"""
...?

The context in which this statement was made is not clear to me, so I
don't know what I would actually defend.

I haven't followed all branches of the thread, and the posting that
contains that statement is a reaction to an analogy that I think is not
a good one. I don't want to dive further into this specific branch. Sorry.


Pascal

 

[Corey Coughlin]

You are mostly correct about Japanese, I took a year of it in college
and it is a fairly standard SOV language. (Like Latin, oddly enough.)
And I'm sure you're right about RPN vs. PN, I always get those
confused. Which is kind of the point, really, having studied math
since I was a kid I got used to stuff like "y = mx + b", can you
blame me if I have an easier time with "y = m*x + b" as opposed to
"(let y (+ (* m x) b))" (Forgive me if the parenthesis on that are
off, the newsreader editor doesn't match them, and maybe I need a
'setq' instead of a 'let' or some other thing, I'm not entirely sure.)
(And again, is the point getting more clear?) And thanks for backing
me up on car's and cdr's, I never could understand why a language
ostensibly designed for 'list processing' has such a bizarre way to
reference items in a list. But is (nth 10 mylist) really as easy as
mylist[10]? My intuition says no, not really.

Sure, I can appreciate looking at things in different ways, and it is
nice to learn new things and see how they apply. But if John Grisham
learns Japanese, does that mean he should write all his books in
Japanese? Or should he stick to English? I suppose if I were a real
CS guy (I'm actually an electrical engineer, the scheme course was one
of the two CS courses I took in college, so I'm mostly self taught) or
if I worked within a big group of Lisp programmers, I would probably
feel more comfortable with it. Since I now mostly work as an isolated
programmer for other engineers, and the last language I was using for
everything was C, Python is a huge improvement, and it doesn't give me
too much of a headache. Sure, it's not perfect. But there's no way
I'm going to adopt Lisp as a perfect language anytime soon. That is,
if I want to keep hitting my deadlines and getting paid. And sure, I
may get comfortable and miss out on cool stuff, but on the upside,
I'll be comfortable.

Oh, and if I'm writing in this thread, I suppose I should comment on
how bad lisp macros are. Except I know nothing about them. But it
seems like most languages have dark corners like that, where you can
do thing above and beyond your standard programming practices. Python
has metaclasses, which give me a headache most of the time, so I don't
really use them at all. But I seem to get plenty of stuff done
without using them, so it works for me. If you really have to use
macros in Lisp to get things done, that sounds kind of troublesome,
but it would be consistent, it always seemed like really working well
in Lisp requires you to really know how everything works all at once,
which always kind of struck me as kind of a downside. But as I said,
I'm not the big CS guru, so Lisp just may not be for me in general.
Ah well, I suppose I'll get by with Python.

 

[Andrew Dalke]

Pascal Costanza:

And, as Paul Graham put it, if you take a language and "add that final
increment of power, you can no longer claim to have invented a new
language, but only to have designed a new dialect of Lisp". (see
http://www.paulgraham.com/diff.html )

What about a language for quantum based computing? I
know the perl folk have a stab at emulating such a language
http://www.perl.com/pub/a/2001/08/08/quantum.html
Are there programming possibilities in manipulating ensembles
of possible expressions? Eg, an O(sqrt(N)) search of all
possible algorithms which solve an optimization problem
might prove interesting.

Hmmm, if a problem can be solved in a language of 32 operators
using, say, 50 of those operators, then it can be found after only
O(sqrt(32**50)) == O(2**30) steps.

I'm sure there are other possibilites which are more realistic though.

The fact that such hardware does not exist shouldn't
be a limitation. Babbage's code worked on a machine that
was never built (*and* it had the potential ability to modify
it's own input, making Lisp a 'dialect' of that ur-language),
and Turing's work was also on a theoretical machine.

But then, I'm just a speaker of a dialect of Proto-Indo-European
and my language hasn't improved on the original over the last
few thousand years.

Andrew
(e-mail address removed)

 

[Andrew Dalke]

Alex Martelli:

Absolutely! But funding such studies would seem hard. Unless some
company or group of volunteers had their own reasons to take some
existing large app coded in Lisp/Python/Perl/Ruby, and recode it in
one of the other languages with essentially unchanged functionality,

I keep thinking that the bioperl/biopython/bioruby/biojava code bases
would be a source of data for such a study, but upon just a bit of
thinking I realized that each of the projects has a different way of
dealing with a problem. There is overlap between the two, but even
at the file parsing level, bioperl's approach is different from biojava's
and quite different from biopython's.

Another possibility is the code needed for some cross-platform
standard, like a DOM. That's possible, but I suspect there are
large variations in code size even amoung the Python DOMs,
as well as little likelihood of full compliance by all code bases.

Andrew
(e-mail address removed)

 

[Andrew Dalke]

Alex:

Well, somebody once asked how
a Pythonista would go about it -- and the answer was unanimous:
print "Just another Python hacker"

It wasn't unanimous. I recall I posted something like
.... def __getattr__(self, word): print word,; return self
.... def __repr__(self): return ""
....Just another Python hacker

My perhaps-not-extensive-enough experience with macros showed them
being used to merge language and meta-language -- in widely different
ways in different labs or even within a given lab -- while at the
same time other firms were using languages without macros (APL and
variants thereof) and processing them with different and separate
metalanguages AND thereby managing to achieve better (intra-firm, at
least) cooperation.

I rather like the phrase I used in private mail: supports mutliple language
paradigm instead of a multiple paradigm language.

Andrew
(e-mail address removed)

 

[Andrew Dalke]

Alex Martelli:

would you kindly set right the guys (such as your
namesake) who (on c.l.lisp with copy to my mailbox but not to here) are
currently attacking me because, and I quote,
"""
Software is a department of mathematics.
"""

And anyone who doesn't think mathematics has its own
culture with ideas and even mistaken preferences for what
is right and wrong should read

The Mystery of the Aleph: Mathematics, the Kabbalah, and the Human Mind

to see how Cantor's ideas of transfinite numbers (and other ideas,
as I recall, like showing there are functions which are everywhere
continuous and nowhere differentiable) were solidly rejected by
most other mathematicians of his time.

Mathematicians are people as well.

Andrew
(e-mail address removed)

 

[Matthew Danish]

I find the Lisp syntax hardly readable when everything looks alike,
mostly words and parentheses, and when every level of nesting requires
parens. I understand that it's easier to work with by macros, but it's
harder to work with by humans like I.

I find the ML/Haskell/infix syntax hardly readable when everything is
visually ambiguous, mostly a jumble of non-alphanumeric characters, and
it is difficult to determine where nesting occurs without looking it up
in a reference. I understand that this gives the code the dubious
distinction of looking a little bit like math notation, perverted into
ASCII, but it's harder to work with by Lisp programmers like myself.


P.S. I've written my fair share of SML, I'd like to think, and while it
could be quite infuriating I did appreciate pattern matching and
currying. Of course, both of these can be done in CL, and more flexibly
too.

 

[prunesquallor]

[narrowing down to c.l.l and c.l.p]

Only if your cut boundaries are at the same lexical level. If you cut
across boundaries, it is no longer clear what should happen at the paste.

The only unclarity would arise from dedents of more than one level on the
corner of your cut region, I think.

Suppose I cut just one arm of a conditional. When I paste, it is
unclear whether I intend for the code after the paste to be part of
that arm, part of the else, or simply part of the same block.

Sure there is. Your editor and immediate visual feedback (no need to remember
to reindent after making the semantic changes).

`immediate visual feedback' = programmer discipline
Laxness at this point is a source of errors.

Well, I'm evidently still not getting it, because my reply would be "and so do
'def ...:' and 'if ...:' in python" (and you also can't tell whether the 8
spaces on the left margin come from a 'def' and and an enclosed 'if' or vice
versa).

I'm not sure I can explain any better, I *think* perhaps some of the other
Lispers might get what I'm saying here.

10 spaces (which BTW I counted in emacs in just the same way that I'd count a
similar number of parens) -- but what has counting random trailing whitespace
got to do with anything?

It is simply an illustration that there is no obvious glyph associated
with whitespace, and you wanted a concrete example of something that can't
be displayed.

Still, I'm sure you're familiar with the following quote (with which I most
heartily agree):

"[P]rograms must be written for people to read, and only incidentally for
machines to execute."

People can't "read" '))))))))'.

Funny, the people you just quoted would disagree with you about parenthesis.
I expect that they would disagree with you about whitespace as well.

Why, do they know someone with a paren-counting brain-implant? If so every
lisper should get one -- no more carpal tunnel syndrome from frantic C-M-\
pressing or alphabet wastage due to scheme-style 'aliasing' of '[...]' to
'(...)'.

Seriously, I really fail to see see what the source of disagreement with the
above statement could be.

I cannot read Abelson and Sussman's minds, but neither of them are
ignorant of the vast variety of computer languages in the world.
Nonetheless, given the opportunity to choose any of them for
exposition, they have chosen lisp. Sussman went so far as to
introduce lisp syntax into his book on classical mechanics.
Apparently he felt that not only *could* people read ')))))))', but
that it was often *clearer* than the traditional notation.

If I gave you a piece of lisp code jotted down on paper that (as those
hypothetical examples usually are) for some reason was of vital importance and
that on typing it in revealed a mismatch between the indentation and the
parenthesation in a key section of the code -- which interpretation would you
hedge your bets on in the absence of other indicators; the one suggested by
the indentation or the 9 trailing parens?

Obviously the indentation. But I'd notice the mismatch.

If I gave you a piece of python code jotted down on paper that (as these
hypothetical examples usually are) for some reason was of vital importance
but I accidentally misplaced the indentation -- how would you know?