Python syntax in Lisp and Scheme

[Joe Marshall]

Apart from that, can you have higher-order macros? With mutual
recursion between a macro and its argument? That is, can you write a
fixpoint operator on macros?

Yes, you can. This occurs when you call MACROEXPAND as part of
computing the expansion of the macro.

 

[Joe Marshall]

That's irrelevant. When it happens doesn't change the fact that this
proves it (multiple dispatch with non-congruent arglists) is possible.
Nothing prevents you from using the same algorithm at run time.

In fact, on a project I'm working on I have to handle overloaded
functions. I used the MOP to adjust how method combination worked
and now the generic functions first perform an arity-based dispatch.

 

[Andrew Dalke]

Pascal Costanza:

I guess this reflects his experiences when he has learned Lisp in the
beginning of the 80's (AFAIK).

But the talk wasn't given in the early 80s. It was given in the early 00s.
And it implies that Lisp is still strange that way. OTOH, you aren't him
so it's not fair of me to ask you all to defend his statements.

Yes, scripting languages have caught up in this regard. (However, note
that Common Lisp also includes a full compiler at runtime.)

However, that's an implementation difference -- the only thing
users should see is that the code runs faster. It doesn't otherwise
provide new functionality.

No, that's not a logical conclusion.

I used those wishy-washy words "reminds me" and "seems".

These abbreviations seem strange to a Lisp outsider, but they are very
convenient, and they are easy to read once you have gotten used to them.
You don't actually "count" the elements in your head every time you see
these operators, but they rather become patterns that you recognize in
one go.

I did know about cddr, etc. from Hofstadter's essays in Scientific
America some 15 years ago.

I don't know how this could be done with 1st, rst or hd, tl respectively.

Okay, I gave alternatives of "." and ">" instead of "car" and "cdr"
"." for "here" and ">" for "the rest; over there". These are equally
composable.

.. == car

== cdr

cadr == >.
caddr == >>.
cddr == >>

Pick your choice. "There is not only one way to do it." (tm)

Perl beat you to it -- "TMTOWTDO" (There's more than one way to
do it.).

Python's reply "There should be one-- and preferably only one --
obvious way to do it."

The learning curve is steeper, but in the long run you become much more
productive.

Which brings us back to the start of this thread.

Andrew
(e-mail address removed)

 

[Lulu of the Lotus-Eaters]

|Something like this seems more logical to me:
|for line in file('input.txt').lines:
| do_something_with(line)
|for byte in file('input.txt').bytes:
| do_something_with(byte)

Well, it's spelled slightly differently in Python:

for line in file('input.txt').readlines():
do_something_with(line)

for byte in file('input.txt').read():
do_something_with(byte)

Of course, both of those slurp in the whole thing at once. Lazy lines
are 'fp.xreadlines()', but there is no standard lazy bytes.

A method 'fp.xread()' might be useful, actually. And taking a good idea
from Dave Benjamin up-thread, so might 'fp.xreadwords()'. Of course, if
you were happy to write your own class 'File' that provided the extra
iterations, you'd only need to capitalize on letter to get these extra
options.

Yours, Lulu...

 

[Kenny Tilton]

Kenny Tilton wrote:




The very 'feature' that was touted by Erann Gat as macros' killer advantage
in the WITH-CONDITION-MAINTAINED example he posted is the crucial
difference: functions (HO or not) and classes only group some existing code
and data; macros can generate new code based on examining, and presumably to
some level *understanding*, a LOT of very deep things about the code
arguments they're given.

Stop, your scaring me. You mean to say there are macros out there whose
output/behavior I cannot predict? And I am using them in a context where
I need to know what the behavior will be? What is wrong with me? And
what sort of non-deterministic macros are these, that go out and make
their own conclusions about what I meant in some way not documeted?

I think the objection to macros has at this point been painted into a
very small corner.

...If all you do with your macros is what you could
do with HOF's, it's silly to have macros in addition to HOF's

There is one c.l.l. denizen/guru who agrees with you. I believe his
position is "evrything can be done with lambda". And indeed, many a
groovy WITHOUT-CELL-DEPENDENCY expands straight into:

(call-with-cell-dependency (lambda () ,yadayadayada))

But code with WITHOUT-CELL-DEPENDENCY looks prettier (I hope we can
agree that that matters, esp. if you are a Pythonista).

-- just

MTOWTDItis encouraging multiple different approaches to solve any given
problem -- this, of course, in turn breeds divergence when compared to a
situation in which just one approach is encouraged. If you do use the
potential implied in that example from Gat, to do things that functions and
classes just couldn't _begin_ to, it's worse -- then you're really
designing your own private divergent language (which most posters from
the Lisp camp appear to assert is an unalloyed good, although admittedly
far from all). This is far from the first time I'm explaining this, btw.

Oh. OK, now that you mention it I have been skimming lately.

Do they do things a HOF or class could do?
Yes.

... If so why bother using such
an over-powered tool as macros instead of HOFs or classes?

Hang on, we just agreed that in this case the only added value is
prettier code. Nothing over-powered going on. (And that is, in this
case, why I bother.)

 

[Doug Tolton]

Doug Tolton wrote:

David Mertz wrote:

There's something pathological in my posting untested code. One more
try:

def categorize_jointly(preds, it):
results = [[] for _ in preds]
for x in it:
results[all(preds)(x)].append(x)
return results

|Come on. Haskell has a nice type system. Python is an application of
|Greespun's Tenth Rule of programming.

Btw. This is more nonsense. HOFs are not a special Lisp thing. Haskell
does them much better, for example... and so does Python.

What is your basis for that statement? I personally like the way Lisp
does it much better, and I program in both Lisp and Python. With Python
it's not immediately apparent if you are passing in a simple variable
or a HOF. Whereas in lisp with #' it's immediately obvious that you are
receiving or sending a HOF that will potentially alter how the call
operates.

IMO, that syntax is far clearner.

I think it's about a single namespace (Scheme, Python, Haskell, ...) vs
CLisp's dual namespaces. People get used pretty fast to having every
object (whether callable or not) "first-class" -- e.g. sendable as an
argument without any need for stropping or the like. To you, HOFs may
feel like special cases needing special syntax that toots horns and
rings bells; to people used to passing functions as arguments as a way
of living, that's as syntactically obtrusive as, say, O'CAML's mandate
that you use +. and not plain + when summing floats rather than ints
(it's been a couple years since I last studied O'CAML's, so for all I
know they may have changed that now, but, it IS in the book;-).

Yeah I'm not a big fan of +. and /. etc. Every operation on floating
point values has to be explicitly specified. You can't even do a 2 +.
3.0. Instead you have do do (float_of_int 2) +. 3.0
I'm not a huge fan of their syntax either. I don't think they've
removed it, because it's in the first chapter of the tutorial on their site.

I also uses HOF's daily, and I don't generally run into problems
specifying that a variable is in fact a function. I like the sharp
quote thouge because it makes it clear that something a little out of
the ordinary is occurring. Honestly though either way is fine with me.
I can see the arguments both ways.

btw CLisp is an implementation of Common Lisp. If you want to shorten
it, it's usually less ambiquous if you use CL.

I'm not going to comment on the dual versus single namespaces, because
that isn't an area I'm very familiar with how Lisp operates.

 

[Thomas F. Burdick]

The very 'feature' that was touted by Erann Gat as macros' killer advantage
in the WITH-CONDITION-MAINTAINED example he posted is the crucial
difference: functions (HO or not) and classes only group some existing code
and data; macros can generate new code based on examining, and presumably to
some level *understanding*, a LOT of very deep things about the code
arguments they're given.

Yes! Macros can generate code, and compiler-macros can transform
perfectly ordinary, easy-to-read code into efficient code. Anyone who
has worked in a domain where efficiency matters has run into the
problem most languages have: abstraction *or* efficiency. Custom,
domain-specific transforms are something you can't always expect the
compiler to do. With Lisp, you're not at the mercy of your vendor; if
you know damn well that some readable code A can be transformed into
equivalent, but efficient code B, you can cause it to happen!

If all you do with your macros is what you could do with HOF's,

But you can do more with macros, so there's no point in looking at the
conclusion to this sentance.

Oh, and if you're one of those who disapprove of Gat's example feel free
to say so, but until I hear a substantial majority denouncing it as idiotic
(and I haven't seen anywhere near this intensity of disapproval for it from
your camp)

Of course not, it's a lovely example of one use of macros.

I'm quite justifyied in taking it as THE canonical example of a
macro doing something that is clearly outside the purview of normal
tools such as functions and classes.

No, you're not justified at all.

--
/|_ .-----------------------.
,' .\ / | No to Imperialist war |
,--' _,' | Wage class war! |
/ / `-----------------------'
( -. |
| ) |
(`-. '--.)
`. )----'

 

[Andrew Dalke]

Björn Lindberg:

Apart from the usual problems with micro benchmarks, there are a few
things to consider regarding the LOC counts on that site:

I wasn't the one who pointed out those micro benchmarks. Kenny
Tilton pushed the idea that more concise code is better and that Lisp
gives the most concise code, and that Perl is much more compact
than Python. He suggested I look at some comparisons, so I followed
his suggestion and found that 1) the Lisp code there was not more
succinct than Python and 2) neither was the Perl code.

* Declarations. Common Lisp gives the programmer the ability to
optimize a program by adding declarations to it.

While OCaml, which has the smallest size, does type inferencing....

since the
micro benchmarks in the shootout are focused on speed of
execution, and they are so small, all of them contains a lot of
declarations, which will increase LOC.

Ahh, good point.

* In many languages, any program can be written on a single
line. This goes for Lisp, ut also for C and other languages.

Absolutely correct. Both Alex Martellli and I tried to dissuade
Kenny Tilton that LOC was the best measure of succinctness and
appropriateness, and he objected.

* I don't think the LOC saving qualities of Lisp is made justice in
micro benchmarks. The reason Lisp code is so much shorter than the
equivalent code in other languages is because of the abstractive
powers of Lisp, which means that the difference will be more
visible the larger the program is.

Agreed. I pointed out elsewhere that there has been no systematic
study to show that Lisp code is indeed "so much shorter than the
equivalent code in other languages" where "other languages" include
Python, Perl, or Ruby.

The closest is
http://www.ipd.uka.de/~prechelt/Biblio/
where the example program, which was non-trivial in size
took about 100LOC in Tcl/Rexx/python/perl and about 250LOC
in Java/C/C++.

There was a repeat of that test at
http://www.flownet.com/gat/papers/lisp-java.pdf
which showed that the average Lisp size was 119 LOC and
277 for Java. I eyeballed the numbers, and I'm not sure if
the counts included comments or not, so call it the same.

In any case, it implies you need to get to some serious sized
programs (1000 LOC? 10000LOC? A million?) before
the advantages of Lisp appear to be significant.

That's not saying that they are more obvious in some subdomain.
For that matter, if I want to do hardware I/O on some
memory mapped ports, it's pretty obvious that C is a good
contender for that domain.

So we're left with depending on gut feelings, guided by
(non-rigorous) observation. Eg, observations that Python
does scale to large projects, observations that the people
who will use my code (computational scientists, not
programmers) find Python easier than Lisp and Tcl-style
commands easier than Python .

Andrew
(e-mail address removed)

 

[Andrew Dalke]

Is portability of code across different language implementations not a priority
for LISP programmers?

james anderson:

there are some things which the standard does not cover.

"The standard" as I understand it, is some document written a decade
ago. Can't a few of you all get together and say "hey, the world's
advanced. Let's agree upon a few new libraries and APIs."?


Andrew
(e-mail address removed)

 

[Erann Gat]

then you're really
designing your own private divergent language (which most posters from
the Lisp camp appear to assert is an unalloyed good, although admittedly
far from all).

FWIW, IMO "designing your own private divergent language" is not an
unalloyed good, but having the *ability* to design your own private
divergent language quickly and easily is. Also, the "private divergent
languages" that most people design using Lisp macros are supersets of
Lisp, so that limits the extent to which divergence happens in practice.

Your position against macros and in favor of HOFs strikes me as being very
similar to those who want to ban model rocketry on the grounds that it is
too dangerous. Yes, macros (and model rockets) can be dangerous. If
you're not careful you can put your eye out with them. But if you are
careful you can do very cool things with them and -- almost as important
-- learn a lot in the process. (And what you learn from Lisp and model
rocketry are deep truths about the world, not a bunch of random kabuki
juju like what you fill your brain with when you learn C++.) This
mindset, that anything that is potentially dangerous ought to be avoided
because it is potentially dangerous, is IMHO a perverse impediment to
progress. There is no reward without risk.

Life is short. It's not hard to produce a reasonable estimate of the
total number of keystrokes that you will be able to execute in a lifetime,
and it's a pretty small number in the grand and glorious scheme of
things. If you have no ambitions beyond writing
yet-another-standard-web-app then macros are not for you. But if your
goals run grander than that then the extra leverage that you get from
things like macros becomes very precious indeed. Once your ambitions pass
a certain point the only option open to you is to teach your computer to
write code for you, because you don't have time to do it yourself. For
example, there is no reason it should take multiple work years to write an
operating system. There is no fundamental reason why one could not build
a computational infrastructure that would allow a single person to write
an operating system from scratch in a matter of days, maybe even hours or
minutes. But such a system is going to have to have a fairly deep
understanding of the relationship of code to hardware. You may want to
write things like:

(define-hardware-type ethernet-controller ...)

or

(define-hardware-standards-hierarchy
(networking
(ethernet
(standard-ethernet
(NE2000 ....))
(fast-ethernet ...)
(gigabit-ethernet ...))
(fddi ...)
(fibre-channel ...)
...)
(mass-storage
(hard-drive
(ide ...)
(scsi ...))

or

(is-a ne2000 standard-ethernet-card)

or

(define-register-layout ...)

God only knows. Only one thing is certain: with macros and readtables you
will be limited only by your imagination. With anything less you will be
limited by something else.

E.

 

[Andrew Dalke]

Me:
Doug Tolton:

That's not exactly my position, rather my position is that just about
anything can and will be abused in some way shape or fashion. It's a
simple fact of working in teams. However I would rather err on the side
of abstractability and re-usability than on the side of forced

restrictions.

You are correct. I misremembered "Tolton" as "Tilton" and confused
you with someone else. *blush*

My answer, btw, that the macro preprocessor in C is something
which is useful and too easily prone to misuse. Eg, my original
C book was "C for native speakers of Pascal" and included in
the first section a set of macros like

#define BEGIN {
#define END }

It's not possible to get rid of cpp for C because the language
is too weak, but it is something which takes hard experience to
learn when not to use.

As for a language feature which should never be used. Alex Martelli
gave an example of changing the default definition for == between
floats, which broke other packages, and my favorite is "OPTION
BASE 1" in BASIC or its equivalent in Perl and other langauges.
That is, on a per-program (or even per-module) basis, redefine
the 0 point offset for an array.

Andrew
(e-mail address removed)

 

[Andrew Dalke]

Jon S. Anthony:

This thing has been debunked for years. No one with a clue takes it
seriously. Even the author(s) indicate that much of it is based on
subjective guesses.

Do you have a reference? And a pointer to something better?

The only one better I know is
http://www.ipd.uka.de/~prechelt/Biblio/jccpprtTR.pdf

which on page 23, Fig. 17, has a mapping from median
work time actual to work time w.r.t the language list.

Of the 6 languages there are two major misorderings.
First, C actually ended up easier to use than Java or C++.
(which is strange since you would think C++ would be
at least as good as C), and second, Tcl actually ends up
much better.

2 of 6 is better than random, so Jones' work can't be
complete bunkum.

Andrew
(e-mail address removed)

 

[Daniel Silva]

james anderson:

"The standard" as I understand it, is some document written a decade
ago. Can't a few of you all get together and say "hey, the world's
advanced. Let's agree upon a few new libraries and APIs."?

Isn't that what repositories like perl's CPAN are for?

Though it would be nice if everyone agreed on a single, simple
foreign-function interface...

- Daniel

 

[Andrew Dalke]

Thomas F. Burdick:

With Lisp, you're not at the mercy of your vendor; if
you know damn well that some readable code A can be transformed into
equivalent, but efficient code B, you can cause it to happen!

Not at the mercy of your vendor unless you want to use something
which isn't in the standard, like unicode (esp "wide" unicode, >16bit),
regular expressions (esp. regexps of unicode), sockets, or ffi?

But that's just flaming -- ignore me.

Andrew
(e-mail address removed)

 

[Jon S. Anthony]

The very 'feature' that was touted by Erann Gat as macros' killer advantage
in the WITH-CONDITION-MAINTAINED example he posted is the crucial
difference: functions (HO or not) and classes only group some existing code
and data; macros can generate new code based on examining, and presumably to
some level *understanding*, a LOT of very deep things about the code
arguments they're given.

Are you really this dense? For crying out loud - functions can
generate code as well and just as easily as macros. In fact macros
are just functions anyway so it really should go without saying.

If all you do with your macros is what you could do with HOF's,
it's silly to have macros in addition to HOF's -- just

No it isn't, because they the mode of _expression_ may be better with
on in context A and better with the other in context B.

MTOWTDItis encouraging multiple different approaches to solve any given
problem -- this, of course, in turn breeds divergence when compared to a

Actually it breeds better solutions to problems. If you don't
understand this, you will never understand much of anything about good
problem solving.

Oh, and if you're one of those who disapprove of Gat's example feel free
to say so, but until I hear a substantial majority denouncing it as idiotic

At the moment the only thing I am willing to denounce as idiotic are
your clueless rants.


/Jon

 

[Rayiner Hashem]

Lisp doesn't let you do that, because it turns out to be a bad idea.
Only if you subscribe to the Java-mentality that power is bad. Operator
overloading is a must in languages that strive to make user-defined types
fully equal to built-in types (like Dylan, and Goo, maybe CLOS). Besides,
prohibiting operator-overloading is not orthogonal in languages with
generic dispatch, because it creates a pointless schism between built-in
operators and user-defined types.

When you go reading someone's program, what you really want is for
the standard operators to be doing the standard and completely
understood thing.

Why? Usually, you cannot overload operators on built-in types. So an integer
plus an integer will always have a specific meaning. However a matrix plus
a matrix can have a user-defined meaning, assuming matricies are
user-defined types. Now, a programmer could go ahead and make an overload
of operator+ that actually subtracted matricies, but he could just as well
write a function "add-matrix" that actually subtracted matricies. And
assuming that the programmer isn't trying to lie to you, its much easier to
understand

(+ m1 m2 m3)

than

(add-matrix m1 m2 m3)

because '+' carries with it an expected set of semantics, while the reader
has to go to the definiton of add-matrix to fully understand its semantics.

Lisp has carefully defined those operations to do all the things
that are both well-understood (like complex numbers) and missing
from languages like C++.

Actually, C++ has a complex number class in its standard library. Why not in
the language proper? There is no need for it to be in the language, because
it can be implemented in the library just as easily. Its the same reason
'loop' is a macro and not built into the language. Besides, what do you do
when you need quaternions? Wait for the next version of the standard?

 

[Andrew Dalke]

Doug Tolton:

I have this personal theory (used in the non-strict sense here) that
given enough time any homogenous group will split into at least two
competing factions.

Reminds me of Olaf Stapledon's "First and Last Men"? His
civilizations often had two roughtly equal but opposing components.

Also reminds me of learning about the blue eyed/brown eyed
experiment in my sociology class in high school. As it turns out,
I was the only blue-eyed person in the class of 25 or so.

Over time it seems to me that human
beings are incapable of remaining as one single cohesive group, rather
that they will always separate into several competing factions. Or at
the very least groups will splinter off the main group and form their
own group.

Not necessarily "competing", except in a very general sense. Is
Australian English in competition with Canadian English?

However in the opensource world I expect splinters to happen frequently,
simply because there is little to no organizational control. Even
Python hasn't been immune to this phenomenon with both Jython and
Stackless emerging.

As well as PyPy and (more esoterically) Vyper.

Excepting the last, all have had the goal of supporting the C Python
standard library where reasonably possible. When not possible
(as the case with Jython and various C extensions), then supporting
the native Java libraries.

"bristly"

Ohh! Good word! I had forgotten about it.

Andrew
(e-mail address removed)

 

[Jon S. Anthony]

Pascal Costanza:

It's undefined on strings -- a type error. Having + doesn't
mean that - must exist.

No, but it makes the semantics odd for an operation named by "+". Of
course it may not be obvious what the semantics should be, but then
the semantics of "hi" + "there" isn't obvious either.

(A more interesting question would be to ask what
the result of "throne" - "one" is. But again, a type error.)

Why? This seems like a likely candidate for a string -.

I understand your point of view. OTOH, it's like when I used to
work with C. It was standardized, but required that I download
a slew of packages in order to do things.

That's why there are _implementations_. It's odd that the obvious
distinction between "compiler" (or interpreter or whatever) and
"language" is so hard to grasp.

appropriate. I know there are good reasons for a standard to
leave out useful packages, but I know there are good reasons for
an implementation to include a large number of useful packages.

Wow, this actually sounds right.

Is there a free Lisp/Scheme implementation I can experiment with
which include in the distribution (without downloading extra
packages; a "moby" distribution in xemacs speak):
- unicode
- xml processing (to some structure which I can use XPath on)
- HTTP-1.1 (client and server)
- URI processing, including support for opening and reading from
http:, file:, and https:
- regular expressions on both 8-bit bytes and unicode
- XML-RPC
- calling "external" applications (like system and popen do for C)
- POP3 and mailbox processing

Yes. Allegro CL (ACL) for one.

As far as I can tell, there isn't. I'll need to mix and match packages

You obviously can't "tell" too well.

No, but there is a good overlap. I believe all of the above are
supported on both implementations.

Interaction with Java (to access it's libararies and whatnot) is also
in ACL.

Which Common Lisp *distribution* provides the above? I

One is pointed out above.


/Jon

 

[Jon S. Anthony]

Ahhh, so make the language easier for computers to understand and
harder for intelligent users to use?

Spoken like a true Python supporter...

/Jon

 

[Erann Gat]

If you do use the
potential implied in that example from Gat, to do things that functions and
classes just couldn't _begin_ to, it's worse -- then you're really
designing your own private divergent language (which most posters from
the Lisp camp appear to assert is an unalloyed good, although admittedly
far from all).

[This may be a duplicate posting -- I wrote a response to this earlier but
it seems to have vanished into the cosmic void.]

FWIW, I do not believe that "designing your own private divergent language
.... is an unalloyed good." I do, however, believe that having the
*ability* to quickly and easily design your own private language is a Good
Thing. In practice, most "private languages" built using macros are
supersets of Lisp, so this limits the extent to which divergence matters
in practice.

(My original post went on with a rant about how risk aversion was an
impediment to progress, but I think I'll take the disappearance of my
original post as a Sign From God and just leave it at that this time
around.)

E.