BIG successes of Lisp (was ...)

[Pascal Costanza]

Indentation in Lisp is not clear enough. Let's look at an example from
http://www-users.cs.umn.edu/~gini/aiprog/graham/onlisp.lisp:

(defun mostn (fn lst)
(if (null lst)
(values nil nil)
(let ((result (list (car lst)))
(max (funcall fn (car lst))))
(dolist (obj (cdr lst))
(let ((score (funcall fn obj)))
(cond ((> score max)
(setq max score
result (list obj)))
((= score max)
(push obj result)))))
(values (nreverse result) max))))

Note that only one pair of adjacent lines is indented by the same amount.
Other alignments are in the middle of lines.

Here is a straightforward translation into my dream language; note that
there aren't a lot of parens despite insignificant indentation and despite
using braces (like C) instead of bracketing keywords (like Pascal):

def mostn Fun [] = [], null;
def mostn Fun (First\List) {
var Result = [First];
var Max = Fun First;
each List ?Obj {
let Score = Fun Obj;
if Score > Max {Max = Score; Result = [Obj]}
if Score == Max {Result = Obj\Result}
};
reversed Result, Max
};

Apparently, Paul Graham doesn't like CLOS nor the LOOP macro. Here is
another verson in Common Lisp (and this is not a dream language :

(defmethod mostn (fn (list (eql nil)))
(declare (ignore fn list))
(values nil nil))

(defmethod mostn (fn list)
(loop with result = (list (car list))
with max = (funcall fn (car list))
for object in (cdr list)
for score = (funcall fn object)
when (> score max) do (setq max score
result (list object))
when (= score max) do (push object result)
finally return (values (nreverse result) max)))


Pascal

 

[Borcis]

Do you often let the behavior of others control how you think
about technical topics?

In a stretched sense of "to control" and "to think" such as is implied by
context, I start with ridding myself of any illusion that my manner of
"thinking about" technical topics could be free from any "control" by the
behavior of other people.

 

[Bud Rogers]

The story goes that in ancient Greece the sophists were recognized
experts on public speaking, who took pride in being able to sway their
audience into any point of view they liked, or got paid for. (Much
like todays "spin doctors", I believe.) This in contrast to the
philosophers, who were the experts on what is True, regardless of
popularity or any pragmatics.

So sophistry usually means something like a lot of fancy talk that
isn't very helpful in establishing what is true, or important, or
right, or bestest of parens and white-space.

In Zen and the Art of Motorcycle Maintenance, Robert Pirsig discusses the
philosopher vs sophist debate and to some extent resurrects the honor of
the sophists. According to Pirsig, the sophists thought the most important
question was 'What is Good?'. The philosophers thought the more important
question was 'What is Truth?'. The philosophers seem to have won the
debate, at least in the West. And as they say, the victors write the
history books, so sophistry came to mean fancy arguments intended to evade
or obscure the truth.

For good or ill, the history of western civilization has been driven in
large part by that outcome. In almost every field of human endeavor, we
have brilliant technical solutions to almost every problem. All too often,
the solution only creates another problem. A modern day sophist (a real
one, not the snake oil salemen who have inherited the name) might say with
a certain smugness, "A clever man will do what makes sense, even if it
isn't any good. A wise man will do what is good, even if it doesn't make
any sense."

 

[Luke Gorrie]

Apparently, Paul Graham doesn't like CLOS nor the LOOP macro. Here is
another verson in Common Lisp (and this is not a dream language :

(defmethod mostn (fn (list (eql nil)))
(declare (ignore fn list))
(values nil nil))

(defmethod mostn (fn list)
(loop with result = (list (car list))
with max = (funcall fn (car list))
for object in (cdr list)
for score = (funcall fn object)
when (> score max) do (setq max score
result (list object))
when (= score max) do (push object result)
finally return (values (nreverse result) max)))

C'mon guys, live a little,

(defun mostn (fn list)
(let ((max (apply #'max (mapcar fn list))))
(values (remove max list :test-not #'= :key fn) max)))

-Luke

 

[james anderson]

Indentation in Lisp is not clear enough. Let's look at an example from
http://www-users.cs.umn.edu/~gini/aiprog/graham/onlisp.lisp:

(defun mostn (fn lst)
(if (null lst)
(values nil nil)
(let ((result (list (car lst)))
(max (funcall fn (car lst))))
(dolist (obj (cdr lst))
(let ((score (funcall fn obj)))
(cond ((> score max)
(setq max score
result (list obj)))
((= score max)
(push obj result)))))
(values (nreverse result) max))))

Note that only one pair of adjacent lines is indented by the same amount.
Other alignments are in the middle of lines.

Here is a straightforward translation into my dream language; note that
there aren't a lot of parens despite insignificant indentation and despite
using braces (like C) instead of bracketing keywords (like Pascal):

def mostn Fun [] = [], null;
def mostn Fun (First\List) {
var Result = [First];
var Max = Fun First;
each List ?Obj {
let Score = Fun Obj;
if Score > Max {Max = Score; Result = [Obj]}
if Score == Max {Result = Obj\Result}
};
reversed Result, Max
};

Apparently, Paul Graham doesn't like CLOS nor the LOOP macro. Here is
another verson in Common Lisp (and this is not a dream language :

(defmethod mostn (fn (list (eql nil)))
(declare (ignore fn list))
(values nil nil))

(defmethod mostn (fn list)
(loop with result = (list (car list))
with max = (funcall fn (car list))
for object in (cdr list)
for score = (funcall fn object)
when (> score max) do (setq max score
result (list object))
when (= score max) do (push object result)
finally return (values (nreverse result) max)))

i must admit, that i share this imputed aversion. you see, loop does something
counterproductive in the conventional formatting: it makes language constructs
which have different semantics and/or different scope appear similar. this is
something which m.kowalczyk, despite the thougtful exposition, fails to
understand. to have not been able to characterize the respective indentation
with any more precision than "in the middle of lines" indicates a deficiency
in reading comprehension.

there is a difference between an operator, an argument, a binding, an
antecedant, a consequent, etc. the conventional lisp indentation enables the
literate reader to assign these roles directly upon comprehending the operator
and - surprise, surprise - the indentation, and without having to comprehend
the content of the dependant form. this means that the reader need attend to
the dependant form only if that is their actual focus.

the form of the loop macro interfers with this.
as does the conventional formatting of the posted python example.

ps, if the cond is that simple, one might consider

....

 

[Marcin 'Qrczak' Kowalczyk]

C'mon guys, live a little,

(defun mostn (fn list)
(let ((max (apply #'max (mapcar fn list))))
(values (remove max list :test-not #'= :key fn) max)))

I can write it shorter that way in Python and in my language too, and it's
not even equivalent because it applies fn twice for each element, so my
point stands.

Yours would be a valid point if Lisp provided a significantly different
paradigm. But it was about the syntax of function calls, local variables,
loops, conditionals, variables etc. which are about as common in Lisp as
in those languages.

def mostn(fn, list):
Max = max(map(fn, list))
return [x for x in list if fn(x) == Max], Max

def mostn Fn List {
let Max = map List Fn->maximum;
select List ?X {Fn X == Max}, Max
};

 

[Terry Reedy]

It turned out that this only lasted for a couple of days. In fact,
once you get it, you realize that every single form in Lisp has
exactly the right number of parentheses and as long as you know the
arglist (tools like ILISP will help with that) it's a no-brainer to
know how many parens you'll need.

Last night I accepted the repeated suggestion to download and read
(some of) OnLisp. Problem: I *don't* know the arglist structure of
every form Graham uses, and since Graham assumes such knowledge, I had
to count parens to reverse engineer function prototypes and determine
what is an argument to what. A Python beginner would more easily do
this with Python code.

What is ILISP. Is there a usable online CL reference that would let
me lookup, for instance, DO and get the prototype and short
description?

I will admit that after a few chapters, the parens did begin to fade a
bit. They are a surface issue. I think some others are more
important.

As others have pointed out it is _essential_ to use Emacs or an Emacs
clone (like Hemlock or the editors coming with the commercial Lisps)
if you're going to write Lisp programs. Your editor should be able to
at least get the indentation right automatically, it should be able to
show matching parentheses,

Lisp-aware editors do not solve the problem of reading Lisp code
embedded in text, as with OnLisp formatted .ps or .pdf.

Terry J. Reedy

 

[Kenny Tilton]

Lisp-aware editors do not solve the problem of reading Lisp code
embedded in text, as with OnLisp formatted .ps or .pdf.

I agree. A big wadge of Lisp is not readable like NL text. And having
seen Graham's MOSTN posted here, wow, that is really verbose lisp. With
any Lisp, the indentation increment is just a few characters. I think
one can extend that in many editors, but then the structured nature of
functional code has one sailing off the right margin before too long.

As for reading code, I do not do that in books much, and when I do they
are pretty short examples. When they get bigger, right, I slow down and
destructure mentally. For example, in MOSTN I saw the earliest few lines
were just handling the edge case of a null input list. I moved a couple
of levels into the tree and forgot that outer check (which was
unnecessary!).

I was recommending Chapter's 1 & 8 for well-written thoughts on the
macro issue.

 

[Edi Weitz]

Last night I accepted the repeated suggestion to download and read
(some of) OnLisp. Problem: I *don't* know the arglist structure of
every form Graham uses, and since Graham assumes such knowledge, I
had to count parens to reverse engineer function prototypes and
determine what is an argument to what. A Python beginner would more
easily do this with Python code.

Well, "On Lisp" is not a book for beginners. It's a book about the
power of CL macros and was written for people who alread know Common
Lisp.

What is ILISP.

<http://www.cliki.net/ILISP>

Is there a usable online CL reference that would let me lookup, for
instance, DO and get the prototype and short description?

Yes, definitely. Common Lisp has an ANSI standard and the ANSI
standard is available online.[1]

<http://www.lispworks.com/reference/HyperSpec/Front/index.htm>

Be warned that this is a language standard, though. It's not a
textbook written for people who want to learn the language.

Tools like ILISP will allow you do put your cursor on a symbol and
open the corresponding page of the HyperSpec with one keystroke. Or

Lisp-aware editors do not solve the problem of reading Lisp code
embedded in text, as with OnLisp formatted .ps or .pdf.

Yes, but experience does.

Edi.

[1] Well, technically this is not the ANSI standard (you have to pay
money to ANSI to get it - it's a printed document) but for all
practical purposes it is.

 

[james anderson]

...
Lisp-aware editors do not solve the problem of reading Lisp code
embedded in text, as with OnLisp formatted .ps or .pdf.

well, to the extent that they indent consistently, they do. one trains ones
eye to ignore the parentheses. perhaps one may have misinterpretated the
resistance to tyrannical indentation. one very effective way to read lisp
attends to the indentation and ignores the parentheses. the compiler does not
do this, but the eye does. different eyes are trained differently, but the
principle remains.


for instance, if we take m.kowalczyk's example. a competent lisp programmer
could just as readily interpret the text below, as interpret the text with
parentheses. the parenthese are redundant information which aids
disambiguation - eg where numerous forms are on a single line, and corrects
errors, but fairly early on one no longer "reads" them.


defun mostn fn lst
if null lst
values nil nil
let result list car lst
max funcall fn car lst
dolist obj cdr lst
let score funcall fn obj
cond > score max
setq max score
result list obj
= score max
push obj result
values nreverse result max

practice. one suggestion would be to find an old mac and download a copy of
mcl. if you can get along with a mac, it really is the easiest of the lisps to
break in with.

....

 

[Wade Humeniuk]

C'mon guys, live a little,

(defun mostn (fn list)
(let ((max (apply #'max (mapcar fn list))))
(values (remove max list :test-not #'= :key fn) max)))

Yes, live a little, get messy. Why is there this insistence that
everything be neat and tidy? (in your dreams) In reality it never is,
that's the beauty of it.

(defun mostn (fn lst)
(let ((mostn-elems nil) (max nil))

(tagbody
find-mostn
(unless (null lst)

(let ((score (funcall fn (car lst))))
(cond
((or (null max) (> score max))
(setf max score mostn-elems (list (car lst))))
((= score max)
(push (car lst) mostn-elems)))

(setf lst (cdr lst))
(go find-mostn))))

(values (nreverse mostn-elems) max)))

Wade

 

[Paul Rubin]

(defun mostn (fn lst)
(let ((mostn-elems nil) (max nil))

(tagbody
find-mostn
(unless (null lst)

(let ((score (funcall fn (car lst))))
(cond
((or (null max) (> score max))
(setf max score mostn-elems (list (car lst))))
((= score max)
(push (car lst) mostn-elems)))

(setf lst (cdr lst))
(go find-mostn))))

(values (nreverse mostn-elems) max)))

This is untested but looks simpler than the lisp example:

def mostn (fn, lst):
max = None
for a in lst:
score = fn(lst)
if max is None or score > max:
max = score
ret = [a]
elif score == max:
ret.append(a)
return ret

 

[Wade Humeniuk]

Paul Rubin wrote:

This is untested but looks simpler than the lisp example:

def mostn (fn, lst):
max = None
for a in lst:
score = fn(lst)
if max is None or score > max:
max = score
ret = [a]
elif score == max:
ret.append(a)
return ret

That's the spirit, now

(defun mostn (fn lst &aux (mostn-elems nil) (max nil))
(tagbody
find-mostn
(unless (null lst)
((lambda (score)
(cond
((or (null max) (> score max))
(setf max score mostn-elems (list (car lst))))
((= score max)
(push (car lst) mostn-elems))))
(funcall fn (car lst)))
(setf lst (cdr lst))
(go find-mostn)))
(values (nreverse mostn-elems) max))

or

(defun mostn (fn lst &aux (mostn-elems nil) (max nil))
(map nil
(lambda (score elem)
(cond
((or (null max) (> score max))
(setf max score mostn-elems (list elem)))
((= score max)
(push elem mostn-elems))))
(mapcar fn lst)
lst)
(values (nreverse mostn-elems) max))

Wade

 

[Edi Weitz]

<http://www.cliki.net/ILISP>
<http://sourceforge.net/projects/ilisp/>

I should have mentioned that Bill Clementson gave a talk about using
Emacs as a Lisp IDE at ILC 2003 (alas, I wasn't there, but I saw a
draft of his paper) last week. It will be partly available from

<http://cl-cookbook.sf.net/emacs-ide.html>

in a couple of days.

Edi.

 

[Pascal Costanza]

Last night I accepted the repeated suggestion to download and read
(some of) OnLisp. Problem: I *don't* know the arglist structure of
every form Graham uses, and since Graham assumes such knowledge, I had
to count parens to reverse engineer function prototypes and determine
what is an argument to what. A Python beginner would more easily do
this with Python code.

Other people have already pointed you to some resources. I have also
liked the book "Common Lisp, The Language - 2nd edition" (CLtL2) by Guy
Steele a lot. It is a snapshot of the then ongoing ANSI standardization
from 1989. The HyperSpec is based on the final ANSI standard, as
published in 1995.

There are some minor differences between CLtL2 and the ANSI standard,
but they are neglectable, especially for a beginner. CLtL2 is generally
more accessible IMHO - better overall structure and more examples. It
feels a little bit like an advanced tutorial. When in doubt, refer to
the HyperSpec.

CLtL2 can be downloaded for free at
http://www-2.cs.cmu.edu/afs/cs.cmu.edu/project/ai-repository/ai/html/cltl/cltl2.html

I will admit that after a few chapters, the parens did begin to fade a
bit. They are a surface issue. I think some others are more
important.

Yes, exactly.

Lisp-aware editors do not solve the problem of reading Lisp code
embedded in text, as with OnLisp formatted .ps or .pdf.

That's right. A good exercise is to type in the code that you find in
the books by yourself. This should make you get a better feel for how to
read Lisp code.


Pascal

 

[Terry Reedy]

Is there a usable online CL reference that would let me lookup, for
instance, DO and get the prototype and short description?

Yes, definitely. Common Lisp has an ANSI standard and the ANSI
standard is available online.[1]

<http://www.lispworks.com/reference/HyperSpec/Front/index.htm>

Be warned that this is a language standard, though. It's not a
textbook written for people who want to learn the language.

Very helpful. I've read the first W&H Lisp on MacLisp, but CL has
lots of new name and syntax symbols, like &rest, :whatever, etc that I
can now look up as needed.

Thanks, Terry J. Reedy

 

[Russell Wallace]

Lisp offers a bit more power than Python (macros), but after a while
with Python, one notices that none of the power that is missing is
actually even needed. Python has just the right
features. Period. There are one or two warts (dangling variable after
list comprehensions comes to mind), but the language as a whole feels
natural. Lisp feels natural if you are writing a compiler or want to
be as orthogonal as possible, Python feels natural for most (all
except drivers/applications with extreme performance requirements) of
the real world applications.

Depends. For a lot of things I agree with you, writing Python code is
very natural and straightforward. (I've decided my aesthetic dislike
of Python is such that I'd never be happy writing anything other than
throwaway scripts in it, but that's a different matter; aesthetics is
in the eye of the beholder.)

However, this really applies only when you're doing things the
language designer anticipated. The great thing I find about Lisp is
that when I'm doing something the language was _not_ designed to
handle, it's not an "oh shit" problem, it's a "well, I suppose I can
do this with a few macros" problem.

 

[Alex Martelli]

Paul Rubin wrote:
...

This is untested but looks simpler than the lisp example:

def mostn (fn, lst):
max = None
for a in lst:
score = fn(lst)

Perhaps "score = fn(a)" ?

if max is None or score > max:
max = score
ret = [a]
elif score == max:
ret.append(a)
return ret

it it all right to raise an exception if lst is empty?
I'm not sure of the lisp example's behavior in that case,
but I suspect it returns an empty list and None. In
that case, we may need to initialize ret to [] too.

Stylistically (but it's just me) I don't much like that
"first-time switch" use -- null in lisp, None in Python.
An alternative might be to treat the empty lst as
an exceptional case:

def mostfn(fn, lst):
try: a = lst[0]
except IndexError: return [], None
ret, max = [], fn(a)
for a in lst:
score = fn(a)
if score > max:
max = score
ret = [a]
elif score == max:
ret.append(a)
return ret, max

a _bit_ more complicated than Paul's code, and calls
fn twice on lst[0], but it caters to my dislike for
"first-time switches"...;-). One could remove the
double-call problem by changing the start to:

def mostfn(fn, lst):
it = iter(lst)
try: a = it,next()
except StopIteration: return [], None
ret, max = [a], fn(a)
for a in it:
...

If taking O(N) auxiliary memory was acceptable, of
course, there would be an alternate possibility:

def mostfn(fn, lst):
if not lst: return [], None
aux = [ (fn(a), a) for a in lst ]
Max = max([ f for f, a in aux ])
return [ a for f, a in aux if f==Max ], Max

Typical problem with all high-level languages --
they often make most attractive an approach that
may not be quite as spare of bytes and cycles as
some lower-level one!-) [A simple test with
fn=lambda x:x%333 and lst=range(9999) shows the
lower-level approach is over 3 times faster,
both with and without psyco.proxy on both].


Alex

 

[Adam Warner]

Hi Marcin 'Qrczak' Kowalczyk,

What about reading (in books, on WWW)? I guess you would say
"indentation"...

Indentation in Lisp is not clear enough. Let's look at an example from
http://www-users.cs.umn.edu/~gini/aiprog/graham/onlisp.lisp:

(defun mostn (fn lst)
(if (null lst)
(values nil nil)
(let ((result (list (car lst)))
(max (funcall fn (car lst))))
(dolist (obj (cdr lst))
(let ((score (funcall fn obj)))
(cond ((> score max)
(setq max score
result (list obj)))
((= score max)
(push obj result)))))
(values (nreverse result) max))))

Note that only one pair of adjacent lines is indented by the same amount.
Other alignments are in the middle of lines.

I have ignored the followup-to because your comment is ignorant. If you
want to make ignorant comments please only make them in comp.lang.lisp so
corrections don't also have to be posted to comp.lang.python.

Here is what a Common Lisp users see from the indentation:

1. IF takes a "test" form, a "then" form and in this case an "else"
form. By the indentation I can instantly see that the "test" form is
(null lst), the "then" form is (values nil nil) and the "else" form is:

(let ((result (list (car lst)))
(max (funcall fn (car lst))))
(dolist (obj (cdr lst))
(let ((score (funcall fn obj)))
(cond ((> score max)
(setq max score
result (list obj)))
((= score max)
(push obj result)))))
(values (nreverse result) max))))

2. LET is binding values to RESULT and MAX. By the indentation I can
instantly see that this binding is in effect for the remainder of the form:

(dolist (obj (cdr lst))
(let ((score (funcall fn obj)))
(cond ((> score max)
(setq max score
result (list obj)))
((= score max)
(push obj result)))))
(values (nreverse result) max))))

3. By the indentation I can also instantly see that the DOLIST is
only operating over:

(let ((score (funcall fn obj)))
(cond ((> score max)
(setq max score
result (list obj)))
((= score max)
(push obj result)))))

4. I can instantly see that SCORE is bound only within this form:

(cond ((> score max)
(setq max score
result (list obj)))
((= score max)
(push obj result)))))

5. And finally I can instantly see that there are two test conditions, the
first being (> score max) and the second (= score max). Again by the
indentation I can see the return results grouped with the test conditions.

In pulling apart these forms there are still dangling parentheses
remaining on the page. That's the point. You can read the semantics
through knowledge of the special forms and the indentation alone. I didn't
attempt to manually delete the parentheses without the assistance
of an appropriate editor because I would probably screw something up. But
it did not stop me from being able to read the code.

Regards,
Adam

 

[Ville Vainio]

However, this really applies only when you're doing things the
language designer anticipated. The great thing I find about Lisp is

When I'm really doing something the language designer didn't
anticipate (and the required features don't already exist), I'm
probably doing something wrong and need to revise my design
anyway. You can do a whole lot in Python, esp. as you can code parts
in C if you really need that.

that when I'm doing something the language was _not_ designed to
handle, it's not an "oh shit" problem, it's a "well, I suppose I can
do this with a few macros" problem.

One can always go to c.l.py and ask, only to see that the problem is
easily solved in plain old Python. I think the benefits of Python far
outweigh the theoretical scenario that I actually needed some feature
that isn't there, and could be done in Lisp. Most often the missing
features involve not having a library to, say, access some database,
and I doubt Lisp has more than Python to offer in that area.

Meanwhile, people are voting with their feet: a lot (thousands? don't
know the exact figure) of people are taught Lisp (well, Scheme, but
anyway) at colleges/whatever every year, and they abandon it in a
blink of an eye after the course (obviously this might be because the
courses emphasize functional programming). Many even think that C++,
of all things, is easier!