Python syntax in Lisp and Scheme

[Doug Tolton]

How do you "screw up indentation"? Actually, being more of a
Python fan, I though you had screwed up parens, since indentation
is absolute for me . Anyway, do you just go to some line and start
pressing spacebar randomly, and not notice this? Maybe
you do need a safety net, then.

Maybe by coding in an environment with multiple programmers?
Alex was saying that Python's syntax and simplicity is so much better
for large scale projects with multiple programmers. With just 4
programmers using Python at work we have problems with indentation
levels. One guy (me) uses emacs, another uses ultra-edit, another
uses Boa and the other uses PythonWin. Indenting can be a major
source of compilation bugs.

But the whitespace already has all the structural information needed.
The whitespace doesn't just disappear suddenly, much like parens don't
magically disappear or mutate.

There can be problems in cutting and pasting code if one mixes tabs
and spaces, like Ingvar said, but generally the rule in Python is to
use 4 spaces for each indentation level. If you break this rule,
you know you had it coming when something fails .

Just because it's defined that way, doesn't make it less a pain in the
ass to fix the problems.

of course it's not a big problem if you are the only consumer of your
code.


Doug Tolton
(format t "~a@~a~a.~a" "dtolton" "ya" "hoo" "com")

 

[Eli Barzilay]

Three words and a hyphen: Higher-Order Functions.

This covers most usages of macros for control structures, but not
all. (For example, goto.)

But there is an additional word to add, no hyphens: Bindings.

This is false. Writing your own macro expander is not necessary for
getting the effect. The only thing that macros give you in this
regard is the ability to hide the lambda-suspensions. To some
people this is more of a disadvantage than an advantage because,
when not done in a very carefully controlled manner, it ends up
obscuring the logic of the code. (Yes, yes, yes, now someone will
jump in an tell me that it can make code less obscure by "canning"
certain common idioms. True, but only when not overdone.)

(This hits one of the major differences between Lisp and Scheme -- in
Lisp I'm not as happy to use HOFs because of the different syntax
(which is an indication of a different mindset, which leads to
performance being optimized for a certain style). Scheme is much more
functional in this respect, for example -- using HOF versions of
with-... compared to Lisp where these are always macros.)

 

[james anderson]

...

(This hits one of the major differences between Lisp and Scheme -- in
Lisp I'm not as happy to use HOFs because of the different syntax

which difference differnt syntax?

(which is an indication of a different mindset, which leads to
performance being optimized for a certain style). Scheme is much more
functional in this respect, for example -- using HOF versions of
with-... compared to Lisp where these are always macros.)

in practice, as a rule, a with- is available at least optionally also as a
call-with-. not just for convenience, but also for maintainability.

in the standard there are but twelve of these. should they be an impediment,
in most cases there is nothing preventing one from writing call-with-...
equivalents. many expand fairly directly to special forms, so they are even
trivial to reimplement if the "double-nesting" were distasteful. i am curious,
however, about the HOF equivalents for macros which expand primarily to
changes to the lexical environment. not everything has a ready equivalence to
lambda. for instance with-slots and with-accessors. what is the HOF equivalent
for something like

? (defclass c1 () ((s1 )))

#<STANDARD-CLASS C1>
? (defmethod c1-s1 ((instance c1))
(with-slots (s1) instance
(if (slot-boundp instance 's1)
s1
(setf s1 (get-universal-time)))))

#<STANDARD-METHOD C1-S1 (C1)>
? (defparameter *c1* (make-instance 'c1))

*C1*
? (describe *c1*)
#<C1 #x69D95C6>
Class: #<STANDARD-CLASS C1>
Wrapper: #<CCL::CLASS-WRAPPER C1 #x69D95A6>
Instance slots
S1: #<Unbound>
? (c1-s1 *c1*)
3274553173
? (describe *c1*)
#<C1 #x69D95C6>
Class: #<STANDARD-CLASS C1>
Wrapper: #<CCL::CLASS-WRAPPER C1 #x69D95A6>
Instance slots
S1: 3274553173
?

 

[Eli Barzilay]

(This hits one of the major differences between Lisp and Scheme --
in Lisp I'm not as happy to use HOFs because of the different
syntax

which difference differnt syntax?
Huh?

(which is an indication of a different mindset, which leads to
performance being optimized for a certain style). Scheme is much more
functional in this respect, for example -- using HOF versions of
with-... compared to Lisp where these are always macros.)

in practice, as a rule, a with- is available at least optionally also as a
call-with-. not just for convenience, but also for maintainability.
[...]

Yes, but I was talking about the difference approaches, for example:

(dolist (x foo)
(bar x))

vs:

(mapc #'bar foo)

i am curious, however, about the HOF equivalents for macros which
expand primarily to changes to the lexical environment. [...]

That was the point I made in the beginning.

 

[james anderson]

(This hits one of the major differences between Lisp and Scheme --
in Lisp I'm not as happy to use HOFs because of the different
syntax

which different [] syntax?

Huh?

that is, what is different about the syntax for higher-order functions in lisp?

(which is an indication of a different mindset, which leads to
performance being optimized for a certain style). Scheme is much more
functional in this respect, for example -- using HOF versions of
with-... compared to Lisp where these are always macros.)

in practice, as a rule, a with- is available at least optionally also as a
call-with-. not just for convenience, but also for maintainability.
[...]

Yes, but I was talking about the difference approaches, for example:

(dolist (x foo)
(bar x))

vs:

(mapc #'bar foo)

are these not two examples of coding in common-lisp. how do they demonstrate
that "scheme is much more functional"?

i am curious, however, about the HOF equivalents for macros which
expand primarily to changes to the lexical environment. [...]

That was the point I made in the beginning.

sorry, i missed that.

 

[Peter Seibel]

Using parentheses and rpn everywhere makes lisp very easy to parse,
but I'd rather have something easy for me to understand and hard for
the computer to parse.

That would be a strong argument--seriously--if the only folks who
benefited from the trivial mapping between Lisp's surface syntax and
underlying were the compiler writers. I certainly agree that if by
expending some extra effort once compiler writers can save their users
effort every time they write a program that is a good trade off.

If the only thing a "regular" programmer ever does with a language's
syntax is read and write it, then the only balance to be struck is
between the perhaps conflicting goals of readability and writability.
(For instance more concise code may be more "writable" but taken to an
extreme it may be "write only".) But "machine parsability" beyond,
perhaps, being amennable to normal machine parsing techniques (LL,
LALR, etc.) should not be a consideration. So I agree with you.

But Lisp's syntax is not the way it is to make the compiler writer's
job easier. In Lisp "regular" programmers also interact with the code
as data. We can easily write code generators (i.e. macros) that are
handed a data representation of some code, or parts of code, and have
only to return a new data structure representing the generated code.
This is such a useful technique that it's built into the compiler--it
will run our code generators when it compiles our code so we don't
have to screw around figuring out how to generate code at runtime and
get it loaded into our program. *That's* why we don't mind, and, in
fact, actively like, Lisp's syntax.

The point is not that the syntax, taking in total isolation from the
rest of the language, is necessarily the best of all possible syntaxi.
The point is that the syntax makes other things possible that *way*
outweigh whatever negatives the syntax may have.

I'd humbly suggest that if you can't see *any* reason why someone
would prefer Lisp's syntax, then you're not missing some fact about
the syntax itself but about how other language features are supported
by the syntax.

-Peter

 

[james anderson]

Matthias Blume wrote:

is the no advantage to being able to do either - or both - as the
occasion dictates?

I can't parse this sentence,
sorry:

is the[re] no advantage to being able to do either - or both - as the
occasion dictates?

but of course you can also use HOFs in Lisp
(all flavours). The interesting part is that most Lisp'ers don't seem
to use them, or even to know that you can use them, and use macros instead.

while the first assertion might well be born out by a statistical analysis of,
for example, open-source code, i'm curious how one reaches the second conclusion.

The only real advantage of macros over HOFs is that macros are guaranteed
to to executed at compile time. A good optimizing compiler (like GHC
for Haskell) might actually also evaluate some expressions including
HOFs at compile time, but you have no control over that.


HOFs can of course be used directly in CL, and you can use macros to
do everything one could use HOFs for (if you really want).

that's a disappointment.

The advantage of HOFs over macros is simplicity: You don't need additional
language constructs

when did common-lisp macros become an "additional language construct"?

(which may be different even for different Lisp
dialects, say), and other tools (like type checking) are available for
free; and the programmer doesn't need to learn an additional concept.

doesn't that last phrase contradict the previous one?

i do admit to infrequent direct use of higher-order functions. one reason is
that there is little advantage to articulating the creation of functions which
have dynamic extent only, so in my use, most hof's are manifest through a
macro interface. it's the same distaste i have about inner and anonymous java
classes. the other reason is that when i moved from scheme to lisp, in the
process of porting the code which i carried over, it occurred to me that much
of what i was using higher-order functions for could be expressed more clearly
with abstract classes and appropriately defined generic function method combinations.

....

 

[Pascal Costanza]

I was never very fond of lisp. I guess I mean scheme technically, I
took the Ableson and Sussman course back in college, so that's what I
learned of scheme, lisp in general I've mostly used embedded in other
things. In general, it always seemed to me that a lot of the design
choices in lisp are driven more by elegance and simplicity than
usability.

You should give Common Lisp a try. Many Lispers think that that's
exactly one of the advantages of Common Lisp over Scheme: it focuses
more on usability than on elegance.

Of course, mileages vary, but you shouldn't draw conclusions about
Common Lisp from your experience with Scheme, and vice versa. Common
Lisp and Scheme are as similar as, say, C++ and Pascal.


Pascal

 

[Eli Barzilay]

Eli Barzilay wrote:

(This hits one of the major differences between Lisp and
Scheme -- in Lisp I'm not as happy to use HOFs because of the
different syntax

which different [] syntax?

Huh?

that is, what is different about the syntax for higher-order
functions in lisp?

funcall, function, #', the double namespace.

are these not two examples of coding in common-lisp. how do they
demonstrate that "scheme is much more functional"?

The first is very popular, the second is hardly known. R5RS has
`for-each' which is exactly like `mapc', but no `dolist' equivalent.
In Scheme, this is not a problem, in Lisp, the syntax makes me worry
for the extra effort in creating a closure.

 

[Pascal Bourguignon]

When you have macros such as loop that allow you to write stuff like:

(loop for color in '(blue white red)

[...]

Well, some people say the "loop" syntax is not very lispish - it's unusual
that it uses many words and few parentheses. It still uses only words and
parentheses, no other punctuation, and it introduces one pair of parentheses
for its one nesting level.

Yes. The point is that the language is rather agnostic about any
topic, even about the syntax. Personnaly I don't like much LOOP, but
I take it as an example by the language designers showing us that it's
even possioble to avoid parethensis if you don't want them.

A richer alphabet is often more readable. Morse code can't be read as fast
as Latin alphabet because it uses too few different symbols. Japanese say
they won't abandon Kanji because it's more readable as soon as you know it -
you don't have to compose words from many small pieces which look alike
but each word is distinct. Of course *too* large alphabet requires long
learning and has technical difficulties, but Lisp expressions are too
little distinctive for my taste.

Well, I would say that kanji is badly designed, compared to latin
alphabet. The voyels are composed with consones (with diacritical
marks) and consones are written following four or five groups with
additional diacritical marks to distinguish within the groups. It's
more a phonetic code than a true alphabet.

I know I can implement infix operators with Lisp macros, but I even don't
know how they feel because nobody uses them (do I have to explicitly open
infix region and explicitly escape from it to regular syntax?), and
arithmetic is not enough.

Most probably, you would write a macro named WITH-INFIX and thus
automatically scope the infix part:

(with-infix 1 / x + 1 / ( x ^ 3 ) + 1 / ( x ^ 5 ) )

and if you write it well:

(with-infix
if a = b then format t "equal ~D~%" a ;
else format t "diff ~D /= ~D~%" a b ; endif ;
for i = 1 to 10 ; print i ; next i )

All Lisp code I've read uses lots of parentheses
and they pile up at the end of each large subexpression so it's hard to
match them (an editor is not enough, it won't follow my eyes and won't
work with printed code).

Syntax is the thing I like the least in Lisp & Scheme.

I'll tell you the secret: yes there are alot of parethesis. This is a
price all lispers pay for greater benefits. It gives us such
advantages that we gladly pay the price.

Otherwise, I would say:

1- don't pay so much attention to the parenthesis!

2- if you have to and it's hard, then it means the code is badly
structured (probably too big a function). Rewrite it, factorize.

3- what do you mean "printed"? A double-click on any parenthesis
selects the enclosed list so it's quite easy to see what it encloses.

 

[prunesquallor]

Well, I supposed this thread has spiralled out of control already anyway


I really don't understand why this is a problem, since its trivial to
transform python's 'globally context' dependent indentation block structure
markup into into C/Pascal-style delimiter pair block structure markup.

Of course it can. Any unambiguous grammar has a parse tree.

Significantly, AFAICT you can easily do this unambiguously and *locally*, for
example your editor can trivially perform this operation on cutting a piece of
python code and its inverse on pasting (so that you only cut-and-paste the
'local' indentation). Prima facie I don't see how you loose any fine control.

Only if your cut boundaries are at the same lexical level. If you cut
across boundaries, it is no longer clear what should happen at the paste.

Also, it is frequently the case that you need to `tweak' the code after
you paste it.

Sorry, I don't understand this sentence, but maybe you mean that the potential
inconsitency between human and machine interpretation is a *feature* for Lisp,
C, Pascal etc!? If so I'm really puzzled.

You misunderstand me. In a python block, two expressions are
associated with each other if they are the same distance from the left
edge. This is isomorphic to having a nametag identifying the scope
of the line. Lines are associated with each other iff they have the
same nametag. Change one, and all must change.

If, instead, you use balanced delimiters, then a subexpression no
longer has to encode its position within the containing expression.

Let me demonstrate the isomorphism. A simple python expression:
(grrr.. I cut and paste it, but it lost its indentation between
the PDF file and Emacs. I hope I redo it right...)

def index(directory):
# like os.listdir, but traverses directory trees
stack = [directory]
files = []
while stack:
directory = stack.pop()
for file in os.listdir(directory):
fullname = os.path.join(directory, file)
files.append(fullname)
if os.path.isdir(fullname) and not os.path.islink(fullname):
stack.append(fullname)
return files

Now the reason we know that ` files.append(fullname)' and
` fullname = os.path.join(directory, file)' are part of the
same block is because they both begin with 12 spaces. The first
four spaces encode the fact that they belong to the same function,
the next four indicate that they belong in the while loop, and
the final four indicate that they belong in the for loop.
The ` return files', on the other hand, only has four spaces, so
it cannot be part of the while or for loop, but it is still part
of the function. I can represent this same information as a code:

t -def index(directory):
d - # like os.listdir, but traverses directory trees
d - stack = [directory]
d - files = []
d - while stack:
dw - directory = stack.pop()
dw - for file in os.listdir(directory):
dwf - fullname = os.path.join(directory, file)
dwf - files.append(fullname)
dwf - if os.path.isdir(fullname) and not os.path.islink(fullname):
dwfi- stack.append(fullname)
d - return files

The letter in front indicates what lexical group the line belongs to. This
is simply a different visual format for the leading spaces.

Now, suppose that I wish to protect the body of the while statement
within a conditional. Simply adding the conditional won't work:

d - while stack:
dw - if copacetic():
dw - directory = stack.pop()
dw - for file in os.listdir(directory):
dwf - fullname = os.path.join(directory, file)
dwf - files.append(fullname)
dwf - if os.path.isdir(fullname) and not os.path.islink(fullname):
dwfi- stack.append(fullname)

because the grouping information is replicated on each line, I have to
fix this information in the six different places it is encoded:

d - while stack:
dw - if copacetic():
dwi - directory = stack.pop()
dwi - for file in os.listdir(directory):
dwif - fullname = os.path.join(directory, file)
dwif - files.append(fullname)
dwif - if os.path.isdir(fullname) and not os.path.islink(fullname):
dwifi- stack.append(fullname)

The fact that the information is replicated, and that there is nothing
but programmer discipline keeping it consistent is a source of errors.

I don't understand why this is any different to e.g. ')))))' in Lisp. The
closing ')' for DEFUN just looks the same as that for IF.

That is because the parenthesis *only* encode the grouping information,
they do not do double duty and encode what they are grouping. The key
here is to realize that the words `DEFUN' and the `IF' themselves look
very different.

I don't understand what you mean. Could you maybe give a concrete example of
the information that can't be displayed?

Sure. Here are five parens ))))) How much whitespace is there here:

Still, I'm sure you're familiar with the following quote (with which I most
heartily agree):

"[P]rograms must be written for people to read, and only incidentally for
machines to execute."

People can't "read" '))))))))'.

Funny, the people you just quoted would disagree with you about parenthesis.
I expect that they would disagree with you about whitespace as well.

 

[james anderson]

...


The first is very popular, the second is hardly known.

somehow i wonder if we're discussing the same language.

R5RS has
`for-each' which is exactly like `mapc', but no `dolist' equivalent.
In Scheme, this is not a problem, in Lisp, the syntax makes me worry
for the extra effort in creating a closure.

what me worry? about syntax?

? (defmacro Barzilay (type operator &rest lists)
`(map ',type (function ,operator) ,@lists))
BARZILAY
? (BARZILAY vector evenp '(1 2 3 4))
#(NIL T NIL T)
? (defmacro Barzilac (operator &rest lists)
`(mapc (function ,operator) ,@lists))
BARZILAC
? (Barzilac (lambda (x) (print x)) '(1 2 3 4))

1
2
3
4
(1 2 3 4)
? (defmacro Barzilar (operator &rest lists)
`(mapcar (function ,operator) ,@lists))
BARZILAR
? (Barzilar + '(1 2 3 4) '(5 6 7 8))
(6 8 10 12)
?

....

 

[Lulu of the Lotus-Eaters]

| 3- what do you mean "printed"? A double-click on any parenthesis
| selects the enclosed list so it's quite easy to see what it encloses.

I kept clicking on the parenthesis in all your samples. All my
newsreader did was go to text-select mode!

Btw. I believe the word "printed" means "printed"... you've seen that
stuff where ink is applied to paper, yes?

Yours, Lulu...

 

[prunesquallor]

when did common-lisp macros become an "additional language construct"?

That's what macros do: they add new language constructs.

I think that many Scheme students inadvertantly get taught `macros = evil'.

the other reason is that when i moved from scheme to lisp, in the
process of porting the code which i carried over, it occurred to me that much
of what i was using higher-order functions for could be expressed more clearly
with abstract classes and appropriately defined generic function method combinations.

I also think that many Scheme students are mislead and inadvertantly
taught that one should avoid everything but LAMBDA.

 

[prunesquallor]

You should give Common Lisp a try. Many Lispers think that that's
exactly one of the advantages of Common Lisp over Scheme: it focuses
more on usability than on elegance.

S&ICP (Abelson and Sussman) use Scheme to illustrate basic concepts by
stripping away things to get at the core of what's going on. But I
think too many people assume that the stripped-down version is
supposed to be `better' in some sense. (Other than the pedagogic
sense, that is.)

It is like learning about internal combustion engines by disassembling
a lawnmower. It is a very simple illustration of everything important
about 4-stroke engines. However, no one would suggest you actually
build a car that way!

 

[Tim Hochberg]

On Tue, 07 Oct 2003 21:59:11 +0200, Pascal Bourguignon wrote: [SNIP]
A richer alphabet is often more readable. Morse code can't be read as fast
as Latin alphabet because it uses too few different symbols. Japanese say
they won't abandon Kanji because it's more readable as soon as you know it -
you don't have to compose words from many small pieces which look alike
but each word is distinct. Of course *too* large alphabet requires long
learning and has technical difficulties, but Lisp expressions are too
little distinctive for my taste.

Well, I would say that kanji is badly designed, compared to latin
alphabet. The voyels are composed with consones (with diacritical
marks) and consones are written following four or five groups with
additional diacritical marks to distinguish within the groups. It's
more a phonetic code than a true alphabet.

[SNIP]

Admittedly, I found the above paragraph pretty hard to parse and my
never stellar knowledge of Japanees has mostly evaporated over time, but
I'm pretty sure you are talking about Hiragana (or Katakana), not Kanji.
Japaneese has three alphabets, which they mix and match in ordinary
writing. Kanji aren't phonetic at all, they're ideograms, and can
typically be read at least two completely different ways depending on
the context, making reading Japanese extra confusing for the non fluent.

A random web search supplies this basic descripion of Hiragana, Katakana
and Kanji:

http://www.kanjisite.com/html/wak/wak1.html

-tim

 

[Eli Barzilay]

somehow i wonder if we're discussing the same language.

These are both Lisp examples. The first is much more popular than the
second. Scheme has an equivalent for the second, not for the first.

Conclusion: the first one is stylistically preferred in Lisp, the
(equivalent of the) second is stylistically preferred in Scheme.

R5RS has `for-each' which is exactly like `mapc', but no `dolist'
equivalent. In Scheme, this is not a problem, in Lisp, the syntax
makes me worry for the extra effort in creating a closure.

what me worry? about syntax?
[...]

You completely missed my point.

 

[Hartmann Schaffer]

You find delimited words more difficult than symbols? For literate
people who use alphabet-based languages, I find this highly suspect.
Maybe readers of only ideogram languages might have different
preferences, but we are writing in English here...

well, there are a few occasions where symbols are preferrable. just
imagine mathematics with words only

hs

 

[prunesquallor]

Most probably, you would write a macro named WITH-INFIX and thus
automatically scope the infix part:

(with-infix 1 / x + 1 / ( x ^ 3 ) + 1 / ( x ^ 5 ) )

and if you write it well:

(with-infix
if a = b then format t "equal ~D~%" a ;
else format t "diff ~D /= ~D~%" a b ; endif ;
for i = 1 to 10 ; print i ; next i )

CGOL is an Algol-like syntax for MacLisp that was designed by Vaughn
Pratt. I believe it has been ported to CL.

ABSTRACT

MACLISP programmers who feel comfortable with ALGOL-like
notation, that is, an algebraic style in which one might write a
matrix multiply routine as

for i in 1 to n do
for k in 1 to n do
(ac := 0;
for j in 1 to n do
ac := ac + a(i,j)*b(j,k);
c(i,k) := ac)

can now write LISP programs in just such a notation. This notation is
essentially transparent to the MACLISP system, and files containing
CGOL code (possibly mixed in with standard code) can be read by the
interpreter and compiled by the compiler just as though they were
written in straight LISP notation.


It has never caught on, though.

I'll tell you the secret: yes there are alot of parethesis. This is a
price all lispers pay for greater benefits. It gives us such
advantages that we gladly pay the price.

Excerpts from real C header files:

#define FromHex(n) (((n) >= 'A') ? ((n) + 10 - 'A') : ((n) - '0'))
#define StreamFromFOURCC(fcc) ((WORD) ((FromHex(LOBYTE(LOWORD(fcc))) << 4) + (FromHex(HIBYTE(LOWORD(fcc))))))
#define ToHex(n) ((BYTE) (((n) > 9) ? ((n) - 10 + 'A') : ((n) + '0')))
#define MAKEAVICKID(tcc,stream) MAKELONG((ToHex((stream) & 0x0f) << 8) | (ToHex(((stream) & 0xf0) >> 4)),tcc)

#define va_arg(AP, TYPE) \
(AP = (__gnuc_va_list) ((char *) (AP) + __va_rounded_size (TYPE)), \
*((TYPE *) (void *) ((char *) (AP) \
- ((sizeof (TYPE) < __va_rounded_size (char) \
? sizeof (TYPE) : __va_rounded_size (TYPE))))))

#define PLAUSIBLE_BLOCK_START_P(addr, offset) \
((*((format_word *) \
(((char *) (addr)) + ((offset) - (sizeof (format_word)))))) == \
((BYTE_OFFSET_TO_OFFSET_WORD(offset))))

 

[james anderson]

Eli Barzilay wrote:
...

R5RS has `for-each' which is exactly like `mapc', but no `dolist'
equivalent. In Scheme, this is not a problem, in Lisp, the syntax
makes me worry for the extra effort in creating a closure.

what me worry? about syntax?
[...]

You completely missed my point.

if the point was about something other than when and if a closure is created
and whether that differentiates lisp from scheme, then yes, i did.

....