S
Steve Williams
Alex Martelli wrote:
[snip]
finally, a breath of fresh air.
[snip]
finally, a breath of fresh air.
C
Coby Beck
Andrew Dalke said:
Now you're obviously just trying to be difficult! Reminds me of a thread a
while back where someone argued that C could so dynamically compile code and
posted an example that wrote a hello-world in a string, wrote it to a file,
called gcc on it and then system-called the .so file!
Or is there something else you mean besides design the language, implement a
parser, write a compiler? Yes you can do that in C.
I don't get your point at all. At least not how it possibly applies to a
discussion of using macros to create domain specific languages.
This is just the same memorizing that applies to application specific
functions, classes etc. Not at all like memorizing @ % $ ; { } & etc.
Really, this argument is based on pure FUD. It is not the norm to use
macros to do anything at all obfuscating. One of the most controversial
macros has got to be loop, and it is ridiculed for something close to what
you seem to be arguing: it creates a very different and unlisp-like
sublanguage.
But even that, it is not:
(loop ^x %% foo==bar -> baz)
it is:
(loop for x from foo upto bar finally return baz)
Not hard to memorize what FROM and UPTO mean.
My head prefers to be full of more important things, that's all.
Just have another look. He did not even say behaviour is the most
important, let alone the only important thing.
But in the global trade of language design (because remember, everything is
a trade-off) behaviour *is* more important than syntax *if* your goals are
primarily technical rather than social.
Nothing wrong with social goals, but you should not be naive when it comes
to considering what you have traded in for personal notions of ease of
learning.
I don't think this is a reasonable discussion point. You are presumably
trying to show us an example of a problem lisp is not flexible enough to
handle, but you have not presented a problem to solve, you have presented a
solution to implement. These are entirely different things.
C
Coby Beck
Andrew Dalke said:
Nothing surprising!
T
Terry Reedy
Pascal Costanza said:
Module level functions are local to the module unless imported by
another module. Nested functions are local to the function they are
nested within unless explicitly returned. Methods are local to
classes and subclasses. Lambda expressions are very local unless
somehow passed around.
I am not sure which best meets your intention.
Yes, named objects, including functions can (locally) shadow
(override) builtins. It is considered a bad habit/practice unless
done intentionally with a functional reason.
Terry J. Reedy
A
Andrew Dalke
Pascal Costanza:
While an interesting topic, it's something I'm not going to worry about.
And if I did, it would be in Python
I bring it up as a counter-example to the idea that all modes of
programming have been and can be explored in a current Lisp.
I conjectured one interesting possibility -- that of handling ensembles
of possible solutions to a given problem.
In retrospect I should have given a more obvious possibility.
As some point I hope to have computer systems I can program
by voice in English, as in "House? Could you wake me up
at 7?" That is definitely a type of programming, but Lisp is
a language designed for text, not speed.
Pascal Costanza:
My spreadsheet program looks different from my word processor
looks different from my chemical structure editor looks different from
my biosequence display program looks different from my image
editor looks different from my MP3 player looks different from my
email reader looks different from Return to Castle Wolfinstein ....
There are a few bits of commonality; they can all open files. But
not much more. Toss out the MP3 player and RtCW and there
is more in common. Still, the phrase "practicality beats purity" is
seems appropriate here.
Do you have any doubt to my answer?
Personally, I would love to write equations on a screen like I
would on paper, with integral signs, radicals, powers, etc. and
not have to change my notation to meet the limitations of computer
input systems.
For Lisp is a language tuned to keyboard input and not the full
range of human expression. (As with speech.)
(I know, there are people who can write equations in TeX as
fast as they can on paper. But I'm talking about lazy ol' me
who wants the covenience.)
Or, will there ever be a computer/robot combination I can
teach to dance? Will I do so in Lisp?
Hmmm.. Is the number '1' an object? Is a function an object?
What about a module? A list? A class?
Where in that example are you aware that you are dealing with classes
and objects?
Which is why I didn't see the point of original statement. My
conjecture is that additional syntax can make some things easier.
That a problem can be solved without new syntax does not
contradict my conjecture.
Yes. Got a URL for documentation on a Lisp providing access
to shared memory? My guess is that the Lisp runtime needs
to be told about the arenas and that the multiple instances of
Lisp sharing the arena must use some extra IPC to handle
the distributed gc.
It gets worse if program X forks copies Y and Z, with shared
memory XY between X and Y (but not Z) and XZ between
X and Z (but not Y). X needs to be very careful on which
data is copied, and it isn't immediately obvious what happens
when some object from XZ is inserted into a list accessible
to Y via XY.
Consider also a "persistent memory" server running in C
(or hardware access to some sort of non-volatile memory.)
You can use standard IPC to get an initially zeroed memory
block and are free to use that memory without restrictions.
It's persistent after program exit so when the program restarts
it can reconnect to shared memory and get the data as it
was at exit.
This service is straight-forward to support in C/C++. It
sounds like for Lisp you are dependent on the implementation,
in that if the implementation doesn't support access to its
memory allocator/gc subsystem then it's very hard to
write code for this hardware on your own. It may be
possible to use an extension (written in C? to read/write
to that persistent memory using some sort of serialization,
but that's the best you can do -- you don't have live objects
running from nonvolatile store -- which is worse than C++.
Andrew
(e-mail address removed)
While an interesting topic, it's something I'm not going to worry about.
And if I did, it would be in Python
I bring it up as a counter-example to the idea that all modes of
programming have been and can be explored in a current Lisp.
I conjectured one interesting possibility -- that of handling ensembles
of possible solutions to a given problem.
In retrospect I should have given a more obvious possibility.
As some point I hope to have computer systems I can program
by voice in English, as in "House? Could you wake me up
at 7?" That is definitely a type of programming, but Lisp is
a language designed for text, not speed.
Pascal Costanza:
My spreadsheet program looks different from my word processor
looks different from my chemical structure editor looks different from
my biosequence display program looks different from my image
editor looks different from my MP3 player looks different from my
email reader looks different from Return to Castle Wolfinstein ....
There are a few bits of commonality; they can all open files. But
not much more. Toss out the MP3 player and RtCW and there
is more in common. Still, the phrase "practicality beats purity" is
seems appropriate here.
Do you have any doubt to my answer?
Personally, I would love to write equations on a screen like I
would on paper, with integral signs, radicals, powers, etc. and
not have to change my notation to meet the limitations of computer
input systems.
For Lisp is a language tuned to keyboard input and not the full
range of human expression. (As with speech.)
(I know, there are people who can write equations in TeX as
fast as they can on paper. But I'm talking about lazy ol' me
who wants the covenience.)
Or, will there ever be a computer/robot combination I can
teach to dance? Will I do so in Lisp?
Hmmm.. Is the number '1' an object? Is a function an object?
What about a module? A list? A class?
Where in that example are you aware that you are dealing with classes
and objects?
Which is why I didn't see the point of original statement. My
conjecture is that additional syntax can make some things easier.
That a problem can be solved without new syntax does not
contradict my conjecture.
If it's a good Lisp library I would expect it to work like this:
(with-allocation-from :shared-memory
...)
Any more questions?
Yes. Got a URL for documentation on a Lisp providing access
to shared memory? My guess is that the Lisp runtime needs
to be told about the arenas and that the multiple instances of
Lisp sharing the arena must use some extra IPC to handle
the distributed gc.
It gets worse if program X forks copies Y and Z, with shared
memory XY between X and Y (but not Z) and XZ between
X and Z (but not Y). X needs to be very careful on which
data is copied, and it isn't immediately obvious what happens
when some object from XZ is inserted into a list accessible
to Y via XY.
Consider also a "persistent memory" server running in C
(or hardware access to some sort of non-volatile memory.)
You can use standard IPC to get an initially zeroed memory
block and are free to use that memory without restrictions.
It's persistent after program exit so when the program restarts
it can reconnect to shared memory and get the data as it
was at exit.
This service is straight-forward to support in C/C++. It
sounds like for Lisp you are dependent on the implementation,
in that if the implementation doesn't support access to its
memory allocator/gc subsystem then it's very hard to
write code for this hardware on your own. It may be
possible to use an extension (written in C?
to read/writeto that persistent memory using some sort of serialization,
but that's the best you can do -- you don't have live objects
running from nonvolatile store -- which is worse than C++.
Andrew
(e-mail address removed)
A
Andrew Dalke
Coby Beck:
Hmm, I think you are correct. This discussion has worn me out
and I'm reacting now more out of crabbishness than thoughtfulness.
I hereby withdraw from this thread. Or at least from cross-posting
outside of c.l.py
Andrew
(e-mail address removed)
Hmm, I think you are correct. This discussion has worn me out
and I'm reacting now more out of crabbishness than thoughtfulness.
I hereby withdraw from this thread. Or at least from cross-posting
outside of c.l.py
Andrew
(e-mail address removed)
K
Kenny Tilton
Andrew said:
Ah, but there was a 75% chance that my remarks were not /completely/
random, so my unfairness towards you can't be complete bunkum.
T
Terry Reedy
Peter Seibel said:
But can't you explicitly quote the test cases for input to the HOF and
eval them within the HOF, so you again have both the literal code and
value generated? Not as pretty, admittedly, and perhaps less
efficient, but workable?
Terry J. Reedy
P
Pascal Costanza
Andrew said:
I don't understand that last sentence. Could you clarify this a bit? You
don't want to say that there is an inherent dichotomy between text and
speed, do you?!?
....but you probably know from the start where to find the menus, what
the shortcuts are for opening and saving files, how to find the online
help, and so forth.
Lisp also has this to a certain degree: It's always clear what
constitutes the meaning of an s-expression, namely its car, no matter
what language "paradigm" you are currently using.
Toss out the MP3 player and RtCW and there
is more in common. Still, the phrase "practicality beats purity" is
seems appropriate here.
Do you have any doubt to my answer?
No, not really.
I know people who have even started to use s-expression for mathematical
notation (on paper), because they find it more convenient.
There is some research going on to extend Lisp even in this regard
(incorporating more ways of expression).
Well, maybe I am wrong. However, in a recent example, a unit test
expressed in Python apparently needed to say something like
"self.assertEqual ...". Who is this "self", and what does it have to do
with testing?
If additional syntax makes specific things easier, then in god's name
just add it! The loop macro in Common Lisp is an example of how you can
add syntax to make certain things easier. This is not rocket science.
The point here is that for most languages, if you want to add some
syntax, you have to change the definition of the language, extend the
grammar, write a parser, extended a compiler and/or interpreter, maybe
even the internal bytecode representation, have wars with other users of
the language whether it's a good idea to change the language that way,
and so forth. In Lisp, you just write a bunch of macros and you're done.
No problems with syntax except if you want them, most of the time no
problems with changes to the language (far less than in other
languages), no messing around with grammars and related tools, no need
to know about compiler/interpreter internals and internal
representation, no wars with other language users, and so forth.
Syntax is boring.
OK, I am sorry that I have lost focus here. You have given this example
as one that shows what probably cannot not be done in Lisp out of the
box. However, most Lisp implementations provide a way to access native
code and in that way deal with specific features of the operating
system. And there is a de-facto standard for so-called foreign function
calls called UFFI that you can use if you are interested in a
considerable degree of portability.
I don't know a lot about the specifics of shared memory, so I can't
comment on your specific questions.
This service is straight-forward to support in C/C++. It
sounds like for Lisp you are dependent on the implementation,
in that if the implementation doesn't support access to its
memory allocator/gc subsystem then it's very hard to
write code for this hardware on your own. It may be
possible to use an extension (written in C?
to that persistent memory using some sort of serialization,
but that's the best you can do -- you don't have live objects
running from nonvolatile store -- which is worse than C++.
This should be possible as a combination of a FFI/UFFI and the CLOS MOP.
AFAIK, you can define the memory layout and the allocation of memory for
specific metaclasses. However, I really don't know the details.
The paper at
http://www-db.stanford.edu/~paepcke/shared-documents/mopintro.ps might
be interesting. For UFFI, see http://uffi.b9.com/
As Paul Graham put it, yes, there is some advantage when you use the
language the operating system is developed in, or it ++.
Pascal
P
Pascal Costanza
Terry said:
Well, this proves that Python has a language feature that is as
dangerous as many people seem to think macros are.
<irony>
What you say is that local function definitions can obscure the meaning
of the Python language and/or its standard library, and this has the
potential to split the language community and make it impossible to read
each other's code. Heck, you really can't rely on the fact that sum(...)
sums its arguments? This means that a local function definition is
really a dangerous language feature, isn't it? Shouldn't it better be
abandoned?
</irony>
That last paragraph sounds as non-sensical to your ears as the arguments
against the inclusion of macros into a language because of their
expressive power sound to our ears.
BTW, do you know the talk "Growing a Language" by Guy Steele? See
http://www.research.avayalabs.com/user/wadler/steele-oopsla98.pdf - read
it, it's very insightful (even though it talks about Java.
Pascal
P
prunesquallor
Andrew Dalke said:
Never met him.
You hang out with dumb programmers?
A
Alex Martelli
Andrew said:
Me neither, for now.
And if I did, it would be in Python
I suspect no existing language would be anywhere near adequate.
But if any current programming concept could stretch there, it might
be that of "unrealized until looked-into set of things", as in, Haskell's
"lazy" (nonstrict) lists. Now lists are sequential and thus quantumly
inappropriate, but perhaps it's a start.
I suspect we may have to map the 'ensembles' down to sets of
items, just as we generally map concurrency down to sets of
sequential actions, in order to be able to reason about them (though
I have no proof of that conjecture). IF you have to map more
complicated intrinsics down to sequential, deterministic, univocal
"things", I'm sure you could do worse than Lisp. As to whether
that makes more sense than dreaming up completely different
languages having (e.g.) nondeterminism or multivocity as more
intrinsic concepts, I pass: it depends mostly on what human beings
will find they need to use in order to reason most effectively in
this new realm -- and quite likely different humans will find they
have different needs in this matter.
Yeah, well, I fear the answer will be yes (it could), but it won't
do so since you haven't _asked_ it to wake you up, only if it
could. ME, I definitely don't want to use natural language with
all of its ambiguity for anything exept communicating with
other human beings, thankyouverymuch.
*blink* what does THAT doubtful assertion have to do with anything
else we were discussing just now...? I think lisp was designed for
lists (as opposed to, say, snobol, which WAS "designed for text") and
that they're a general enough data structure (and supplemented in
today's lisps with other good data structures) that they'll be quite good
for all kinds of 'normal' (deterministic &c) programming. As for speed,
I'm sure it's easier to get it out of lisp than out of python right now.
So what's your point, and its relation to the above...?
Yes, I agree this IS generally accepted (with, of course, some dissenters,
but in a minority).
Sure, so do mine, but their menus are quite similar -- in as much as
it makes sense for them to have similar operations -- and ditto ditto
for their toolbars, keyboard shortcuts, etc etc. I.e. the differences
only come "when needed for a specific domain" just as Pascal just
said. So I don't know what you're intending with this answer.
Uniformity is more practical than diversity: e.g. ctrl-c as the Copy
operation everywhere means my fingers, even more than my brain, get
used to it. If you assign ctrl-c to some totally different operation in
your gui app "because you think it's more practical" you're gonna
drive me crazy, assuming I have to use your app. (That already
happens to me with the -- gnome based i think -- programs using
ctrl-z for minimize instead of undo -- I'm starting to have frayed
nerves about that even for GVIM, one of the programs I use most
often...
.Do you have any doubt to my answer?
Given the difficulty I'm having understanding your stance
this post, I do. My own answer would be that syntax sugar is
in people's head anyway because of different contexts -- infix
arithmetic taught since primary school, indentation in outlines
and pseudocode, etc etc -- so, following that already-ingrained
set of conventions takes no "room to spare in people's heads" --
indeed, the contrary -- it saves them effort. If people's head
started as "tabula rasa" it might be different, but they don't, so
that's a moot issue.
That much being said, I _do_ like prefix syntax. In some cases
I need to sum a+b+c+d and repeating that silly plus rather than
writing (+ a b c d) grates. Or I need to check a<b<c<d and
again I wish I could more summarily write (< a b c d). When I
designed my own language for bridge-hands evaluation, BBL, I
used prefix notation, though in the form operator ( operands )
[which I thought would have been easier for other bridge players
to use], e.g.:
& ( # weak NT opener requires AND of two things:
s ( 1 g 4 3 3 3 # shape 4333 (any), or
2 g 4 4 3 2 # 4432 (any), or
3 3- 3- 3- 5 # 5332 with 5 clubs, or
4 3- 3- 5 3- # 5332 with 5 diamonds
)
< ( 12 # as well as, 13-15 range for
\+ SHDC c( 4 3 2 1 0) # normal Milton-Work pointcount
16
)
)
Maybe readers are starting to understand why I don't WANT to
use a language I design myself;-). Anyway, the language was
NOT enthusiastically taken up, until I wrote code generators with
a GUI accepting conditions in more "ordinary looking" notations
and building this, ahem, intrinsically "better" one;-) -- then, but only
then, did other players start using this to customize hand generators
and the like. (Yes, I did have macros, but puny enough that they
still required operator(operands) syntax -- they basically served only
to reduce duplication, or provide some little abstraction, not to
drastically change the language syntax at all). Ah well -- maybe I
should just put the BBL (Turbo Pascal) implementation and (Italian-
language) reference manual online -- it still moves nostalgia in me!-)
My APL experience tells me this is false: conveniently expressing
solutions is HALF the problem -- you (and others!) have to be
able to read them back and maintain and alter them later too.
So jot your equations on a tablet-screen and look for a good
enriched text recognition system. What's programming gotta
do with it?
Python even more so on the output side -- try getting a screen-reader to
do a halfway decent job with it. But what does this matter here?
You may want to teach by showing and having the computer
infer more general rules from example. Whether the inference
engine will be best built in lisp, prolog, ocaml, mozart, whatever,
I dunno. I don't think it will be optimally done in Python, though.
"Horses for courses" is my philosophy in programming.
Given the "everything is an object" (classes included) and every object
belongs to a class, you could indeed say that -- in much the same sense
as you may be said to always be aware that you're breathing air in
everyday life. Such awareness is typically very subconscious, of course.
I'm not sure in what sense "python forces you to see" that, e.g.,
the number 2 is an object -- or how can that fail to "contribute to
the solution". Care to exemplify?
Yes to all of the above, in Python. I don't get your point.
Me? At every step -- I know 'sum' names a builtin object that is a
function (belonging to the class of builtin functions) taking one argument
which is a sequence, 'range' names another builtin object returning
a list object, etc. I'm not directly dealing with any of their classes --
I know they belong to classes, like any object does, but I have no need
to think about them in this specific statement (in fact, I hardly ever do;
signature-based polymorphism is what I usually care about, not class
membership, far more often than not).
But I don't get your points -- neither Andrew's nor Pascal's. How does
this differ from the awareness I might have in some macro-enhanced
lisp where I would type (print (sum (range 100))) or the like?
But even if we provisionally concede your conjecture we are still
left wondering: is the degree of easing so high that it overcomes
the inevitable increase in complication, needed for a language to
have N+1 syntax forms where previously it only had N? I.e., it's
in general a difficult engineering tradeoff, like many in language
design -- which is why I'd rather delegate the decisions on these
tradeoffs to individuals, groups and processes with a proven track
record for making a lot of them with complexive results that I find
delightful, rather than disperse them to myself & many others
(creating lots of not-quite-congruent language dialects).
Alex
A
Alexander Schmolck
Andrew Dalke said:
I think this is vital point. CL's inaccessibility is painted as a feature of
CL by many c.l.l denizens (keeps the unwashed masses out), but IMO the CL
community stunts and starves itself intellectually big time because CL is (I
strongly suspect) an *extremely* unattractive language for smart people
(unless they happen to be computer geeks).
Apart from the fact that this yields a positive feedback loop, I'd think that
even the smart computer geeks are likely to suffer from this incestuousness in
the midrun.
'as
P
prunesquallor
Andrew Dalke said:Pascal Costanza:
While an interesting topic, it's something I'm not going to worry about.
And if I did, it would be in Python
I bring it up as a counter-example to the idea that all modes of
programming have been and can be explored in a current Lisp.
I conjectured one interesting possibility -- that of handling ensembles
of possible solutions to a given problem.
Oops, try again.
http://hampshire.edu/lspector/qgame.html
http://www.het.brown.edu/people/andre/qlambda/
http://mitpress.mit.edu/sicp/full-text/sicp/book/node88.html
Oops, try again.
http://www.hpl.hp.com/techreports/94/HPL-94-30.html
http://dynamo.ecn.purdue.edu/~qobi/software.html
http://citeseer.nj.nec.com/siskind93screamer.html
If you were to select a language that has been used for *more* different
kinds of programming paradigms, you'd be hard pressed to find something
better than lisp.
Oops, it turns out that Lisp is used for handwriting recognition
in the banking industry. I've heard rumors of Lisp being used
for voice recognition.
You want to teach a robot to dance? I would prefer a cute woman,
myself. Suit yourself, but you can use Lisp:
http://citeseer.nj.nec.com/lee95programming.html
http://www.double.co.nz/creatures/developer/sharedmemory.htm
Actually, it is obvious with a little thought. Objects sharable
between X and Y must reside in the XY address space. Objects
sharable between X and Z must reside in the XZ address space.
Since neither Y nor Z share their entire address space with X, there
exist outgoing edges in the heap of Y that do not refer to objects
within Y. Presumably the author of the system thought of that.
(If not, he's in trouble before Z even exists.) An object common
with X and Z is as opaque to Y as any other object in X.
Why do you think this? The api would be straighforward. a
shared persistent memory would at least have these calls:
allocate, which takes an allocation amount and returns
some name for the object allocated,
retrieve, which takes some name for an object and recovers
the object itself (for use upon restarting)
dereference, which takes an object and a field and returns
or assigns the field.
An obvious way to integrate this into lisp is to make a
displaced array.
P
Peter Seibel
Terry Reedy said:
Well, if you eval the test cases, they are evaled in what's known as
the "null" lexical environment. So if you do something like:
(let ((x 10))
(check '(= (foo x) (* 2 x))))
where check is a function that evals its argument, rather than a
macro, then the evaluation will not be able to "see" the local binding
of the variable x so is unlikely to do what you think.
Python's eval--as I understand it--handles this differently. Common
Lisp's EVAL may be the way it is partially because it is not needed
for things like this given the existence of macros. There's are also
some semantic difficulties of what lexical environment the EVAL should
occur in. Given that the quoted expression above is just a piece of
data there's no particular way to attach the lexical environment to it
so that a subsequent EVAL can use it.
-Peter
A
Alexander Schmolck
Peter Seibel said:
Well, maybe he's seen things like IF*, MVB, RECEIVE, AIF, (or as far as
simplicity is concerned LOOP)...?
I'm not saying that macros always have ill-effects, but the actual examples
above demonstrate that they *are* clearly used to by people to create
idiosyncratic versions of standard functionality. Do you really think clarity,
interoperability or expressiveness is served if person A writes
MULTIPLE-VALUE-BIND, person B MVB and person C RECEIVE?
I think the example isn't a bad one, in principle, in practice however I guess
you could handle this superiorly in python.
I develop my testing code like this:
# like python's unittest.TestCase, only that it doesn't "disarm"
# exceptions
TestCase = awmstest.PermeableTestCase
#TestCase = unittest.TestCase
class BarTest(TestCase):
...
def test_foos(self):
assert foo(1,2,3) = 42
assert foo(4,5,6) = 99
Now if you run this in emacs/ipython with '@pdb on' a failure will raise an
Exception, the debugger is entered and emacs automatically will jump to the
right source file and line of code (I am not mistaken in thinking that you
can't achieve this using emacs/CL, right?) and I can interactively inspect the
stackframes and objects that were involved in the failure.
I find this *very* handy (much handier than having the wrong result printed
out, because in many cases I'm dealing with objects such as large arrays wich
are not easily visualized).
Once the code and test code works I can easily switch to mere reporting
behavior (as described by andrew dalke) by uncommenting unittest.TestCase back
in.
'as
P
Peter Seibel
Andrew Dalke said:
Yup. I'd certainly be loath to claim that there's anything that *only*
be done using macros. And it *is* the case that dynamic and
introspective features make lots of things that might otherwise be
done with macros less painful.
So here's another example of using macros to define a domain specific
language. In this case the domain is lexing and parsing. In this case
I wrote a parser generator (like ANTLR in Java on YACC in C) by
defining the macros DEFPROD, DEFCHARTYPE, and DEFLEXE macros that
allow me to define the gramatical rules for a lexer in an s-expression
notation and then stitch them together into a procedure that
implements a tokenizer based on the grammar rules. The notation may
look a bit funny if you're not used to Lisp syntax but you can
probably observe that what I have hear is pretty much a translation
from the BNF in the Java language standard to my own s-exp notation.
(I'll give just a sample of the production rules, the rest are
similar. The code required to implement the lexer proper is 91 lines
of actual code, not counting a preprocessor function that translates
Java's special unicode escape syntax.)
In other languages parser generators usually have to define their own
grammar language and write a program that parsers *that* format in
order to generate the parsing code. Macros let me do the same thing
with much less work because Lisp does parsing for me--what follows
*is* a Lisp program given that I've defined the macros it uses.
As with DEFTEST, the point is not that this is the only way to do it
but rather that macros give me a way to concisely and directly express
the stuff I *care* about (i.e. what is the grammar I'm trying to
parse) while abstracting away the mechanims by which it gets
translated into efficient code that actually does the work.
(A sample of the productions from the code that generates a Java lexer.)
;; 3.4 Line terminators
(defprod line-terminator () (/ #\newline (#\return (? #\newline))))
(defchartype input-character
'(and character (not (member #\newline #\return))))
;; 3.5 Input Elements and Tokens
(defprod input () ((* input-element) (? #\Sub)))
(defprod input-element () (/ white-space comment token))
(defprod token () (/ identifier java-keyword literal separator operator))
;; 3.6 White space
(defprod white-space () (/ #\space #\tab #\page line-terminator))
;; 3.9 Keywords
(defprod java-keyword ()
(/
"abstract" "boolean" "break" "byte" "case" "catch" "char" "class" "const"
"continue" "default" ("do" (? "uble")) "else" "extends" ("final" (? "ly"))
"float" "for" "goto" "if" "implements" "import" "instanceof"
("int" (? "erface")) "long" "native" "new" "package" "private" "protected"
"public" "return" "short" "static" "strictfp" "super" "switch"
"synchronized" "this" ("throw" (? "s")) "transient" "try" "void"
"volatile" "while"))
;; 3.10.2 Floating-Point Literals
(defprod floating-point-literal ()
(/
((+ digit)
(/
(#\. (* digit) (? exponent-part) (? float-type-suffix))
(exponent-part (? float-type-suffix))
float-type-suffix))
(#\. (+ digit) (? exponent-part) (? float-type-suffix))))
(defprod exponent-part () (exponent-indicator signed-integer))
(defchartype exponent-indicator '(member #\e #\E))
(defprod signed-integer () ((? sign) (+ digit)))
(defchartype sign '(member #\+ #\-))
(defchartype float-type-suffix '(member #\f #\F #\d #\D))
;; 3.12 Operators
(defprod operator ()
(/
":"
"?"
"~"
("!" (? "="))
("%" (? "="))
("&" (? (/ "=" "&")))
("*" (? "="))
("+" (? (/ "=" "+")))
("-" (? (/ "=" "-")))
("/" (? "="))
("<" (? "<") (? "="))
("=" (? "="))
(">" (? ">" (? ">")) (? "="))
("^" (? "="))
("|" (? (/ "=" "|")))))
;; This macro actually expands into a fuction java-lexer that takes
;; a string of text and tokenizes it according to the rules defined
;; above, starting from the "input" rule and returning as tokens
;; objects that represent identifiers, java-keywords, literals,
;; separators, and operators. Comments and whitespace are, thus,
;; discarded.
(deflexer java-lexer
(
start-rule input)tokens identifier java-keyword literal separator operator)))Sure, you can put any boolean expression in a check and have it
treated as a test case. The macro code can figure out whether it's a
function call and if it is arranges to evaluate the arguments itself
so it can see what their values are and then passes the values to the
function. Because all the comparators such as =, EQL (object
string< said:
Yeah. Though that's just because I didn't get around to implementing
that. As long as the expression consists of a tree of function calls,
I could do the same thing to sub-expressions I do to the top-level
expressions in the CHECK, and display them as well. If I really wanted
I could write a GUI to browse through the whole tree of function
calls. But I decided that the 80/20 rule applies and if I can't figure
out from the top-level expressions what's going on then I can always
resort to normal debugging. (Also the test framework can dump you into
the debugger at the point of a test failure so you can poke around
with the debugger to see any values you care about.)
The other difference is that runtime introspection happens at runtime.
That's probably fine for a test framework. I've written several Java
test frameworks that use reflection in similar ways so I know about
that approach too. But for other tasks, such as my parsing example
above, even if they could be done using introspection, there's a big
advantage for doing a lot of the work once, at compile time.
[snip]
Heh. That's just a sort of, pardon my French, kludgy reimplementation
of macros. Macros are nothing more (or less) than a mechanism whereby
you can--at compile time--get a hold of the parse tree in a form that
your code can manipulate it to generate other code. (Your's is still
at runtime which makes it somewhat less useful, though it does work
for the test framework case.)
Sure. I could use Lisp's meta object protocol (MOP) to define all
sorts of crazy things. But I'd probably still end up wrapping them in
some nice syntactic abstractions (macros) to make them as expressive
as possible. The thing is, I'm not looking for a "non-macro way" to do
these things because macros (in Lisp) are no more problematic than
functions--they're just another way of definining abstractions. Like
any mechanism for defining abstractions they need to be used with good
design sense because bad abstractions can be worse than no
abstractions at all--at least if things are concrete you can see how
they work, even if you do have to wade through pages of code to see
it. But good abstractions--whether functions, classes, or macros--make
it even easier to understand the code.
-Peter
P.S. Pythonistas--I was originally following this thread in c.l.lisp.
Somewhere along the lines the follow-ups got set to c.l.python which
is fine with me as the Lisp guys (should) already know this. But feel
free let me know if you want me to shut up about this.
P
Paolo Amoroso
Andrew said:
If I understand things correctly, ITA Software's Orbitz system does
that. See the slides of Rodney Daughtrey's ILC 2002 talk "ITA Software
and Orbitz: Lisp in the Online Travel World" (International Lisp
Conference 2002 Proceedings, page 606).
A toy example is contained in Paul Graham's book "ANSI Common
Lisp". See section 13.5 "Example: Pools" on page 226.
Paolo
A
Alex Martelli
Pascal Costanza wrote:
...
Indeed, a chorus of "don't do that" is the typical comment each
and every time a newbie falls into that particular mis-use. Currently,
the --shadow option of PyChecker only warns about shadowing of
_variables_, not shadowing of _functions_, but there's really no
reason why it shouldn't warn about both. Logilab's pylint does
diagnose "redefining built-in" with a warning (I think they mean
_shadowing_, not actually _redefining_, but this may be an issue
of preferred usage of terms).
"Nailing down" built-ins (at first with a built-in warning for overriding
them, later in stronger ways -- slowly and gradually, like always, to
maintain backwards compatibility and allow slow, gradual migration of the
large existing codebase) is under active consideration for the next version
of Python, expected (roughly -- no firm plans yet) in early 2005.
So, yes, Python is not perfect today (or else, we wouldn't be planning a
2.4 release.... While it never went out of its way to give the user "as
much rope as needed to shoot oneself in the foot", neither did it ever
spend enormous energy in trying to help the user avoid many possible errors
and dubious usage. Such tools as PyChecker and pylint are a start, and
some of their functionality should eventually be folded back into the
core, just as tabnanny's was in the past with the -t switch. I don't think
the fundamental Python will ever nag you for missing comments or
docstrings, too-short names, etc, the way pylint does by default (at
least, I sure hope not...!-), but there's quite a bit I _would_ like to have
it do in terms of warnings and, eventually, error messages for
"feechurs" that only exist because it was once simple to allow than
to forbid them, not by a deliberate design decision to have them there.
Note that SOME built-ins exist SPECIFICALLY for the purpose of
letting you override them. Consider, for example, __import__ -- this
built-in function just exposes the inner mechanics of the import
statement (and friends) to let you get modules from some other
place (e.g., when your program must run off a relational database
rather than off a filesystem). In other word, it's a rudimentary hook
in a "Template Method" design pattern (it's also occasionally handy
to let you import a module whose name is in a string, without
going to the bother of an 'exec', so it will surely stay for that purpose
even though we now have a shiny brand-new architecture for
import hooks -- but that's another story). Having a single hook of
global effect has all the usual downsides, of course (which is exactly
why we DO have that new architecture;-): two or more complicated
packages doing import-hooks can't necessarily coexist within the
same Python application program (the only saving grace which let
us live with that simplistic hook for so many years is that importing
from strange places is typically a need of a certain deployment of
an overall application, not of a package -- still, such packages DO
exist, so the previous solution was far from perfect).
Anyway, back to your contention: I do not think that the fact that
the user can, within his functions, choose very debatable names,
such as those which shadow built-ins, is anywhere as powerful,
and therefore as dangerous, as macros. My own functions using
'sum' will get the built-in one even if yours do weird things with
that same name as a local variable of their own. The downsides
of shadowing are essentially as follows...
a newbie posts some fragment of his code asking for guidance,
and among other things that fragment has
for i in range(length(thenumbers)):
total = total + thenumbers
he will receive many suggestions on how to make it better,
including the ideal one:
total = sum(thenumbers, total)
But then he tries it out and reports "it breaks" (newbies rarely
are clueful enough to just copy and paste error messages). And
we all waste lots of time finding out that this is because... the
hapless newbie had named HIS OWN FUNCTION 'sum', so
this was causing runaway recursion. Having met similar issues
over and over, one starts to warn newbies against shadowing
and get sympathetic with the idea of forbidding it.
That doesn't really compare to an extra feature in the language
that is deliberately designed to let reasonably-clueful users do
their thing, isn't deprecated nor warned against by anybody at
all (with a few isolated voices speaking about "abuse" of macros
in this thread, but still with an appreciation for macros when
_well_ used), and is MEANT to do what newbies _accidentally_
do with shadowing & much more besides;-).
Alex
...
Indeed, a chorus of "don't do that" is the typical comment each
and every time a newbie falls into that particular mis-use. Currently,
the --shadow option of PyChecker only warns about shadowing of
_variables_, not shadowing of _functions_, but there's really no
reason why it shouldn't warn about both. Logilab's pylint does
diagnose "redefining built-in" with a warning (I think they mean
_shadowing_, not actually _redefining_, but this may be an issue
of preferred usage of terms).
"Nailing down" built-ins (at first with a built-in warning for overriding
them, later in stronger ways -- slowly and gradually, like always, to
maintain backwards compatibility and allow slow, gradual migration of the
large existing codebase) is under active consideration for the next version
of Python, expected (roughly -- no firm plans yet) in early 2005.
So, yes, Python is not perfect today (or else, we wouldn't be planning a
2.4 release...
. While it never went out of its way to give the user "as much rope as needed to shoot oneself in the foot", neither did it ever
spend enormous energy in trying to help the user avoid many possible errors
and dubious usage. Such tools as PyChecker and pylint are a start, and
some of their functionality should eventually be folded back into the
core, just as tabnanny's was in the past with the -t switch. I don't think
the fundamental Python will ever nag you for missing comments or
docstrings, too-short names, etc, the way pylint does by default (at
least, I sure hope not...!-), but there's quite a bit I _would_ like to have
it do in terms of warnings and, eventually, error messages for
"feechurs" that only exist because it was once simple to allow than
to forbid them, not by a deliberate design decision to have them there.
Note that SOME built-ins exist SPECIFICALLY for the purpose of
letting you override them. Consider, for example, __import__ -- this
built-in function just exposes the inner mechanics of the import
statement (and friends) to let you get modules from some other
place (e.g., when your program must run off a relational database
rather than off a filesystem). In other word, it's a rudimentary hook
in a "Template Method" design pattern (it's also occasionally handy
to let you import a module whose name is in a string, without
going to the bother of an 'exec', so it will surely stay for that purpose
even though we now have a shiny brand-new architecture for
import hooks -- but that's another story). Having a single hook of
global effect has all the usual downsides, of course (which is exactly
why we DO have that new architecture;-): two or more complicated
packages doing import-hooks can't necessarily coexist within the
same Python application program (the only saving grace which let
us live with that simplistic hook for so many years is that importing
from strange places is typically a need of a certain deployment of
an overall application, not of a package -- still, such packages DO
exist, so the previous solution was far from perfect).
Anyway, back to your contention: I do not think that the fact that
the user can, within his functions, choose very debatable names,
such as those which shadow built-ins, is anywhere as powerful,
and therefore as dangerous, as macros. My own functions using
'sum' will get the built-in one even if yours do weird things with
that same name as a local variable of their own. The downsides
of shadowing are essentially as follows...
a newbie posts some fragment of his code asking for guidance,
and among other things that fragment has
for i in range(length(thenumbers)):
total = total + thenumbers
he will receive many suggestions on how to make it better,
including the ideal one:
total = sum(thenumbers, total)
But then he tries it out and reports "it breaks" (newbies rarely
are clueful enough to just copy and paste error messages). And
we all waste lots of time finding out that this is because... the
hapless newbie had named HIS OWN FUNCTION 'sum', so
this was causing runaway recursion. Having met similar issues
over and over, one starts to warn newbies against shadowing
and get sympathetic with the idea of forbidding it
.That doesn't really compare to an extra feature in the language
that is deliberately designed to let reasonably-clueful users do
their thing, isn't deprecated nor warned against by anybody at
all (with a few isolated voices speaking about "abuse" of macros
in this thread, but still with an appreciation for macros when
_well_ used), and is MEANT to do what newbies _accidentally_
do with shadowing & much more besides;-).
Alex
A
Andrew Dalke
Alex:
Pshaw. My hypothetical house of the 2050s or so will know
that "could" in this context is a command.
But what if computers someday become equally capable
as humans in understanding uncontrained speech? It
can be a dream, yes?
An unfortunate typo. I meant "speech" instead of "speed" but
my fingers are too used to typing the latter. Here I would like
a computer to ask "um, did you really mean that?" -- so long as
the false positive rate was low enough.
Loosing ... focus ... mind .. numb .. must stop answering thread.
It's a knee-jerk reaction and an example that I'm no longer
thinking before I start to reply.
The conjecture that computer programming languages are
contrained by the form of I/O and that other languages, based
on speech, free-form 2D writing, or other forms of input may
be more appropriate, at least for some domain.
This was in response to the idea that Lisp is the most appropriate
language for all forms of programming.
It was a rhetorical set of questions, to see what Pascal meant
about all the time being aware that we are dealing with classes
and object. When I wrote it, I didn't see the ambiguity that I
could be pointing out that these aren't classes/objects; in part
because I just didn't think about that alternative.
That was my point.
Good point. And as I no longer feel like following up on this
thread, I'll leave it at that.
Andrew
(e-mail address removed)
Pshaw. My hypothetical house of the 2050s or so will know
that "could" in this context is a command.
But what if computers someday become equally capable
as humans in understanding uncontrained speech? It
can be a dream, yes?
An unfortunate typo. I meant "speech" instead of "speed" but
my fingers are too used to typing the latter. Here I would like
a computer to ask "um, did you really mean that?" -- so long as
the false positive rate was low enough.
Loosing ... focus ... mind .. numb .. must stop answering thread.
It's a knee-jerk reaction and an example that I'm no longer
thinking before I start to reply.
The conjecture that computer programming languages are
contrained by the form of I/O and that other languages, based
on speech, free-form 2D writing, or other forms of input may
be more appropriate, at least for some domain.
This was in response to the idea that Lisp is the most appropriate
language for all forms of programming.
It was a rhetorical set of questions, to see what Pascal meant
about all the time being aware that we are dealing with classes
and object. When I wrote it, I didn't see the ambiguity that I
could be pointing out that these aren't classes/objects; in part
because I just didn't think about that alternative.
That was my point.
Good point. And as I no longer feel like following up on this
thread, I'll leave it at that.
Andrew
(e-mail address removed)