Can elisp call C functions without recompilation of emacs?

[Fren Zeee]

In 1996, Stephen Eglin asked the following question which was never
answered but is of interest to people in C , Lisp and users of Emacs :

http://groups.google.com/group/gnu....a2368ff6f5ea1b7b?q="Writing+Emacs+Primitives"

Newsgroups: comp.emacs, gnu.emacs.help
From: (e-mail address removed) (Stephen Eglen)
Date: 1996/04/10
Subject: Can elisp call C functions without recompilation of emacs?
Reply to author | Forward | Print | Individual message | Show original
| Report this message | Find messages by this author
Can elisp call functions written in C without recompilation of emacs?

I think what I am asking is not possible, but here goes anyway...

Here at Sussex we have an environment called POP-11, which is a lisp
style interactive AI programming environment. One of its main
features is that you can call C functions from POP-11 after the shared
object files containing the C functions have been loaded into POP-11.
This can be done interactively, and does not require either POP-11 to
be recompiled or for the C functions to be compiled with any special
options. So for example, if I write a function called add(x,y) in C,
and then compile it into a shared object, I can then call this C
function from POP-11 once I have loaded in the object file storing the
add function. Of course,one of the restrictions made on the POP:C
interface is that I think the arguments to a c function must be ints,
floats, doubles and chars, rather than abstract c data types.

This is a very useful facility, especially when debugging c functions,
since you can write the test functions in POP-11, which is an
interactive language, and so only the C functions need compiling, and
not the test functions. (I'm not sure exactly how the mechanism
works
in POP-11, but I think it uses some form of incremental compiler?)

My question therefore is does emacs offer this facility of calling c
functions from elisp? I notice in the elisp info files (Writing Emacs
Primitives) that of course some emacs primitives are written in c, but
am I right in thinking that once the c function has been written, the
whole of emacs needs recompiling? If this is the case, then I guess
the compile time of emacs would be a big overhead, and probably not
worth it. Also, it looks like the c functions have to be written
especially with elisp in mind (eg use of the DEFUN macros.)

I only have access to emacs 19.27 at the moment, but if it works for
other versions, I would be interested in hearing about them.

Thanks in advance, Stephen Eglen.
--
Stephen Eglen, DPhil Student,
(e-mail address removed)
School of Cognitive and Computing Sciences, Fax: +44-(0)1273
671320
University of Sussex, Phone: +44-
(0)1273-678524
Falmer, BRIGHTON, BN1 9QH, UK http://www.cogs.susx.ac.uk/users/stephene/

 

[Fren Zeee]

Sources :::
[1] http://delysid.org/emacs/xemacs.html
[2] http://www.jwz.org/doc/lemacs.html
[3] http://en.wikipedia.org/wiki/XEmacs


Message-ID: <[email protected]>
Date: Mon, 22 Jun 92 21:37:51 PDT
From: (e-mail address removed) (Richard P. Gabriel)
To: (e-mail address removed), (e-mail address removed)
Cc: (e-mail address removed), (e-mail address removed), (e-mail address removed)
Subject: Epoch and EMACS merger

I read RMS's message with sadness. Over the past two years I have
tried very hard to figure out a way to work with FSF on Emacs, and for
a lot of that time I thought we were working together, but apparently
I failed. For those of you who wish to read about that effort, you can
read the postscript to this note, which outlines Lucid's involvement -
a sort-of history of the events.

We still want to work with FSF and have our contributions become part
of FSF Emacs, and we are eager to hear suggestions on how to proceed.

-rpg-

P.S. RMS's letter paints an unflattering and unfair picture of Lucid's
involvement with Emacs, and in particular on the motives and efforts
of our hackers.

The Free Software Foundation is a remarkable organization - for many
years I have supported it personally (and still do), and my company
supports it, its goals, and its products. This support is in terms of
time, effort, and dollars.

RMS says:

However, the people who wrote the Lucid version didn't try to to
make the changes usable for us. They charged ahead without discussing
the design with me or with the main Emacs maintainer. They
reimplemented things we had already done. I asked them to do some work
making parts usable for us, but they said no.

This is not how we at Lucid remember events, although it may be a
difference without a distinction. I'll recount the events as I recall
them and let the reader supply the characterization.

Lucid set a course 4 years ago to develop a C/C++ programming
environment, using Emacs as a key component. We needed a small number
of specific things:

* it had to run under X using multiple windows

* it had to support multiple fonts (at least a little)

* it had to support embedded glyphs (icons)

* it had to support attaching active regions to contiguous blocks
of characters (to support a hypertext-like facility)

* it had to support menubars

* it had to support readonly regions of text

My goals with respect to FSF were these:

* do anything (including paying money) to get our changes into the
official FSF sources

* not sacrifice quality or the schedule (which was lengthy, and
therefore apparently safe)

For the first two years we prototyped the functionality we needed in
an X-widget-based text editor and in Epoch.

By the Summer of 1990, Emacs 19 appeared to be right around the
corner, so we entered discussions with the person at FSF whom we were
led to believe was the official Emacs 19 coder - Joe Arceneaux. It
appeared that our goals and FSF's were in line, and so he proposed
that we pay him a consulting fee to speed up Emacs 19 development plus
get our changes in. We agreed, wrote a contract, and volunteered
manpower to help.

The contract period started October 1990, was to last approximately 9
months. We were to pay him approximately $50k. He was paid on
milestones, on which he appeared to deliver. In addition, we provided
him office space and his own Sparcstation at Lucid. (At the conclusion
of this period, it appeared that Joe Arceneaux was no longer the
official Emacs 19 coder.)

In the Spring of 1991 it became apparent that the effort was going
badly - Emacs 19 as a text editor was slow, and full of bugs. The only
people at Lucid who dared use it were Arceneaux and myself. Moreover,
Emacs 19 performed hideously in the new areas outlined above. Its
behavior was especially bad with respect to active regions -- even
worse than our seat-of-the-pants prototype in Epoch.

Nevertheless, we persevered: We re-arranged our Emacs team and
priorities to work more closely with Arceneaux, and we were confident
the arrangement could work. To be safe, I put 4 of our best hackers on
it with Arceneaux. By Summer of last year we became aware that RMS had
hired/appointed Jim Blandy as the official Emacs 19 person.

At that time, one of our hackers went out to Cambridge to talk RMS
about how to proceed, and it appeared we reached a reasonable working
arrangement. But, as time went on we determined that the FSF code for
Emacs 19 was too unwieldy, bug-ridden, and convoluted to work with, so
we started rewriting parts of it.

In addition to not listening to any design ideas we had, RMS's policy
was to require us to provide source diffs and to explain every changed
line. Because our efforts towards release were mounting, and we were
re-writing massive amounts of the system, source diffs didn't make
sense, but RMS and Blandy were unwilling or unable to work at a higher
level. I traveled to Cambridge to work things out, and my team had
numerous phone conversations with RMS.

One area that was completely rewritten was the implementation of
active regions. Various attempts were made to discuss the design with
RMS, including a 2 hour telephone conversation from one of my hackers,
but he was completely unwilling to consider another point of view. At
first, he apparently didn't understand the differences between
attaching active regions to text and attaching fonts to text, and when
he finally did, he was so wedded to his design for putting fonts in
the text that he wanted us to implement BOTH his design for fonts and
ours for other active region support.

Our hacker wanted RMS to cooperate to the extent that we would first
implement our version to support all of these features, and see how
well it worked, but he wouldn't even agree to that. It is worth noting
that with the original FSF version, in spite of some rather elaborate
tree-balancing code and other hackery for precomputing display
characteristics, Emacs redisplay spent more than 40% of its time in
the active regions code. In our much simpler version it spends
significantly less than 1% of the redisplay time in the corresponding
code, even for buffers with a few thousand active regions. (This isn't
to say that the original design couldn't be made to work, but several
man-months of effort had gone into their original version, and it took
less than 2 man-months to implement and debug the code that we shipped
with our version. Finally, our version is approximately 1/2 the number
of lines of the original, it is more general, and it uses less
memory.)

Early last Fall, after spending approximately $70k on Arceneaux, we
let his contract expire, but we did put a leased 38.4 kb digital line
and an X terminal in Arceneaux's apartment in San Francisco (35 miles
away from Lucid), so he could continue to work on Emacs 19 for FSF
with Lucid support.

When we learned last Summer that FSF had established an Emacs steering
committee, and no one from my organization was put on that, or even
informed of its existence, we came to the conclusion that the FSF had
no interest in working with us, and we stopped trying. The punch line
of RMS' statement is true -- we wouldn't do things they way he wanted.
But we didn't ``charge ahead'' without talking to FSF. While we did
reimplement some things that they had already done, the things we
reimplemented were completely broken. It's true that in the end our
work has been unusable by the FSF, but that seems to us to have more
to do with their willingness to use it than with anything else.

RMS concludes:

The result is that not much of the code in Lucid's version will
ever appear in the FSF's version. Some of the features are worth
having, but need to be reimplemented. If you'd like this to happen
sooner, one way is to volunteer to do some of the work.

I have trouble imagining how anyone could have worked with with them
on this. We may have failed in this attempt, but we have a history of
trying, and we have succeeded in other areas.

In addition to working on Emacs, and contributing to Cygnus initial
implementations of some new features for gdb, such as a malloc that
supports multiple heaps, code to save the internal gdb ``symbol
table'' state for a given executable file, and a various C++ debugging
features, (as well as paying $100K to have these features officially
incorporated into gdb), during the past year we dedicated
approximately 1 person-year, spread over 3 people, to producing a new
Emacs manual. In addition we have spent a lot of time trying to figure
out how to work amicably with FSF.

We have also supported FSF and RMS by selecting several FSF components
for our product, an endorsement of their quality and usefulness. Since
1987 Lucid has regularly contributed code (and occasionally money) to
FSF. Lucid paid over $200k to support FSF products over the last 2
years. We contributed over 8 hacker-years to improve FSF products.

I personally am a member of LPF (not related to FSF, but related to
RMS). I travel about debating various people in RMS's name, and I have
actively supported RMS in various private ways. I couldn't figure out
how to work with FSF and RMS even though I had years to do it,
hundreds of thousands of dollars to spend, and 8 man-years to
volunteer. What would you have done differently?

We offer a free Emacs 19 to anyone who wants it, under exactly the
same conditions as the FSF, providing commercial support to our paying
customers (all improvements arising out of this support become part of
the free version as well, of course), as well as answering questions
and incorporating changes received from users on the net, and we will
continue to develop it. We will rewrite the redisplay code, and we
will very probably do work to internationalize Emacs. We also are
using scarce documentation resources on the Emacs manual (copyright
still held jointly with RMS). However, unlike the FSF, we will also
listen to ideas from other people. For one thing, if someone
implements an improved version of a feature, we aren't arrogant or
inflexible enough to refuse to consider using it. The hackers that
worked on our version are concerned about doing a good job for the
sake of Emacs, not just for the sake of our company or their egos.

By this time I've ceased expecting that we will work closely with FSF
on Emacs, but we would still like to do so. As far as we can tell, the
reason that FSF won't use any of our work is that they aren't
interested in working *with* us (rather they want us to work *for*
them, doing precisely what they tell us to do the way they say to do
it, and on their own schedule). Perhaps they assume that our designs
and implementations are substandard, but as far as I know they have
never really looked at what we have done.