C scripting (continued)

[jacob navia]

You post a revised version of a program that takes about
100 lines of C code to a) compilee, b) link and c) run the
C programs in a certain directory.

d) Gather statistics about what parts of those process failed
e) print them

And, as has been pointed
out, there are still some issues with the program.

No, there is just a transcript error from the program
to the post. That's all . NO issues were raised.

?So I am
wondering is what you actually want to prove with this?

That a program that takes just 5072 bytes of executable size
I can do the same thing that in other scripting languages with the
same style of security.

I could come up with less then 10 lines of a Makefile or
similar number of limes of a Perl or bash script to do
more or less the same,

rather less since it would be surprising that Makefiles
gather statistics about which things failed

and I am rather sure many of the
other "regulars" could do the same (or they rather might
know an even more concise way to do it). So it's no a
very convincing example of why one should write scripts
in C.

For you. Your opinion. My opinion differs.

If you want to write everything in C (and I don't blame you,
I like C and also write a lot of stuff in C, but not every-
thing) that's fine with me. What I don't understand is why
you seem to see C as the one and only "true way" of writing
programs.

What I like in C is precisely this minimalist approach that simplifies
software construction and allows to produce powerful and small programs.

In this days of trying to make of everything a monster program that
takes megabytes to run it is refreshing to see a standalone
program that takes just 5K.

C is, at least to me, a nice. small and often ele-
gant language, but, alas, at times bogged down with histo-
rical decisions that can't be easily undone and simply have
to be accepted. It's very useful for a lot of things I do
(especially if speed is an issue or you have to write e.g.
libraries or device drivers). But there are things that
are too time-consuming and complicated to write in C and then,
if possible, I tend to write that in a different language.

Your choice.

[snip]

But, if I understand what you write at other times, you
don't seem to like C for what it is: you want to extend
it in all kinds of directions, e.g. making it more like
C++, with operator and function overloading etc., just to
name a few examples.

Surely not as C++, I never proposed that. C++ exists, and
it is a model of what should not be done.

I get the feeling that you actually
want to develop your own language and the restrictions of
C are keeping you back all of the time, making you unhappy
more than necessary.

There are three main points that I want to add:

(1) Abstract types like lists/extensible arrays and others.
(2) Non zero terminated strings
(3) To use the above two a simple language extension is
necessary.

Why don't you simple cast off all the
baggage and start your own language? Other people have done
so before and have been very successful, with C++ as the
most obvious example?

Implicit here is the idea that C should NOT evolve. That is dead
and any improvement (even getting rid of gets() or asctime())
is just a waste.

C has a very important role nowadays as a lingua franca.
Many kernels, device drivers etc. are written in C, as
well as all kinds of other languages and libraries. But
that's also what makes C so hard to change fundamentally.
How much you rave and swear, it won't get changed easily.

Your opinion.

And if you don't like it come up with something that's
really better and gets accepted!

lcc-win has a download rate of several hundreds per day.
We passed the million downloads recently. lcc-win is accepted.

You obviously have tried
to with lcc, but just having tried, unfortunately, isn't
good enough.

I want to make this extension public and discuss about them in this
forum.

There are lots of people that have written
excellent programs or tools that never got the recognition
they deserved. So why not make your lcc a compiler for a
language you really like and feel to be superior to C,
dropping all pretensions of it being "just another C
compiler"? It obviously isn't meant to be one. You don't
seem to want to write just a C compiler that's faster than
all others or just produces faster executables - from all
you write I take that you want to write a compiler for what
you see as a better language than (that's just similar to
C). So just do that and don't look back - don't get bogged
down by people pointing out that it may have some issues as
a standard conforming C compiler.

I do not get bogged down. I am finishing the C99 implementation,
I have improved in the 32 bit compiler the handling of 64 bit
data, released the 64 bit version, started the defining of
improved complex operations code generation, ported
the 32 bit version to linux again etc etc.

 

[Bartc]

Here's my version, written in Perl, based on your C program. ....
It's roughly half the size of your C version, it performs more
checking, and because of the way it invokes system() it's not
vulnerable to attacks involving file names with metacharacters. It
took me a few minutes to write it.

In defence of the C version, that included an implementation of a
line-reading function fggets() that really has no business being in the
script; it should be buried in a library somewhere. That reduces the line
count by a third.

Some other tidying of the C version (such as using compile()/link() as
library functions) would make it about the same size as the perl code.

 

[Bartc]

Jens Thoms Toerring wrote:

That a program that takes just 5072 bytes of executable size
I can do the same thing that in other scripting languages with the
same style of security.

That's commendable. But it does rely on the presence of a C compiler and
linker, which must be 1MB or so, by themselves, and which might also require
some 25MB of include and library files (that's for lcc-win).

And for creating or maintaining scripts such as these, you again will need a
C system with it's include files and libraries, still 25MB which compares
well with the 800KB you mentioned for Ruby.

 

[Jens Thoms Toerring]

That's commendable. But it does rely on the presence of a C compiler and
linker, which must be 1MB or so, by themselves, and which might also require
some 25MB of include and library files (that's for lcc-win).

I think that's a bit unfair since then you would have to compare
that to all the resources that are required to run a program in a
scripting language. If I look e.g. just for my /usr/lib/perl/ and
/usr/lib//perl5/ directories where all the modules are that one
may need I already come up with something in the order of 50 MB.
And then one might argue that, since a C compiler is needed to
compile Perl, it also should be counted in;-)

And for creating or maintaining scripts such as these, you again will need a
C system with it's include files and libraries, still 25MB which compares
well with the 800KB you mentioned for Ruby.

I think the 800 kB for Ruby were just the run time size, not the
whole infrastructure that probably also has to exist for Ruby (I
can't say for sure since I haven't used Ruby, but I would guess
that there's more than just 800 kB to a full install of Ruby).

Regards, Jens

 

[Keith Thompson]

In defence of the C version, that included an implementation of a
line-reading function fggets() that really has no business being in
the script; it should be buried in a library somewhere. That reduces
the line count by a third.

In Perl, as in most scripting language, the equivalent of the
line-reading function is built into the language. Furthermore, the
Perl script reads the directory directly using readdir(), so it
doesn't need to read lines; the C version could have done something
similar, but the code would have been much more verbose.

And the standard C library has no function that can read arbitrarily
long lines; if you want portability, that function has to be part of
the program.

Some other tidying of the C version (such as using compile()/link() as
library functions) would make it about the same size as the perl code.

Do you know of some implementation that provides compile() and link()
as library functions? I don't. The code that invokes the compiler
and linker has to be part of the program, even if it's in a separate
library. And if you're organizing the program into libraries, or
otherwise using multiple translation units, then it's a fairly
substantial programming task, not just a "script".

If you don't count custom libraries as part of the program, you can
always write:

#include "customlib.h"
int main(void) {
run_tests();
return 0;
}

 

[Jens Thoms Toerring]

d) Gather statistics about what parts of those process failed
e) print them

No, there is just a transcript error from the program
to the post. That's all . NO issues were raised.

Ok, I had make four changes to get it to compile on my
(Linux) machine (with a few warnings remaining), and I
couldn't easily distinguisg between "transcript errors"
and "issues". But that we should be bothered about.

That a program that takes just 5072 bytes of executable size
I can do the same thing that in other scripting languages with the
same style of security.

Having small programs is nice, definitely. And that you
can do the same things in C you can do in a scripting lan-
guages is, I think, uncontentious (especially since many
scripting languages are actually based on C). It just
doesn't convince me that C is very useful for scripting
since IMHO there are other priorities than just code size.

rather less since it would be surprising that Makefiles
gather statistics about which things failed

Right, though I would prefer the direct output of make any
time since it shows what went wrong and not just how many
times something went wrong;-)

For you. Your opinion. My opinion differs.

That's right, it's definitely my opinion.

What I like in C is precisely this minimalist approach that simplifies
software construction and allows to produce powerful and small programs.

In this days of trying to make of everything a monster program that
takes megabytes to run it is refreshing to see a standalone
program that takes just 5K.

As I said, small code size is nice. But when writing a script
that's actually rather low on my priority list. What's high on
my list is get the job done as fast as possible and with the
least hassle. And many scripting languages simply make a lot
of things simpler. I don't know what you think about tools
like awk or sed etc. All you can do with them you can also
write in C, but it would seem to me to be a huge PITA doing
that compared to just writing one or a few more lines in e.g.
awk to get a job done. And with e.g. Perl I can do basically
everything I could do in awk, just with a lot of other nice
things thrown in. If it takes a megabyte more doesn't bother
me much.

Your choice.

Yes, of course. But I thought you wanted to convince me that
there are advantages in writing scripts in C (the mere fact
that you can do that has, as far as I can see, never been
questioned).

[snip]

But, if I understand what you write at other times, you
don't seem to like C for what it is: you want to extend
it in all kinds of directions, e.g. making it more like
C++, with operator and function overloading etc., just to
name a few examples.

Surely not as C++, I never proposed that. C++ exists, and
it is a model of what should not be done.

There are three main points that I want to add:

(1) Abstract types like lists/extensible arrays and others.
(2) Non zero terminated strings
(3) To use the above two a simple language extension is
necessary.

Aren't you missing a garbage collector, operator overloading,
default arguments, arbitrary precision numbers etc. from that
list? At least that's what I remember you to have proposed
quite a few times here.

I have to admit that I never felt that zero-terminated strings
are that evil as you tend to paint them. They're just an array
with one of the elements being set to zero. That fits well
into what I see as making C a slim, effective language. I can
see the security improvements when using counted strings. On
the other hand, as far as I can see, to achieve this security
you then need to have bound checking done on each and every
access to a char in such a string - something that for a good
reason isn't part of the way things are done in C. I thus would
prefer to see this left to a library and not made a core part
of the C language.

I am also not really convinced that introducing abstract types
into the language is beneficial. Until now C has a number of
simple data types, basically numbers and chars, arrays of
them, plus structures that allow you to make up abstract data
types. And, in my opinion, that's a good deal of what makes C
such a nice and relatively simple language. Adding abstract
data types would, again in my opinion, go against the grain
of what makes C what it is.

Of course, that are my private opinions. And, of course, I also
have a few things I would love to have added to C and which I
can't simply do via a library like exceptions...

Implicit here is the idea that C should NOT evolve. That is dead
and any improvement (even getting rid of gets() or asctime())
is just a waste.

Well, the gets() and ascitime() stuff is just a red hering. I
actually couldn't care less if the exist, remain or not (and
I can't remember having seen any real code use e.g. gets()).
And, yes, I don't think that C needs much evolving. For me
it works quite fine most of the time. And the direction of
"evolution" you propose would, again in my opinion, lead to
a different language that would cast off a lot of the advan-
tages is see with C. I find it important that with C I still
can "feel" what's happening behind the scenes, there are no
unexpected calls of functions like you would need for over-
loading, constructors etc. So I use C for stuff where I care
about that. And for other stuff I use a different language.

The main difference seems to me that you seem to want C to be
more similar to higher level languages, so that you can do
everything in C. I would prefer C to stay as "near to the
metal" as possible and when that's unimportant for a task
and writing it in C is inconvenient I'd use something more
suitable. Unfortunately, I think it's going to be difficult
that we at one point will come to hold the same opinion...

Your opinion.

I thought that's not my opinion but your experience;-)

lcc-win has a download rate of several hundreds per day.
We passed the million downloads recently. lcc-win is accepted.

Congratulations! I didn't know it's that popular. It's a pity
you seem not to be able to publish a Linux version. I'm sure
I would give it a try. And, actually, you seem to have created
a new language, with all the additions you made (sorry, I tried
to have a closer look at the documentation, but whenever I try
to download something I get a "The change you wanted was re-
jected" error message). And having a one million user base
seems to me to be pretty impressive! And if it can be used
as a standard comformant C compiler beside it's even more
impressive.
Regards, Jens

 

[jacob navia]

Aren't you missing a garbage collector, operator overloading,
default arguments, arbitrary precision numbers etc. from that
list? At least that's what I remember you to have proposed
quite a few times here.

A garbage collector is furnished with the standard distribution
of lcc-win, but it doesn't need anything in the syntax or
in the language to be changed. It is just another library like the
full perl regular expression library or the 352 bits precision
floats or any other library.

Operator overloading is the simple language extension I was speaking
about. It boils down to giving you the capacity of building new
types of numbers and use the array notation to access user
defined structures.

For instance if you use counted strings now (as opposed to
zero terminated strings) you have to:

String s; // String is a counted string

assign(s,12,'b');

instead of

s[12] = 'b';

Overloading the operator [ ] allows to keep most of the
code written for zero terminated strings to be used with
the new strings with minimal effort.

I have to admit that I never felt that zero-terminated strings
are that evil as you tend to paint them. They're just an array
with one of the elements being set to zero. That fits well
into what I see as making C a slim, effective language. I can
see the security improvements when using counted strings.

The problem is that they are too error prone! Take a programmer like
Heathfield for instance. We recently discussed a bug he had
using realloc+zero terminated strings. And he is an experienced
C programmer. I think that nobody can reasonably say that zero
terminated strings are ok!

On
the other hand, as far as I can see, to achieve this security
you then need to have bound checking done on each and every
access to a char in such a string - something that for a good
reason isn't part of the way things are done in C. I thus would
prefer to see this left to a library and not made a core part
of the C language.

Since HOW the operator overloading works is left to the user
that writes his/her own routines we can have a debug version
and a fast version with and without bounds checking as you
want!

I am also not really convinced that introducing abstract types
into the language is beneficial. Until now C has a number of
simple data types, basically numbers and chars, arrays of
them, plus structures that allow you to make up abstract data
types. And, in my opinion, that's a good deal of what makes C
such a nice and relatively simple language. Adding abstract
data types would, again in my opinion, go against the grain
of what makes C what it is.

You never use a list? A flexible array? A hash table?
Come on...

For instance

stdlist *directory_stat(char *specs);

Wouldn't it be nice to have standard interfaces for those
things?


Thanks for your input.

 

[jameskuyper]

....
... And having a one million user base
seems to me to be pretty impressive!

Having one million downloads is not quite the same as having one
million users. Many people might have downloaded it, taken a look at
it, decided they weren't interested, and discarded it. That is not
intended as a comment about the quality of lcc-win; that's an issue
with all freely downloadable software.

When I installed my Linux system, it came with the option of
installing compilers for several different computer languages, include
a few I'd never even heard of, must less used. I installed them all,
on the off-chance that I might someday want to learn one or more of
those languages, and the system regularly downloads updates as they
become available, but I've still never made any use of most of those
compilers.

Also, some people might be sharing a single download, while other
users have probably made multiple downloads to keep up-to-date on the
latest release - but those two effects might tend to cancel each other
out.

 

[CBFalconer]

.... snip ...

In defence of the C version, that included an implementation of a
line-reading function fggets() that really has no business being
in the script; it should be buried in a library somewhere. That
reduces the line count by a third.

It's available in suitable form, in public domain. See:

<http://cbfalconer.home.att.net/download/ggets.zip>

 

[jacob navia]

I suspect the industry primarily use other criterias.

For instance Microsoft, a small company in Seattle.

They are trying to improve the safety of C with their
proposal of a safe library

For instance gcc with their warnings about gets() when
linking

The trend towards more security in software is obvious to anyone
but the people that do not want to see it.

The argumentation of Gwyn defending gets() and the argumentation
of committee members defending the buffer overflow in asctime()
goes against all proposals that have been done by the industry
in the last years.

 

[Richard Bos]

For instance Microsoft, a small company in Seattle.

Redmond, actually.

They are trying to improve the safety of C with their
proposal of a safe library

You *can* *not* be serious.

Microsoft and safety go together like the English and sexiness, the
French and accuracy, or the Dutch and friendliness: if they coincide,
that is entirely by accident, and not by design.

Richard

 

[jameskuyper]

Richard Bos wrote:
....

Microsoft and safety go together like ... the Dutch and friendliness: if they coincide,
that is entirely by accident, and not by design.

I spent a week in Amsterdam a few years ago. From what I saw, if what
you're saying is correct, then they must have been having an awful lot
of accidents.

 

[jacob navia]

First, I doubt removing gets() would make C more "interesting".

At least would eliminate one bug in the language.

Then, my opinion is that you misread what they were arguing. If I
remember correctly, they agree that there is a problem with the
interface of the function, but once the function was recognized as an
existing practice and moved to C89, now, after two decades passed, there
is almost nothing to do with it.

This point of view (no change can be done to the language since
there are already programs that use that interface) is another way
of freezing any development of C.

Since you included comp.std.c in the discussion I should recall the
basis of my argumentation for the people that did not read the
context.

The C library is full of badly specificated functions like gets()
or asctime(). Moreover it fails to specify error conditions. That is
why it needs an important work to be done with it. A start was
done with the Technical Report that Microsoft proposed. Even if
many points in that document can be discussed, it represents a huge
step forward in the specifications of the library.

It proves that the industry *IS* interested in developing better
functions in the C standard library, and is a good start since
all functions have an exhaustive ERROR ANALYSIS.

Again, if I remember correctly, the only thing you said you would expect
from removing gets() ASAP is that that would help to avoid gets() in new
programs.

Yes. And that is an important point.

However, having in mind the implementations that care about
such things already provide this kind of help and most implementors
still have reasons to continue supply gets() whatever a subsequent
standard says, that seem to be a weak proposition, with no sensible
benefits, but a (yet another) contradiction with how the language used.

No, since gets() would no be part of the language. Anyone a beginner can
write gets() in a matter of a few minutes. There is absolutely no
sophisticated algorithm, no special problems in writing such a thing

Finally, the extensions from Microsoft you referred are published as a
TR and thus, strictly speaking, are not part of the language.

I never said they were. I am arguing precisely that we use the way those
functions are specified as a guideline to ensure that we have secure
counterparts to the traditional functions in the language.

 

[Richard Bos]

Richard Bos wrote:
...

I spent a week in Amsterdam a few years ago. From what I saw, if what
you're saying is correct, then they must have been having an awful lot
of accidents.

Was money involved? The Dutch are quite willing to feign friendliness if
we see money coming at the end of it.

Richard

 

[jameskuyper]

Was money involved? The Dutch are quite willing to feign friendliness if
we see money coming at the end of it.

Well, I was a tourist . However, friendliness feigned in the hopes
of gaining money is most definitely "by design". Some nationalities (I
won't be as specific as you) have a reputation of not even bothering
to feign it, not even for tourists.

 

[Joachim Schmitz]

Was money involved? The Dutch are quite willing to feign friendliness
if we see money coming at the end of it.

You're drawing a far to bad picture of your people.
I know quite a few friendly dutch.

Bye, Jojo

 

[Kenny McCormack]

On Tue, 31 Mar 2009 18:07:02 +0200, jacob navia

[snip C code]

Below is a 33 line Lakota script. (Lakota is a scripting
language that I developed ~15 years ago.) I would like to make a
few points. First of all I opine that it is clearer and more
readable than your C code - even if you have never seen Lakota
code before. Secondly, it is more concise by a factor of three.
Thirdly, it doesn't require compilation. Fourthly, it is free of
a number of limitation and hacks in the C code.

You have to admire the hights of hilarity that we have reached here.

Here you (and other 'regs') are bashing Jacob for posting C code in
comp.lang.c (!!! - let the hilarity of that sink in for a bit!).
Worse than that, you are yourselves posting totally off-topic gibberish
from various other languages. How can you do that?

Jacob is doing exactly the right thing (by the reg's light, a group in
which, I assume you would count yourself) in suggesting and posting a
C-language solution to his general problem. If Jacob had posted his
idea in any other language, the regs would be blasting him from here to
next Thursday. When in Rome, ya know...?

Another example of the double standard at work.

 

[jacob navia]

The point of view is not just that there are programs that use gets(),
but that if the function is removed right now, it will result into
nothing except there will be yet another point where the Standard
doesn't reflect the existing practice, which is a much more serious
thing than omitting special support for writing new programs.

Microsoft declared it deprecated and their compiler will warn about
its usage

Gcc has done the same

And you tell us that eliminating it would make the standard
not reflect the existing practice.

OK. I can see now your logic. You have convinced me.

KEEP GETS().

As you say:

There is no point in discussing this further.

 

[Ian Collins]

The point of view is not just that there are programs that use gets(),
but that if the function is removed right now, it will result into
nothing except there will be yet another point where the Standard
doesn't reflect the existing practice, which is a much more serious
thing than omitting special support for writing new programs.

Do what? Who's existing practice is to use gets()?

 

[TonyMc]

Do what? Who's existing practice is to use gets()?

Yes, and a sonick screwdriver.

Tony