Need suggestions for C links.

[Chris Hills]

On Sun, 26 Feb 2006 07:48:36 +0000, Chris Hills wrote:
(e-mail address removed)> writes
[...]
You can get a PDF copy of the C standard for about $18 (I think the FAQ
has links). Or you can get a free copy of n1124.pdf (Google it), which
incorporates the entire C99 standard plus TC1 and TC1.

That's assuming you want C99. I think you can also find drafts of the
C89/C90 standard; I'm not certain that the C90 standard itself is still
available.

It is but only in hard copy at around 30 GAP or 50 US.

It's also listed for soft copy purchase as a foreign-ratified Standard
in a few places, some collected on the wiki at:
<http://clc-wiki.net/wiki/Basics_Of_The_C_Standard#Obtaining_the_Standard>.

I don't know whether any of those sources are actually "in stock", but if
not, their websites are out of date.

I stock the hard copy version

Oh, I didn't twig from your previous post that you actually sell it.

Hi,

I *used to* sell it. It appears that all I was doing was using up BSI's
old stock. There is no stock left :-(

The only other version they have is 90+ GBP!!!! Around 150 USD. Not
good.

I am working on a different solution for reintroducing C90+A1+the TC's
at a sensible price of 20-30 GBP (around 30-50 USD)

Unfortunately there are no electronic versions... well that is not
strictly true. The one I have is over 28megabytes as each page was
scanned in as a tiff and is a graphic. Not easy to cut and past or
search :-(

Do
you have a link that could be added to the wiki page? I searched
through your site a bit without turning one up, bar an invitation to
email you for a copy in your column vol 11.7.

That was some time ago :-( A pity BSI can't get any more.

I also noticed that in that column you mention the decline of compiler
certification due to expensiveness. There's recently been suggestion in
c.s.c that a test suite for compilers be part of the C Standard as it is
for Ada's Standard, to avoid the perceived meaninglessness of the "one
program" requirement of 5.2.4 - is that something you'd advocate?

Not at all. There is a hell of a lot more to it than just a test suite
and test suites are very expensive to do properly. The Ada one was paid
fro AFAIK by the US government (who don't have to show a profit on it)

 

[Kevin Bagust]

[...] Hi,

I *used to* sell it. It appears that all I was doing was using up
BSI's old stock. There is no stock left :-(

The only other version they have is 90+ GBP!!!! Around 150 USD. Not
good.

I am working on a different solution for reintroducing C90+A1+the
TC's at a sensible price of 20-30 GBP (around 30-50 USD)

Unfortunately there are no electronic versions... well that is not
strictly true. The one I have is over 28megabytes as each page was
scanned in as a tiff and is a graphic. Not easy to cut and past or
search :-(

If that is the same version that I have a copy of it actually has both
the tiffs and the text but with the text hidden. But that version is
only 13Mb.It is possible to search or cut and paste it.

Kevin.

 

[Netocrat]

]

Hi,

I *used to* sell [the hardcopy C90 version of the Standard]. It appears
that all I was doing was using up BSI's old stock. There is no stock
left :-(

The only other version they have is 90+ GBP!!!! Around 150 USD. Not
good.

I am working on a different solution for reintroducing C90+A1+the TC's
at a sensible price of 20-30 GBP (around 30-50 USD)

Unfortunately there are no electronic versions... well that is not
strictly true. The one I have is over 28megabytes as each page was
scanned in as a tiff and is a graphic. Not easy to cut and past or
search :-(

It's ironic then that C90 is apparently available through ANSI as
EN 29899:1993 - in soft-copy. It's listed there as an 8.34Mb download, so
it seems more likely to be in a searchable and navigable format such as
pdf. It's quite a bit more expensive than any of the options you've
listed above, although some of the other links collected on the wiki page
that I posted previously are more reasonable (the deal you're working
towards would be the best of the lot).

Incidentally, amendment 1 is also listed as available in soft-copy through
ANSI's store as EN 29899:1993/A1:1996; I haven't checked the availability
of the TCs on that site but they are available in soft-copy elsewhere.

[...]

Not at all. There is a hell of a lot more to it than just a test suite
and test suites are very expensive to do properly. The Ada one was paid
fro AFAIK by the US government (who don't have to show a profit on it)

When you write that there's a lot more to "it" than a test suite, which
perspective are you coming from - embedded or in general?

 

[Mark McIntyre]

A parallel html version would be nice for the uses that _don't_ involve
printing, though. Maybe a multi-file version like most html
documentation using docbook or info2html or anything else uses.

I hesitate to point out that it took me around ten mins to generate an
html version, though admittedly the formatting was curious. I guess
ISO didn't want to pay to have it re-typeset more accurately.

And for those wanting plain-text, its worth noting that Acrobat 7.07
actually has an option to save in that format.

And we think we're good with computers...
Mark McIntyre

 

[Jordan Abel]

When you write that there's a lot more to "it" than a test suite, which
perspective are you coming from - embedded or in general?

It's a lot easier, I'm sure, to write a test suite to prove an
implementation incorrect than to prove it correct. The question is
whether this is worth doing.

Also, a test suite could not easily be automated, since, what are you
going to write it in, Standard C? So you're left with feeding the
programs in and examining the output by hand.

 

[Chris Hills]

On 4 Mar 2006 01:37:59 GMT, in comp.lang.c , Jordan Abel

I hesitate to point out that it took me around ten mins to generate an
html version, though admittedly the formatting was curious. I guess
ISO didn't want to pay to have it re-typeset more accurately.

ISO are not expecting people to turn it into HTML.

And for those wanting plain-text, its worth noting that Acrobat 7.07
actually has an option to save in that format.

only if the PDF file has the security set to permit it.

 

[Keith Thompson]

It's a lot easier, I'm sure, to write a test suite to prove an
implementation incorrect than to prove it correct. The question is
whether this is worth doing.

I suspect it's impossible to write a test suite to prove an
implementation correct.

Based on my experience with Ada, having a de-facto standard and freely
available test suite is extremely valuable. Of course developing it
would be expensive. Then again, developing a working C implementation
is also expensive, and a number of those are freely available. On the
other other other hand, it's probably harder to find skilled people
willing to donate their time developing a test suite than to find
people willing to develop a compiler. As a practical matter,
*someone* would have to pay for it (in time or money).

I encourage someone else to invest the time and money to develop this
so I can sit back, be lazy, and reap the benefits without doing any of
the work.

Also, a test suite could not easily be automated, since, what are you
going to write it in, Standard C? So you're left with feeding the
programs in and examining the output by hand.

The tests themselves would be in standard C. There's no good portable
way to invoke a C compiler in standard C, though I suppose with
judicious use of system() and a configuration file you might throw
something together. But developing a (system-specific) framework to
invoke the tests and check their output wouldn't be too difficult. If
the tests are properly written, there's no need to manually run the
tests or examine the output.

The Ada validation test suite (formerly called ACVC, now called ACATS)
contains three classes of tests. Some are just required to compile
without error (warnings are ok). Some contain deliberate errors that
the compiler is required to detect, and are required *not* to compile
(Ada has stricter rules than C about programs with errors). And some
are required to compile and execute correctly. Each test in the last
category checks internally for the correct behavior, and prints a
formatted "PASSED" or "FAILED" message as its last line of output; the
test passes if and only if the "PASSED" line appears.

Anyone interested in doing something like this for C should take a
look at <http://www.ada-auth.org/acats.html>.

 

[Keith Thompson]

Unfortunately there are no electronic versions... well that is not
strictly true. The one I have is over 28megabytes as each page was
scanned in as a tiff and is a graphic. Not easy to cut and past or
search :-(

(Referring to the C89/C90 standard.)

I have an electronic copy of the C90 standard. I paid $18 to NA for
it. It looks like it was scanned from a paper document, but it is
possible to search and copy-and-paste from it -- but the result of
copy-and-pasting often has small errors. Apparently somebody ran some
kind of optical character recognition on the scanned copy.

For example, here's a copy-and-paste from the document (reformatted
for line length but

In a freestanding environment (in which C program execution may
take place without any benetit of an operating system), the name
and type of the function called at program startup are
implementation-detined There are otherwise no reserved external
identitiers. Any library tacilities available to a freestanding
propram are implementation-defined.


Note the misspellings "detined", "identitiers", and "tacilities, and
the missing '.' after the first sentence, all things visually similar
to the correct text.

The file size is 13624364 bytes 13 MBytes); the name is
"ansi+iso+9899-1990+(r1997).pdf". I don't know whether it's still
available.

 

[Chris Hills]

I suspect it's impossible to write a test suite to prove an
implementation correct.

Define correct

Based on my experience with Ada, having a de-facto standard and freely
available test suite is extremely valuable. Of course developing it
would be expensive.

it was VERY expensive but the US government paid for it.

Then again, developing a working C implementation
is also expensive, and a number of those are freely available.

There is a vast difference between developing a C implementation and
doing a validation suite. In any event several exist but they are VERY
expensive.

On the
other other other hand, it's probably harder to find skilled people
willing to donate their time developing a test suite than to find
people willing to develop a compiler. As a practical matter,
*someone* would have to pay for it (in time or money).

exactly there is no such thing as a free lunch. Also there is a LOT more
to it than the time developing. AFAIK the people who currently do
validations suites are also major players on the ISO C panels and have
been for some time....

Look to a minimum of 5 days a year national meetings and 10 days a year
on the international meetings. At least one block of 5 days will be in
Europe or Australia etc. SO there is a lot of time (and travel)
involved there apart from the continual email discussions....

SO that is about three weeks working on standards (half of which are
abroad) before you start to write the validation suite.

Still it saves you having to spend 18USD on a PDF for the standard

I encourage someone else to invest the time and money to develop this
so I can sit back, be lazy, and reap the benefits without doing any of
the work.

Plum Hall and Perennial have already done it.... ask them for a free
copy

The tests themselves would be in standard C.

So all the compilers will fail. 100% of them miss something out of C99
and 100% of them have extensions.

There's no good portable
way to invoke a C compiler in standard C, though I suppose with
judicious use of system()

!!!! System!??? that is a bad starting point!

and a configuration file you might throw
something together. But developing a (system-specific) framework to
invoke the tests and check their output wouldn't be too difficult. If
the tests are properly written, there's no need to manually run the
tests or examine the output.

A lot of ifs (and hard work) there....

The Ada validation test suite (formerly called ACVC, now called ACATS)
contains three classes of tests. Some are just required to compile
without error (warnings are ok). Some contain deliberate errors that
the compiler is required to detect, and are required *not* to compile
(Ada has stricter rules than C about programs with errors). And some
are required to compile and execute correctly. Each test in the last
category checks internally for the correct behavior, and prints a
formatted "PASSED" or "FAILED" message as its last line of output; the
test passes if and only if the "PASSED" line appears.

Anyone interested in doing something like this for C should take a
look at <http://www.ada-auth.org/acats.html>.

Also ask the US government how many man hours it took to do the Ada
one....

 

[Keith Thompson]

Define correct
Conforming.


it was VERY expensive but the US government paid for it.


There is a vast difference between developing a C implementation and
doing a validation suite. In any event several exist but they are VERY
expensive.

Just to be clear, you're saying that developing a validation suite is
vastly more expensive than developing a C implementation?

[...]

So all the compilers will fail. 100% of them miss something out of C99
and 100% of them have extensions.

Since the standard allows extensions, extensions shouldn't cause any
tests to fail.

If no existing compilers conform to C99, then all compilers will fail
some tests; that's the whole point of a test suite. (Knowing *which*
tests fail could still be useful.)

!!!! System!??? that is a bad starting point!

If you want to invoke a compiler in purely standard C, system() is the
only way to do it. I wasn't suggesting that the framework needs to be
in purely standard C.

A lot of ifs (and hard work) there....
Certainly.


Also ask the US government how many man hours it took to do the Ada
one....

A lot, presumably.

 

[Chris Hills]

[attributions unwrapped]

Jordan Abel <[email protected]> writes:

exactly there is no such thing as a free lunch. Also there is a LOT more
to it than the time developing. AFAIK the people who currently do
validations suites are also major players on the ISO C panels and have
been for some time [...] SO that is about three weeks working on
standards (half of which are abroad) before you start to write the
validation suite.

You seem to be suggesting that a suite written for inclusion by someone(s)
not directly involved in the Standard would be ill-received by WG14. If
that's the case, surely sponsored (by oversight not money) development
would be possible.

I am suggesting that you would really need more than just the sight of
the standards to do a validations suite. Ie the DRs and TRs and the
internal discussions on what the hell was really meant by various
clauses.

You don't have to be a member of the ISO C panel to do a C validation
suite but you probably would for credibility. If you are doing a test
suite you will have lots of queries and the only place many could be
answered with any authority would be on the ISO panels.

There are a few that claim C99 compliance, are you certain that 100% of
compilers miss something?

Yes.

Extensions aren't disallowed by the Standard though, so long as portable
programs compile correctly in standards mode.

Portability is not required by most users. You need reliability in the
smallest economical space.

For freestanding implementations it wouldn't always be available, but
given a reasonable config file it seems a workable approach on hosted
implementations.

There are vastly more free-standing implementations that not. Most
embedded systems are free-standing, use C90+ and extensions for the
architecture these are the systems that require validation usually for
legal and product liability reasons......

You are prepared to defend your validation suite in a court of law?

It's not impossible though.

No, just of VERY little use if it is just standard C.

 

[Chris Hills]

Conforming.

To what exactly?

Just to be clear, you're saying that developing a validation suite is
vastly more expensive than developing a C implementation?

Yes. Well a good one that is worth having.

Since the standard allows extensions, extensions shouldn't cause any
tests to fail.

OK.... the extensions won't but which standard are you expecting them to
conform to?

If no existing compilers conform to C99, then all compilers will fail
some tests; that's the whole point of a test suite. (Knowing *which*
tests fail could still be useful.)

I agree....

If you want to invoke a compiler in purely standard C, system()

No it is not. You just tick the box that says "strict ISO/ANSI C, no
extensions"

A lot, presumably.

I did hear the figure the other day it was very, very large. I can't
remember exactly as it was in passing in a conversation of costs of
doing things. EG MISRA, ADA-validation suite etc

 

[Richard Heathfield]

Chris Hills said:

Define correct

Right!

 

[Netocrat]

[attributions unwrapped]

The tests themselves would be in standard C.
So all the compilers will fail. 100% of them miss something out of C99

There are a few that claim C99 compliance, are you certain that 100% of
compilers miss something?

Yes.

Oh. Greg Comeau has in the past claimed full C99 compliance for his
compiler, and Perennial have certified not only it but several others
according to its website:
http://www.peren.com/pages/cvsa_isocvpl.htm

Are you excluding those compilers?

Extensions aren't disallowed by the Standard though, so long as portable
programs compile correctly in standards mode.

Portability is not required by most users. You need reliability in the
smallest economical space. [...]
There are vastly more free-standing implementations that not. Most
embedded systems are free-standing, use C90+ and extensions for the
architecture these are the systems that require validation usually for
legal and product liability reasons......

OK, so you're saying that "standard" extensions should be the subject of
testing too, for non-"Standard" reasons. I understand (at least somewhat)
now what you meant by there being a lot more to it.

You are prepared to defend your validation suite in a court of law?

If it were part of the Standard, a validation suite wouldn't require legal
defence...

[A validation suite is] of VERY little use if it is just standard C.

....but such a suite is clearly unappealing to you.

OK, Standard-compliance doesn't imply real-world usefulness. A call to
malloc() requesting 1024 bytes, that ends up claiming 20Mb of system
memory and taking 2 minutes to complete isn't prohibited by the Standard,
but that's a quality of implementation issue that I wouldn't expect to be
part of a Standard test suite anyway, except perhaps as an extension.

There seem to be (at least) three ways a compiler could be tested: for
compliance with the Standard, for QOI, and for robustness of its
extensions.

The issue I originally brought up covered only a part of the first type of
testing (arguably a part of QOI too) - a suite incorporated into the
Standard specifically to counter the claim that 5.2.4's "one program"
requirement is meaningless.

 

[CBFalconer]

. SNIP ...

[A validation suite is] of VERY little use if it is just standard C.

...but such a suite is clearly unappealing to you.

OK, Standard-compliance doesn't imply real-world usefulness. A
call to malloc() requesting 1024 bytes, that ends up claiming 20Mb
of system memory and taking 2 minutes to complete isn't prohibited
by the Standard, but that's a quality of implementation issue that
I wouldn't expect to be part of a Standard test suite anyway,
except perhaps as an extension.

There seem to be (at least) three ways a compiler could be tested:
for compliance with the Standard, for QOI, and for robustness of
its extensions.

By definition, extensions are not standardized, and therefore there
is no basis for testing them. If the criterion for robustness is
"can it crash" then we have to reject a large portion of the
standard library. Similarly, QOI is in the eye of the beholder. I
have been criticized for my practice of using NULL as a synonym for
an empty string for input. Some prefer a violent crash. I
consider the practice robustness.

This leaves standards compliance. Just do it.

--
"If you want to post a followup via groups.google.com, don't use
the broken "Reply" link at the bottom of the article. Click on
"show options" at the top of the article, then click on the
"Reply" at the bottom of the article headers." - Keith Thompson
More details at: <http://cfaj.freeshell.org/google/>
Also see <http://www.safalra.com/special/googlegroupsreply/>

 

[Keith Thompson]

If it were part of the Standard, a validation suite wouldn't require legal
defence...

I wouldn't want the suite to be part of the standard. Again, I'm
biased by my experience with the Ada validation suite. That suite is
not part of the standard, and where a test conflicts with the
standard, the standard wins. You also want the test suite to be able
to change much more quickly than the standard itself; whenever a test
is successfully disputed, it has to be modified or withdrawn.

[A validation suite is] of VERY little use if it is just standard C.

In the Ada community, passing the validation suite is considered the
*minimal* criterion for acceptance. Validation doesn't imply good
quality. (Since the Ada language is larger than C, validation against
the standard is probably more significant for Ada than it would be for
C.)

A validation suite is also a valuable tool for compiler developers.
If you introduce an obscure bug, there's a decent chance that you'll
catch it next time you run the validation tests.

 

[Keith Thompson]

To what exactly?

To the standard.

I must be missing something here. It seems obvious to me that the
point of a validation suite would be to test an implementation's
conformance to the language standard. Why do you keep asking what I
mean by correctness?

[...]

OK.... the extensions won't but which standard are you expecting them to
conform to?

Extensions may not violate the language standard (they can't alter the
behavior of any strictly conforming program). That is the extent of
any possible testing for extensions (and, like testing for anything
else, it will be imperfect).

If you're talking about testing that an extension works as intended,
that would have to be up to the vendor or other body that defines the
extension.

[...]

No it is not. You just tick the box that says "strict ISO/ANSI C, no
extensions"

Sorry, I really don't understand your point.

My full comment was:
] If you want to invoke a compiler in purely standard C, system() is the
] only way to do it. I wasn't suggesting that the framework needs to be
] in purely standard C.

I mentioned system() only in passing, as the only way to implement a
testing framework (the system that runs the validation suite and
summarizes the results) *if* you insist on implementing the testing
framework using only standard C.

The hypothetical test suite would consist of a set of tests, provided
as C source files. To use the test suite, you have to attempt to
compile and execute each test and verify that it produces the correct
output (or, for some tests, verify that it produces a required
diagnostic). The manner in which this is done is a side issue, and is
*not* part of the test suite itself.

It might be sensible to provide testing frameworks for specific
operating systems (for Unix, provide the path to the compiler and a
few other parameters, and it runs the suite for you), but as long as
you have *some* way to run the tests, it really doesn't matter how you
do it. Again, there's no particular reason to implement the testing
framework in strictly standard C, or in C at all. (I'd probably use
Perl myself.)

[...]

 

[Chris Hills]

If it were part of the Standard, a validation suite wouldn't require legal
defence...

I wouldn't want the suite to be part of the standard. Again, I'm
biased by my experience with the Ada validation suite. That suite is
not part of the standard, and where a test conflicts with the
standard, the standard wins. You also want the test suite to be able
to change much more quickly than the standard itself; whenever a test
is successfully disputed, it has to be modified or withdrawn.

[A validation suite is] of VERY little use if it is just standard C.

In the Ada community, passing the validation suite is considered the
*minimal* criterion for acceptance. Validation doesn't imply good
quality. (Since the Ada language is larger than C, validation against
the standard is probably more significant for Ada than it would be for
C.)

A validation suite is also a valuable tool for compiler developers.
If you introduce an obscure bug, there's a decent chance that you'll
catch it next time you run the validation tests.

So go out and use the ones that exist already.

 

[Richard Bos]

This is actually VERY undesirable in any formal document. You do NOT
want pages reflowing, diagrams and tables being orphaned, weird
linebreaks or pagebreaks appearing, etc etc when you print them. You
want them to appear EXACTLY as you laid them out. Trust me on this.


Rigidly fixed is precisely what you want with an actual document.

Exactly. And if it doesn't fit on your irregular paper size, you can
always rescale it - keeps all the page numbers, tables and graphics the
same, only a bit smaller or larger.

Richard

 

[CBFalconer]

.... snip ...

So go out and use the ones that exist already.

Which effectively limits you to the gcc test suite which, AFAIK, is
not especially geared to standards compliance nor to generality.
All others are proprietary, again AFAIK.

--
"If you want to post a followup via groups.google.com, don't use
the broken "Reply" link at the bottom of the article. Click on
"show options" at the top of the article, then click on the
"Reply" at the bottom of the article headers." - Keith Thompson
More details at: <http://cfaj.freeshell.org/google/>
Also see <http://www.safalra.com/special/googlegroupsreply/>