terminal designed for C

[James Kuyper]

On 10/25/2012 12:09 AM, kerravon wrote:
....

I also raised a "please explain what you were smoking" to
the X3J11 committee when I saw them write "there is no
escape in EBCDIC", but I think the letter might have got
lost as I still haven't received a reply.

It's been explained to you that while there is a character named ESC in
EBCDIC, almost the only thing it shares with the ASCII ESC is the
spelling of it's name. If I've followed the discussion correctly, it's
barely even used, and not for any of the same purposes as the ASCII ESC.

I doubt that you'll ever receive a fuller explanation than that. I doubt
that there is a fuller explanation to be given. If that were in fact the
case, what kind of reply, beyond what you've already been given, would
be both truthful and satisfying to you?

 

[Eric Sosman]

[...]
I also raised a "please explain what you were smoking" to
the X3J11 committee

Does that committee still exist? It's mentioned nowhere
in or on ISO/IEC 9899:201x (admittedly not the actual Standard,
but reputed to be pretty close), and a search on ANSI's own
web site turns up "No Results." How did you contact them?
Was there a seance?

when I saw them write "there is no
escape in EBCDIC",

Where did X3J11, or any other C-involved Committee, write
any such thing?

but I think the letter might have got
lost as I still haven't received a reply.

You'll probably wait a long time for a reply from the
Committee Afterlife, especially for a reply to someone who
starts from contrary-to-fact claims.

 

[kerravon]

Where did X3J11, or any other C-involved Committee, write
any such thing?

The C89 Committee considered proposals to add the character constant '\e' to represent the
ASCII ESC ('\033') character. This proposal was based upon the use of ESC as the initial
character of most control sequences in common terminal driving disciplines such as ANSI
X3.64. However, this usage has no obvious counterpart in other popular character codes such as
EBCDIC.


That's where they wrote it - in the ANSI rationale.

As to it actually being used to drive terminals - why
on earth would that be necessary? Just because a
PANSY terminal has not been created in real life
(because IBM quite correctly decided that there
would be less CPU impact if terminals operated in
block mode) doesn't mean that a PANSY terminal
couldn't have been created the following week.
This has nothing to do with EBCDIC not having an
escape. It was purely because only mainframes
were using EBCDIC, and it so happens that only
mainframes were serious enough machines that they
didn't drive terminals directly like that.

In addition, the standard full-screen TPUT begins
with hex 02,27,ccw,wcc,orders/text...
where the 27 is an escape!


BFN. Paul.

 

[Eric Sosman]

The C89 Committee considered proposals to add the character constant '\e' to represent the
ASCII ESC ('\033') character. This proposal was based upon the use of ESC as the initial
character of most control sequences in common terminal driving disciplines such as ANSI
X3.64. However, this usage has no obvious counterpart in other popular character codes such as
EBCDIC.

That's where they wrote it - in the ANSI rationale.

Yes, I know about the Rationale: I quoted this exact same
passage in a reply to you a couple days ago. But where does
this say "there is no escape in EBCDIC?"

 

[kerravon]

Yes, I know about the Rationale: I quoted this exact same
passage in a reply to you a couple days ago. But where does
this say "there is no escape in EBCDIC?"

They didn't use those exact words, but that is what
is implied. Perhaps you can use your own words,
speaking on behalf of a sane X3J11, why \e was not
provided as part of C89?

Thanks. Paul.

 

[Ben Bacarisse]

They didn't use those exact words, but that is what
is implied.

Maybe, in the heat of your determination that something is very wrong
here, you just misread the paragraph? It says that it is "this usage"
not "this character" that has not obvious counterpart in EBCDIC.

Now that may be wrong, but it is certainly consistent with what little I
remember about how IBM terminal equipment was driven. Maybe there is a
document you can point to that shows EBCDIC ESC (or, indeed, some other
character) being used as a control sequence prefix?

If you read section 5.2.2 of the C standard, it's clear that the
committee wants to reserve \X sequences for characters that serve some
clearly definable purpose. I am not sure that even ASCII ESC fits this
limited criterion since its role is rather vague, but that alone did not
seem to the main reason for rejecting \e.

Perhaps you can use your own words,
speaking on behalf of a sane X3J11, why \e was not
provided as part of C89?

I don't see why this is such a big issue. Porting a display editor from
one environment to another is going to involve a considerable amount
re-writing. Having to define the value of ESC is going to be the most
trivial part of it, is it not?

 

[James Kuyper]

They didn't use those exact words, but that is what
is implied.

No, that is not what is implied. They did not say that there is no
escape in EBCDIC. They were talking about "this usage", referring to
"use of ESC as the initial character of most control sequences in common
terminal driving disciplines", when they said that it "has no obvious
counterpart in ... EBCDIC."

Everyone here who has claimed more experience than I have in EBCDIC
(which is a trivial requirement to meet) has said that the EBCDIC ESC
character is NOT so used. Could you prove them wrong by identifying any
"common terminal driving discipline" where "most control sequences"
have EBCDIC ESC as "the initial character"? You've talked about a
discipline for a hypothetical terminal which doesn't actually exist yet,
but a non-existent one could hardly qualify as "common", could it?
Standards are written primarily to standardize existing practice, so
whether or not a practice is "common", or even "existing" is certainly
one of the things the committee is supposed to take into consideration
when making its decisions.

 

[Keith Thompson]

They didn't use those exact words, but that is what
is implied. Perhaps you can use your own words,
speaking on behalf of a sane X3J11, why \e was not
provided as part of C89?

Ben and James have already pointed out that that's not what the
rationale says or implies.

I understand you'd like the language to support '\e' (I wouldn't
mind it myself, and there's precedent in Perl). But the fact is that
it doesn't. The reasons have already been explained to some extent.
Would understanding the reasons in a bit more detail really help
you in any way?

My suggestion: Consider the amount of effort that would be
required to add a '\e' escape to the C standard, for *all* compiler
implementers to implement it, and for the change to be tested and
deployed widely enough for you to use it. Compare this to the
amount of effort required to write:

#ifdef EBCDIC
#define ESCAPE '\x27'
#else
#define ESCAPE '\x1b'
#endif

Ok, I know that the difficulty of typing those lines is not really
the issue; you're more concerned with being able to express the
escape character portably. But IMHO it's really not that big a deal.
If you want a language in which all concepts can be expressed 100%
portably, you're looking for something other than C.

 

[kerravon]

If you want a language in which all concepts can be expressed 100%
portably, you're looking for something other than C.

It's more I'm asking the question - what concepts are
inherently portable, and given that, can I express
that in C? E.g. one of the things that drove me to
work on GCCMVS for years was the basic principle that
at its heart, GCC took in a bunch of text files (C
code), and output another text file (assembler code),
and something is very wrong if text file to text file
is not portable, ie won't run on an EBCDIC mainframe.

BFN. Paul.

 

[kerravon]

Maybe, in the heat of your determination that something is very wrong
here, you just misread the paragraph?

I don't think so. I've read it multiple times and I'm
a native English speaker, and it implies to me that
they're looking for an escape character in EBCDIC that
would do the equivalent of the ASCII version.

It says that it is "this usage"
not "this character" that has not obvious counterpart in EBCDIC.

Now that may be wrong, but it is certainly consistent with what little I
remember about how IBM terminal equipment was driven. Maybe there is a
document you can point to that shows EBCDIC ESC (or, indeed, some other
character) being used as a control sequence prefix?

1. I don't think it should be necessary for equipment
marketted by IBM to actually *use* the clearly-defined
control character. The actual *use* of \e is determined
by some coincidental marketing factors such as:

A. Is EBCDIC used on mainframes or PCs?
B. Are character-mode terminals efficient use of scarce
mainframe cycles?

2. Here's what a mainframe expert said:

If you look at BTAM, TCAM, or VTAM, you'll note that
many terminal types use escape. In particular, the standard
full-screen TPUT begins with hex 02,27,ccw,wcc,orders/text...
where the 27 is an escape!

3. Here's what the IBM manual says:

http://publibz.boulder.ibm.com/cgi-.../IKJ2B703/10.2?SHELF=&DT=19930804072754&CASE=

1. If the first character in your message is an escape control
character (X'27'), the two characters following it are treated as
a command code and as a write control character by the 3270. Note
that the command code should always be for a remote 3270. If
necessary, TPUT will convert the code to that for a local 3270.
If the first character is not an escape character, a default write
command and a write control character are added to the beginning
of the message. Any attachment-dependent characters required for
correct transmission of the data stream are provided by the access
method.

I don't see why this is such a big issue. Porting a display editor from
one environment to another is going to involve a considerable amount
re-writing. Having to define the value of ESC is going to be the most
trivial part of it, is it not?

If micro-emacs cannot be written portably, for some
yet-to-be-explained technical reason, then my Plan
B is to write a replacement for micro-emacs that can
theoretically be ported, even if I need to change/
invent new hardware and change every operating system
to cope with my use of the C language.

I note that the MSDOS "edlin" program appears to be
the sort of thing that is portable. So why not an
editor that is better that edlin? Why is edlin the
limit?

BFN. Paul.

 

[Keith Thompson]

If micro-emacs cannot be written portably, for some
yet-to-be-explained technical reason, then my Plan
B is to write a replacement for micro-emacs that can
theoretically be ported, even if I need to change/
invent new hardware and change every operating system
to cope with my use of the C language.

Or you can write your C code so that it can be configured to use
either the ASCII escape character or the EBCDIC escape character.
It may not be an ideal solution, but I'm at a loss to understand
why you think inventing new hardware is better.

I note that the MSDOS "edlin" program appears to be
the sort of thing that is portable. So why not an
editor that is better that edlin? Why is edlin the
limit?

Micro-emacs requires the ability to read single characters from
the keyboard without waiting for an end-of-line character (unless
you radically change the user experience). That cannot be done
in 100% portable C (though it's typically straightforward to do in
non-portable C).

 

[kerravon]

Or you can write your C code so that it can be configured to use
either the ASCII escape character or the EBCDIC escape character.
It may not be an ideal solution, but I'm at a loss to understand
why you think inventing new hardware is better.

I am transfixed with the C90 standard, and expect the
world to revolve around that, than vice-versa.

Micro-emacs requires the ability to read single characters from
the keyboard without waiting for an end-of-line character (unless
you radically change the user experience). That cannot be done
in 100% portable C (though it's typically straightforward to do in
non-portable C).

Instead of getting carried away with saying that the
C90 standard can be interpreted differently by different
operating systems, let's instead posit that the
operating systems will instead interpret the C90
standard the same way - with everyone interested in
the purity of the C90 standard.

Under those circumstances, I am quite happy to define
stdin and stdout as _IONBF which should get the
desired character-at-a-time return.

The fact that existing terminal hardware needs to be
replaced in order for this to work is to me a minor
concern. My major concern is whether I can code
according to the C90 specs and have a DECENT operating
system do something DECENT.

BFN. Paul.

 

[Ian Collins]

The fact that existing terminal hardware needs to be
replaced in order for this to work is to me a minor
concern. My major concern is whether I can code
according to the C90 specs and have a DECENT operating
system do something DECENT.

I would expect a decent contemporary operating system to follow C99...

 

[kerravon]

I would expect a decent contemporary operating system to follow C99...

In order to do C99, C90 has to be covered as well. I'll
settle for the universality of decent interpretations
of C90.

BFN. Paul.

 

[Ben Bacarisse]

I don't think so. I've read it multiple times and I'm
a native English speaker, and it implies to me that
they're looking for an escape character in EBCDIC that
would do the equivalent of the ASCII version.

Right. We are both reading the rationale the same way now. This is not
the same as "there is no escape in EBCDIC".

1. I don't think it should be necessary for equipment
marketted by IBM to actually *use* the clearly-defined
control character. The actual *use* of \e is determined
by some coincidental marketing factors such as:

A. Is EBCDIC used on mainframes or PCs?
B. Are character-mode terminals efficient use of scarce
mainframe cycles?

2. Here's what a mainframe expert said:

If you look at BTAM, TCAM, or VTAM, you'll note that
many terminal types use escape. In particular, the standard
full-screen TPUT begins with hex 02,27,ccw,wcc,orders/text...
where the 27 is an escape!

3. Here's what the IBM manual says:

http://publibz.boulder.ibm.com/cgi-.../IKJ2B703/10.2?SHELF=&DT=19930804072754&CASE=

1. If the first character in your message is an escape control
character (X'27'), the two characters following it are treated as
a command code and as a write control character by the 3270. Note
that the command code should always be for a remote 3270. If
necessary, TPUT will convert the code to that for a local 3270.
If the first character is not an escape character, a default write
command and a write control character are added to the beginning
of the message. Any attachment-dependent characters required for
correct transmission of the data stream are provided by the access
method.

That seems a little too specific to be classed as a similar usage, but I
don't have the knowledge to evaluate it properly. If (and I think it's
a big if) this usage is almost universal among EBCDIC display devices
then I'd agree that the committee could have treated the two escapes as
having similar usage.

If micro-emacs cannot be written portably, for some
yet-to-be-explained technical reason, then my Plan
B is to write a replacement for micro-emacs that can
theoretically be ported, even if I need to change/
invent new hardware and change every operating system
to cope with my use of the C language.

That's a very odd statement. My notion of portable software is software
that does not require new hardware and re-written operating systems to
run. In what sense would your editor be portable if I could not run it
without installing your changed OS and your newly invented hardware?

I note that the MSDOS "edlin" program appears to be
the sort of thing that is portable. So why not an
editor that is better that edlin? Why is edlin the
limit?

It isn't any kind of limit. For one thing edlin is not universally
portable -- I doubt it would have been possible to get anything even
remotely similar to run on the IBM RAX system that I first used.

Portability is a relative property not an absolute one. How portable do
you want your software to be[1] and to what systems? Once you have that
pinned down you can investigate the lowest common denominator of those
systems to see what you have to work with.

As Keith has pointed out, C alone does not give you enough to get very
far at all. Just getting un-buffered input requires more that C
provides, and controlling all the different terminal hardware out there
is well nigh impossible.

[1] By "how portable do you want your software to be" I mean, for
example, do you want exactly the same source to build and run on all
your targets, or will you accept some small effort of conversion? Will
you accept different build systems provided the source is unchanged?
and so on.

 

[kerravon]

That's a very odd statement. My notion of portable software is software
that does not require new hardware and re-written operating systems to
run. In what sense would your editor be portable if I could not run it
without installing your changed OS and your newly invented hardware?

Good question. I want to posit the existence of a
rectangular screen that follows the ANSI X3.64
specs, adjusted for character set differences. That
ties down the screen hardware. In order to follow
the ANSI X3.64 specs I need the operating system
to be able to address the terminal appropriately.
That ties down the operating system.

Other than that, the C90 language gives the flexibility
to address such a real/theoretical terminal. It does
indeed allow unbuffered input/output. I consider it to
be the operating system's job to respond to the C90
request for unbuffered input/output. I only expect my
software to be portable to hardware that meets the
X3.64 standard and operating systems that respond
logically to the unbuffered input/output requests.
I don't expect my source code to change.

Thankyou for helping me to pin down my unstated assumptions.

BFN. Paul.

 

[James Kuyper]

Actually, that's not something that's necessarily up to the OS, though
some OS's do make a C compiler one of the standard utilities.

In order to do C99, C90 has to be covered as well.

No, that isn't the case. A C99 compiler need not conform to C90, and
need not even have a option to do so. It may even be impossible to
conform to both simultaneously, though I'd have to think about that for
a while to be sure.

 

[Ben Bacarisse]

Good question. I want to posit the existence of a
rectangular screen that follows the ANSI X3.64
specs, adjusted for character set differences. That
ties down the screen hardware. In order to follow
the ANSI X3.64 specs I need the operating system
to be able to address the terminal appropriately.
That ties down the operating system.

That's got very little to do with the OS -- all it has to do is send a
character stream to the terminal. You'll probably be *removing*
portability from MircoEmacs to do this since I imagine it can drive
other terminal types as well as ANSI X3.64 ones.

Will you consider terminal emulators? These may have curious properties
such as the ability to change size during program execution.

Other than that, the C90 language gives the flexibility
to address such a real/theoretical terminal. It does
indeed allow unbuffered input/output.

Not in the way you want. You can set some stream-level buffering but
you can't get at the line-buffering that most OSes put between your
keyboard and the program. How easy or hard it is to alter this will
vary from system to system.

Once you've done that you need to consider what the OS gives you. It
might simply provide raw scan codes or it may give you something
higher-level like a stream of processed bytes (maybe even processed
characters -- e.g. turning dead-key combinations into accented
characters). Whatever it does, MicroEmacs will probably need much more
than a plain character stream. Full Emacs needs to be able to tell what
modifiers are pressed so you can distinguish between ctrl-x and
ctrl-alt-x and so on. MicroEmacs is probably similar.

I consider it to
be the operating system's job to respond to the C90
request for unbuffered input/output. I only expect my
software to be portable to hardware that meets the
X3.64 standard and operating systems that respond
logically to the unbuffered input/output requests.

X3.64 does not address the input side.

I don't expect my source code to change.
Thankyou for helping me to pin down my unstated assumptions.

There are some file system issues as well. C's file handling is very
basic so most editors will use OS features directly.

 

[Keith Thompson]

Actually, that's not something that's necessarily up to the OS, though
some OS's do make a C compiler one of the standard utilities.


No, that isn't the case. A C99 compiler need not conform to C90, and
need not even have a option to do so. It may even be impossible to
conform to both simultaneously, though I'd have to think about that for
a while to be sure.

Hmm. I think it would be possible for a compiler to conform to both C90
and C99, though it would be ugly. (The following is of no particular
practical use or relevance to the current discussion.)

For example, this:

int main(void) {
int restrict;
return 0;
}

is a strictly conforming C90 program, but a syntax error in C99.
But the standard only requires a diagnostic for a syntax error
or constraint violation; the compiler could issue a diagnostic
and then translate the program as if it were a C90 compiler.
(Extra diagnostics are permitted.) It could behave similarly in
the other direction for things like implicit int.

It would have to define __STDC_VERSION__ as 199901L for C99
conformance; C90 doesn't explicitly forbid that, since it doesn't
mention __STDC_VERSION__. (This makes a compiler that conforms to both
C99 and C11 impossible, I think.)

I hope nobody actually does this.

 

[James Kuyper]

Hmm. I think it would be possible for a compiler to conform to both C90
and C99, though it would be ugly. (The following is of no particular
practical use or relevance to the current discussion.)

For example, this:

int main(void) {
int restrict;
return 0;
}

is a strictly conforming C90 program, but a syntax error in C99.
But the standard only requires a diagnostic for a syntax error
or constraint violation; the compiler could issue a diagnostic
and then translate the program as if it were a C90 compiler.
(Extra diagnostics are permitted.) It could behave similarly in
the other direction for things like implicit int.

For each version of the standard, you can say one of the following about
any particular piece of code:
1. The range of permitted behavior for the code is specified by the
standard; as a special case of this, there might be only one permitted
behavior.
2. The standard mandates the issuance of at least one diagnostic.
3. The behavior of the code is undefined by the standard.
4. The standard prohibits successful translation of the program.

As far as I can remember (never having had an official copy of the C90
standard), most of the differences between C90 and C99 shifted code from
one of those categories to another. Most of those categories are
compatible with each other, except that category 1 is incompatible with
category 4. There's only one construct in category 4: a #error directive
that survives conditional compilation. That was not in category 1 for
C90, so the fact that it's in category 4 for C99 does not prevent
simultaneous conformance.

There's only one other thing that could prevent simultaneous
conformance: behavior that is in category 1 according to both standards,
but where the range of behavior permitted by C90 doesn't overlap the
range permitted by C99. For C99 and C2011, one such case is the one you
mentioned, involving the value of __STDC_VERSION__. I couldn't come up
with any examples of such a difference between C90 and C99 - but I don't
feel confident in claiming that there are no such differences.