What is Expressiveness in a Computer Language

[Xah Lee]

in March, i posted a essay “What is Expressiveness in a Computer
Languageâ€, archived at:
http://xahlee.org/perl-python/what_is_expresiveness.html

I was informed then that there is a academic paper written on this
subject.

On the Expressive Power of Programming Languages, by Matthias
Felleisen, 1990.
http://www.ccs.neu.edu/home/cobbe/pl-seminar-jr/notes/2003-sep-26/expressive-slides.pdf

Has anyone read this paper? And, would anyone be interested in giving a
summary?

thanks.

Xah
(e-mail address removed)
∑ http://xahlee.org/

 

[Joe Marshall]

in March, i posted a essay "What is Expressiveness in a Computer
Language", archived at:
http://xahlee.org/perl-python/what_is_expresiveness.html

I was informed then that there is a academic paper written on this
subject.

On the Expressive Power of Programming Languages, by Matthias
Felleisen, 1990.
http://www.ccs.neu.edu/home/cobbe/pl-seminar-jr/notes/2003-sep-26/expressive-slides.pdf

Has anyone read this paper? And, would anyone be interested in giving a
summary?

The gist of the paper is this: Some computer languages seem to be
`more expressive' than
others. But anything that can be computed in one Turing complete
language can be computed in any other Turing complete language.
Clearly the notion of
expressiveness isn't concerned with ultimately computing the answer.

Felleisen's paper puts forth a formal definition of expressiveness in
terms of semantic
equivilances of small, local constructs. In his definition, wholescale
program transformation is
disallowed so you cannot appeal to Turing completeness to claim program
equivalence.

Expressiveness isn't necessarily a good thing. For instance, in C, you
can express the
addresses of variables by using pointers. You cannot express the same
thing in Java, and
most people consider this to be a good idea.

 

[Simon Richard Clarkstone]

The gist of the paper is this: Some computer languages seem to be
`more expressive' than others. But anything that can be computed in
one Turing complete language can be computed in any other Turing
complete language. Clearly the notion of expressiveness isn't
concerned with ultimately computing the answer.

Felleisen's paper puts forth a formal definition of expressiveness in
terms of semantic equivilances of small, local constructs. In his
definition, wholescale program transformation is disallowed so you
cannot appeal to Turing completeness to claim program equivalence.

I suspect that the small, local transformations versus global
transformations is also to do with the practice of not saying the same
thing twice. Everything from subroutines to LISP macros also helps
here, increasing language expressiveness.

Expressiveness isn't necessarily a good thing. For instance, in C,
you can express the addresses of variables by using pointers. You
cannot express the same thing in Java, and most people consider this
to be a good idea.

Assuming the more-expressive feature does not preclude the
less-expressive one, good/bad depends on the programmer. I know *I*
can't be trusted with pointers ;-) , but I know many programmers benefit
greatly from them. Of course, knowing that the programmer cannot do
something does help the compiler stop you shooting yourself in the foot.

 

[PofN]

Xah Lee wrote:
[the usual toff-topic trolling stuff]

Shit, da troll is back. Abuse reports need to go to
abuse [] pacbell.net and abuse [] swbell.net this time.

 

[Kaz Kylheku]

Has anyone read this paper? And, would anyone be interested in giving a
summary?

Not you, of course. Too busy preparing the next diatribe against UNIX,
Perl, etc.

 

[Ken Tilton]

The gist of the paper is this: Some computer languages seem to be
`more expressive' than
others. But anything that can be computed in one Turing complete
language can be computed in any other Turing complete language.
Clearly the notion of
expressiveness isn't concerned with ultimately computing the answer.

Felleisen's paper puts forth a formal definition of expressiveness in
terms of semantic
equivilances of small, local constructs. In his definition, wholescale
program transformation is
disallowed so you cannot appeal to Turing completeness to claim program
equivalence.

Expressiveness isn't necessarily a good thing. For instance, in C, you
can express the
addresses of variables by using pointers. You cannot express the same
thing in Java, and
most people consider this to be a good idea.

Thanks for the summary.

Me, I would like to see a definition of expressiveness that would
exclude a programming mechanism from "things to be expressed".

If the subject is programmer productivity, well, I write programs to get
some behavior out of them, such as operating an ATM cash dispenser. If I
need to keep a list of transactions, I need to express the abstraction
"list" in some data structure or other, but below that level of
abstraction I am just hacking code, not expressing myself -- well, that
is the distinction for which I am arguing.

heck, in this case I will even give you as "thing to express" getting
back multiple values from a function. That comes up all the time, and it
can be an aggravation or a breeze. But then I would score C down because
it does not really return multiple values. One still has some heavy
lifting to do to fake the expressed thing. But I would still give it an
edge over java because Java's fakery would have to be a composite object
-- one could not have a primary return value as the function result and
ancillary values "somewhere else".

kt

 

[Xah Lee]

hi Joe,

Joe Marshall wrote:
« Expressiveness isn't necessarily a good thing. For instance, in C,
you can express the addresses ...»

we gotta be careful here, because soon we gonna say binaries are the
most expressive. For instance, in assembly, you can express the
registers and stuff.

Expressiveness, with respect to — for lack of refined terms —
semantics, is a good thing, period. When discussing a language's
semantical expressiveness, it goes without saying that a “domainâ€
are understood, or needs to be defined. This is almost never mentioned
because it is very difficult. Put it in driveler's chant for better
understanding: we can't “compare apples with orangesâ€.

Let me give a example. Let's say i invented a language, where, there's
no addition of numbers, but phaserfy and realify with respective
operators ph and re. So, in my language, to do 1+2, you write “ph 1
re ph 2â€, which means, to phaserfy 1, and phaserfy 2, then realify
their results, which results in 3. Now, this language is the most
expressive, because it can deal with concepts of phaserfy and realify
that no other lang can.

This may seem ridiculous, but is in fact a lot imperative languages do.
I won't go long here, but for instance, the addresses or references of
C and Perl is such. And in Java and few other OOP langs, there's
“iterator†and “enumerator†things, are likewise immaterial.

As to OOP's iterator and enumerator things, and the general perspective
of extraneous concepts in languages, i'll have to write a essay in
detail some other day.

----

Thanks for the summary.

Is there no one else who are able to read that paper?

Xah
(e-mail address removed)
∑ http://xahlee.org/

 

[=?iso-8859-1?q?Torben_=C6gidius_Mogensen?=]

The gist of the paper is this: Some computer languages seem to be
`more expressive' than others. But anything that can be computed in
one Turing complete language can be computed in any other Turing
complete language. Clearly the notion of expressiveness isn't
concerned with ultimately computing the answer.

Felleisen's paper puts forth a formal definition of expressiveness
in terms of semantic equivilances of small, local constructs. In
his definition, wholescale program transformation is disallowed so
you cannot appeal to Turing completeness to claim program
equivalence.

I think expressiveness is more subtle than this. Basically, it boils
down to: "How quickly can I write a program to solve my problem?".

There are several aspects relevant to this issue, some of which are:

- Compactness: How much do I have to type to do what I want?

- Naturality: How much effort does it take to convert the concepts of
my problem into the concepts of the language?

- Feedback: Will the language provide sensible feedback when I write
nonsensical things?

- Reuse: How much effort does it take to reuse/change code to solve a
similar problem?

Compactness is hard to measure. It isn't really about the number of
characters needed in a program, as I don't think one-character symbols
instead of longer keywords make a language more expressive. It is
better to count lexical units, but if there are too many different
predefined keywords and operators, this isn't reasonable either.
Also, the presence of opaque one-liners doesn't make a language
expressible. Additionally, as mentioned above, Turing-completeness
(TC) allows you to implement any TC language in any other, so above a
certain size, the choice of language doesn't affect size. But
something like (number of symbols in program)/log(number of different
symbols) is not too bad. If programs are allowed to use standard
libraries, the identifiers in the libraries should be counted in the
number of different symbols.

Naturality is very difficult to get a grip on, and it strongly depends
on the type of problem you want to solve. So it only makes sense to
talk about expressiveness relative to a set of problem domains. If
this set is small, domain-specific languages win hands down, so if you
want to compare expressiveness of general-purpose languages, you need
a large set of very different problems. And even with a single
problem, it is hard to get an objective measure of how difficult it is
to map the problem's concepts to those of the language. But you can
normally observe whether you need to overspecify the concept (i.e.,
you are required to make arbitrary decisions when mapping from concept
to data), if the mapping is onto (i.e., can you construct data that
isn't sensible in the problem domain) and how much redundancy your
representation has.

Feedback is a mixture of several things. Partly, it is related to
naturality, as a close match between problem concepts and language
concepts makes it less likely that you will express nonsense (relative
to the problem domain) that makes sense in the language. For example,
if you have to code everything as natural numbers, untyped pure lambda
calculus or S-expressions, there is a good chance that you can get
nonsense past the compiler. Additionally, it is about how difficult
it is to tie an observation about a computed result to a point in the
program.

Measuring reuse depends partly on what is meant by problems being
similar and also on whether you at the time you write the original
code can predict what types of problems you might later want to solve,
i.e., if you can prepare the code for reuse. Some languages provide
strong mechanisms for reuse (templates, object hierarchies, etc.), but
many of those require that you can predict how the code is going to be
reused. So, maybe, you should measure how difficult it is to reuse a
piece of code that is _not_ written with reuse in mind.

This reminds me a bit of last years ICFP contest, where part of the
problem was adapting to a change in specification after the code was
written.

Expressiveness isn't necessarily a good thing. For instance, in C,
you can express the addresses of variables by using pointers. You
cannot express the same thing in Java, and most people consider this
to be a good idea.

I think this is pretty much covered by the above points on naturality
and feedback: Knowing the address of a value or object is an
overspecification unless the address maps back into something in the
problem domain.

On a similar note, is a statically typed langauge more or less
expressive than a dynamically typed language? Some would say less, as
you can write programs in a dynamically typed language that you can't
compile in a statically typed language (without a lot of encoding),
whereas the converse isn't true. However, I think this is misleading,
as it ignores the feedback issue: It takes longer for the average
programmer to get the program working in the dynamically typed
language.

Torben

 

[Raffael Cavallaro]

It takes longer for the average
programmer to get the program working in the dynamically typed
language.

Though I agree with much of your post I would say that many here find
the opposite to be true - it takes us longer to get a program working
in a statically typed language because we have to keep adding/changing
things to get the compiler to stop complaining and actually compile and
run a program which would be perfectly permissible in a dynamically
typed language such as common lisp - for example - heterogeneous lists
and forward references to as yet non-existent functions.

 

[Rob Thorpe]

On a similar note, is a statically typed langauge more or less
expressive than a dynamically typed language? Some would say less, as
you can write programs in a dynamically typed language that you can't
compile in a statically typed language (without a lot of encoding),
whereas the converse isn't true. However, I think this is misleading,
as it ignores the feedback issue: It takes longer for the average
programmer to get the program working in the dynamically typed
language.

From the point of view purely of expressiveness I'd say it's rather

different.

If a language can express constraints of one kind that is an increase
in expressiveness.
If a language requires constraint to be in one particular way thats a
decrease in expressiveness.

So I would say languages that can be statically typed and can be
dynamically typed are the most expressive. Languages that require
static typing or are dynamic but cannot express static typing are less
expressive.

This meets my experience of what useful in practice too, static typing
for everything is painful for writing simple code. Pure dynamic typing
is painful when writing complex code because it makes impossible a
layer of error checking that could otherwise be useful.

 

[Joachim Durchholz]

For example,
if you have to code everything as natural numbers, untyped pure lambda
calculus or S-expressions, there is a good chance that you can get
nonsense past the compiler.

Also past peer review and/or debugging runs. And, most importantly, past
your own mental review processes.

Regards,
Jo

 

[Joachim Durchholz]

Though I agree with much of your post I would say that many here find
the opposite to be true - it takes us longer to get a program working in
a statically typed language because we have to keep adding/changing
things to get the compiler to stop complaining and actually compile and
run

I think Torben was assuming a language with type inference. You write
only those type annotations that really carry meaning (and some people
let the compiler infer even these).

a program which would be perfectly permissible in a dynamically
typed language such as common lisp - for example - heterogeneous lists
and forward references to as yet non-existent functions.

Um... heterogenous lists are not necessarily a sign of expressiveness.
The vast majority of cases can be transformed to homogenous lists
(though these might then contain closures or OO objects).

As to references to nonexistent functions - heck, I never missed these,
not even in languages without type inference

I don't hold that they are a sign of *in*expressiveness either. They are
just typical of highly dynamic programming environments such as Lisp or
Smalltalk.

Regards,
Jo

 

[Joachim Durchholz]

If a language can express constraints of one kind that is an increase
in expressiveness.
Agreed.

If a language requires constraint to be in one particular way thats a
decrease in expressiveness.

Unless alternatives would be redundant.
Having redundant ways to express the same thing doesn't make a language
more or less expressive (but programs written in it become more
difficult to maintain).

So I would say languages that can be statically typed and can be
dynamically typed are the most expressive. Languages that require
static typing or are dynamic but cannot express static typing are less
expressive.

Note that this is a different definition of expressiveness.
(The term is very diffuse...)

I think Felleisen's paper defines something that should be termed
"conciseness".
Whether there's a way to express constraints or other static properties
of the software is something different. I don't have a good word for it,
but "expressiveness" covers too much for my taste to really fit.

Regards,
Jo

 

[Pascal Costanza]

On a similar note, is a statically typed langauge more or less
expressive than a dynamically typed language? Some would say less, as
you can write programs in a dynamically typed language that you can't
compile in a statically typed language (without a lot of encoding),
whereas the converse isn't true.

It's important to get the levels right here: A programming language with
a rich static type system is more expressive at the type level, but less
expressive at the base level (for some useful notion of expressiveness .

However, I think this is misleading,
as it ignores the feedback issue: It takes longer for the average
programmer to get the program working in the dynamically typed
language.

This doesn't seem to capture what I hear from Haskell programmers who
say that it typically takes quite a while to convince the Haskell
compiler to accept their programs. (They perceive this to be worthwhile
because of some benefits wrt correctness they claim to get in return.)


Pascal

 

[Pascal Bourguignon]

Um... heterogenous lists are not necessarily a sign of
expressiveness. The vast majority of cases can be transformed to
homogenous lists (though these might then contain closures or OO
objects).

In lisp, all lists are homogenous: lists of T.


--
__Pascal Bourguignon__ http://www.informatimago.com/

ADVISORY: There is an extremely small but nonzero chance that,
through a process known as "tunneling," this product may
spontaneously disappear from its present location and reappear at
any random place in the universe, including your neighbor's
domicile. The manufacturer will not be responsible for any damages
or inconveniences that may result.

 

[Raffael Cavallaro]

Um... heterogenous lists are not necessarily a sign of expressiveness.
The vast majority of cases can be transformed to homogenous lists
(though these might then contain closures or OO objects).

As to references to nonexistent functions - heck, I never missed these,
not even in languages without type inference

I don't hold that they are a sign of *in*expressiveness either. They
are just typical of highly dynamic programming environments such as
Lisp or Smalltalk.

This is a typical static type advocate's response when told that users
of dynamically typed languages don't want their hands tied by a type
checking compiler:

"*I* don't find those features expressive so *you* shouldn't want them."

You'll have to excuse us poor dynamically typed language rubes - we
find these features expressive and we don't want to give them up just
to silence a compiler whose static type checks are of dubious value in
a world where user inputs of an often unpredictable nature can come at
a program from across a potentially malicious internet making run-time
checks a practical necessity.

 

[Raffael Cavallaro]

In lisp, all lists are homogenous: lists of T.

CL-USER 123 > (loop for elt in (list #\c 1 2.0d0 (/ 2 3)) collect
(type-of elt))
(CHARACTER FIXNUM DOUBLE-FLOAT RATIO)

i.e., "heterogenous" in the common lisp sense: having different dynamic
types, not in the H-M sense in which all lisp values are of the single
union type T.

 

[=?iso-8859-1?q?Torben_=C6gidius_Mogensen?=]

CL-USER 123 > (loop for elt in (list #\c 1 2.0d0 (/ 2 3)) collect
(type-of elt))
(CHARACTER FIXNUM DOUBLE-FLOAT RATIO)

i.e., "heterogenous" in the common lisp sense: having different
dynamic types, not in the H-M sense in which all lisp values are of
the single union type T.

What's the difference? Dynamically types values _are_ all members of
a single tagged union type. The main difference is that the tages
aren't always visible and that there are only a fixed, predefined
number of them.

Torben

 

[=?iso-8859-1?q?Torben_=C6gidius_Mogensen?=]

It's important to get the levels right here: A programming language
with a rich static type system is more expressive at the type level,
but less expressive at the base level (for some useful notion of
expressiveness .


This doesn't seem to capture what I hear from Haskell programmers who
say that it typically takes quite a while to convince the Haskell
compiler to accept their programs. (They perceive this to be
worthwhile because of some benefits wrt correctness they claim to get
in return.)

That's the point: Bugs that in dynamically typed languages would
require testing to find are found by the compiler in a statically
typed language. So whil eit may take onger to get a program thatgets
past the compiler, it takes less time to get a program that works.

Torben

 

[Joachim Durchholz]

Um... heterogenous lists are not necessarily a sign of expressiveness.
The vast majority of cases can be transformed to homogenous lists
(though these might then contain closures or OO objects).

As to references to nonexistent functions - heck, I never missed
these, not even in languages without type inference

[[snipped - doesn't seem to relate to your answer]]

This is a typical static type advocate's response when told that users
of dynamically typed languages don't want their hands tied by a type
checking compiler:

"*I* don't find those features expressive so *you* shouldn't want them."

And this is a typical dynamic type advocate's response when told that
static typing has different needs:

"*I* don't see the usefulness of static typing so *you* shouldn't want
it, either."

No ad hominem arguments, please. If you find my position undefendable,
give counterexamples.
Give a heterogenous list that would to too awkward to live in a
statically-typed language.
Give a case of calling nonexistent functions that's useful.
You'll get your point across far better that way.

Regards,
Jo