Boost Workshop at OOPSLA 2004

[David B. Held]

[...]
So I don't understand it. Even if it only takes a single man year to
implement export, I can understand companies like Digital Mars having
some hesitancy -- I would imagine that even one man year represents a
significant part of your development budget. I can understand the
problem by g++ as well -- they have more than three developpers, but
since you don't feed a family on what you make working on g++, their
developers can't normally work full time on it. But even supposing that
it takes five man years (although others have done it in less), there
are vendors out there for whom five man years is less that a tenth of
one percent of their advertising budget. What's their excuse?

I find it particularly hypocritical that some of these companies (and
there isn't just one of them) are extremely active and visible in the
standardization effort, and then choose to totally ignore the results
when it doesn't suit them. Personally, it gives me the impression that
this activity is just a sophisticated form of advertising. False
advertising, in fact.

In N1426, EDG makes the important point that implementing export would
likely stall development on any other features for a given compiler,
because it so pervasively affects the entire codebase. So it's not 5
man-years in parallel with other feature additions. It is 5 man-years
that form a bottleneck for development of the entire product. Since
there are plenty of other goodies to add to C++ (and since most other
vendors still have a ways to go merely to be conforming without regards
to export), it makes sense that most vendors, regardless of size, would
tend to favor the features actually requested by users, rather than the
albatross mandated by the standard with virtually no noticeable demand
from the market.

Dave

 

[Hyman Rosen]

Do you really expect programmers to make macros for every class they
make? What next, a macro generator ?

Why not? You're just doing it in one place, in the implementation file.

Since you have so many possibilities, you'd end up with way too many
macros in source files. I am not very confortable with that. But, as
time goes by, I might

But they're just local macros in a single source file. That file doesn't
get included anywhere else, and users of the library never see it.

However, in case I want to create a portable library, and want to make
use of the export feature (for the compilers it's implemented on), for
each templated class, I'll have to provide the definitions in a
different file, and if export is not available, #include it. This is
sooooo tedious, and will make code harder to read and maintain IMO.

Then don't do it. One of the reasons compilers have become so capable of
compiling complex template code is the fact that Alexandrescu published a
totally uncompromising and terrific library which demanded standards
conformance. If we want to drive them to implement export, then the way to
do it is to make a terrific library that uncompromisingly demands export.
Make sure to use lots of macros and anonymous namespaces

 

[Gabriel Dos Reis]

| David Abrahams wrote:
| > most actual templates that people write don't have weird name-capturing problems.
| >
| > Doesn't that tend to argue that export isn't solving a real problem?
|
| No. I just meant that I believe that in most template code,
| you are not going to find calls to free non-dependent functions
| that vary by instantiation context. But export lets you write
| the template method definitions without requiring them to be
| safe for bodily inclusion into arbitrary compilation units.
| For example, library writers can write their template code
| without having to prefix every name in sight with '__' to avoid
| potential name clashes with user code.

Yes, that is true. You may have to make sure that you do really want
ADL where you use unqualified names in ADL-enabling contexts though.
But, that is a different set of concerns.

I wish we have the #nospam directive.

 

[Peter C. Chapin]

But export lets you write
the template method definitions without requiring them to be
safe for bodily inclusion into arbitrary compilation units.
For example, library writers can write their template code
without having to prefix every name in sight with '__' to avoid
potential name clashes with user code.

Just to clarify... is it true that the only name clashes one is really
protecting in this way are clashes with macro names? For example

namespace my_namespace {

template<class __Type>
void swap(__Type &__x, __Type &__y)
{
__Type __temp(__x);
__x = __y;
__y = __temp;
}

}

Ordinarly there is no way the names __Type, __x, __y, or __temp would be
exposed to potential clashes with user names. They are all local to the
template. It's only the preprocessor that causes problems here (right?).
If that's the case the approach of decorating names with underscores
only really works if macros are defined with a suitable naming
convention that avoids such underscores. If one supposes such a naming
convention, one might be able to suppose a macro naming convention that
would allow the template programmer to use more natural names. For
example if macros are all uppercase then the template programmer should
be free to use undecorated local names that are all lower case or that
are mixed case. In any event, solving this problem by introducing a way
to control the scope of macro names would seem to me to be more direct
and more decisive.

Peter

 

[kanze]

No, you have it exactly reversed.

Yup. In D, g(int) is called in both instances, for the reason you give.

OK. That seems a bit wierd to me, but possibly just because I learned
templates with CFront, where it was just the opposite.

But there must be a way to tell the compiler that I want a function
looked up in the context of the instantiation. Consider an extreme
case:

template< typename Traits >
class C
{
public:
void f()
{
Traits::g() ;
}
} ;

How does D do the name lookup of g() here?

 

[Walter]

No. I just meant that I believe that in most template code,
you are not going to find calls to free non-dependent functions
that vary by instantiation context. But export lets you write
the template method definitions without requiring them to be
safe for bodily inclusion into arbitrary compilation units.
For example, library writers can write their template code
without having to prefix every name in sight with '__' to avoid
potential name clashes with user code.

Or they could use namespaces, which were specifically designed to fix that
problem.

 

[Walter]

Given that Digital Mars C++ doesn't have export, what are you
suggesting the times are compared against?

Compare an EDG-based compiler using export vs DMC++ using precompiled
headers.

 

[Walter]

How do you call operator overloads on template parameters? e.g.
consider the D equivalent of this C++:

template <class T>
T sum(T lhs, T rhs)
{
return lhs + rhs;
}

How is the operator+ found if lookup only occurs at the point of
definition?

All operator overloads are member functions.

I have a horrible feeling you are about to tell me that
all operator overloads are member functions in D,

and members of
template parameters of course considered.

Of course.

If that's the case, how
about if the operator in question is a member of a different class
than the template parameter?

It first tries to find an operator overload as a member of the left operand.
If there isn't one, it attempts to find a "reverse" operator overload member
of the right operand. This may sound odd at first, and it's a bit more
complex than I just described, but it works out quite nicely and neatly
avoids the need for ADL.

 

[Walter]

It's also worth pointing out that the company in question only has three
technical employees, and that they also managed to implement a Java
compiler at the same time. Now, I don't know about Digital Mars, but
I'm pretty sure that some of the other vendors do have a few more
technical employees. For some, by several magnitudes even.

It is also fair to point out that said company only develops front ends. It
doesn't develop optimizers, code generators, linkers, librarians, runtime
libraries, debuggers, or IDEs, all of which are part of a compiler product,
and all of which consume a lot of resources to create, enhance, and
maintain, and all of which customers ask for improvements in.

If all you do is front end work, then it's common sense to expend one's
limited resources on the front end. There is no point to working on the
(non-existent) code generator instead.

BTW, I've written a Java compiler. It's more than an order of magnitude less
work than doing C++.

 

[Jean-Marc Bourguet]

Try your compilation speed benchmarks with Digital Mars C++
precompiled headers!

So you want me to compare compilation speed between different
compilers on different OS on different computers? I already know the
result: I can easily built a test case for which como win for
recompilation after modification of the template implementation: como
will have to recompile one file and do the link when Digital Mars C++
will have to recompile thousands of them before the link.

The less artificial benchmark could have been more interesting
(finding the cross point for a more realistic template), but it is too
dependant on Unix and porting it to Windows would be a too important
effort for getting difficult to interpret results. And the result of
a compilation from scratch of the very artificial test above would
also be very difficult to interpret (difference in the compiler
architecture: como compiles to C and use gcc to compile the result, I
assume that Digital Mars compiles to object code, difference in
caching algorithm of the OS and in disk speed could make either win,
difference in optimisation level will also be a factor).

Yours,

 

[tom_usenet]

Compare an EDG-based compiler using export vs DMC++ using precompiled
headers.

How is that a useful comparison? Apples and oranges. How about
comparing Comeau C++ using precompiled headers and the inclusion model
against Comeau C++ using precompiled headers + export?

Tom

 

[Jean-Marc Bourguet]

news:<[email protected]>... [...]
I've been thinking about this myself. Just yesterday, I ran into
what seems to be a frequent scenario in my code, where the template
class itself consists of a lot of small, simple functions, and one
or two very complex functions, and the template is used in a number
of different source modules, instantiated over the same type. Every
time I make the slightest modification in the complicated function,
I touch the header file, which means that my make system recompiles
all of the sources which include that header. And that is slow.

I would expect export to improve this, for the simple reason that I
don't expect to have to specify the implementation file in my
dependencies in the make file. I hope not, anyway; the dependencies
are generated automatically depending on the files explicitly
included. So the automatically generated dependencies will NOT
include the implementation file for exported templates.

With como, you'll need dependencies on implementation file for
exported template for every compilation unit which is responsible of
providing an instanciation. As far as I remember, in my tests all the
dependancies (even on exported template implementation) where
generated automatically by como.

In an earlier thread, I raised the same point, and David Vandevoorde
(the author of the export implementation used in Como) assured me
that this was a scenario where build times would be improved.

That's what I've experimented.

That said, and to be honest, I suspect that the improvement is
really because it is the compiler handling the dependencies, instead
of make, and that it handles them more intelligently; that is, it
would probably be possible to obtain the same improvements without
export (but David would be in a better position than I am to confirm
or deny this).

You could probably have the same improvements by carefully controlling
the compilation units in which you include the template
implementation. The only one I known which tried that approach on a
sizable project gave up after loosing something like one week of work
(that someone tried so long to do that is a sign by itself).

Yours,

 

[Hyman Rosen]

the albatross mandated by the standard with virtually no noticeable demand
from the market.

The standard mandated lots of things not demanded by the market, such as STL.
Export is no albatross. From the user point of view, it presents a perfectly
simple concept - put your declarations in a header file and your code in an
implementation file, and compile. It's the inclusion model which is nasty and
broken and counter-intuitive.

And it's hardly the only difficult thing in C++ for implementors to deal with.
Go look at g++'s ABI document to see what kind of dance implementations have
to go through just to lay out classes, and to manipulate virtual tables during
object construction and destruction, and to implement exceptions without speed
penalties. I find it hard to believe that writing export is harder than doing
all the unwinding correctly when an exception is thrown.

The reason vendors haven't implemented export is that users can muddle along
with the inclusion model, so there's no pressure on them to get it working.
The only reason templates are in as good a shape as they are is _Modern C++
Design_ - that library drove enough user demand that implementors felt that
they had to get this part of the standard implemeneted. Otherwise they just
can't be bothered. For God's sake, until the recent version 7, Microsoft's
exception handling was totally broken. Maybe that was an albatross too?

 

[Hyman Rosen]

Just to clarify... is it true that the only name clashes one is really
protecting in this way are clashes with macro names?
Yes.

If that's the case the approach of decorating names with underscores
only really works if macros are defined with a suitable naming
convention that avoids such underscores.

There is a subset of names which are reserved to the implementation,
described in 17.4.3.1.2, namely those which begin with an underscore
followed by a capital letter and those which contain two consecutive
underscores. So templates which are part of the implementation of the
standard use those names, because user code may not.

For example if macros are all uppercase

That would be a change that would break existing code.

In any event, solving this problem by introducing a way to control the
scope of macro names would seem to me to be more direct and more decisive.

That would solve the clunky name problem. It would do nothing about to
allowing anonymous namespaces, or avoiding the need to include all of
the template implementation's header files into the body of every file
that uses the template, or about the need to recompile the template
instantiation code for every compilation unit that uses the template.

You don't make every non-template method inline. Why in the world should
every template method be inline?

 

[P.J. Plauger]

However, in case I want to create a portable library, and want to make
use of the export feature (for the compilers it's implemented on), for
each templated class, I'll have to provide the definitions in a
different file, and if export is not available, #include it. This is
sooooo tedious, and will make code harder to read and maintain IMO.

Not necessarily, but it does make it harder to write initially. We
produce the export version of our library, for use with the EDG
compiler, by decorating our existing headers with suitable hints.
We have always generated multiple products from a common baseline
by using edit scripts that key off these decorations. It's no harder
to make the export source files and headers to match than to produce
our EC++ and Abridged libraries. And I believe the code is fairly
readable in all cases. It's also not to bad to maintain because the
decorations are simple enough.

Having said all that, I would *not* recommend that this process be
widely adopted. We're in the unique position of producing the basic
libraries used by multiple compilers. It's worthwhile for us to invest
in efforts that let us keep a common baseline. For most other products,
the extra investment in portability will not have the same payoff.
And it's certainly premature to produce an export-only library,
unless you're fortunate enough to work in an EDG-only world.

P.J. Plauger
Dinkumware, Ltd.
http://www.dinkumware.com

 

[Hyman Rosen]

Or they could use namespaces, which were specifically designed to fix that
problem.

No, because users may define any non-reserved name as an
arbitrary macro, so any code forced in by the inclusion
model can confict in that way. Furthermore, any headers
needed by the template implementation are also forced
into the users code, causing more potential conflict.

And template implementations can't use techniques based
upon anonymous namespaces under the inclusion model for
fear of violating the ODR.

 

[Hyman Rosen]

All operator overloads are member functions.

Can these operators be member templates? If not, you've
thrown away the possibility of building a units system
as described by Barton & Nackman.

What about overloading operators on enumeration types?

If there isn't one, it attempts to find a "reverse" operator
overload member of the right operand.

What is the "reverse" of subtraction and division?

 

[Gabriel Dos Reis]

| | > David Abrahams wrote:
| > > most actual templates that people write don't have weird name-capturing
| problems.
| > >
| > > Doesn't that tend to argue that export isn't solving a real problem?
| >
| > No. I just meant that I believe that in most template code,
| > you are not going to find calls to free non-dependent functions
| > that vary by instantiation context. But export lets you write
| > the template method definitions without requiring them to be
| > safe for bodily inclusion into arbitrary compilation units.
| > For example, library writers can write their template code
| > without having to prefix every name in sight with '__' to avoid
| > potential name clashes with user code.
|
| Or they could use namespaces, which were specifically designed to fix that
| problem.

Macros don't obey scope rules, therefore anything namespaces are doing
are irrelevant to macro name leakage.

 

[Walter]

So you want me to compare compilation speed between different
compilers on different OS on different computers?

No, between different compilers on the same machine and the same OS.

I already know the
result: I can easily built a test case for which como win for
recompilation after modification of the template implementation: como
will have to recompile one file and do the link when Digital Mars C++
will have to recompile thousands of them before the link.

This can be dealt with by adjusting the dependencies specified in the
makefile. Such could also theoretically be added to precompiled headers, but
nobody has done that to the best of my knowledge.

The less artificial benchmark could have been more interesting
(finding the cross point for a more realistic template), but it is too
dependant on Unix and porting it to Windows would be a too important
effort for getting difficult to interpret results.

I agree that would be a problem, but Comeau's compiler is available on
Windows, so a direct comparison is possible.

And the result of
a compilation from scratch of the very artificial test above would
also be very difficult to interpret (difference in the compiler
architecture: como compiles to C and use gcc to compile the result, I
assume that Digital Mars compiles to object code, difference in
caching algorithm of the OS and in disk speed could make either win,
difference in optimisation level will also be a factor).

If DMC++ can, using precompiled headers, compile faster than a compiler
optimized with export, then that shows that exported templates are not
needed for boosting compile speed. On the same machine and OS, and with
compiler switch settings for max compile speed, of course.

 

[Walter]

But there must be a way to tell the compiler that I want a function
looked up in the context of the instantiation. Consider an extreme
case:

template< typename Traits >
class C
{
public:
void f()
{
Traits::g() ;
}
} ;

How does D do the name lookup of g() here?

g is looked up in Traits' symbol table. I.e. it works as one would expect.