Implementing library-wide build switches

[Kaba]

Hi,

The problem: I am seeing two approaches to implementing library-wide
build switches (e.g. DEBUG_MODE). But both of them are problematic. Can
you see a way out of this?

### Approach 1: As header file switches

In this approach the build switches are located in a such .h file which
is included in every file in the library. In this file the appropriate
preprocessor defines can be set to the desired values, e.g.
`#define DEBUG_MODE 1`, before the build.

The problem with this approach is that each configuration of the library
(e.g. debug and release) needs a different set of these defines. Setting
the switches in a header file does not work well with build tools where
you must be able to select any configuration to build.

### Approach 2: As compiler switches

In this approach the build switches are given to the compiler when
building the library, e.g. `/DDEBUG_MODE`. This will avoid the problems
with the header file approach: you can now select your build switches
freely and in co-operation with your build system. However, it then
creates new problems. Consider a library B using a library A. Then the
library B needs to remember the build switches that were used to build
the library A and use those same switches to build B, in addition to its
own build switches. In general, if there are n libraries in a sequence,
then the m:th library needs to remember m build switch sets. Clearly
this approach does not scale well.

 

[Pavel]

Hi,

The problem: I am seeing two approaches to implementing library-wide build
switches (e.g. DEBUG_MODE). But both of them are problematic. Can you see a way
out of this?

### Approach 1: As header file switches

In this approach the build switches are located in a such .h file which is
included in every file in the library. In this file the appropriate preprocessor
defines can be set to the desired values, e.g.
`#define DEBUG_MODE 1`, before the build.

The problem with this approach is that each configuration of the library (e.g.
debug and release) needs a different set of these defines. Setting the switches
in a header file does not work well with build tools where you must be able to
select any configuration to build.

### Approach 2: As compiler switches

In this approach the build switches are given to the compiler when building the
library, e.g. `/DDEBUG_MODE`. This will avoid the problems with the header file
approach: you can now select your build switches freely and in co-operation with
your build system. However, it then creates new problems. Consider a library B
using a library A. Then the library B needs to remember the build switches that
were used to build the library A and use those same switches to build B, in
addition to its own build switches. In general, if there are n libraries in a
sequence, then the m:th library needs to remember m build switch sets. Clearly
this approach does not scale well.

It is probably an off-topic here, but an important practical question (and many
people struggle from same problem), so: take a look at libtool discipline.

Also, Approach 1 has all the drawbacks of approach 2 for A-using-B case.

Also, a library does not have to always be built with the exactly same compiler
options as its dependent library -- e.g. 99% of the times you build debugging
code against release libc on *nix, 99% of big binaries are built with at least
one library built with different optimization options etc. (the above does not
mean it is a good idea to mix up libraries built with different *important*
options -- "multi-threaded" is one of these)


HTH
-Pavel

 

[Marcel Müller]

### Approach 1: As header file switches

In this approach the build switches are located in a such .h file which
is included in every file in the library. In this file the appropriate
preprocessor defines can be set to the desired values, e.g.
`#define DEBUG_MODE 1`, before the build.

This one is unusual, since you have to change the source code all the
time. This might raise problems with version control tools.

### Approach 2: As compiler switches

In this approach the build switches are given to the compiler when
building the library, e.g. `/DDEBUG_MODE`. This will avoid the problems
with the header file approach: you can now select your build switches
freely and in co-operation with your build system.

This is common practice. Usually the make file (or build environment)
does the job.

However, it then
creates new problems. Consider a library B using a library A. Then the
library B needs to remember the build switches that were used to build
the library A and use those same switches to build B, in addition to its
own build switches.

I do not see this constraint. It is a good advise to keep the API binary
compatible between debug and release builds. So you could use the debug
version of an application with non-debug builds of well tested base
libraries. In fact this could be preferable in some cases.

In general, if there are n libraries in a sequence,
then the m:th library needs to remember m build switch sets. Clearly
this approach does not scale well.

If you really want to operate this way, create two working copies of the
entire source tree, synchronized by some version control tool. Build one
of them with debug support and the other one without.


Marcel

 

[Juha Nieminen]

The problem: I am seeing two approaches to implementing library-wide
build switches (e.g. DEBUG_MODE).

With regard to that, generally speaking using compile-time options in
libraries is seldom a good idea. Perhaps the only exception to this
is an option that's intended to be used exclusively for developing the
library itself and nothing else. (Although, to be fair, "DEBUG_MODE"
sounds like such an option.)

The major problem with compile-time options is that they hinder creating
precompiled libraries (.dll, .so, .a, ...) This especially so if the option
is something that the end user might want to be able to choose. (Not only
does it hinder the ability of making such choices, but it also can create
compatibility problems between different builds of the library, if they
use different options.)

 

[Kaba]

19.6.2012 14:09, Juha Nieminen kirjoitti:

With regard to that, generally speaking using compile-time options in
libraries is seldom a good idea. Perhaps the only exception to this
is an option that's intended to be used exclusively for developing the
library itself and nothing else. (Although, to be fair, "DEBUG_MODE"
sounds like such an option.)

The compile-time options my library uses are (roughly) as follows:

PASTEL_DEBUG_MODE - Conditional inclusion of assert-like checks.

PASTEL_ENABLE_OMP - Conditional inclusion of OpenMp header file omp.h.

PASTEL_DYNAMIC_LIBRARIES - Conditional inclusion of dll-keywords on
Windows (sprinkle __declspec(dllexport) and __declspec(dllimport)).

PASTEL_LARGE_INTEGER - Whether the abstract 'integer' type used in the
library should be 64-bit or 32-bit (roughly). The former allows
practically unbounded data structures, while the latter preserves memory
when it is known that the amount of data in each data structure keeps
under 2^31 (the way it was with 32-bit machines).

Considering Visual Studio in Windows (the library is cross-platform), it
has the switch _ITERATOR_DEBUG_LEVEL (which some may previously remember
by _SECURE_SCL and _HAS_ITERATOR_DEBUGGING). Different values of this
result in different definitions of iterator types in the STL. This
breaks ABI compatibility (and may cause horrible crashes, although they
did add a linker check later to avoid these). I don't know whether the
debug and release modes of STL are ABI compatible after setting
_ITERATOR_DEBUG_LEVEL to 0. I would assume they are.

So we see that the build switch problem already begins from the STL
level (on Windows). If you set _ITERATOR_DEBUG_LEVEL to x, then you need
to be consistent on that for all the libraries that come after.
Similarly for PASTEL_LARGE_INTEGER. The PASTEL_DEBUG_MODE and
PASTEL_ENABLE_OMP on the other hand are different in that they can't
break ABI compatibility. However, forgetting to add PASTEL_ENABLE_OMP
can mean serious performance losses when the parallelization does not
get enabled.

Thus it seems that the build switches divide into two groups: those that
break ABI compatibility and those that do not. The former are critical
to get right. Getting the latter wrong 'only' affects secondarily by
missing or extra features (such as missing performance or missing
checks). This can be very bad too.

The major problem with compile-time options is that they hinder creating
precompiled libraries (.dll, .so, .a, ...) This especially so if the option
is something that the end user might want to be able to choose. (Not only
does it hinder the ability of making such choices, but it also can create
compatibility problems between different builds of the library, if they
use different options.)

Not sure what you are saying here. It would be helpful if you could
apply this to the situation described above.

 

[Kaba]

19.6.2012 6:22, Pavel kirjoitti:

It is probably an off-topic here, but an important practical question
(and many people struggle from same problem), so: take a look at libtool
discipline.

I read some of the documentation of libtool. If the answer is there
somewhere, I am not able to find it. Could you summarize the approach or
otherwise point me to the exact position in the documentation?

 

[Pavel]

19.6.2012 6:22, Pavel kirjoitti:

I read some of the documentation of libtool. If the answer is there somewhere, I
am not able to find it. Could you summarize the approach or otherwise point me
to the exact position in the documentation?

Read on using deplibs_check_method with file_magic (admittedly, there is no much
documentation around, you will probably end up reading libtool.m4 source). The
idea is for a library built with particular set of options to define some
constant identifying the set of used switches and for file_magic_cmd to
determine if it is what you want for your library that depends on that first
library. Libtool does not provide any magic here but just a common interface so
that others can build packages from your sources using more-or-less standard
"configure" interface.

Again, it's more like a discipline than the "answer in the software"; that is,
if you keep to it, you will be able to build a library only against *your* other
libraries built according to that your discipline (that is, with desired
switches) -- but you won't be able to detect whether a 3-rd party library was
compiled with the switches you want (which I am not even sure is possible in
general).

HTH
-Pavel