Windows Compilation Madness

[Luis Lavena]

I'm not a Windows programmer and I didn't really want to cause a
rehash of all the discussion, I've seen before. I was just trying to
get the 'big picture' to understand why Windows is so problematic in
this area.

Another attempt at a big picture summary:

*nix Binary Distribution
-- limited to enclaves (Solaris, MacOSX, Linux) and has all
the same library version problems as with Windows but on any
particular platform there is more continuity between different
libc versions and only a single prevalent compiler avoiding
various object code linkage problems.

Windows Binary Distribution
-- same situation as Unix but the enclaves are Windows variations
of NT, XP, Vista *and* all the different compiler/object code
incarnations. Less common ground to work with. The CLR
is yet another windows binary context to be considered.

On *nix, all the binaries must be compiled and linked to the same
version of the C runtime.

On Windows, different programs can use its own copy of the C runtime
(MSVCRT), use the system wide one or different version available also
system-wide.

Share libraries (dlls) loaded by some application cannot mix the C
runtimes.

*nix Source Distribution
-- standards and common build-chains makes source code
and build processes compatible across wide variety of
systems. Complex packages composed of components from
multiple 3rd parties can be compiled and linked via the
same 'common' build-tool chain (because there is only one
per platform). I realize that autoconf is hideous
but it is one of the reasons for the broad source code
compatibility across *nix platforms.

Windows Source Distribution
-- lack of prevalent compiler and common build-chains makes it
difficult to author and distribute a source code package
that builds and installs correctly in all the different
Windows environments

-- the lack of a common build-chain means that it can be
very difficult to mix-and-match 3rd party source distributions,
which is a much rarer problem in *nix environments.

It seems like the nut to crack is the build-chain environment
in Windows. If the build-chain isn't predictable then it is
going to be pretty hard to avoid an n*m amount of work to get
arbitrary collection of n-packages to work together in m different
Windows environments.

I'll like to quote your message in a future blog post regarding this
issue

Sounds like the right place to focus developer energy. Also
seems like that a Ruby/Rake combo that could be built with
an absolute minimum of external dependencies would be a great
tool for bootstrapping a full-featured build-chain.

Been under that road

Here is a really crazy idea. Instead of trying to construct
a build environment on each and every Windows box what if the
build environment was available via the Internet? So a small
tool would query the local system and then instruct a remote
system to build the appropriate dlls/applications for the local
machine. Lots of security implications since you would have
to trust the code that was being delivered by the remote
build tool but of course if you are downloading and installing
a build-tool environment you are *already* extending trust to
the provider of the build-tool.

I was about to provide a windows "tinderbox" so gem developers can get
windows builds of their gems for free, without the need of get windows
+development tools and the pain it is.

There is also CruiseControl as CI tool, that could be used too.

Thank you Gary for your comments,

Regards,

 

[Gary Wright]

No. The build scripts themselves, are ok. The backing toolchains
are an issue.

I wasn't actually differentiating between the two. Having common
scripts but not common tool chains isn't really all that useful,
right? It still means that you've got n-different contexts to work
with instead of 1.

There are plenty of prevalent compilers, but there are more options
than on *nix, and the options have more differences.

OK, my dictionary says that I should have said 'no prevailing
compiler' as in one particular compiler that prevails among all the
prevalent ones.
Contrast this to *nix platforms were there is generally just a single
compiler choice and not a choice among many prevalent ones.

Source distributions are fine. Again, it's purely a tool chain and
version linkage issue.

OK, but for a source distribution to be useful in a wide variety
of environments, the tool chain must be consistent, which it isn't
on Windows. I suppose trivial applications can be distributed that
don't have a dependency on a tool chain but I wasn't talking about
them. Maybe we are just partitioning the problem space with
different terminology but if source distributions worked well
under Windows we wouldn't be having this discussion, right?

Thanks for all the Windows folks who have been patient with my
questions.

Gary Wright

 

[Gary Wright]

On *nix, all the binaries must be compiled and linked to the same
version of the C runtime.

Yep. My point wasn't that Unix was different in this regard but
that there is generally only one *style* of C runtime because
there is generally only a single prevailing compiler/build/tool
on any particular *nix platform. There is a multiplicative factor
in the Windows context that doesn't exist in the Unix context.

Gary Wright

 

[Austin Ziegler]

As for the MSVCRT issues, I have read many many mixed reports. The
summary is roughly like this, AFAIK:
- You *can* mix msvcrt versions safely *if* you don't pass structs /
pointers across the boundaries for which the api has changed. In
reality, for some applications, this is unworkable - however there are
several production ready gems out there that are working just fine on
the OCI (MSVCRT) which are compiled with VS 2005 (MSVCR7/8 (I can't
remember the exact version -> VS pairings, sorry).

Um. Yes and no. You can mix them *if* the libraries compiled against
those versions manage their own memory and file handles. In Unix, it's
common to see a library that malloc()s memory that you later have to
free(). It's *safer*, though, if you have a foo_free() in your libfoo
so you can let libfoo be responsible for freeing memory. (It's not
just memory; it's other resources, too. But memory is one of the big
issues.)

-austin