[EVALUATION] - E02 - Support for MinGW Open Source Compiler

[Bruno Desthuilliers]

Stephen Kellett a écrit :

He is already badgering the Ruby guys. Without about as much success as
this newsgroup. When he doesn't get what he wants a post along the lines
of

"XXXX lang is only suited to small projects and not real world
industrial projects"
or
"the community doesn't care"
will appear.

Its quite incredible - in the time he has spent complaining he could
have done his own research and written some useful tools. I know how
long it took me to write my first major C++ app that interfaced with
Python and Ruby. It was less time than he has spent complaining - and
that included rebuilding Python/Ruby, inspecting the source for what I
needed and performing many experiments before I succeeded.

Just google for Ilias Lazaridis... You may observe that our new troll
has already played the same game on many newsgroups, forums and whatever...

 

[Pat]

Stephen Kellet said:

<quote>
Pat, could you include some context in your replies? I have no idea if
you are replying to my comments about Visual Studio Express or someone
else? The only text I see in your replies is what you write, no text
from the posting you are replying to. As it is I've ignored all your
replies so far as I'm not sure I'm the person you are addressing (until
I saw the above, now I'm confused).
</quote>

Sorry about that. I'm replying using Google Groups and making a total
mess of things. :-(

David Fraser asked if I had tried to compile a Python extension for
Python 2.4 using minGW. I said that I had not (but I have for Python
2.3.5). Here is the rest of my reply, for future reference:

We ran into some issues with Python 2.4 that caused us
to return to Python 2.3.5. But I would really like to upgrade to
Python 2.4. So I started researching the subject before I did
anything.

If you are telling me that minGW can compile extensions that are
compatible with the Python 2.4 that uses msvcr71.dll, then that is good
news indeed. Is there anything that needs to be configured or patched
to make this happen? And how does minGW know which dll to link? What
if I have both versions of Python installed - 2.3.5 and 2.4? Is there
an easy way to detect this and switch between the two dlls?

If I'm asking questions already answered elsewhere, I'd love a link to
that resource, if you have it.

Patrick K. O'Brien
Orbtech http://www.orbtech.com
Schevo http://www.schevo.org
Pypersyst http://www.pypersyst.org

 

[Stephen Kellett]

That answers the cost question (assuming that your interpretation of
the licensing is correct, since I'm not a lawyer nor qualified to
render much of an opinion on that). But there is still the issue of
going through a bunch of configuration hassle that scares me away from

What configuration hassle? Can't be any harder than specifying a
different CRT surely?

Stephen

 

[Stephen Kellett]

if I have both versions of Python installed - 2.3.5 and 2.4? Is there
an easy way to detect this and switch between the two dlls?

Easy? Depends what you call easy.

a) You just need to detect if pythonNN.dll is implicitly linked to
msvcrt.dll or msvcrXX.dll (where XX indicates a VS studio number,
currently XX can only be 71, but if Python is done for Visual Studio
2005 (8.0) then XX may also have a value of 80).

b) GetModuleHandle(name of DLL from a) to get the DLL handle.

c) GetProcAddress on the respective to get the C runtime function you
want to call (for example, fopen).

d) call the function.

You either need (a) or all of (a)...(d) depending on what you are doing.
Those embedding Python but no idea of the Python version in advance will
do the latter.

If I'm asking questions already answered elsewhere, I'd love a link to
that resource, if you have it.

I'm probably not the best person to answer such a question. I'm mainly a
C/C++ (with Java under protest) person dabbling in Python/Ruby.

Stephen

 

[Pat]

"Pat"wrote:
code

the python source or the extension source?


would indicate the latter. setup.py handles that just fine, if you have the
right tools. no pain at all.


if your problem is compiling C extension modules, I suggest getting a
good compiler. I've done that, and compiling C extension modules is
no problem at all.

if your problem is that you don't want to use a good compiler, or that
your company cannot afford to buy you a compiler, or you have other
reasons to chose the "pain in the ass" way over the "it just works" way,
I'm afraid I cannot help you. my time's too precious to waste on inferior
tools.

Okay, I think we are pretty much talking about the same thing. My
problem is not that I'm unable or unwilling to purchase a good
compiler. My problem is that I don't want to make it a requirement of
my users. The twist is that my users will be working out of a
Subversion repository that includes source code for extensions written
in C. Those extensions need to be compiled, and the code changes too
frequently for me to want to deal with supplying compiled binaries.

So I'm looking for options for Windows users who do not have, and are
unwilling to get, a Microsoft compiler. For some users, minGW is an
attractive option. For those who want to use minGW, I'm trying to
establish whether or not minGW is a viable option, particularly for
Python 2.4. I was under the impression that there were possible
advantages to compiling Python itself using minGW, but that that
required some patches that hadn't been applied to Python (for reasons
unknown to me). Said patches are available here:

http://jove.prohosting.com/iwave/ipython/pyMinGW.html

I'm also under the impression that there are possible disadvantages to
compiling Python itself with anything other than the Microsoft's
optimizing compiler, such as performance. But I haven't seen any
numbers on that, so I don't have information one way or the other.

It may also be the case that minGW can now correctly compile extension
that link to the newer runtime used by Python 2.4, but that wasn't
always the case. If that's true, that would eliminate the primary
reason for wanting to compile Python itself with minGW, though
philosophical differences with MS might motivate some to want to avoid
the MS compilers altogether.

Perhaps the only thing that's a mess is my understanding of the
situation. But even if that's the case, I don't think I'm alone, based
on all the threads I've read over the past couple of days. ;-)

Thanks in advance for any help anyone can offer to reduce my confusion.

Patrick K. O'Brien
Orbtech http://www.orbtech.com
Schevo http://www.schevo.org
Pypersyst http://www.pypersyst.org

 

[Pat]

In message <[email protected]>, Pat
from

What configuration hassle? Can't be any harder than specifying a
different CRT surely?

I don't want to have to ask users of my code to have to go through
this:

http://www.vrplumber.com/programming/mstoolkit/

What I want is to provide everything a user would need within my own
Subversion repository. Any C code would be compiled behind the scenes
using free tools that I'm able to include in my repository (without
having to hire a lawyer first).

Having to tell users that they need to download, install, and configure
all this additional compiler stuff is asking too much from my potential
user base, since I'm also targeting novices and developers from other
languages for whom C compiler stuff is going to be a barrier to entry.

I hope that makes sense.

 

[A.B., Khalid]

There have been extensive discussions about these issues on the
Python-Dev mailing list over the past couple of months (mostly in
December, but continuing to the present - see
http://mail.python.org/pipermail/python-dev/2004-December/thread.html
as a starting point), which seem to have fizzled out or at least
haven't resolved much. The discussions made reference to work that has
already been done to allow Python to be compiled with minGW:

pyMinGW is a patch to Python source that aims to get Python to compile
under MinGW

http://jove.prohosting.com/iwave/ipython/pyMinGW.html

I've not seen any commentary on the quality of this patch, so that
doesn't appear to be the reason it hasn't been officially adopted.
Reading all the threads from Python-Dev has not enlightened me at all
as to what the underlying reason is for not adopting these changes.
Maybe there are good reasons, I just couldn't find them, and I'm
usually pretty good with Google.

If passing all the regression tests of the official Windows Python
distribution is an indication of the quality of patch-- and pyMinGW
patched and MinGW built Python does pass all of them-- then one is
inclined to say that pyMinGW is a good patch.

The reason why it is, on the other hand, not included in the official
distribution is threefold.

1. Contrary to what many might imagine, I don't think enough people use
MinGW to frankly justify any extra effort beyond pyMinGW.
2. Given number 1 above, this patch, I believe, and I could be
mistaken, must not rush to be included in Python's core; people like
your esteemed person should test it (note that it is designed not to
interfere with your trusted and working official Python, if any); it is
only when enough people do such testing that there will be a case for
it to be included in Python's core.
3. Finally. there is nothing wrong with third-party patches if they get
the job done, which I believe is the case with pyMinGW.


Regards,
Khalid

 

[Stephen Kellett]

Easy? Depends what you call easy.

a) You just need to detect if pythonNN.dll is implicitly linked to
msvcrt.dll or msvcrXX.dll (where XX indicates a VS studio number,
currently XX can only be 71, but if Python is done for Visual Studio
2005 (8.0) then XX may also have a value of 80).

I should have also mentioned that if you don't know how to do get the
list of imported DLLs you can grab the source code for the PE Explorer
DLL from:

http://www.objmedia.demon.co.uk/freeSoftware/peFileDLL.html

If you want to see how to use this DLL you can also grab:

http://www.objmedia.demon.co.uk/freeSoftware/peFileExplorer.html

Stephen

 

[Fredrik Lundh]

Okay, I think we are pretty much talking about the same thing. My
problem is not that I'm unable or unwilling to purchase a good
compiler. My problem is that I don't want to make it a requirement of
my users. The twist is that my users will be working out of a
Subversion repository that includes source code for extensions written
in C. Those extensions need to be compiled, and the code changes too
frequently for me to want to deal with supplying compiled binaries.

here's what I do to supply compiled binaries for 2.1 through 2.4:

> mkall
> upload dist

So I'm looking for options for Windows users who do not have, and are
unwilling to get, a Microsoft compiler. For some users, minGW is an
attractive option. For those who want to use minGW, I'm trying to
establish whether or not minGW is a viable option, particularly for
Python 2.4.

here's what I just did (time in minutes):

+00: googled for the mingw home page
+00: found the mingw download page
+02: finally figured out what to download
+03: noticed that my usual SF site only offered 1K/s; aborted download
+07: finished downloading the mingw kit from another SF site
+17: finished installing
+18: added \mingw\bin to the path
+18: typed "python setup.py install --compiler=mingw32"
+18: got a linker error; googled for help
+19: copied python24.dll to \mingw\lib
+20: finished building the sample library (cElementTree); all tests pass

so what's your excuse for not doing this? ;-)

</F>

 

[Fredrik Lundh]

Having to tell users that they need to download, install, and configure
all this additional compiler stuff is asking too much from my potential
user base, since I'm also targeting novices and developers from other
languages for whom C compiler stuff is going to be a barrier to entry.

I hope that makes sense.

telling them to download and configure subversion just to be able to use
your stuff doesn't make much sense, no.

(not that subversion is hard to install, though, but it's a lot harder than just
running a bdist installer).

</F>

 

[Stephen Kellett]

I don't want to have to ask users of my code to have to go through
this:

http://www.vrplumber.com/programming/mstoolkit/

OK. I misunderstood your original intent. I thought the original
question was can we redistribute this DLL? I answered this bit. I wasn't
attempting to answer the larger question about building extensions etc.
My development environment has all the things in that article by default
- I wouldn't even start work without them, so I had them "taken for
granted" in my thoughts. I can see your problem but I have no easy
answer. Sorry.

 

[Fredrik Lundh]

If you are telling me that minGW can compile extensions that are
compatible with the Python 2.4 that uses msvcr71.dll, then that is good
news indeed. Is there anything that needs to be configured or patched
to make this happen? And how does minGW know which dll to link? What
if I have both versions of Python installed - 2.3.5 and 2.4? Is there
an easy way to detect this and switch between the two dlls?

just use the right Python to run the setup.py script, and distutils will take
care of the rest.

</F>

 

[Fredrik Lundh]

Easy? Depends what you call easy.

in the context of "building a C extension when you have multiple versions
installed", the answer is "easy". setup.py automatically finds the right DLL,
mingw seems take care of the rest.

a) You just need to detect if pythonNN.dll is implicitly linked to msvcrt.dll or msvcrXX.dll
(where XX indicates a VS studio number, currently XX can only be 71, but if Python is done for
Visual Studio 2005 (8.0) then XX may also have a value of 80).

b) GetModuleHandle(name of DLL from a) to get the DLL handle.

c) GetProcAddress on the respective to get the C runtime function you want to call (for example,
fopen).

d) call the function.

You either need (a) or all of (a)...(d) depending on what you are doing. Those embedding Python
but no idea of the Python version in advance will do the latter.

if someone needs code that does it all, check the exemaker sources:

http://effbot.org/zone/exemaker.htm

</F>

 

[Pat]

here's what I do to supply compiled binaries for 2.1 through 2.4:



here's what I just did (time in minutes):

+00: googled for the mingw home page
+00: found the mingw download page
+02: finally figured out what to download
+03: noticed that my usual SF site only offered 1K/s; aborted download
+07: finished downloading the mingw kit from another SF site
+17: finished installing
+18: added \mingw\bin to the path
+18: typed "python setup.py install --compiler=mingw32"
+18: got a linker error; googled for help
+19: copied python24.dll to \mingw\lib
+20: finished building the sample library (cElementTree); all tests pass

so what's your excuse for not doing this? ;-)

A few things. Primarily the fact that I'm not very experienced in C
(the extensions that I need to have compiled are not written by me).
Secondarily, the fact that the discussion threads I read made it seem
much more complicated than what you just described. Third, the fact
that some of the code we've tried to compile didn't compile cleanly,
the way your cElementTree did (but I can't remember what exactly the
problem was and I didn't do the compiling). And, finally, an aversion
to trial-and-error solutions. I prefer to Google and ask questions
when I'm out of my element.

Thanks for the info.

 

[Stephen Kellett]

I like to synchronize any efforts with the existing ones.

I assume the reason for doing that would be to avoid duplicating effort?
If that is the case why do you want lots of people all to answer your
questionnaire. Thats a huge duplication of effort.

A much more effective use of effort would be for you to do the research
and when you are done to ask people to comment on the *results* of the
research, not the questions.

This accomplishes several things:
o Reduction in duplication of effort.
o Demonstrates to everyone else that you are prepared to do some work
before asking questions.
o When you find that the response is much more welcoming than you have
had so far you will have learnt that you need to put some effort in to
get some reward out (which is what most of us have been trying to tell
you).

Of course, based on your behaviour here (and currently in comp.lang.ruby
where you are spinning a nice yarn in not bothering to read up on the
answers people give you, even when their answers are detailed), I have
not much hope of you taking the above approach.

Stephen

 

[jfj]

Ilias Lazaridis wrote:


Why should they? It already runs on Windows with a freely available
compiler.

The point is that the freely available compiler wouldn't be free if
it wasn't for gcc. Just for that I _believe_ python, being open source,
should support mingw as the default. But I *don't care* and I don't
mind, really


jfj

 

[Fredrik Lundh]

A few things. Primarily the fact that I'm not very experienced in C
(the extensions that I need to have compiled are not written by me).
Secondarily, the fact that the discussion threads I read made it seem
much more complicated than what you just described.

from two posts at the top of this thread:

"Writing a setup.py and running
python setup.py build_ext --compiler=mingw32
works for me *without* any more work. Things can't get much
simpler."

and

"The mingw compiler *is* supported through distutils. distutils
can straightforwardly be configured to build extensions with
mingw."

(now go read Ilias replies to those posts)

Third, the fact that some of the code we've tried to compile didn't compile
cleanly, the way your cElementTree did (but I can't remember what exactly
the problem was and I didn't do the compiling).

was that code tested under gcc? code that compiles under visual C doesn't
necessarily compile silently under gcc, but fixing that is usually pretty trivial
(no worse than porting mostly portable code between platforms).

And, finally, an aversion to trial-and-error solutions. I prefer to Google and
ask questions when I'm out of my element.

sure didn't sound that way when you entered this thread:

"So in an effort to make some headway, I'm going to try to summarize the
current state of affairs. The bottom line is that compiling C extension modules
on the Windows platform for Python 2.4 is, today, a royal pain in the ass.
Period. Here's why. /.../"

now go download MinGW and figure out what's wrong with your C code.
if you get stuck, post the error messages, and I'm sure some c.l.pythoneer
will help you sort it out.

</F>

 

[Steve Horsley]

LOL Falling off my chair!!!!!!

I think the expression you are looking for is ROFL!

 

[Mike Meyer]

If it is a programming language, the requirement "using an open-source
toolchain" is a rational and valid one.

It is. However, mingW has nothing to do with "using an open-sourcer
toolchain".

Python runs in an environment with a full, open-source tool chain. You
can use it on Linux or any of the various BSDs. It also runs in an
environment where you can't have an open-source toolchain: Windows.

Once you've given up on an open-source environment, arguing about
trivia like which free compiler you are going to use is a waste of
time. Either use a real open-source environment, or live with the
closed-source tools/environment that someone is willing to support.

Or, of course, support the environment you want yourself.

<mike

 

[Ilias Lazaridis]

I assume the reason for doing that would be to avoid duplicating effort?

[...] - (off-topic suggestions processing model)

Your suggestions were irrelevant to me.

Please avoid further off-topic posts.

..