Pyrex installation on windows XP: step-by-step guide

[Julien Fiore]

Do you wand to install Pyrex on Windows ?

Here is a step-by-step guide explaining:

A) how to install Pyrex on Windows XP.
B) how to compile a Pyrex module.

Julien Fiore,
U. of Geneva

-------------------------------------------

### A) Pyrex installation on Windows XP ###


# step A.1 #
Install Python (we used version 2.4.2)


# step A.2 #
Run the windows installer for Pyrex (e.g. Pyrex-0.9.3.1.win32.exe),
available on the Pyrex homepage
(http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/)


# step A.3 #
Install Mingw, the gcc compiler for Windows, available at
http://www.mingw.org/download.shtml. (we downloaded the file
MinGW-5.0.2.exe and installed only the "base tool" (this includes
mingw-runtime 3.9, w32api-3.6, binutils 2.15.91 and gcc-core 3.4.2).
Add Mingw path ("C:\MinGW\bin") to the Windows "Path" environment
variable. If you already have cygwin installed, add C:\MinGW\bin before
the Cygwin path.


# step A.4 #
Create or edit the file "c:/Python2x/Lib/distutils/distutils.cfg" and
put the following into it:
[build]
compiler = mingw32

-------------------------------------------


### B) Create a Pyrex module ###


# step B.1 #
Create a working directory (e.g. D:\pyrex_module\). Write a pyrex
module and save it with a "pyx" extension (e.g. "primes.pyx", code
available on the Pyrex homepage)


# step B.2 #
Write the following python script and save it as "setup.py" in your
working directory.

from distutils.core import setup
from distutils.extension import Extension
from Pyrex.Distutils import build_ext
setup(
name = "PyrexGuide",
ext_modules=[
Extension("primes", ["primes.pyx"])
],
cmdclass = {'build_ext': build_ext}
)

If you want to compile several modules, duplicate the line starting
with "Extension" and replace "primes" by your module names.


# step B.3 #
In your working directory, create a batch file called
"build_and_install.bat" containing the following lines, where
"PythonXX" should be replaces by your Python version (e.g. "Python24").

C:\Python24\python.exe setup.py build_ext install
pause

To run the batch, double-click the file. You will see many "Warning"
messages during the building process: do not worry, it is normal.


# step B.4 #
Mission completed. The file "primes.pyd" (a "pyd" is a Python Extension
DLL, equivalent of .so in Unix) is now located in
"C:\Python24\Lib\site-packages" and the "primes" module is available in
Python. In your working directory, you can delete the file "primes.c"
and the "build" folder created by the building process.

Test your new module at the python shell:
[2, 3, 5, 7, 11, 13, 17, 19, 23, 29]

--------------------------------------------

 

[Michel Claveau]

Merci beaucoup !
Thank you very much!

 

[vj]

Can you use the stock python build or do you have to build python from
scratch with mingw to use pyrex modules built with mingw?

 

[Julien Fiore]

To install Pyton, I simply used the python Windows installer
(python-2.4.2.msi) available at python.org.

Julien

 

[Boris Borcic]

Do you wand to install Pyrex on Windows ?

Here is a step-by-step guide explaining:

A) how to install Pyrex on Windows XP.
B) how to compile a Pyrex module.

Julien Fiore,
U. of Geneva

Thanks. One detail missing : for this (step b3) to work smoothly, one needs to
make sure that (a copy of) eg python24.dll resides in Python24\libs\

-------------------------------------------

### A) Pyrex installation on Windows XP ###


# step A.1 #
Install Python (we used version 2.4.2)


# step A.2 #
Run the windows installer for Pyrex (e.g. Pyrex-0.9.3.1.win32.exe),
available on the Pyrex homepage
(http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/)


# step A.3 #
Install Mingw, the gcc compiler for Windows, available at
http://www.mingw.org/download.shtml. (we downloaded the file
MinGW-5.0.2.exe and installed only the "base tool" (this includes
mingw-runtime 3.9, w32api-3.6, binutils 2.15.91 and gcc-core 3.4.2).
Add Mingw path ("C:\MinGW\bin") to the Windows "Path" environment
variable. If you already have cygwin installed, add C:\MinGW\bin before
the Cygwin path.


# step A.4 #
Create or edit the file "c:/Python2x/Lib/distutils/distutils.cfg" and
put the following into it:
[build]
compiler = mingw32

-------------------------------------------


### B) Create a Pyrex module ###


# step B.1 #
Create a working directory (e.g. D:\pyrex_module\). Write a pyrex
module and save it with a "pyx" extension (e.g. "primes.pyx", code
available on the Pyrex homepage)


# step B.2 #
Write the following python script and save it as "setup.py" in your
working directory.

from distutils.core import setup
from distutils.extension import Extension
from Pyrex.Distutils import build_ext
setup(
name = "PyrexGuide",
ext_modules=[
Extension("primes", ["primes.pyx"])
],
cmdclass = {'build_ext': build_ext}
)

If you want to compile several modules, duplicate the line starting
with "Extension" and replace "primes" by your module names.


# step B.3 #
In your working directory, create a batch file called
"build_and_install.bat" containing the following lines, where
"PythonXX" should be replaces by your Python version (e.g. "Python24").

C:\Python24\python.exe setup.py build_ext install
pause

To run the batch, double-click the file. You will see many "Warning"
messages during the building process: do not worry, it is normal.


# step B.4 #
Mission completed. The file "primes.pyd" (a "pyd" is a Python Extension
DLL, equivalent of .so in Unix) is now located in
"C:\Python24\Lib\site-packages" and the "primes" module is available in
Python. In your working directory, you can delete the file "primes.c"
and the "build" folder created by the building process.

Test your new module at the python shell:
[2, 3, 5, 7, 11, 13, 17, 19, 23, 29]

 

[sturlamolden]

# step A.3 #
Install Mingw, the gcc compiler for Windows, available at
http://www.mingw.org/download.shtml. (we downloaded the file
MinGW-5.0.2.exe and installed only the "base tool" (this includes
mingw-runtime 3.9, w32api-3.6, binutils 2.15.91 and gcc-core 3.4.2).
Add Mingw path ("C:\MinGW\bin") to the Windows "Path" environment
variable. If you already have cygwin installed, add C:\MinGW\bin before
the Cygwin path.

I don't think this is safe. MinGW links with msvcrt.dll whereas the
main Python distribution links with msvcr71.dll (due to Visual C++
2003). It is not safe to mix and blend different C runtime libraries.
If you are to build a C extension with MinGW, you also need to build
Python against msvcrt.dll, i.e. you have to use a Python built with
MinGW or Visual C++ 6.0. There other option is to make MinGW link
against msvcr71.dll. I don't know if that is feasible.

 

[Julien Fiore]

Thanks for your remark, Sturlamolden.

Is there a free version of the "Visual C++ 2003" compiler available on
the web? I have found "Visual C++ 2005 Express edition"
(http://msdn.microsoft.com/vstudio/express/visualc/). According to
Micrsoft, it replaces VC++2003
(http://msdn.microsoft.com/visualc/vctoolkit2003/). Is VC++2005ee the
good compiler to compile a Pyrex module (or any Python extension) ?
Does it link with msvcr71.dll ?

 

[sturlamolden]

Thanks for your remark, Sturlamolden.

Is there a free version of the "Visual C++ 2003" compiler available on
the web? I have found "Visual C++ 2005 Express edition"
(http://msdn.microsoft.com/vstudio/express/visualc/). According to
Micrsoft, it replaces VC++2003
(http://msdn.microsoft.com/visualc/vctoolkit2003/). Is VC++2005ee the
good compiler to compile a Pyrex module (or any Python extension) ?
Does it link with msvcr71.dll ?

The bad news is that "Visual C++ 2005 Express" links with msvcr80.dll,
which incompatible with both msvcrt.dll and msvcr71.dll. What you need
is the "Microsoft .NET Framework SDK Version 1.1". It contains version
7.1 of Microsoft's C/C++ compiler and links with the correct CRT.

http://tinyurl.com/5flob

I am not sure if this is an optimizing compiler. Microsoft did not give
away their optimizing compiler prior to "Visual C++ 2005 Express". Even
the standard version of Visual Studio did not have an optimizing
compiler, it only shipped with the professional and enterprise
versions. If this compiler does not optimize, you may try to make
"Visual C++ 2005 Express" use the import library for msvcr71.dll which
ships with the .NET SDK.

Now you know the meaning of the word "DLL HELL".

 

[sturlamolden]

I don't think this is safe. MinGW links with msvcrt.dll whereas the
main Python distribution links with msvcr71.dll (due to Visual C++
2003).

In order to make minGW link with msvcr71.dll, edit the text file

c:\mingw\lib\gcc\mingw32\3.2.4\specs

and change "-lmsvcrt" to "-lmsvcr71".

Now MinGW will link with the same CRT as Python 2.4.

 

[Julien Fiore]

edit the text file "c:\mingw\lib\gcc\mingw32\3.2.4\specs"
and change "-lmsvcrt" to "-lmsvcr71".

Thank you very much sturlamolden,

This procedure should be added to the "step-by-step" guide (see 1st
message of this thread) as "step A.5".

For ignorant people like me, CRT = "C runtime library".

 

[Julien Fiore]

I added "step A.5" to the guide and published it on the Python wiki, so
that anyone can update it easily:
http://wiki.python.org/moin/PyrexOnWindows

 

[Jim Lewis]

and change "-lmsvcrt" to "-lmsvcr71".

But then I get this error: Python was built with version 7.1 of Visual
Studio, and extensions need to be built with the same version of the
compiler, but it isn't installed.
I want to use mingw.

 

[=?ISO-8859-1?Q?Gonzalo_Monz=F3n?=]

sturlamolden escribió:

Julien Fiore wrote:




I don't think this is safe. MinGW links with msvcrt.dll whereas the
main Python distribution links with msvcr71.dll (due to Visual C++
2003). It is not safe to mix and blend different C runtime libraries.
If you are to build a C extension with MinGW, you also need to build
Python against msvcrt.dll, i.e. you have to use a Python built with
MinGW or Visual C++ 6.0. There other option is to make MinGW link
against msvcr71.dll. I don't know if that is feasible.

I use Python 2.4.3 (msvcrt71) and I succesfully installed the last
version of binutils, pyrex and MinGW, some weeks ago, using Julien Fiore
step-by-step guide, so "my" MinGW is linking with msvcrt71.dll, with the
default configuration.

I don't understand why do you say MinGW links with msvcrt.dll... perhaps
you've got an older version than the ones Julien posted?

Hope it helps,
Gonzalo.

 

[Robert Kern]

In order to make minGW link with msvcr71.dll, edit the text file

c:\mingw\lib\gcc\mingw32\3.2.4\specs

and change "-lmsvcrt" to "-lmsvcr71".

Now MinGW will link with the same CRT as Python 2.4.

However, MinGW's header files are still written for MSVCRT.dll. IIRC, isupper()
and friends use a jump table called _ctype in MSVCRT.dll which is missing in
MSVCR71.dll. Some extensions will compile okay, some won't.

The order in which distutils adds 'msvcr71' to the list of libraries seems to
make sure that the PYD picks up free, malloc, printf, and other important
functions from MSVCR71.dll first before trying MSVCRT.dll. However, C++ modules
that use std::cout, for example, will crash.

AFAICT, there is currently no general solution for compiling Python 2.4
extension modules with MinGW. There probably won't be one unless MinGW is
patched to fully use MSVCR71.dll.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Robert Kern]

sturlamolden escribió:

I use Python 2.4.3 (msvcrt71) and I succesfully installed the last
version of binutils, pyrex and MinGW, some weeks ago, using Julien Fiore
step-by-step guide, so "my" MinGW is linking with msvcrt71.dll, with the
default configuration.

I don't understand why do you say MinGW links with msvcrt.dll... perhaps
you've got an older version than the ones Julien posted?

Nope. He said it because it is true. Use Dependency Walker
(http://www.dependencywalker.com) to see for yourself what DLLs are linked in.
In fact, *both* DLLs are linked in. Sometimes this works fine, other times it
does not.

If you would like a more visceral demonstration, write a small C++ extension
that uses std::cout.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Robert Kern]

But then I get this error: Python was built with version 7.1 of Visual
Studio, and extensions need to be built with the same version of the
compiler, but it isn't installed.
I want to use mingw.

You have to tell distutils to use mingw:

python setup.py build_ext --compiler=mingw32

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Guest]

Robert Kern escribió:

Gonzalo Monzón wrote:



Nope. He said it because it is true. Use Dependency Walker
(http://www.dependencywalker.com) to see for yourself what DLLs are linked in.
In fact, *both* DLLs are linked in. Sometimes this works fine, other times it
does not.

If you would like a more visceral demonstration, write a small C++ extension
that uses std::cout.

I see there are both libraries linked in my pyrex modules... However,
when one should expect to have problems with this "dll nightmare", if
you always provide the right msvcr71.dll bundled within your project -I
mean, using py2exe by example- ?

Of course if you didn't bundle the right crt, it does depend on the
available crt on target system... then you would be in trouble if user
doesn't have msvcrt71, or anyway, could be in trouble linking the
"right" library?

Thanks for any explanation.

Regards,
Gonzalo

 

[Robert Kern]

I see there are both libraries linked in my pyrex modules... However,
when one should expect to have problems with this "dll nightmare", if
you always provide the right msvcr71.dll bundled within your project -I
mean, using py2exe by example- ?

Of course if you didn't bundle the right crt, it does depend on the
available crt on target system... then you would be in trouble if user
doesn't have msvcrt71, or anyway, could be in trouble linking the
"right" library?

If you use std::cout in a C++ extension module that was built with the stock
Python 2.4 distutils and the stock MinGW, you will crash the interpreter.
"Bundling the right CRT" is not the issue. *Building* with the right CRT is.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Jim Lewis]

Thanks. I had done that but it seems I had to remove "install". Now it
works.

 

[Guest]

Robert Kern escribió:

Gonzalo Monzón wrote:




If you use std::cout in a C++ extension module that was built with the stock
Python 2.4 distutils and the stock MinGW, you will crash the interpreter.
"Bundling the right CRT" is not the issue. *Building* with the right CRT is.

Ok, I left building out of the equation... I misunderstood the issue. Of
course, you need to build always using the same crt.

Regards,
Gonzalo