Pyrex installation on windows XP: step-by-step guide

[Claudio Grondi]

The bad news is that "Visual C++ 2005 Express" links with msvcr80.dll,
which incompatible with both msvcrt.dll and msvcr71.dll. What you need
is the "Microsoft .NET Framework SDK Version 1.1". It contains version
7.1 of Microsoft's C/C++ compiler and links with the correct CRT.

http://tinyurl.com/5flob

I am not sure if this is an optimizing compiler.

The free available Visual C++ 2003 compiler is the optimizing one.
You have to get it separately downloading the Visual C++ Toolkit 2003
which comes with the Professional version of Microsoft Visual C++ .NET
2003 compiler and linker.

Claudio

 

[Jim Lewis]

Still problems :-(

I have a directory c:\data\code\test\pyrex containing:

build_and_install.bat:
"C:\program files\Python\python.exe" setup.py build_ext
--compiler=mingw32
pause

setup.py:
from distutils.core import setup
from distutils.extension import Extension
from Pyrex.Distutils import build_ext
setup(
name = "PyrexGuide",
ext_modules=[
Extension("worldimport", ["world.pyx"])
],
cmdclass = {'build_ext': build_ext}
)

world.pyx - pyrex code

run.py:
import worldimport
....
========================
The batch file creates a build folder with subfolders containing:
world.pyd, world.def, world.o, worldimport.def

But running run.py gives: "ImportError: No module named worldimport"

Should worldimport.def be going to C:\program
files\python\Lib\site-packages?
Why isn't it?

 

[Robert Kern]

Still problems :-(

I have a directory c:\data\code\test\pyrex containing:

build_and_install.bat:
"C:\program files\Python\python.exe" setup.py build_ext
--compiler=mingw32
pause

But running run.py gives: "ImportError: No module named worldimport"

Should worldimport.def be going to C:\program
files\python\Lib\site-packages?
Why isn't it?

You also have to run the distutils install command as well to place the results
of the build in site-packages.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Jim Lewis]

Thanks but now another problem :-(
Examples in books show importing a global so why does below give:
AttributeError: 'module' object has no attribute 'globalvar'

primes.pyx:
from run import globalvar
def prime(int kmax):
result = [run.globalvar]
....

run.py:
from primes import prime
globalvar = 999
while True:
print prime (10)

 

[sturlamolden]

I use Python 2.4.3 (msvcrt71) and I succesfully installed the last
version of binutils, pyrex and MinGW, some weeks ago, using Julien Fiore
step-by-step guide, so "my" MinGW is linking with msvcrt71.dll, with the
default configuration.

A successful build (compile and linkage) does not imply that you are
linking with the same crt as Python. It just means that there are no
syntax errors (successful compile) or unresolved external symbols
(successful linkage) in your code.

I don't understand why do you say MinGW links with msvcrt.dll... perhaps
you've got an older version than the ones Julien posted?

Because it does not! What happens is that MinGW links with the import
library for msvcrt.dll. The compiler does not complain bacause there
are no syntax errors. The linker does not complain bacuase there are no
unresolved external symbols (after all, msvcrt is a standard crt). When
you run your program, Windows search the search path for msvcrt.dll,
finds the dll in c:\windows\system32, and loads it into your process
image. This produces no errors either. So intitially everything seems
to be ok.

But... The process image for your Pyrex extension is shared with the
Python interpreter. Previously, msvcrt71.dll was loaded into into the
porcess image when Python started. Now both crts reside in the process,
and Python and your Pyrex extension starts to call their respective
crts. The crts interfere with each other and you get very unpredictable
results.

This is not just a Python problem. Microsoft created this problem when
they started to publish multiple versions of their crt. Whenever a
process has imported different crts, there will inevitably be run-time
conflicts. Windows is a "component" based operating system. If you are
e.g. using in proc COM objects (also known as "ActiveX Components"),
you can only pray to God that the author did not use a different crt
than you. If you are loading a precompiled DLLs, you can only pray to
God that the author did not use a different crt than you. Previously
this was not an issue, as Windows had exactly one shared crt. Now this
is all messed up. Why did Microsoft decide to create "C runtime DLL
Hell"? I have no idea.

 

[=?ISO-8859-1?Q?Gonzalo_Monz=F3n?=]

Hi,

sturlamolden escribió:

A successful build (compile and linkage) does not imply that you are
linking with the same crt as Python. It just means that there are no
syntax errors (successful compile) or unresolved external symbols
(successful linkage) in your code.





Because it does not! What happens is that MinGW links with the import
library for msvcrt.dll. The compiler does not complain bacause there
are no syntax errors. The linker does not complain bacuase there are no
unresolved external symbols (after all, msvcrt is a standard crt). When
you run your program, Windows search the search path for msvcrt.dll,
finds the dll in c:\windows\system32, and loads it into your process
image. This produces no errors either. So intitially everything seems
to be ok.

Does this happen if you're releasing, i.e. a built exe with py2exe,
where you supply the right crt? or if you do supply the right crt on the
application folder? ... I see my pyrex extension imports with both,
msvcrt and msvcrt71, and when I do execute the python program, I see
with dependency tool that only msvcr71 gets loaded and I suppose no more
crt's are loaded at runtime, as it is shared by my extension and the
Python interpreter & std. extensions.

In a posterior email on this thread i asked about this issue:

I see there are both libraries linked in my pyrex modules... However,
when one should expect to have problems with this "dll nightmare", if
you always provide the right msvcr71.dll bundled within your project -I
mean, using py2exe by example- ?

Of course if you didn't bundle the right crt, it does depend on the
available crt on target system... then you would be in trouble if user
doesn't have msvcrt71, or anyway, could be in trouble linking the
"right" library?

Thanks for any explanation.

Regards,
Gonzalo

But... The process image for your Pyrex extension is shared with the
Python interpreter. Previously, msvcrt71.dll was loaded into into the
porcess image when Python started. Now both crts reside in the process,
and Python and your Pyrex extension starts to call their respective
crts. The crts interfere with each other and you get very unpredictable
results.

Why msvcrt is being loaded later than msvcrt71? why both dll are being
loaded it migw links to both and one is already loaded by python
interpreter? perhaps the only way to achieve total reliability is to
compile using the same where Python & standard extensions where
compiled? In the case, does any sense to compile using mingw for
standard python >= 2.4 custom extensions?

This is not just a Python problem. Microsoft created this problem when
they started to publish multiple versions of their crt. Whenever a
process has imported different crts, there will inevitably be run-time
conflicts. Windows is a "component" based operating system. If you are
e.g. using in proc COM objects (also known as "ActiveX Components"),
you can only pray to God that the author did not use a different crt
than you. If you are loading a precompiled DLLs, you can only pray to
God that the author did not use a different crt than you. Previously
this was not an issue, as Windows had exactly one shared crt. Now this
is all messed up. Why did Microsoft decide to create "C runtime DLL
Hell"? I have no idea.

I'm aware of this, but I am not about every detail.

Regards,
Gonzalo

 

[sturlamolden]

Does this happen if you're releasing, i.e. a built exe with py2exe,
where you supply the right crt? or if you do supply the right crt on the
application folder? ...

It does not matter which crt you "supply". The dynamic linker will
attempt to load the crt specified by the import library that you linked
when the extension was built. What will happen, is that Windows will
search through its "search path" (i.e. the current folder and folder
specified by the PATH environment variable) until it find the crt it
wants.

I see there are both libraries linked in my pyrex modules... However,
when one should expect to have problems with this "dll nightmare", if
you always provide the right msvcr71.dll bundled within your project -I
mean, using py2exe by example- ?

Windows will e.g. look in its own directories as well. You can bundle
msvcr71.dll, but msvcrt.dll will always be available in
c:\windows\system32. If you used an import library for msvcrt.dll, then
msvcrt.dll will be loaded.

Also you cannot legally bundle msvcr71.dll unless you hold a license
for Visual Studio 2003. There are no other way to legally redistribute
msvcr71.dll. Other versions of Visual Studio allows redistribution of
other versions of the crt. Visual C++ Toolkit 2003 only allows a
statically linked crt.

Of course if you didn't bundle the right crt, it does depend on the
available crt on target system...

No it does not. It always depends on the CRT specified by the import
library. If you linked against msvcr71.lib, then the extension needs to
load msvcr71.dll. If Windows cannot find msvcr71.dll on the search
path, the process will be killed. It is your responsibility to make
sure msvcr71.lib reside on the search path. If it cannot be found,
Windows will not load a different crt instead. Windows will pop up a
warning message telling you that msvcr71.dll cannot be found, and then
kill the process.

If you linked against msvcrt.lib, then the extension will load
msvcrt.dll.

 

[greg]

Thanks but now another problem :-(
Examples in books show importing a global so why does below give:
AttributeError: 'module' object has no attribute 'globalvar'

primes.pyx:
from run import globalvar
def prime(int kmax):
result = [run.globalvar]
...

run.py:
from primes import prime
globalvar = 999
while True:
print prime (10)

The first thing run does is import primes, which
immediately imports run and tries to access the
name globalvar in it, before it has been defined.

There are ways of fixing that, but for various
reasons it's usually a bad idea for subsidiary
modules to try to import things from the main
module of a program. It would be better to move
globalvar into a third module.

Even better again would probably be not to use
a global at all, but pass it in as a parameter
to the prime() function.

 

[Dennis Lee Bieber]

Thanks but now another problem :-(
Examples in books show importing a global so why does below give:
AttributeError: 'module' object has no attribute 'globalvar'

primes.pyx:
from run import globalvar
def prime(int kmax):
result = [run.globalvar]

One: You imported only the name "globalvar", you did not import the
"run" namespace, so there is no "run.globalvar" (I'm not sure why you
are defining result as a one-element list)

run.py:
from primes import prime
globalvar = 999
while True:
print prime (10)

Does Pyrex behave differently from normal Python?

python run.py

runs with the implied namespace of "__main__", not of "run", so even if
you had set globalvar /before/ the import, it would not be found. You
have
__main__.globalvar = 999
but are looking for
run.globalvar
--
Wulfraed Dennis Lee Bieber KD6MOG
(e-mail address removed) (e-mail address removed)
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: (e-mail address removed))
HTTP://www.bestiaria.com/