Windows Debugging w/o MS

[Christopher Anderson]

Hello all. I have a fairly fundamental question that I have been
googling like crazy, but I can only seem to find really outdated or
unreliable info:

I am trying to build an extension module written in C++ on windows XP.
I am trying to use only open-source tools, such as mingw. I'm using
the Python 2.5 official, and it seems to compile my module just fine
(I'm using the --compiler=mingw32). I can also import the module. The
problem is, when I attempt to use it, I get a segfault. Now, I'm
pretty sure this segfault is just a bug in my C++ code. So of course,
I would like to debug this thing somehow. I tried using the mingw gdb
giving it my python.exe to run, but obviously this python has no debug
info, and wasn't even compiled with mingw. I was hoping it would still
somehow debug my extension module right, but when I do a backtrace
after it segfaults, I get nothing useful.

I've tried compiling python from source, and my extension module,
using MSVC8 (free express version), and I managed to get this to work.
The thing is, I don't want to have to recompile every single python
package I need (wxPython, SciPy, etc).

So basically, I need some general advice for how to compile and debug
extension modules on windows using only open-source tools. I am still
amazed that MSVC7.1 was chosen for the official python release and not
mingw, has anyone explained this?

Thank you so much for your time,
Chris Anderson

 

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

I am trying to build an extension module written in C++ on windows XP.

I am trying to use only open-source tools, such as mingw. I'm using
the Python 2.5 official, and it seems to compile my module just fine
(I'm using the --compiler=mingw32). I can also import the module. The
problem is, when I attempt to use it, I get a segfault.

Does it segfault if you import it, or just when you invoke functions
of the module?

You should get it in a state where you can import it just fine, e.g.
by removing stuff from the init function.

Now, I'm
pretty sure this segfault is just a bug in my C++ code. So of course,
I would like to debug this thing somehow. I tried using the mingw gdb
giving it my python.exe to run, but obviously this python has no debug
info, and wasn't even compiled with mingw. I was hoping it would still
somehow debug my extension module right, but when I do a backtrace
after it segfaults, I get nothing useful.

If you can import safely, you should do this:
1. start gdb, with python.exe as the program
2. r(un) it, in gdb
3. import your_module
4. Ctrl-c (or otherwise get back into the debugger)
5. see whether you can set breakpoints on your function

gdb should be able to figure out that your extension module was
loaded and has debug information. Set a breakpoint and see whether
it shows line numbers. If it doesn't, it's because your module does
not have debug information.

I've tried compiling python from source, and my extension module,
using MSVC8 (free express version), and I managed to get this to work.
The thing is, I don't want to have to recompile every single python
package I need (wxPython, SciPy, etc).

You should also be able to compile Python with the mingw compiler,
giving you gdb debug information for all modules.

So basically, I need some general advice for how to compile and debug
extension modules on windows using only open-source tools. I am still
amazed that MSVC7.1 was chosen for the official python release and not
mingw, has anyone explained this?

Python uses the "system compiler" on all systems, and the system
compiler on Microsoft Windows, is, well, Microsoft C.

Using that compiler works better in a number of important use cases.
For example, it is absolutely necessary to be able to compile
extension modules that use C++, COM, and the Microsoft Foundation
Classes. Such extension modules must be compiled with MSC, since
g++ is not capable of compiling them (at least for MFC). One
important extension module that relies on such technology is
Mark Hammond's PythonWin.

If Python was build with mingw, it might be that such modules could
not work anymore, depending on how precisely the mingw build is
done (e.g. what CRT you would use).

Regards,
Martin

 

[Thomas Lenarz]

problem is, when I attempt to use it, I get a segfault. Now, I'm
pretty sure this segfault is just a bug in my C++ code. So of course,
I would like to debug this thing somehow. I tried using the mingw gdb
giving it my python.exe to run, but obviously this python has no debug
info, and wasn't even compiled with mingw. I was hoping it would still
somehow debug my extension module right, but when I do a backtrace
after it segfaults, I get nothing useful.

I have no experience in writing python extension-modules. However, I
would give the following scheme a try. It helped myself a lot while
debugging segfaulting modules plugged into a framework, for which I
haven't had the source-code. It is done easyly and quickly. If it
doesn't work you won't lose a lot of time:

1. At the earliest stage of your own C++-Code, code an infinite loop
depending on a single variable.

e.g.: int i=0; for(;i==0;

2. When you start your test now and it does not segfault before your
code is reached, the process will be stuck in this loop.

3. Attach gdb to the process using the --pid option. (Unfortunately I
did this only on Unix with dbx. Hope it works on windows and gdb as
well.

4. Look if you are now able to see your source in gdb.

5. Set a breakpoint immediately after the infinite loop.

6. Free the loop by setting the variable i to a value different from
0.

7. You should be able to single step now.

I am not sure if this will work. But it might be worth a try.

Thomas

 

[olsongt]

I've tried compiling python from source, and my extension module,
using MSVC8 (free express version), and I managed to get this to work.
The thing is, I don't want to have to recompile every single python
package I need (wxPython, SciPy, etc).

Debug builds are incompatible with release builds. You'll need to
build every binary extension in debug mode (assuming the original
authors don't provide debug builds).

 

[Christopher Anderson]

Debug builds are incompatible with release builds. You'll need to

build every binary extension in debug mode (assuming the original
authors don't provide debug builds).

Right, and this is what I would like to avoid having to do.

Thanks,
Chris

PS. Sorry for the duplicate olsongt

 

[olsongt]

Right, and this is what I would like to avoid having to do.

Thanks,
Chris

PS. Sorry for the duplicate olsongt

Well I guess what I'm saying is, it's a pain to get a python debug
environment up and running, regardless of your toolchain, but there
isn't really any way of avoiding it. (Although I guess this is the
case with any project that uses a bunch of different 3rd party
components.) So you might as well buckle down and do it with whatever
environment you like to use.

I'd say maybe you could figure things out with a map file, but I'm
guessing at least some .dlls are getting their addresses relocated
when they're loaded.