saving Python process state for later debugging

[yossi.kreinin]

Hi!

Is there a way to save the state of a Python process for later
inspection with a debugger? One way to do this is to dump core, but is
the result usable for debugging with pdb (it can be debugged by gdb,
which can inspect PyObjects in a CPython core, for example, but it's
not much fun)?

If there is no way to do this today, are there essential difficulties
in implementing this on top of an OS support for "raw" core dumps?

TIA,
Yossi

 

[aspineux]

Hi!

Is there a way to save the state of a Python process for later
inspection with a debugger? One way to do this is to dump core, but is
the result usable for debugging with pdb (it can be debugged by gdb,
which can inspect PyObjects in a CPython core, for example, but it's
not much fun)?

If there is no way to do this today, are there essential difficulties
in implementing this on top of an OS support for "raw" core dumps?

Pylon has something like that.
http://pylonshq.com/docs/0.9.4.1/interactive_debugger.html

Turbogears has the same with option tg.fancy_exception

You can navigate into the stack trace from the web interface

 

[yossi.kreinin]

Pylon has something like that.http://pylonshq.com/docs/0.9.4.1/interactive_debugger.html

Turbogears has the same with option tg.fancy_exception

I could get it wrong, but these things seem to be about debugging
crashed processes "online", not saving snapshots to files for later
inspection. Can you e-mail a process snapshot to a different machine
with them, for example? I understood that you are supposed to debug
the original process, which is kept alive, via the web. I'm talking
about a situation where you have a Python program deployed to a user
who is not running a web server, and have the user send you a snapshot
as a bug report.

-- Yossi

 

[aspineux]

I could get it wrong, but these things seem to be about debugging
crashed processes "online", not saving snapshots to files for later
inspection. Can you e-mail a process snapshot to a different machine
with them, for example? I understood that you are supposed to debug
the original process, which is kept alive, via the web. I'm talking
about a situation where you have a Python program deployed to a user
who is not running a web server, and have the user send you a snapshot
as a bug report.

-- Yossi

A context in python is no more than 2 dictionaries ( globals() and
locals()).
You can easily serialize both to store them.
You can navigate into the python stack using module inspect and
generate the context for all
the functions in the stack trace.

This is probably no more than 50 lines of code, maybe 20

You can find sample of how to get these info and use them in the
sample I was reffering before.

 

[yossi.kreinin]

A context in python is no more than 2 dictionaries ( globals() and
locals()).
You can easily serialize both to store them.

I don't think it will work with objects defined by extension modules,
except if they somehow support serialization, will it? I guess I can
dump core for debugging these objects, and serialize the Python
context for debugging the rest. Finding the extension objects in the
core dump can be a pain though, as will be figuring out the stack with
interlaced Python & native code. And then there are the lovely cases
when CPython crashes, like "deletion of interned string failed. Abort
(core dumped)", where you get to shovel through CPython state with a
native debugger.

The thing with mixing native code with Python is that when native code
misbehaves, it's a big problem, and if Python code misbehaves, it's
still a problem, although a smaller one (serializing the native state
& navigating through it). Maybe the best way around this is to spawn
sub-processes for running native code...