Python Crashes

[koranthala]

Hi,
I have a twisted based application based on Python 2.4.3. I also
have one thread in this application.

I found that my program crashes repeatedly after a random interval
(ranging from 10 min to 3 hr). When I say crash, it is not just that
the program dies. Rather in WinXP - a window comes mentioning that
'Python.exe unexpectedly crashed'.
I tried putting in catching the exception by using try: except -
and logging everything. But the logging code is also not hit, i.e.
even the 'except' code is not hit. I feel that the base python
interpreter itself is crashing due to some bug in my program.
Earlier, I had put code as
try:
except:
log
At this time, my whole machine also froze. It was so slow that
one mouse movement happened after 7-8 minutes after I moved the
mouse.

Then I modified the code to
try:
except Exception:
log

Then, the machine no longer freezes. But, still there is no
logging of errors happening too.

Could anyone guide me on this? I have been facing this issue for a
day, and cannot seem to solve it.

P.S -> I am not allowed to post the code of application - so I hope
I have explained it enough.

 

[Philip Semanchuk]

Hi,
I have a twisted based application based on Python 2.4.3. I also
have one thread in this application.

I found that my program crashes repeatedly after a random interval
(ranging from 10 min to 3 hr). When I say crash, it is not just that
the program dies. Rather in WinXP - a window comes mentioning that
'Python.exe unexpectedly crashed'.

Are you using any 3rd party modules?

 

[James Mills]

Hi,
I have a twisted based application based on Python 2.4.3. I also
have one thread in this application.

I found that my program crashes repeatedly after a random interval
(ranging from 10 min to 3 hr). When I say crash, it is not just that
the program dies. Rather in WinXP - a window comes mentioning that
'Python.exe unexpectedly crashed'.
I tried putting in catching the exception by using try: except -
and logging everything. But the logging code is also not hit, i.e.
even the 'except' code is not hit. I feel that the base python
interpreter itself is crashing due to some bug in my program.
Earlier, I had put code as
try:
except:
log
At this time, my whole machine also froze. It was so slow that
one mouse movement happened after 7-8 minutes after I moved the
mouse.

Then I modified the code to
try:
except Exception:
log

Then, the machine no longer freezes. But, still there is no
logging of errors happening too.

Could anyone guide me on this? I have been facing this issue for a
day, and cannot seem to solve it.

P.S -> I am not allowed to post the code of application - so I hope
I have explained it enough.

Do you use any 3rd party extensions (DLLs) ?

cheers
James

 

[koranthala]

Do you use any 3rd party extensions (DLLs) ?

cheers
James

I am using 9 3rd party modules and there are some DLLs too - PIL etc.
But the problem is that the issue comes only after 3 hrs usually.
When I checked the memory using taskmanager, I found that it is not
going too high.
Is there any tool or anything which will help me debug this?

 

[Martin P. Hellwig]

koranthala wrote:
<cut explanation>

This does sounds more to me like a windows/hardware problem, what you
could do is check the windows log for errors, especially look for read
errors from the hard disk.

Windows sometimes can behave very strangely especially if the external
libs don't behave well on a multi process machine or when the execution
prevention gets in the way.

Both problems can be worked around though (setting a uniprocessor flag
at the binary, adding an exception for the execution prevention) but
should not be done without proper reason.

Good luck.

 

[Terry Reedy]

I am using 9 3rd party modules and there are some DLLs too - PIL etc.
But the problem is that the issue comes only after 3 hrs usually.
When I checked the memory using taskmanager, I found that it is not
going too high.

I know of 3 general ways to crash the interpreter.

1. Use 3rd party modules. By far most common. I would try to do
everything but use one of them, possibly by replacing import with stub.

2. Use ctypes. This can bypass all protections.

3. Write a subtle infinite recursion in a class method that is not
checked for infinite recursion. The following illustrates the idea
without testing whether this crashes.

class c():
def __add__(self, other): return self+other # calls this __add__


I believe some of these vulnerabilities existed in 2.4 but many were
crushed in 2.5. Backported to 2.4.3? No idea. This could hit so fast
you never see it in task manager.

If your 3rd party modules work with 2.5, consider upgrading to 2.5.3.
Many bug fixes.

tjr

 

[David Bolen]

Could anyone guide me on this? I have been facing this issue for a
day, and cannot seem to solve it.

We had a scheduling system that had a similar "once in a long while
hard Windows-process crash" which after a bunch of work to try to
track down the source, the most robust solution was just to trap the
failure and restart, as the system ran off a persistent state that was
already engineering to be robust in the case of a hardware crash
(power outage, etc...) While I agree with others that it's most
likely a fault in an extension that gets tickled over time, as was
likely in our case, we needed all our extensions and were using latest
versions at the time. So if your application is such that just
restarting it is practical, it may be a sufficient temporary (or not
so temporary - our system ran for years this way) workaround for you.

What you can do is execute your script from beneath control of another
script, and trap process failures, restarting the script on
non-standard exits. This can be in addition to any top level
exception handling of the child script itself, where it can provide
more graceful support for internal failures.

The trick, under Windows, is to ensure that you disable any pop-up
windows that may occur during the crash, otherwise the monitoring task
never gets a chance to get control and restart things.

With the pywin32 extension, something like:

import win32api, win32con

old_mode = win32api.SetErrorMode(win32con.SEM_FAILCRITICALERRORS |
win32con.SEM_NOGPFAULTERRORBOX |
win32con.SEM_NOOPENFILEERRORBOX)

Or with ctypes:

import ctypes

SEM_FAILCRITICALERRORS = 1
SEM_NOGPFAULTERRORBOX = 2
SEM_NOOPENFILEERRORBOX = 0x8000

old_mode = ctypes.windll.kernel32.SetErrorMode(SEM_FAILCRITICALERRORS |
SEM_NOGPFAULTERRORBOX |
SEM_NOOPENFILEERRORBOX)

at any point prior to starting the child process will ensure that hard
process errors will silently terminate the process and return control
the parent, as well as not popping up any dialog boxes that require
intervention by a person.

Should the process exit harshly, the exit code should be fairly
clear (I forget, but I think it's in the 0xC000xxxx range, maybe
0xC0000005 for a typical GPF), and you can decide on restarting the
task as opposed to just exiting normally.

This will also prevent any pop-ups in the main monitoring process.
You can restore old behavior there after starting the child by making
another call to SetErrorMode using old_mode as the argument.

-- David