Third party script debugging on remote server ...

[dbee]

Right. I've got a really, really annoying/difficult/time consuming
problem with my development environment. I'm using django to build a
web app with paypal integration. My server is hosted remotely, and it
is receiving IPN (payment notifications) POST requests from Paypal.
I've checked on google and irc and this is my last shot at solving
this before I go mad ... :-(

The problem is that I can't debug those POST requests. Browser
debugging is out of the question obviously cause I'm not at the
computer, ( and it doesn't have X ).

I've tried cgitb but that doesn't seem to work at all ...

import cgitb; cgitb.enable(display=0,logdir='/tmp/',format='plain')
import cgitb; cgitb.enable()

.... neither of those commands have any effect. Although I do log other
parts of the script to /tmp, so I know that it's reachable...

mod_python absolutely refuses to error_log to the apache error_log. I
have restarted it and it still won't flush whatever error buffer it
may ( or may not ) have stored.

I can re-constitute the requests in my browser using a GET request.
But frankly, that's kinda messy - there are lots of paypal IPN
combinations and I may have to integrate other applications with
paypal. So ideally speaking I'm looking for a proper debugging
environment for this kind of thing ...

Basically, I either want mod_python to start error_logging properly,
or I want some type of working traceback environment to be available.
Help !

Server version: Apache/2.0.52
Server built: Aug 13 2006 03:29:43
CentOS4.x: (RedHat Clone)
mod_python.i386 3.1.3-5.1
installed

# httpd.conf

<VirtualHost *:80 >

ServerName mydomain.biz
ServerAlias www.mydomain.biz
SetHandler mod_python
PythonPath "['/home/babo/django'] + sys.path"
PythonHandler django.core.handlers.modpython
SetEnv DJANGO_SETTINGS_MODULE mydomain.settings

<Location "/libs/media/">
SetHandler None
Options None
</Location>

</VirtualHost>


My python.conf: ( seems pretty normal )

#
# Mod_python is a module that embeds the Python language interpreter
# within the server, allowing Apache handlers to be written in Python.
#

LoadModule python_module modules/mod_python.so

# Override type-map handler for /var/www/manual
<Directory "/var/www/manual/mod/mod_python">
<Files *.html>
SetHandler default-handler
</Files>
</Directory>

# This will cause files beneath /var/www/html with the extension .spam
# to be handled by the Python script /var/www/html/eggs.py
#
#<Directory /var/www/html>
# AddHandler python-program .spam
# PythonHandler eggs
#</Directory>

# This will cause all requests to the /python heirachy of your
# webserver to be handled by the python script /path/to/myhandler.py
#
#<Location /python>
# SetHandler python-program
# PythonPath "sys.path + ['/path/to']"
# PythonHandler myhandler
#</Location>

# This will cause all requests to the /python heirachy of your
# webserver to be handled by mod_python's Publisher handler
# (see http://localhost/manual/mod/mod_python/hand-pub.html)
#
# This will cause the output of all requests to files beneath
# /var/www/html with the extension .flt to be filtered through
# the Python script /var/www/html/filter.py
#
#<Directory /var/www/html>
# PythonOutputFilter filter MYFILTER
# AddOutputFilter MYFILTER .flt
#</Directory>

 

[Graham Dumpleton]

After calling whatever it is that is writing to standard output/error,
do:

import sys
sys.stdout.flush()
sys.stderr.flush()

This should cause any buffered data to be immediately flushed to the
main Apache error log, ie., don't look in any virtual host logs, check
the main one.

Graham

Right. I've got a really, really annoying/difficult/time consuming
problem with my development environment. I'm using django to build a
web app with paypal integration. My server is hosted remotely, and it
is receiving IPN (payment notifications) POST requests from Paypal.
I've checked on google and irc and this is my last shot at solving
this before I go mad ... :-(

The problem is that I can't debug those POST requests. Browser
debugging is out of the question obviously cause I'm not at the
computer, ( and it doesn't have X ).

I've tried cgitb but that doesn't seem to work at all ...

import cgitb; cgitb.enable(display=0,logdir='/tmp/',format='plain')
import cgitb; cgitb.enable()

... neither of those commands have any effect. Although I do log other
parts of the script to /tmp, so I know that it's reachable...

mod_pythonabsolutely refuses to error_log to the apache error_log. I
have restarted it and it still won't flush whatever error buffer it
may ( or may not ) have stored.

I can re-constitute the requests in my browser using a GET request.
But frankly, that's kinda messy - there are lots of paypal IPN
combinations and I may have to integrate other applications with
paypal. So ideally speaking I'm looking for a proper debugging
environment for this kind of thing ...

Basically, I either wantmod_pythonto start error_logging properly,
or I want some type of working traceback environment to be available.
Help !

Server version: Apache/2.0.52
Server built: Aug 13 2006 03:29:43
CentOS4.x: (RedHat Clone)mod_python.i386 3.1.3-5.1
installed

# httpd.conf

<VirtualHost *:80 >

ServerName mydomain.biz
ServerAliaswww.mydomain.biz
SetHandlermod_python
PythonPath "['/home/babo/django'] + sys.path"
PythonHandler django.core.handlers.modpython
SetEnv DJANGO_SETTINGS_MODULE mydomain.settings

<Location "/libs/media/">
SetHandler None
Options None
</Location>

</VirtualHost>

My python.conf: ( seems pretty normal )

#
#Mod_pythonis a module that embeds the Python language interpreter
# within the server, allowing Apache handlers to be written in Python.
#

LoadModule python_module modules/mod_python.so

# Override type-map handler for /var/www/manual
<Directory "/var/www/manual/mod/mod_python">
<Files *.html>
SetHandler default-handler
</Files>
</Directory>

# This will cause files beneath /var/www/html with the extension .spam
# to be handled by the Python script /var/www/html/eggs.py
#
#<Directory /var/www/html>
# AddHandler python-program .spam
# PythonHandler eggs
#</Directory>

# This will cause all requests to the /python heirachy of your
# webserver to be handled by the python script /path/to/myhandler.py
#
#<Location /python>
# SetHandler python-program
# PythonPath "sys.path + ['/path/to']"
# PythonHandler myhandler
#</Location>

# This will cause all requests to the /python heirachy of your
# webserver to be handled bymod_python'sPublisher handler
# (seehttp://localhost/manual/mod/mod_python/hand-pub.html)
#
# This will cause the output of all requests to files beneath
# /var/www/html with the extension .flt to be filtered through
# the Python script /var/www/html/filter.py
#
#<Directory /var/www/html>
# PythonOutputFilter filter MYFILTER
# AddOutputFilter MYFILTER .flt
#</Directory>

 

[dbee]

After calling whatever it is that is writing to standard output/error,
do:

import sys
sys.stdout.flush()
sys.stderr.flush()

This should cause any buffered data to be immediately flushed to the
main Apache error log, ie., don't look in any virtual host logs, check
the main one.

Graham

Right. I've got a really, really annoying/difficult/time consuming
problem with my development environment. I'm using django to build a
web app with paypal integration. My server is hosted remotely, and it
is receiving IPN (payment notifications) POST requests from Paypal.
I've checked on google and irc and this is my last shot at solving
this before I go mad ... :-(

The problem is that I can't debug those POST requests. Browser
debugging is out of the question obviously cause I'm not at the
computer, ( and it doesn't have X ).

I've tried cgitb but that doesn't seem to work at all ...

import cgitb; cgitb.enable(display=0,logdir='/tmp/',format='plain')
import cgitb; cgitb.enable()

... neither of those commands have any effect. Although I do log other
parts of the script to /tmp, so I know that it's reachable...

mod_pythonabsolutely refuses to error_log to the apache error_log. I
have restarted it and it still won't flush whatever error buffer it
may ( or may not ) have stored.

I can re-constitute the requests in my browser using a GET request.
But frankly, that's kinda messy - there are lots of paypal IPN
combinations and I may have to integrate other applications with
paypal. So ideally speaking I'm looking for a proper debugging
environment for this kind of thing ...

Basically, I either wantmod_pythonto start error_logging properly,
or I want some type of working traceback environment to be available.
Help !

Server version: Apache/2.0.52
Server built: Aug 13 2006 03:29:43
CentOS4.x: (RedHat Clone)mod_python.i386 3.1.3-5.1
installed

# httpd.conf

<VirtualHost *:80 >

ServerName mydomain.biz
ServerAliaswww.mydomain.biz
SetHandlermod_python
PythonPath "['/home/babo/django'] + sys.path"
PythonHandler django.core.handlers.modpython
SetEnv DJANGO_SETTINGS_MODULE mydomain.settings

<Location "/libs/media/">
SetHandler None
Options None
</Location>

My python.conf: ( seems pretty normal )

#
#Mod_pythonis a module that embeds the Python language interpreter
# within the server, allowing Apache handlers to be written in Python.
#

LoadModule python_module modules/mod_python.so

# Override type-map handler for /var/www/manual
<Directory "/var/www/manual/mod/mod_python">
<Files *.html>
SetHandler default-handler
</Files>
</Directory>

# This will cause files beneath /var/www/html with the extension .spam
# to be handled by the Python script /var/www/html/eggs.py
#
#<Directory /var/www/html>
# AddHandler python-program .spam
# PythonHandler eggs
#</Directory>

# This will cause all requests to the /python heirachy of your
# webserver to be handled by the python script /path/to/myhandler.py
#
#<Location /python>
# SetHandler python-program
# PythonPath "sys.path + ['/path/to']"
# PythonHandler myhandler
#</Location>

# This will cause all requests to the /python heirachy of your
# webserver to be handled bymod_python'sPublisher handler
# (seehttp://localhost/manual/mod/mod_python/hand-pub.html)
#
# This will cause the output of all requests to files beneath
# /var/www/html with the extension .flt to be filtered through
# the Python script /var/www/html/filter.py
#
#<Directory /var/www/html>
# PythonOutputFilter filter MYFILTER
# AddOutputFilter MYFILTER .flt
#</Directory>

Hi Graeme,

Thanks for the suggestion. Unluckily for me, it doesn't seem to be
working though ...

I've tried it a number of different ways. I guess if I put the code
after an exception, then the code won't be called.
So I ran an error in the python script and then I called the
sys.stderr.flush() from the python shell. No luck though.

I called on it's own, still no luck,

Then I tried ...

try:

raise Exception('Please log this error')

except:

import sys

sys.stderr.flush()
sys.stdout.flush()

The error log still has no error info in it :-( ....

Is this problem unique to me ?

This is startup message that gets posted to error_log on apache
restart ... (The IIS string is a mod_sec Security Signature)
[Wed Apr 18 19:58:25 2007] [notice] mod_python: (Re)importing module
'django.core.handlers.modpython'
[Wed Apr 18 19:58:32 2007] [notice] mod_python: (Re)importing module
'django.core.handlers.modpython'
DEBUG: InitPostgres
DEBUG: InitPostgres
[Wed Apr 18 19:59:56 2007] [notice] caught SIGTERM, shutting down
[Wed Apr 18 19:59:57 2007] [notice] suEXEC mechanism enabled
(wrapper: /usr/sbin/suexec)
[Wed Apr 18 19:59:57 2007] [notice] mod_security/1.9.4 configured -
Apache/2.0.52 (CentOS)
[Wed Apr 18 19:59:58 2007] [notice] Digest: generating secret for
digest authentication ...
[Wed Apr 18 19:59:58 2007] [notice] Digest: done
[Wed Apr 18 19:59:58 2007] [notice] LDAP: Built with OpenLDAP LDAP SDK
[Wed Apr 18 19:59:58 2007] [notice] LDAP: SSL support unavailable
[Wed Apr 18 19:59:58 2007] [notice] mod_python: Creating 4 session
mutexes based on 256 max processes and 0 max threads.
[Wed Apr 18 19:59:58 2007] [notice] Microsoft-IIS/5.0 configured --
resuming normal operations

There are no python debug messages ...

Also, I've reinstalled mod_python, but it makes no difference. And put
this into my VirtualHost, just in case ...

PythonDebug On

Is there any answer for my problem ?

 

[dbee]

After calling whatever it is that is writing to standard output/error,
do:

import sys
sys.stdout.flush()
sys.stderr.flush()

This should cause any buffered data to be immediately flushed to the
main Apache error log, ie., don't look in any virtual host logs, check
the main one.

Graham

Right. I've got a really, really annoying/difficult/time consuming
problem with my development environment. I'm using django to build a
web app with paypal integration. My server is hosted remotely, and it
is receiving IPN (payment notifications) POST requests from Paypal.
I've checked on google and irc and this is my last shot at solving
this before I go mad ... :-(
The problem is that I can't debug those POST requests. Browser
debugging is out of the question obviously cause I'm not at the
computer, ( and it doesn't have X ).
I've tried cgitb but that doesn't seem to work at all ...
import cgitb; cgitb.enable(display=0,logdir='/tmp/',format='plain')
import cgitb; cgitb.enable()
... neither of those commands have any effect. Although I do log other
parts of the script to /tmp, so I know that it's reachable...
mod_pythonabsolutely refuses to error_log to the apache error_log. I
have restarted it and it still won't flush whatever error buffer it
may ( or may not ) have stored.
I can re-constitute the requests in my browser using a GET request.
But frankly, that's kinda messy - there are lots of paypal IPN
combinations and I may have to integrate other applications with
paypal. So ideally speaking I'm looking for a proper debugging
environment for this kind of thing ...
Basically, I either wantmod_pythonto start error_logging properly,
or I want some type of working traceback environment to be available.
Help !
Server version: Apache/2.0.52
Server built: Aug 13 2006 03:29:43
CentOS4.x: (RedHat Clone)mod_python.i386 3.1.3-5.1
installed
# httpd.conf
<VirtualHost *:80 >
ServerName mydomain.biz
ServerAliaswww.mydomain.biz
SetHandlermod_python
PythonPath "['/home/babo/django'] + sys.path"
PythonHandler django.core.handlers.modpython
SetEnv DJANGO_SETTINGS_MODULE mydomain.settings
<Location "/libs/media/">
SetHandler None
Options None
</Location>
</VirtualHost>
My python.conf: ( seems pretty normal )
#
#Mod_pythonis a module that embeds the Python language interpreter
# within the server, allowing Apache handlers to be written in Python.
#
LoadModule python_module modules/mod_python.so
# Override type-map handler for /var/www/manual
<Directory "/var/www/manual/mod/mod_python">
<Files *.html>
SetHandler default-handler
</Files>
</Directory>
# This will cause files beneath /var/www/html with the extension .spam
# to be handled by the Python script /var/www/html/eggs.py
#
#<Directory /var/www/html>
# AddHandler python-program .spam
# PythonHandler eggs
#</Directory>
# This will cause all requests to the /python heirachy of your
# webserver to be handled by the python script /path/to/myhandler.py
#
#<Location /python>
# SetHandler python-program
# PythonPath "sys.path + ['/path/to']"
# PythonHandler myhandler
#</Location>
# This will cause all requests to the /python heirachy of your
# webserver to be handled bymod_python'sPublisher handler
# (seehttp://localhost/manual/mod/mod_python/hand-pub.html)
#
# This will cause the output of all requests to files beneath
# /var/www/html with the extension .flt to be filtered through
# the Python script /var/www/html/filter.py
#
#<Directory /var/www/html>
# PythonOutputFilter filter MYFILTER
# AddOutputFilter MYFILTER .flt
#</Directory>

Hi Graeme,

Thanks for the suggestion. Unluckily for me, it doesn't seem to be
working though ...

I've tried it a number of different ways. I guess if I put the code
after an exception, then the code won't be called.
So I ran an error in the python script and then I called the
sys.stderr.flush() from the python shell. No luck though.

I called on it's own, still no luck,

Then I tried ...

try:

raise Exception('Please log this error')

except:

import sys

sys.stderr.flush()
sys.stdout.flush()

The error log still has no error info in it :-( ....

Is this problem unique to me ?

This is startup message that gets posted to error_log on apache
restart ... (The IIS string is a mod_sec Security Signature)
[Wed Apr 18 19:58:25 2007] [notice] mod_python: (Re)importing module
'django.core.handlers.modpython'
[Wed Apr 18 19:58:32 2007] [notice] mod_python: (Re)importing module
'django.core.handlers.modpython'
DEBUG: InitPostgres
DEBUG: InitPostgres
[Wed Apr 18 19:59:56 2007] [notice] caught SIGTERM, shutting down
[Wed Apr 18 19:59:57 2007] [notice] suEXEC mechanism enabled
(wrapper: /usr/sbin/suexec)
[Wed Apr 18 19:59:57 2007] [notice] mod_security/1.9.4 configured -
Apache/2.0.52 (CentOS)
[Wed Apr 18 19:59:58 2007] [notice] Digest: generating secret for
digest authentication ...
[Wed Apr 18 19:59:58 2007] [notice] Digest: done
[Wed Apr 18 19:59:58 2007] [notice] LDAP: Built with OpenLDAP LDAP SDK
[Wed Apr 18 19:59:58 2007] [notice] LDAP: SSL support unavailable
[Wed Apr 18 19:59:58 2007] [notice] mod_python: Creating 4 session
mutexes based on 256 max processes and 0 max threads.
[Wed Apr 18 19:59:58 2007] [notice] Microsoft-IIS/5.0 configured --
resuming normal operations

There are no python debug messages ...

Also, I've reinstalled mod_python, but it makes no difference. And put
this into my VirtualHost, just in case ...

PythonDebug On

Is there any answer for my problem ?

I came across this logging class on the web. I've written it into the
app, but it doesn't seem to work as I'd hoped ...

It seems to log errors, but not actual system errors ....

/home/babo/django/flowerhour/flowerhour1/views.py:40: SyntaxWarning:
import * only allowed at module level
def index(request):
DEBUG: InitPostgres

Is this class supposed to override the mod_python standard error
functions ? And log to /tmp instead ?
I couldn't find the __debug__ builtin in the docs, and I'm relatively
new to python ...

class Log:
def __init__(self, file, mode):
if __debug__:
delf.__descriptor = open(file, mode)

def __del__(self):
self.close()

def append(self, string):
self.__descriptor.write(string + "\n\n")
# add some more information if you please

def close(self):
self.__descriptor.close()


# Variables
__descriptor = []
#

global log
log = Log("log.txt")

 

[Graham Dumpleton]

Hi Graeme,

Thanks for the suggestion. Unluckily for me, it doesn't seem to be
working though ...

I've tried it a number of different ways. I guess if I put the code
after an exception, then the code won't be called.
So I ran an error in the python script and then I called the
sys.stderr.flush() from the python shell. No luck though.

I called on it's own, still no luck,

Then I tried ...

try:

raise Exception('Please log this error')

except:

import sys

sys.stderr.flush()
sys.stdout.flush()

The error log still has no error info in it :-( ....

Is this problem unique to me ?

Those calls don't generate output themselves and exception details
don't get output at the point of a try/except block. If you want to
generate to Apache error log details of exception from try/except,
use:

import traceback
import sys

try:
xxx
except:
traceback.print_exc()
sys.stderr.flush()

Using these flush functions from interactive debugger isn't going to
show you anything as this trick is only needed with mod_python. All
the functions do is force the output of any buffered data that was
written out previously.

Graham