Best way to capture output from an exec'ed (or such) script?

[exscape]

Hey everyone,
I'm writing a tiny web server, mostly to learn - security and such is
a non-issue, but learning how to make things secure is of course nice
too. To be clear - I will most likely not even use it myself.

In any case. I've added some minor scripting support, so that you can
write dynamic pages in Python. To do this, I use execfile(), and pass
the script a dictionary with some basic variables. The script then
sets a "ret" variable that's sent back to the browser. That's some
major ugliness right there! If I do a "print" inside the script, then
it'll end up on the server console. I want it to end up in the web
browser.

How would I accomplish this, i.e. redirect the standard output to
somewhere else, such as a variable in the calling script? If it's not
easy (as in, quite trivial) just say so. I've been thinking about
popen(2,3,4) but that's almost as ugly, if not even worse, than the
current "solution".

Thanks in advance.

 

[Gabriel Genellina]

In any case. I've added some minor scripting support, so that you can
write dynamic pages in Python. To do this, I use execfile(), and pass
the script a dictionary with some basic variables. The script then
sets a "ret" variable that's sent back to the browser. That's some
major ugliness right there! If I do a "print" inside the script, then
it'll end up on the server console. I want it to end up in the web
browser.

If `print` were a function, this would be easy: just provide a replacement
into the dictionary you pass to the script. But print is a statement, and
this becomes a bit harder.

If your web server only processes a single request at a time, you can
replace sys.stdout (and perhaps sys.stderr) with a suitable object having
a write() function: a true open file, or a StringIO instance, or even a
custom object that collects "printed" lines into a list.

If your web server is multithreaded (or you use some other way to process
many simultaneous requests) you have to be more careful - remember that
sys.stdout is global, you must find a way to distinguish between output
from different processes all going into the same collector.

 

[Cameron Laird]

If `print` were a function, this would be easy: just provide a replacement
into the dictionary you pass to the script. But print is a statement, and
this becomes a bit harder.

If your web server only processes a single request at a time, you can
replace sys.stdout (and perhaps sys.stderr) with a suitable object having
a write() function: a true open file, or a StringIO instance, or even a
custom object that collects "printed" lines into a list.

If your web server is multithreaded (or you use some other way to process
many simultaneous requests) you have to be more careful - remember that
sys.stdout is global, you must find a way to distinguish between output
from different processes all going into the same collector.

.
.
.
While we're on the subject of Web servers so small
as to be educational, I'll recommend <URL:
http://www.ibm.com/developerworks/web/library/wa-ltwebserv/ >.

 

[gregpinero]

If your web server is multithreaded (or you use some other way to process
many simultaneous requests) you have to be more careful - remember that
sys.stdout is global, you must find a way to distinguish between output
from different processes all going into the same collector.

Any ideas on how to do this? Is it even possible?

-Greg

 

[gregpinero]

Any ideas on how to do this? Is it even possible?

-Greg

I'm actually worried about this now. Does anyone know of any
potential solutions? Anything to at least get me started?

-Greg

 

[Gabriel Genellina]

En Sat, 04 Aug 2007 18:52:16 -0300, (e-mail address removed)

I'm actually worried about this now. Does anyone know of any
potential solutions? Anything to at least get me started?

You can use threading.currentThread() to distinguish between threads, and
a Lock (or RLock) to ensure the output doesn't get mixed:

def write(msg):
t = time.strftime("%x %X", time.localtime())
who = threading.currentThread().getName()
line = "%s (%-15.15s) %s\n" % (t, who, msg)
OutputDebugString("%s (%-15.15s) %s\n" % (t, who, msg))
loglock.acquire()
try:
with logfile() as f:
f.write(line)
finally:
loglock.release()

 

[gregpinero]

En Sat, 04 Aug 2007 18:52:16 -0300, (e-mail address removed)



You can use threading.currentThread() to distinguish between threads, and
a Lock (or RLock) to ensure the output doesn't get mixed:

def write(msg):
t = time.strftime("%x %X", time.localtime())
who = threading.currentThread().getName()
line = "%s (%-15.15s) %s\n" % (t, who, msg)
OutputDebugString("%s (%-15.15s) %s\n" % (t, who, msg))
loglock.acquire()
try:
with logfile() as f:
f.write(line)
finally:
loglock.release()

Thanks, Gabriel. I'm not actually making the threads myself though.
Rather it's a simple single threaded script running under FastCGI.
Would your method still work? I don't know too much about how FastCGI
works. Perhaps that's where I should start.

-Greg