cgi long running jobs

[Vineeth S]

Hi,

I did post this to the tutor list at python.org but there were no
replies.

I have a web app set up, it is one long job and goes on for about 20
or 30 seconds. When the request comes in, I throw a "please wait", set
the job on its way, and want to periodically update the "please wait"
thrown.

What I tried initially was a client-pull, where with the first "please
wait" I would send a refresh header with a flag as parameter, and then
use that flag to check for completion, else send a new "please wait
<blah>". This did not work, I am guessing, because the request and the
subsequent process has been tied to that session, am I correct ?

Going by that reasoning, I tried a fork-exec to run the long job, and
periodically check for completion from the parent. This does not work
either, and this has foxed me. Is it because fork-exec in the cgi
context has some other behaviour ?

Or, is there some other way I should be doing this ?

Regards and TIA
Vineeth

 

[Thomas Guettler]

Am Sun, 03 Oct 2004 07:01:50 -0700 schrieb Vineeth S:

Hi,

I have a web app set up, it is one long job and goes on for about 20
or 30 seconds. When the request comes in, I throw a "please wait", set
the job on its way, and want to periodically update the "please wait"
thrown.

What I tried initially was a client-pull, where with the first "please
wait" I would send a refresh header with a flag as parameter, and then
use that flag to check for completion, else send a new "please wait
<blah>". This did not work, I am guessing, because the request and the
subsequent process has been tied to that session, am I correct ?

Hi,

How do you check the flag?

You can do it like this:
One script or method called "startProcess": Start the
real work in background (os.system(".... 2>&1 &")).
This returns a redirect to "waitForProcess?id=..."

The second scripts redirects itself to the same URL
after 1 second (in the head tag of the html), if
the process has not completed. Otherwise return the result.
You can add some progress meter which gets updated
every second.

Going by that reasoning, I tried a fork-exec to run the long job, and
periodically check for completion from the parent. This does not work
either, and this has foxed me. Is it because fork-exec in the cgi
context has some other behaviour ?

What kind of session-management do you use?

HTH,
Thomas

 

[Vineeth S]

Am Sun, 03 Oct 2004 07:01:50 -0700 schrieb Vineeth S:


Hi,

How do you check the flag?

I check to see if the form has this variable

You can do it like this:
One script or method called "startProcess": Start the
real work in background (os.system(".... 2>&1 &")).
This returns a redirect to "waitForProcess?id=..."

Ah, I didnt think of backgrounding the job. But I am wondering will
apache release control to another script or page ? In the sequential
nature of things it should. I will try this.

What kind of session-management do you use?

I did not understand what exactly you meant by session-management.

HTH,
Thomas

Thanks and regards
Vineeth

 

[Michael Foord]

I check to see if the form has this variable


Ah, I didnt think of backgrounding the job. But I am wondering will
apache release control to another script or page ? In the sequential
nature of things it should. I will try this.

I don't *know* but many servers are set up not to allow background
processes and not to allow any CGI process to run for more than a set
length of time. On servers this is 1 minute, on others it can even be
30 seconds.

Can you split your process into several parts ? You an then send a
redirect header causing the browser to send a new request which
triggers the next part - effectively chaining requests through
redirect headers.

I did not understand what exactly you meant by session-management.

How do you save the state of each process in between accesses.
The answer is probably that you aren't doing any. If you follow my
suggestion you will have to 'save the state' of your process part way
through for it to pick up in the next request. You will then have to
work out some way of dealing with saved stuff that never gets
completed - this is all 'session management'.

Regards,


Fuzzy

http://www.voidspace.org.uk/atlantibots/pythonutils.html

 

[Vineeth S]

How do you save the state of each process in between accesses.
The answer is probably that you aren't doing any. If you follow my
suggestion you will have to 'save the state' of your process part way
through for it to pick up in the next request. You will then have to
work out some way of dealing with saved stuff that never gets
completed - this is all 'session management'.

Ah. (hit self on head), when the process takes off, it keeps writing
its results to a tmp file which is identified on the basis of the pid
that took off, and the session state is managed through this pid.