[Pyro] ConnectionClosedError

[George Sakkis]

Didn't have much luck with this in the Pyro mailing list so I am
trying here, just in case. I have a Pyro server running as a daemon
process and occasionally (typically after several hours or days of
uptime) a ConnectionClosedError is raised when a client calls a remote
method. Both client and server run on the same machine, so I don't
think it's an actual network connectivity issue. The problem is that
once this happens, it is persistent; retrying to call a few times does
not work although the server process is still alive. The current
"solution" is to manually kill the server and restart it but obviously
this is not ideal. Is there a way to either prevent or at least
recover automatically the server when it hangs ?

George

 

[Hendrik van Rooyen]

Didn't have much luck with this in the Pyro mailing list so I am
trying here, just in case. I have a Pyro server running as a daemon
process and occasionally (typically after several hours or days of
uptime) a ConnectionClosedError is raised when a client calls a remote
method. Both client and server run on the same machine, so I don't
think it's an actual network connectivity issue. The problem is that
once this happens, it is persistent; retrying to call a few times does
not work although the server process is still alive. The current
"solution" is to manually kill the server and restart it but obviously
this is not ideal. Is there a way to either prevent or at least
recover automatically the server when it hangs ?

If restarting the server sorts it, why don't you run the server as a
subprocess in a higher level script, and exit with an error code
if the error strikes?

something like:

While True:
error = os.system('python current_server_name.py')
if error:
print 'Restarting the server - error returned was:',error
continue
else:
break

It would obviously need a try - except in the existing
code to catch the exception, and some jiggery-pokery
if the code is threaded to ensure a "clean error exit",
but apart from that, I can't think of a reason for it not
to work.

It may be that the reason for your hang up is also a
threading issue - it could be that the server is alive but
that some critical thread has been killed by the error.
But here I am guessing...

- Hendrik

 

[George Sakkis]

If restarting the server sorts it, why don't you run the server as a
subprocess in a higher level script, and exit with an error code
if the error strikes?

Well as I mentioned the process doesn't exit, it is just unresponsive
from the client's side, so this doesn't solve it.

It may be that the reason for your hang up is also a
threading issue - it could be that the server is alive but
that some critical thread has been killed by the error.
But here I am guessing...

I don't create explicitly any thread in my server code but Pyro itself
is multithreaded. Unfortunately I don't have the resources to start
digging in Pyro's internals..

George