p2esp said:
Hello,
I'm using the xmlrpclib module to contact an XMLRPC
server that takes a long time to send results back. My
client timeouts.
The question is whether there is a way to have an
xmlrpclib client that never timeouts. I have been
searching for some code examples on how to do that,
but I could not find any in the xmlrpclib
documentation.
Thanks for any hints.
"Client that never times out" is not a good idea, generally. Nothing in
life is certain, but even moreso in distributed systems. Timeouts are there
to protect you from all kinds of problem conditions (busy server, crashed
server, etc.). You really don't want your client to be in a position of
waiting for a response that is never going to come.
You are better off turning your synchronous "wait forever" client into an
asynchronous "wait for callback" client. Have your client create a listener
object, and pass a reference to this object as part of your long-running
method call. This allows the XMLRPC call to complete, since it is a brief
submission of work, instead of waiting for the work to complete. When the
server is finished, it uses the callback object to send the results back to
the client.
This does make things more complicated - the server has to save the callback
object so that it knows who to send the results to. The client is probably
going to be multithreaded, so that the client isn't blocked while waiting
for the response (although, given your initial design, this may not bother
you). AND, the server may have to do some exception recovery (or at least
logging), if the client has disconnected before it was able to send back
results.
Another alternative is the "job ticket" model. In this case, the client
calls the server just as you have it coded now, but the return value is some
form of job ticket, or job id. The client then uses this id to periodically
poll the server to see if the results are ready yet. Once the server
replies that results are ready, the client uses the job id as the argument
to a getResults() call on the server. This solution is simpler to code, but
may have implications for network loading (lots of extra polling messages,
as clients repeatedly check if their results are ready), and the server has
to handle the case of clients that disconnect and never pick up their
laundry, um, that is, results.
There are a number of good reference books about client/server programming.
One that I like is "Advanced CORBA Programming with C++" by Henning and
Vinoski. Although it uses CORBA and C++ as its implementation platform, so
you'll have to map the examples to Python and XMLRPC, the concepts are much
the same. And don't be too quick to skip the memory management
discussions - even though Python does its own memory management (in place of
C++'s new/delete model), it is easy to have distributed memory leaks between
client and server. (I have seen a Java memory leak happen this way, even
though Java uses a garbage collection memory management model too.) Another
good resource is Douglas Schmidt's set of web pages at Washington Univ of
St. Louis - see
http://www.cs.wustl.edu/~schmidt/patterns.html for some
distributed system patterns papers.
Good luck - distributed systems can be nerve-wracking, but they are also
some of the really cool problems to solve.
-- Paul
