XML-RPC + SimpleHTTPServer question

[jbrewer]

I'm currently implementing an XML-RPC service in Python where binary
data is sent to the server via URLs. However, some clients that need
to access the server may not have access to a web server, and I need to
find a solution. I came up with the idea of embedding a simple HTTP
server in the XML-RPC clients so that files can be sent in the
following way:

1. Start an HTTP server on the client using e.g SImpleHTTPServer on a
user allowed port
2. Call XML-RPC server with the URL to the file on the client to
upload
3. Get XML-RPC server response, then shut down HTTP server on client

Does this sound reasonable? I know that XML-RPC can send binary data,
but I presume that the URL method would be faster because the XML
parser could skip reading the binary file (is it base64 encoded as a
string like in SOAP?).

Anyway, I'm unsure of how to implement this in Python. In particular,
how do I shut down a web server once I've started it with
server_forever()? Should I run the server in a separate thread or run
it asynchronously? Since I'm only uploading one file I don't really
need to handle multiple clients; I just need to be able to shut the
server down once remote call has finished.

Any help would be appreciated since I'm new to network programming.

Jeremy

 

[Fredrik Lundh]

I'm currently implementing an XML-RPC service in Python where binary
data is sent to the server via URLs. However, some clients that need
to access the server may not have access to a web server, and I need to
find a solution. I came up with the idea of embedding a simple HTTP
server in the XML-RPC clients so that files can be sent in the
following way:

1. Start an HTTP server on the client using e.g SImpleHTTPServer on a
user allowed port
2. Call XML-RPC server with the URL to the file on the client to
upload
3. Get XML-RPC server response, then shut down HTTP server on client

Does this sound reasonable?

why not just use an ordinary HTTP POST request ?

</F>

 

[jbrewer]

why not just use an ordinary HTTP POST request ?

Sorry for such a simple question, but how would I do this? XML-RPC
runs on top of HTTP, so can I do a POST without running a separate HTTP
server?

Jeremy

 

[Fredrik Lundh]

Sorry for such a simple question, but how would I do this? XML-RPC
runs on top of HTTP, so can I do a POST without running a separate HTTP
server?

the XML-RPC protocol uses HTTP POST, so if you can handle XML-RPC, you
should be able to handle any POST request. what server are you using ?

</F>

 

[jbrewer]

What server are you using?

Just SimpleXMLRPCServer from the standard library.

 

[Fredrik Lundh]

Just SimpleXMLRPCServer from the standard library.

which means that you should be able to do something like

from SimpleXMLRPCServer import SimpleXMLRPCServer,\
SimpleXMLRPCRequestHandler

class MyRequestHandler(SimpleXMLRPCRequestHandler):

def do_POST(self):

if self.path != "/data":
return SimpleXMLRPCRequestHandler.do_POST(self)

# handle POST to /data

bytes = int(self.headers["content-length"])

# copy 'bytes' bytes from self.rfile (in some way)
data = self.rfile.read(bytes)
# ... deal with data here ...

response = "OK"

self.send_response(200)
self.send_header("Content-type", "text/plain")
self.send_header("Content-length", str(len(response)))
self.end_headers()
self.wfile.write(response)
self.wfile.flush()
self.connection.shutdown(1)

SimpleXMLRPCServer((host, port), requestHandler=MyRequestHandler)

</F>

 

[jbrewer]

the XML-RPC protocol uses HTTP POST, so if you can handle XML-RPC, you
should be able to handle any POST request. what server are you using ?

I need some clarification of your suggestion. Instead of sending URLs,
I could read the file as a string, create a Binary object, and send
that via XML-RPC. The parameters will be sent to the server via HTTP
POST. However, the file will be encoded as a base64 string and
included in the body of the XML-RPC message, so it will have to be
parsed by the server. In my experience with SOAP, I have found this to
be extremely inefficient.

Are you suggesting sending the file separately thought a 2nd HTTP POST
with no XML-RPC message body? I guess the POST request would look
something like:

POST /path/file HTTP/1.0
From: ...
User-Agent: ...
Content-Type: /application/binary
Content-Length: <file size>

<file contents>

I'm not sure how to add a 2nd request like this. How would I alter a
simple call like that below to inlcude the 2nd post? Do I need to use
httplib and the request() method of HTTPConnection? Or can I make
POSTs through a ServerProxy object?

import xmlrpclib
server = xmlrpclib.ServerProxy("http://myserver")
result = server.my_function(file, params)

Jeremy

 

[jbrewer]

OK, I posted my previous message before I saw your reply on how to
handle the server side. On the client side, should I use
httplib.HTTPConnection.request() to upload the data or can I do this
through xmlrpc.ServerProxy objects?

Jeremy

 

[Tal Einat]

I have recently implemented a system where clients connect to an RPC
server (RPyC in my case), run a webserver on the RPC server, and close
the webserver when they're done with it.

To do this I wrote a ServerThread class which wraps a SimpleHTTPServer,
runs as a thread, and can be signalled to stop. The ServerThread class
doesn't use the server_forever() method (which is just "while 1:
self.handle_request()"), instead it has a while loop which checks a
flag which is signalled from outside.

We need to set a timeout for the handle_request() call. The first thing
I tried was to use Python's socket object's new 'set_timeout' method,
but I found it caused mysterious errors to occur on my WindowsXP
machine Instead I call select() on the server's listening socket to
check for incoming requests, and only then call handle_request().
select()'s timeout works


# This is the heart of the code - the request handling loop:
while not self.close_flag.isSet():
# Use select() to check if there is an incoming request
r,w,e = select.select([self.server.socket], [], [],
self.timeout)
if r:
self.server.handle_request()

# The stop method should be called to stop the request handling loop
def stop(self, wait=False):
self.close_flag.set()
if wait:
while self.isAlive(): # isAlive() is inherited from
threading.Thread
time.sleep(self.timeout/10.0)

(in my case, self.stop_flag is a threading.Event object)


Good luck!

 

[jbrewer]

Thank you very much, Fredrik. Your code and suggestion worked
perfectly. I haven't benchmarked the plain HTTP post vs Binary
wrapper, but strangely even using the naive Binary wrapper in Python
sends files much faster than how Java + Axis wraps byte arrays in SOAP
messages.

Jeremy