Mongrel/XMLRPC For Ruby - Max Post Size

[BJ Dierkes]

Hello List...

Ruby: 1.8.6
Mongrel= 1.0.1


I have created a Ruby XMLRPC server (xmlrpc4r with mongrel) that is
working great. Now I am working on posting large files that are
base64 encoded and am having problems. The following is a sample of
how the Mongrel/XMLRPC is setup and a sample handler:

======================================================================
class XMLRPCHandler < Mongrel::HttpHandler
def initialize()
@xmlrpc_server = XMLRPC::BasicServer.new
[...]
end
[...]
end

@handler = XMLRPCHandler.new()

@handler.xmlrpc_server.add_handler('file_transfer') do | token,
source_name, encoded_source, source_md5 |
[...]
end
======================================================================

Note that The code within the handler is un-important... since the
transfer fails before anything within the block is called. On the
client side I have something like:

======================================================================
if(File.exists?(self.source))
source_name = File.basename(source)
encoded_source = ''
data = ''
File.open(source) do |f|
data = f.read
end
source_md5 = Digest::MD5.hexdigest(data)
encoded_source = Base64.encode64(data)
else
raise ArgumentError, "source file [#{source}] does not exist!"
end

res = @server.call('file_transfer', @session.token, source_name,
encoded_source, source_md5)
======================================================================


When performing this call with a small text file, everything works as
expected, md5sums match... etc. However, I am trying to transfer RPM
files... the smallest of which is around 500K (not that big). Upon
attempting to transfer I get the following error:

Sun Sep 16 17:50:52 -0500 2007: ERROR: undefined method `string' for
#<File:/tmp/mongrel.16916.0>


Like I said, everything works fine for a rather small text file... I
then tried with just a simple file created from /dev/zero. At a size
of 64K the file transfers fine. At 128K I again get the error above.

I'm wondering, is there some sort of limit that anyone can explain
with regards to Mongrel and/or XMLRPC?

Any feedback is much appreciated.

Thank you.



Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential use of the
individual or entity to which this message is addressed, and unless otherwise
expressly indicated, is confidential and privileged information of Rackspace
Managed Hosting. Any dissemination, distribution or copying of the enclosed
material is prohibited. If you receive this transmission in error, please
notify us immediately by e-mail at (e-mail address removed), and delete the
original message. Your cooperation is appreciated.

 

[BJ Dierkes]

My apologies all. I have found where my issue is. Per the Rdoc for
Mongrel::HttpRequest, if the POST size is over Const::MAX_BODY, then
the input is spit out to a tmp file, and then read in chunks:

"The HttpRequest.initialize method will convert any request that is
larger than Const::MAX_BODY into a Tempfile and use that as the body.
Otherwise it uses a StringIO object. To be safe, you should assume it
works like a file."


This is fine, however I am actually overriding the
HttpHandler.process method in order to log every API call:

======================================================================
if(base.config['log_api_calls'] == 'true')
request_xml = request.body.string
[...]
end
======================================================================


The request.body.string call works fine for small transfers because
it is a StringIO object. However, on larger files it is a File
object... and it borks. Note the last line... 'to be safe, you
should assume it works like a file'. That is they key.

Hello List...

Ruby: 1.8.6
Mongrel= 1.0.1


I have created a Ruby XMLRPC server (xmlrpc4r with mongrel) that is
working great. Now I am working on posting large files that are
base64 encoded and am having problems. The following is a sample
of how the Mongrel/XMLRPC is setup and a sample handler:

======================================================================
class XMLRPCHandler < Mongrel::HttpHandler
def initialize()
@xmlrpc_server = XMLRPC::BasicServer.new
[...]
end
[...]
end

@handler = XMLRPCHandler.new()

@handler.xmlrpc_server.add_handler('file_transfer') do | token,
source_name, encoded_source, source_md5 |
[...]
end
======================================================================

Note that The code within the handler is un-important... since the
transfer fails before anything within the block is called. On the
client side I have something like:

======================================================================
if(File.exists?(self.source))
source_name = File.basename(source)
encoded_source = ''
data = ''
File.open(source) do |f|
data = f.read
end
source_md5 = Digest::MD5.hexdigest(data)
encoded_source = Base64.encode64(data)
else
raise ArgumentError, "source file [#{source}] does not exist!"
end

res = @server.call('file_transfer', @session.token, source_name,
encoded_source, source_md5)
======================================================================


When performing this call with a small text file, everything works
as expected, md5sums match... etc. However, I am trying to
transfer RPM files... the smallest of which is around 500K (not
that big). Upon attempting to transfer I get the following error:

Sun Sep 16 17:50:52 -0500 2007: ERROR: undefined method `string'
for #<File:/tmp/mongrel.16916.0>


Like I said, everything works fine for a rather small text file...
I then tried with just a simple file created from /dev/zero. At a
size of 64K the file transfers fine. At 128K I again get the error
above.

I'm wondering, is there some sort of limit that anyone can explain
with regards to Mongrel and/or XMLRPC?

Any feedback is much appreciated.

Thank you.



Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential
use of the
individual or entity to which this message is addressed, and unless
otherwise
expressly indicated, is confidential and privileged information of
Rackspace
Managed Hosting. Any dissemination, distribution or copying of the
enclosed
material is prohibited. If you receive this transmission in error,
please
notify us immediately by e-mail at (e-mail address removed), and delete the
original message. Your cooperation is appreciated.

Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential use of the
individual or entity to which this message is addressed, and unless otherwise
expressly indicated, is confidential and privileged information of Rackspace
Managed Hosting. Any dissemination, distribution or copying of the enclosed
material is prohibited. If you receive this transmission in error, please
notify us immediately by e-mail at (e-mail address removed), and delete the
original message. Your cooperation is appreciated.

 

[Marcin Raczkowski]

Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential use
of the
individual or entity to which this message is addressed, and unless
otherwise
expressly indicated, is confidential and privileged information of
Rackspace
Managed Hosting. Any dissemination, distribution or copying of the enclosed
material is prohibited. If you receive this transmission in error, please
notify us immediately by e-mail at (e-mail address removed), and delete the
original message. Your cooperation is appreciated.

well i suggest you fix this no use for confidential notice when you
post to public group.