XMLRPC problems

[Phil Tomson]

We've been trying to run the following XMLRPC client code to connect to a
server written in C with xmlrpc-c:

require "xmlrpc/client"
server_url,class_name = *ARGV


if ARGV.length != 2
puts "USAGE: ruby queryclass.rb SERVER_URL CLASS_NAME"
#exit
end

puts "server_url is: #{server_url}"
puts "class_name is: #{class_name}"
server = XMLRPC::Client.new("localhost", "/RPC2", 8081)
result = server.call("queryClass", class_name)
puts "result.class is: #{result.class}"

We're getting this error:

ruby queryclass.rb localhost services

server_url is: localhost
class_name is: services
/usr/local/lib/ruby/1.8/net/protocol.rb:133:in `sysread': Connection
reset by peer (Errno::ECONNRESET)
from /usr/local/lib/ruby/1.8/net/protocol.rb:133:in `rbuf_fill'
from /usr/local/lib/ruby/1.8/timeout.rb:56:in `timeout'
from /usr/local/lib/ruby/1.8/timeout.rb:76:in `timeout'
from /usr/local/lib/ruby/1.8/net/protocol.rb:132:in `rbuf_fill'
from /usr/local/lib/ruby/1.8/net/protocol.rb:104:in `read_all'
from /usr/local/lib/ruby/1.8/net/http.rb:2186:in `read_body_0'
from /usr/local/lib/ruby/1.8/net/http.rb:2139:in `read_body'
from /usr/local/lib/ruby/1.8/net/http.rb:2164:in `body'
... 6 levels...
from /usr/local/lib/ruby/1.8/xmlrpc/client.rb:535:in `do_rpc'
from /usr/local/lib/ruby/1.8/xmlrpc/client.rb:420:in `call2'
from /usr/local/lib/ruby/1.8/xmlrpc/client.rb:410:in `call'
from queryclass.rb:13

any ideas?

I'm running on Suse Linux 10.0.

We've got an equivilent perl script which uses Frontier::Client that works
fine:

#!/usr/bin/perl -w

use Frontier::Client;

my ($server_url, $class_name) = @ARGV;

if (@ARGV != 2) {
print "USAGE queryClass.pl SERVER_URL CLASS_NAME\n";
exit;
}

my $server = Frontier::Client->new(url => $server_url);
$result = $server->call("queryClass", $class_name);

my @a = @$result;

foreach (@a) {
my %h = %$_;

print $h{"id"} . "; " . $h{"type"} . "; " . $h{"state"} . "\n";
}

(equivilent except that I hardcoded some of the values in the Ruby version)


Any idea why the Ruby version is having trouble?

Phil

 

[Daniel Berger]

We've been trying to run the following XMLRPC client code to connect to a
server written in C with xmlrpc-c:

require "xmlrpc/client"
server_url,class_name = *ARGV


if ARGV.length != 2
puts "USAGE: ruby queryclass.rb SERVER_URL CLASS_NAME"
#exit
end

puts "server_url is: #{server_url}"
puts "class_name is: #{class_name}"
server = XMLRPC::Client.new("localhost", "/RPC2", 8081)
result = server.call("queryClass", class_name)
puts "result.class is: #{result.class}"

We're getting this error:
server_url is: localhost
class_name is: services
/usr/local/lib/ruby/1.8/net/protocol.rb:133:in `sysread': Connection
reset by peer (Errno::ECONNRESET)
from /usr/local/lib/ruby/1.8/net/protocol.rb:133:in `rbuf_fill'
from /usr/local/lib/ruby/1.8/timeout.rb:56:in `timeout'
from /usr/local/lib/ruby/1.8/timeout.rb:76:in `timeout'
from /usr/local/lib/ruby/1.8/net/protocol.rb:132:in `rbuf_fill'
from /usr/local/lib/ruby/1.8/net/protocol.rb:104:in `read_all'
from /usr/local/lib/ruby/1.8/net/http.rb:2186:in `read_body_0'
from /usr/local/lib/ruby/1.8/net/http.rb:2139:in `read_body'
from /usr/local/lib/ruby/1.8/net/http.rb:2164:in `body'
... 6 levels...
from /usr/local/lib/ruby/1.8/xmlrpc/client.rb:535:in `do_rpc'
from /usr/local/lib/ruby/1.8/xmlrpc/client.rb:420:in `call2'
from /usr/local/lib/ruby/1.8/xmlrpc/client.rb:410:in `call'
from queryclass.rb:13

any ideas?

I'm running on Suse Linux 10.0.

What version of Ruby?

Line 133 of protocol.rb is this:

@rbuf << @io.sysread(1024)

The full method is rbuf_fill:

def rbuf_fill
timeout(@read_timeout) {
@rbuf << @io.sysread(1024)
}
end

EOF errors are ignored in the read_all method, so I don't think that's
it.

Can you step through it and see anything more specific?

Regards,

Dan

 

[rubyfan]

What version of Ruby?
1.8.4


Line 133 of protocol.rb is this:

@rbuf << @io.sysread(1024)

The full method is rbuf_fill:

def rbuf_fill
timeout(@read_timeout) {
@rbuf << @io.sysread(1024)
}
end

EOF errors are ignored in the read_all method, so I don't think that's
it.

Can you step through it and see anything more specific?

Here's another problem I ran into when trying to make a simpler
testcase (I'm starting to wonder how well Ruby's XMLRPC implementation
interoperates with other implementations of XMLRPC like xmplrpc-c ):

I'm running the example xmlrpc server that is mentioned on the xmlrpc-c
website ( http://xmlrpc-c.sourceforge.net/doc/#serverexamplepp ):
//begin xmlrpc_serve.cpp
#include <cassert>
#include <xmlrpc-c/base.hpp>
#include <xmlrpc-c/registry.hpp>
#include <xmlrpc-c/server_abyss.hpp>
using namespace std;

class sampleAddMethod : public xmlrpc_c::method {
public:
sampleAddMethod() {}
void
execute(xmlrpc_c:aramList const& paramList,
xmlrpc_c::value * const retvalP) {

int const addend(paramList.getInt(0));
int const adder(paramList.getInt(1));

paramList.verifyEnd(2);

*retvalP = xmlrpc_c::value_int(addend + adder);
}
};

int
main(int const argc, const char ** const argv) {
xmlrpc_c::registry myRegistry;
xmlrpc_c::methodPtr const sampleAddMethodP(new sampleAddMethod);
myRegistry.addMethod("sample.add", sampleAddMethodP);
xmlrpc_c::serverAbyss myAbyssServer(
myRegistry,
9999, // TCP port on which to listen
"/tmp/xmlrpc_log" // Log file
);
myAbyssServer.run();
// xmlrpc_c::serverAbyss.run() never returns
assert(false);
return 0;
} //end xmlrpc_serve.cpp

Now when I run this server i can test it with the xmlrpc utility that
comes with the xmlrpc-c distribution:

xmlrpc http://localhost:9999/RPC2 sample.add i/3 i/5

Result:
Integer: 8

So the server seems to be running fine.

Now I try the following client code in Ruby:
require "xmlrpc/client"
server = XMLRPC::Client.new( "http://localhost","/RPC2", "9999" )
result = server.call("sample.add",1,2)

And I get:
SocketError: getaddrinfo: Name or service not known
from /usr/local/lib/ruby/1.8/net/http.rb:562:in `initialize'
from /usr/local/lib/ruby/1.8/net/http.rb:562:in `connect'
from /usr/local/lib/ruby/1.8/timeout.rb:56:in `timeout'
from /usr/local/lib/ruby/1.8/timeout.rb:76:in `timeout'
from /usr/local/lib/ruby/1.8/net/http.rb:562:in `connect'
from /usr/local/lib/ruby/1.8/net/http.rb:553:in `do_start'
from /usr/local/lib/ruby/1.8/net/http.rb:542:in `start'
from /usr/local/lib/ruby/1.8/net/http.rb:1031:in `request'
from /usr/local/lib/ruby/1.8/net/http.rb:988:in `post2'
from /usr/local/lib/ruby/1.8/xmlrpc/client.rb:535:in `do_rpc'
from /usr/local/lib/ruby/1.8/xmlrpc/client.rb:420:in `call2'
from /usr/local/lib/ruby/1.8/xmlrpc/client.rb:410:in `call'
from (irb):6


I looked into this and found that line 562 of http.rb is:
s = timeout(@open_timeout) { TCPSocket.open(conn_address(), \
conn_port()) }

So then I tried the following in irb:
irb(main):001:0> require 'socket'
=> true
irb(main):002:0> TCPSocket.open("http://localhost",9999)
SocketError: getaddrinfo: Name or service not known
from (irb):2:in `initialize'
from (irb):2

....oookay... then after playing around a bit I found that it's the
"http://" part that isn't liked. TCPSocket.open("localhost",9999)
works.

Now I went back to the original little ruby client and got rid of the
"http://":
require "xmlrpc/client"
server = XMLRPC::Client.new( "localhost","/RPC2", "9999" )
result = server.call("sample.add",1,2)

But running that gives a different error:
/usr/local/lib/ruby/1.8/xmlrpc/client.rb:546:in `do_rpc': HTTP-Error:
400 Bad Request (RuntimeError)
from /usr/local/lib/ruby/1.8/xmlrpc/client.rb:420:in `call2'
from /usr/local/lib/ruby/1.8/xmlrpc/client.rb:410:in `call'
from littleclient.rb:3

OK, we're making a bit of progress, I suppose...

Here's the corresponding Perl client script that does seem to work:
#!/usr/bin/perl -w
use Frontier::Client;
my $server_url = "http://localhost:9999/RPC2";
my $class_name = "sample.add";
my $server = Frontier::Client->new(url => $server_url);
$result = $server->call($class_name,1,2);
my @a = @$result;


So what's different? Is Ruby's XMLRPC really interoperable with
implementations in other languages? If not, then it's not very useful.
Hopefully, it's just something simple that I've overlooked. (but the
original problem I posted about yesterday is distinct from this one,
and still remains)

Phil

 

[rcoder]

Phil,

In your second example, you're passing 'http://localhost' as a
*hostname*, not a URL fragment, when you call the XML-RPC client
constructor. Try using the 'new2' constructor, which accepts a URL
string, rather than separate hostname, port, and path params.

-Lennon

 

[Phil Tomson]

Phil,

In your second example, you're passing 'http://localhost' as a
*hostname*, not a URL fragment, when you call the XML-RPC client
constructor. Try using the 'new2' constructor, which accepts a URL
string, rather than separate hostname, port, and path params.

-Lennon

Yeah, I just use 'localhost' now.

I found that when I downgraded back to the original Ruby that came with the
Suse 10.0 distro we're using things worked fine.

Is XMLRPC broken on Ruby 1.8.4? I thought about some other possibilities:
There was a thread a while back about compiling Ruby with gcc 4.0.x (I had
compiled Ruby 1.8.4 with gcc 4.0.2 )and how that breaks networking, however,
as I recall that was on OS X not on Linux. I asked others in our group what
versions of Ruby they were running and how they were compiled. So far a couple
of people have said that xmlrpc is broken for them on 1.8.4 (and for them 1.8.4
came with their distro - I think they're running fedora ). Nothing
conclusive yet, but it seems that xmlrpc might be broken on Ruby 1.8.4. It
would be good to get more datapoints.

Phil

 

[rcoder]

Add one more datapoint: I'm running 1.8.4 on my Windows laptop at home,
and the Apple-supplied 1.8.2 on my PowerBook at work, and haven't had
any issues with the XML-RPC library using either version.

-Lennon

 

[rubyfan]

Back to the original problem: interoperability problems with a server
written in C/C++ using the xmlrpc-c library.
I found (with Lennon's help) that the xmlrpc/client.rb that comes with
Ruby 1.8.4 has the following defined in the header:

"Content-Type" => "text/xml; charset=utf-8",

In 1.8.2 this is defined as:
"Content-Type" => "text/xml ",

(no charset=utf-8 in the 1.8.2 version which works)

As an experiment I removed the "charset=utf-8" from the version of
client.rb that comes with Ruby 1.8.4 and it worked fine with the
xmlrpc-c server. So apparently the xmlrpc-c library doesn't like
'charset=utf-8' encoding. I took at look the xmlrpc spec at
http://www.xmlrpc.com/spec and they always show requests with content
type:
Content-Type: text/xml
(no charset=utf-8 ).

Is Ruby's xmlrpc/client.rb out of compliance with the spec?

 

[rubyfan]

Back to the original problem: interoperability problems with a server
written in C/C++ using the xmlrpc-c library.
I found (with Lennon's help) that the xmlrpc/client.rb that comes with
Ruby 1.8.4 has the following defined in the header:

"Content-Type" => "text/xml; charset=utf-8",

In 1.8.2 this is defined as:
"Content-Type" => "text/xml ",

(no charset=utf-8 in the 1.8.2 version which works)

As an experiment I removed the "charset=utf-8" from the version of
client.rb that comes with Ruby 1.8.4 and it worked fine with the
xmlrpc-c server. So apparently the xmlrpc-c library doesn't like
'charset=utf-8' encoding. I took at look the xmlrpc spec at
http://www.xmlrpc.com/spec and they always show requests with content
type:
Content-Type: text/xml
(no charset=utf-8 ).

Is Ruby's xmlrpc/client.rb out of compliance with the spec?

 

[gregarican]

Is Ruby's xmlrpc/client.rb out of compliance with the spec?

It sounds like it in respect to v1.8.4. Personally after trying to get
various XMLRPC implementations talking to each other I have found
subtle differences between them that break interoperability. An example
is one of Java's XMLRPC clients and a Dolphin Smalltalk XMLRPC client
talking to a Ruby 1.8.2 XMLRPC server. The clients were based on
Apache's XMLRPC client, and some of the header information it was
passing and expecting back didn't match. Either the clients weren't
passing along the Content-Type portion of the header to the server or
else they weren't interpreting the <data> response tag correctly. All
of the fixing amounted to was a handful of source code tweaks here and
there but they were definitely needles in a haystack. Like what you
found with the utf-8 deal

 

[rubyfan]

Actually, it turns out that Ruby's xmlrpc is probably correct whereas
xmlrpc-c's server is not in compliance according to this:

http://www.w3.org/TR/2004/REC-xml-20040204/#charencoding

"4.3.3 Character Encoding in Entities

Each external parsed entity in an XML document MAY use a different
encoding for its characters. All XML processors MUST be able to read
entities in both the UTF-8 and UTF-16 encodings. The terms "UTF-8" and
"UTF-16" in this specification do not apply to character encodings with
any other labels, even if the encodings or labels are very similar to
UTF-8 or UTF-16."

At any rate, it's pretty simple to get it to work with xmlrpc-c's
server:
s = XMLRPC::Client.new( url,"/RPC2", port ).http_header_extra =
{"Content-Type" => "text/xml " }

Phil

 

[gregarican]

Actually, it turns out that Ruby's xmlrpc is probably correct whereas
xmlrpc-c's server is not in compliance according to this:

http://www.w3.org/TR/2004/REC-xml-20040204/#charencoding


"4.3.3 Character Encoding in Entities


Each external parsed entity in an XML document MAY use a different
encoding for its characters. All XML processors MUST be able to read
entities in both the UTF-8 and UTF-16 encodings. The terms "UTF-8" and
"UTF-16" in this specification do not apply to character encodings with
any other labels, even if the encodings or labels are very similar to
UTF-8 or UTF-16."

Interesting. Here's a link to a discussion thead on the ruby-core ML
dealing with including charset= in the header -->
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/5127.

Here's a similar quandry that appeared on the Python ML -->
http://mail.python.org/pipermail/python-bugs-list/2004-May/022865.html.

From what I read they apparently dismissed including the charset= in

the header since would break HTTP standards?

At any rate details like these certainly explain why various XMLRPC
implementations between languages aren't 100% compatible.

 

[rubyfan]

Turns out the problem is fixed in a newer version of xmlrpc-c (1.03.13,
we were using 1.03.07). From the changelog for that version:

"Abyss XML-RPC server accepts (ignores) parameters after
"text/xml" in Content-type header from client."

Phil