Concurrent Ruby webservices from wsdl2ruby

[Catherine S.]

Hi,

as a ruby-newbie I'm currently stuck with a problem (possibly more
problems?) with the setup of an webservice environment on Windows.

1st try: Ruby/wsdl2ruby
I've created two webservices from wsdls and I am able to run them
'concurrently', the client - written in C++ with gSoap - is able to
connect and get the data.

One problem was that I didn't know how to get an entrypoint like
"http://<IP>:<port>/Path/to/my/webservice?" like it is originally stated
in the wsdls, but I used em-proxy to create two transparent proxies in a
seperate 'application' to redirect the requests to the webservices (So:
another possible cause for the problem I noticed - problem stated below)


*** Problem ***
When running both webservices and both proxies the system is 'stable'
for 2 minutes, then it starts to get stuck. Client is getting problems,
webservice-responses delay and are delivered 'clustered' back to client
(which is not good for client). With wireshark the communication looks
ok, so perhaps a printout-problem (1). Or a thread problem
(native/green?)(2)?

This is the main problem.

While investigating it I noticed with TCPView that there are lots of
ports in TIME_WAIT (kept by System process '0'). This seems to be caused
by the webservices. Not much data is sent on them (1kb) - another
problem.

2nd try: JRuby/wsdl2ruby
With the possible thread problem I switched to jruby and ran my
applications with this. On TCPView I saw that there are instances of
jruby started, finished and simultaneously another port goes to
TIME_WAIT. They remain there until my process dies/gets killed. As far
as I read, this is 'normal', but I'm not really proud of this ressources
usage and until now I didn't find the reason for starting another
jruby-instance...

I moved my proxies to the webservices application, it's now containing
mainthread, 2 proxythreads, 2 webservice-threads and all is started, but
problems exist of course still... so, questions:

Is the design approach in its basics - wsdl2ruby generated webservices
in a multithreaded way and transparent proxies - okay? I would
appreciate not to use a whole server (tomcat), but how can I investigate
(or solve?) the existing problems? One of my guesses is that webservices
gets interrupted while some operations... or the webservices need some
'configuration' data (connect_timeout, recv_timeout - the gsoap-client
used something like that)?

If it is not okay... are there any other 'easy' approaches for two
webservices from wsdl? The environment is just needed for testing, but
it is necessary to have it stable/runnable in an acceptable time... so
especially setting it up should go fast...


Best regards and hoping for good suggestions,
CS

 

[Robert Klemme]

Hi,

as a ruby-newbie I'm currently stuck with a problem (possibly more
problems?) with the setup of an webservice environment on Windows.

1st try: Ruby/wsdl2ruby
I've created two webservices from wsdls and I am able to run them
'concurrently', the client - written in C++ with gSoap - is able to
connect and get the data.

One problem was that I didn't know how to get an entrypoint like
"http://<IP>:<port>/Path/to/my/webservice?" like it is originally stated
in the wsdls, but I used em-proxy to create two transparent proxies in a
seperate 'application' to redirect the requests to the webservices (So:
another possible cause for the problem I noticed - problem stated below)

Yes, I think the issue might be with the proxies.

*** Problem ***
When running both webservices and both proxies the system is 'stable'
for 2 minutes, then it starts to get stuck. Client is getting problems,
webservice-responses delay and are delivered 'clustered' back to client
(which is not good for client). With wireshark the communication looks
ok, so perhaps a printout-problem (1). Or a thread problem
(native/green?)(2)?

I rather not believe in a thread issue. After all you have only two
clients and most activity seems to be IO.

This is the main problem.

While investigating it I noticed with TCPView that there are lots of
ports in TIME_WAIT (kept by System process '0'). This seems to be caused
by the webservices. Not much data is sent on them (1kb) - another
problem.

So you are saying that the orphan ports are those of the process that
contains the WS? Connection establishment is relatively expensive
compared to message exchange so this could well explain the slowdown
you are seeing. Maybe also the proxy has difficulties handling so
many orphan connections and this makes him slow down.

2nd try: JRuby/wsdl2ruby
With the possible thread problem I switched to jruby and ran my
applications with this. On TCPView I saw that there are instances of
jruby started, finished and simultaneously another port goes to
TIME_WAIT. They remain there until my process dies/gets killed. As far
as I read, this is 'normal', but I'm not really proud of this ressources
usage and until now I didn't find the reason for starting another
jruby-instance...

I moved my proxies to the webservices application, it's now containing
mainthread, 2 proxythreads, 2 webservice-threads and all is started, but
problems exist of course still... so, questions:

Is the design approach in its basics - wsdl2ruby generated webservices
in a multithreaded way and transparent proxies - okay? I would
appreciate not to use a whole server (tomcat), but how can I investigate
(or solve?) the existing problems? One of my guesses is that webservices
gets interrupted while some operations... or the webservices need some
'configuration' data (connect_timeout, recv_timeout - the gsoap-client
used something like that)?

If it is not okay... are there any other 'easy' approaches for two
webservices from wsdl? The environment is just needed for testing, but
it is necessary to have it stable/runnable in an acceptable time... so
especially setting it up should go fast...


Best regards and hoping for good suggestions,

Unfortunately I don't know the proxies you are using. Maybe there is
a setting which allows to keep connections to the WS process open
(HTTP 1.1 vs. HTTP 1.0 comes to mind).

Or the WS itself only supports HTTP 1.0 or is configured to close
connections after each reply. I believe you should try to keep
connections alive, especially between proxies and WS.

HTH

Kind regards

robert

 

[Catherine S.]

Hi,

thanks for your answer!

The proxies are stupidly sending the request to another port on the same
machine where the webservice is, like provided in an example (Simple
port forwarding proxy) of em-proxy gem
(http://github.com/igrigorik/em-proxy)

Something interesting - if I don't use the proxy and run a rubyclient
from the same wsdl generated as the server the problem with the
long-timed-orphaned ports still exists and there's also the jruby.exe
which gets started and finished all the time the server is running (and
producing more orphans), so possibly not the proxy, but the
webservice...

I'm taking a look for the http-issue

 

[Robert Klemme]

The proxies are stupidly sending the request to another port on the same
machine where the webservice is, like provided in an example (Simple
port forwarding proxy) of em-proxy gem
(http://github.com/igrigorik/em-proxy)

Something interesting - if I don't use the proxy and run a rubyclient
from the same wsdl generated as the server the problem with the
long-timed-orphaned ports still exists and there's also the jruby.exe
which gets started and finished all the time the server is running (and
producing more orphans), so possibly not the proxy, but the
webservice...

Yes, might be. IIRC you can set up a simplistic WS with WEBrick.
Maybe you should try that first.

http://www.devx.com/enterprise/Article/28101/1954
Scroll down to "Implement an XML-RPC-Based Web Service".

You'll probably find more searching

http://www.google.com/search?q=ruby+webrick+"web+service"+soap4r

I'm taking a look for the http-issue

Good luck!

Kind regards

robert

 

[Charles Oliver Nutter]

With the possible thread problem I switched to jruby and ran my
applications with this. On TCPView I saw that there are instances of
jruby started, finished and simultaneously another port goes to
TIME_WAIT. They remain there until my process dies/gets killed. As far
as I read, this is 'normal', but I'm not really proud of this ressources
usage and until now I didn't find the reason for starting another
jruby-instance...

It's possible you tried a version of JRuby that had some issues with
selectors that caused sockets to be kept open in TIME_WAIT for a
while. What version did you try? Where did you read that the TIME_WAIT
was normal?

- Charlie

 

[Catherine S.]

Charles Nutter wrote in post #956232:

It's possible you tried a version of JRuby that had some issues with
selectors that caused sockets to be kept open in TIME_WAIT for a
while. What version did you try? Where did you read that the TIME_WAIT
was normal?

- Charlie

Hi,

I'm having some success, currently with Ruby 1.8.6 and wsdl2ruby
generated webservices (no proxy anymore/at the moment, client is using
webservice directly), but my client isn't receiving the data still in
appropriate time.
Appropriate time is 2-3 seconds, but it takes over 60 seconds to receive
the soap answer from the webservice! (watched with Wireshark)

The delay is between accept and close of a socket, but not in the
implementation of the webservice functionality - need to investigate
more, perhaps threading? (still my favourite and non-favoured guess...)

But with the 'opening' and 'working on' various requests (especially at
startup of client) Ruby seems to have a problem and I'm still thinking
that my basic approach might be faulty?

@Charles Nutter:
JRuby version is:
jruby 1.5.3 (ruby 1.8.7 patchlevel 249) (2010-09-28 7ca06d7) (Java
HotSpot(TM) Client VM 1.6.0_22) [x86-java]

This behaviour is described - for example - here:
http://hype-free.blogspot.com/2006/10/what-to-do-if-you-have-many-timewait.html

I think, something like this is happening in JRuby... new instance, new
instance finishs, system process takes over now-orphaned port,...

Best regards,
CS