Unit testing servers

[Luke Kanies]

Hi all,

I've got an xmlrpc server and client that I can't figure out how to test.

The methods I've tried so far, with their results are:

1) Server and client in separate threads. I get a deadlock, no matter what I
try to do.

2) Server in thread, client in main program. Another deadlock.

3) Server in forked process. For some reason the server fails in the forked
process with weird errors even though it works just fine if I don't fork.
Also, I end up with two sets of results from Test::Unit -- one from the main
program and one from the fork. I tried using Kernel.exit! but I still got
two result statements

4) Write an external script that starts the server and start this external
program separately. The problem here is that it becomes quite difficult to
understand what's going on within the external program.

Is there a standard pattern for testing server code without resorting to
external executables? It seems like it should be straightforward, but, um,
I can't figure it out, and it's driving me nuts.

Any help would be greatly appreciated.

--
: We are looking at a newspaper clipping labelled "Huffington
: Herald 11/12/96" in which "Huffington" states, in part:
: "The inhabitants of Tiera del Fuego ... have a single word
: that means 'to look at each other hoping that either will offer to do
: something that both parties desire but are unwilling to do.'"
: Does anyone know that word?
Management?

 

[Joel VanderWerf]

Hi all,

I've got an xmlrpc server and client that I can't figure out how to test.

The methods I've tried so far, with their results are:

1) Server and client in separate threads. I get a deadlock, no matter what I
try to do.

What about trying a simple TCP client/server in this configuration,
maybe starting with the sample/{tsvr,clnt}.rb in the ruby distribution
modified to run in two threads. I'm using single-process multithreaded
tests for TCP client/server code and it works well. It's easier to work
in one process with the state of the whole system before, during, and
after tests. But I don't know what else might be going on with xmlrpc.

 

[Adam Keys]

What about trying a simple TCP client/server in this configuration,
maybe starting with the sample/{tsvr,clnt}.rb in the ruby distribution
modified to run in two threads. I'm using single-process multithreaded
tests for TCP client/server code and it works well. It's easier to
work
in one process with the state of the whole system before, during, and
after tests. But I don't know what else might be going on with xmlrpc.

I'm doing something pretty similar to this. I've got a class that
knows how to start and stop Web Services servers (XMLRPC, SOAP,
etc.) in their own thread. I start a fresh server in TestCase#setup
and kill it in TestCase#teardown. I riffed off the soap4r test cases
[1]. I'd be happy to go into more detail if you'd like.

[1] http://dev.ctor.org/soap4r/file/trunk/test/soap/helloworld/
test_helloworld.rb

 

[Luke Kanies]

What about trying a simple TCP client/server in this configuration,
maybe starting with the sample/{tsvr,clnt}.rb in the ruby distribution
modified to run in two threads. I'm using single-process multithreaded
tests for TCP client/server code and it works well. It's easier to work
in one process with the state of the whole system before, during, and
after tests. But I don't know what else might be going on with xmlrpc.

I've thought about that, but success in that case doesn't really help
me, because it's not useful for me to test a simple client/server. That
is, failure of the simple case, assuming it works for others, informs me
of something, but success only tells me that it works for one system but
not necessarily mine.

Do you have sample code you'd be willing to share, showing how you set
your threads up? I've gotten quite different behaviour depending on how
I start the threads and whether I have sleep statements in them and
such, so I'm pretty curious how you do yours.

I'll create a degenerate test, though, just to see if it's all just me.

 

[Luke Kanies]

I'm doing something pretty similar to this. I've got a class that
knows how to start and stop Web Services servers (XMLRPC, SOAP,
etc.) in their own thread. I start a fresh server in TestCase#setup
and kill it in TestCase#teardown. I riffed off the soap4r test cases
[1]. I'd be happy to go into more detail if you'd like.

You know, I had a test that looked a lot like that and worked for small
transfers but failed for transfers above about 20k, but now it doesn't
even work for small transfers (even as small as one byte). I had the
server in a separate thread and the client in the main process, and it
worked. When I tried using the same process for passing large files
around, though, I got a deadlock every time.

I really am assuming that I'm just retarded and there's something weird
going on with my code, but if everything works fine in a separate
process (which it does) then it's not worth spending days figuring out
how to use threads purely for testing.

I did finally get a forked process to work for testing, as long as I use
the fork with a block, rather than a pure fork, and I'll stick with that
unless I can find some clear failure in my threading code.

Incidentally, here's my current thread test code:

http://reductivelabs.com/cgi-bin/puppet.cgi/file/language/trunk/test/server/tc_server.rb

The threading test is disabled, because, well, it deadlocks (and the
timeouts don't seem to work with deadlocks, which I suppose makes
sense).

I've also tried it with the client in the main thread, not in a separate
thread, but it doesn't seem to make a difference.