ruby real time web server multi threaded probs

[Roger Pack]

So...I seem to have run into a few problems in a ruby multi-threaded
socket based program that, after quite awhile, I am still scratching my
head at these. Has any body encountered/overcome...[sorry if these are
old, or not real bugs]

1) Exceptions that are 'jump their bounds' Sometimes a 'normal'
exception, typically caught, will just bypass its begin rescue blocks!
Brings the whole system to a crash.

2) Crossed streams. You open up a socket and start receiving data meant
for another socket.
Related to that sometimes when doing "server.accept" when under duress
it throws exception like 'not a socket' or 'bad file descriptor' or
'argument invalid' when it is a GOOD file descriptor, and is a socket.
Weirder than weird.
Also, I've had the following happen:
r,e,w = select([socket],nil,nil,5) # succeeds
then r is set to something other than [socket]! [very rare--once or
twice has that ever happened, but it has!]
or
r is set to [socket], then you when you run "socket.recv" it comes back
with "" -- in other words I think it read from some other socket that
was closed.

2) An app just freezes(at least on win32). Sometimes (too many open
sockets?) Ruby will just freeze. There it is. 0% cpu use. ~100 threads
exist. Ruby is frozen [1.8.6]
Or, related, a read from a socket 'freezes' all threads in the whole
program, waiting for a "recv" it come back [and if it doesn't....)
[pretty rare but happened once].


Thus far the 'answer' to these problems has been to...not run a multi
threaded app. Mongrel runs several different apps, for example. Any
ideas or experiences like this?
If nobody recognizes these then I will try to recreate them and submit
them as bugs. This is nuts.
Thanks!
-Roger

 

[Roger Pack]

Thanks for the replies!

One other 'ruby nasty' I thought of was the fact that sometimes
functions will end up 'defined' in a different thread than you would
anticipate them to be. Like if you do a 'require X' then it loads X --
into some other thread. Very odd. Anyone ever had any work arounds for
that? Any trouble with it? [The biggest trouble with the things I've
mentioned is that they are variable--they happen only every so often,
which makes them tough bugs to stomp out.]

Anyway that's about it! Thanks again!

-Roger

Arg, Francis is just too fast for me. I would also highly recommend
EventMachine instead of doing your own socket code (it's Francis'
library
).

Also, it seems like you are expecting Ruby threads to be system threads.
This is simply not the case. Ruby threads are what's called Green
threads,
or handled by the Ruby interpreter only. The OS only ever sees one
thread,
the Ruby process. If you have ~100 Ruby threads going, you are probably
running out of processing power to handle the constant switching.

So...if they were native threads would anticipate the load not being the
problem? I.e. is the only problem with threads inefficiency in context
switching of Ruby versus an OS? BTW when I say '100 threads' most of
them are sleeping or what not, don't worry I'm not insane [or maybe I
am]