Ruby/OpenSSL... bug? [LONG]

[Nathaniel Talbott]

Well, there are three variables in this problem now (there were four, but I
eliminated DRb), and one of them is broken:

1. OpenSSL
2. Ruby binding to OpenSSL
3. My head

I'm betting the problem isn't #1, since it's so widely used. My hope is that
the problem is #3, but I'm betting on #2. Here's the code:

s.rb:

require 'socket'
require 'openssl'

#require 'patch'

key = OpenSSL:Key::RSA.new(512)
cert = OpenSSL::X509::Certificate.new
cert.not_before = Time.now
cert.not_after = Time.now + 3600
cert.public_key = key.public_key
cert.sign(key, OpenSSL:igest::SHA1.new)

ctx = OpenSSL::SSL::SSLContext.new
ctx.key = key
ctx.cert = cert

svr = TCPServer.new('localhost', 5777)
STDOUT.sync = true
loop do
ns = svr.accept
ssl = OpenSSL::SSL::SSLSocket.new(ns, ctx)
ssl.accept
ssl.sync = true
loop do
IO.select([ssl], nil, nil)
size = ssl.read(5)
if(size)
size = size.to_i
else
break
end
print(".")
ssl.write("a" * size)
end
ssl.close
ns.close
end


c.rb:

require 'socket'
require 'openssl'
require 'timeout'

STDOUT.sync = true

def ssl_open
s = TCPSocket.new('localhost', 5777)
ssl = OpenSSL::SSL::SSLSocket.new(s, OpenSSL::SSL::SSLContext.new)
ssl.sync = true
ssl.connect
yield(ssl)
ssl.close
s.close
end

def try(ssl, i)
ssl.write("%05d" % i)
begin
timeout(5) do
ssl.read(i)
end
rescue TimeoutError
return false
end
return true
end

def try_out(*sizes)
ssl_open do |ssl|
sizes.each do |size|
print("#{size}... ")
puts(try(ssl, size) ? "OK" : "FAILED")
end
end
end


puts("BLOCK_SIZE is #{Buffering::BLOCK_SIZE}.")
try_out(2, 1000, 50000, 2000)


Buffering::BLOCK_SIZE = 4096
puts("BLOCK_SIZE is #{Buffering::BLOCK_SIZE}; note that 2000 is OK now.")
try_out(2, 2000, 50000, 5000)


Buffering::BLOCK_SIZE = 16384
puts("BLOCK_SIZE is #{Buffering::BLOCK_SIZE}, which is the SSL window
size. Everything works now.")
try_out(2, 2000, 5000, 17000, 50000, 99999)

Here's the (c.rb) output:

ntalbott@proxytest:~$ ruby -v c.rb
ruby 1.8.0 (2003-08-04) [i386-linux]
BLOCK_SIZE is 1024.
2... OK
1000... OK
50000... OK
2000... FAILED
c3.rb:44: warning: already initialized constant BLOCK_SIZE
BLOCK_SIZE is 4096; note that 2000 is OK now.
2... OK
2000... OK
50000... OK
5000... FAILED
c3.rb:49: warning: already initialized constant BLOCK_SIZE
BLOCK_SIZE is 16384, which is the SSL window size. Everything works now.
2... OK
2000... OK
5000... OK
17000... OK
50000... OK
99999... OK

The problem clearly revolves around BLOCK_SIZE... and as soon as BLOCK_SIZE
is increased to 16k, which according to
http://www.openssl.org/docs/ssl/SSL_read.html is the maximum record size for
SSLv3/TLSv1, everything always works. For now, I'm going to just use a
BLOCK_SIZE of 16k, but it seems that the underlying problem should be hunted
down and fixed.

Thanks,


Nathaniel

P.S. Oh, and I don't know why 50k always works... your guess is as good as
mine.

<(><

 

[GOTOU Yuuzou]

Hi Nathaniel,

The problem clearly revolves around BLOCK_SIZE... and as soon as BLOCK_SIZE
is increased to 16k, which according to
http://www.openssl.org/docs/ssl/SSL_read.html is the maximum record size for
SSLv3/TLSv1, everything always works. For now, I'm going to just use a
BLOCK_SIZE of 16k, but it seems that the underlying problem should be hunted
down and fixed.

I couldn't trace the problem from OpenSSL code, but your
indication seems right. I didn't read a manual properly.
A while ago, BLOCK_SIZE was increased to 16k on CVS.
Thanks for your great help.