Sockets: Fine Regularly, Bad With SSL

[Hal Vaughan]

The code for this is long and I can post the entire thing or post parts as
needed. I have a hunch the issue is probably not so much one of code, but
some "obvious" point I've missed that newbies working with SSL don't get.

I have a port forwarding program. It has a server port and a client port.
I'm using netcat to test it. When I run the program, I start with a
listening copy of netcat acting as a server, run my program and the client
socket in my program connects to the listening netcat. Then I run another
netcat that connects with the ServerSocket in my program. I type something
in the 2nd instance of netcat, my program prints it out (it tracks what it
is forwarding), then the listening instance of netcat also echos it. It
works the same in reverse. I can, without stopping or staring any of the
programs, also type in from the first instance of netcat (the listening
one), watch my program print it out, and watch it echoed on the other
instance of netcat.

So when I use a regular ServerSocket or Socket, it works fine. I'm not
using non-blocking IO, I'm just reading and writing from the InputStream
and OutputStream on each socket. I have noticed I need to use a regular
OutputStream on both sockets for writing so data is not delayed by a
BufferedOutputStream, but I am using a BufferedInputStream wrapped around
the InputStreams.

Once it worked, I added another forwarding program instance to it, like so:

(The numbers in parenthesis are port numbers.)

netcat (in/out: 6000) -->
forwarder (in: 6000, out: 6005) -->
forwarder (in: 6005, out: 6010) -->
netcat (in/out: 6010)

I added to this the ability to make either the server or client socket an
SSLServerSocket or SSLSocket, as needed. I followed a book (Java Tutorial
1.4) for directions on creating an SSLContext and using the right keystores
and key authorities for this. I have set, on the server
SSLServerSocket.setNeedClientAuth(true). At this point, since I easily
have keys for both, I'm requiring two way authentication. Now it looks
like this:

netcat (in/out: 6000) -->
forwarder (in: 6000, SSLout: 6005) -->
forwarder (SSLin: 6005, out: 6010) -->
netcat (in/out: 6010)

Basically, I've made the first forwarder use SSL on a client socket that
connects to an SSLServerSocket on the 2nd forwarder.

This is where it stops working. I type in a word in netcat, it is echoed by
the first forwarder, but immediately after that, the next line in the
program writes to the OutputStream of the client SSLSocket. After that
write (OutputStream.write(byte[]) statement I added a flush() to it and a
print statement for debugging. It turns out my forwarder gets the input,
echoes it with a print statement, but does NOT finish writing to the
OutputStream.

It is the same on the other end: when the 2nd forwarder receives, on its
client SSLSocket, data, it echoes it, but freezes on writing it to the
SSLServerServerSocket's OutputStream.

So what is going on? Why is it there's no problem sending data back and
forth, but once I change them into SSLSockets, write operations to the
OutputStreams lock up. I don't get any error messages, just a freeze up.
If anyone has any ideas what is causing this, I'd appreciate any help.
I've found it very difficult to get help on this entire topic, so I suspect
most people don't spend a lot of time programming sockets, so any help,
links, or info is definitely appreciated.

Thank you!

Hal

 

[Martin Gregorie]

So what is going on? Why is it there's no problem sending data back and
forth, but once I change them into SSLSockets, write operations to the
OutputStreams lock up. I don't get any error messages, just a freeze up.
If anyone has any ideas what is causing this, I'd appreciate any help.
I've found it very difficult to get help on this entire topic, so I suspect
most people don't spend a lot of time programming sockets, so any help,
links, or info is definitely appreciated.

Its an obvious question, but was the encryption handshake successful
and were you able to start an SSL session? A quick look at the SSLSocket
documentation says you'd get the hangup you're seeing if either of these
failed to complete.

 

[Hal Vaughan]

Its an obvious question, but was the encryption handshake successful
and were you able to start an SSL session? A quick look at the SSLSocket
documentation says you'd get the hangup you're seeing if either of these
failed to complete.

I always tend to miss the obvious if I've been working on something for a
good while.  I get so used to Googling for obscure sites and info that
after a while I get in a "mode" and forget about the easy answers like the
documentation.  I added a HandshakeCompletedListener and found out that's
what's going on.  The book I was using indicated that once one set up an
SSLContext and used that to get the SSLSocketFactory, there was no need to
worry about differences between an SSLSocket and a regular Socket.  So now
I just need to find out why the handshake isn't happening, which is much
easier than a freeze that seems unexplained.

Thanks!

Hal

 

[Hal Vaughan]

Its an obvious question, but was the encryption handshake successful
and were you able to start an SSL session? A quick look at the SSLSocket
documentation says you'd get the hangup you're seeing if either of these
failed to complete.

I added a HandshakeCompleted Listener and found the handshake was not being
completed. I wasn't sure quite what to do next, since much of my code was
like the code I was using as a guide. I found I did not do a
System.out.flush() before I initialized the SecureRandom, so I added that
and after that my HandshakeCompleted Listener started saying the handshake
was completed, but I still have the same problem: Data is being sent from
one secure socket to the other, but there is no indication it's ever
received by the other socket -- and this happens both ways, from client to
server and from server to client.

So if the handshake is being completed, are there other likely causes for
data to not be received by an SSLSocket?

Any help or links are appreciated. I've tried Googling for info on
SSLSockets in Java with handshaking, but cannot find much that is helpful.

Thanks for any help.

Hal

 

[Hal Vaughan]

I added a HandshakeCompleted Listener and found the handshake was not
being
completed. I wasn't sure quite what to do next, since much of my code was
like the code I was using as a guide. I found I did not do a
System.out.flush() before I initialized the SecureRandom, so I added that
and after that my HandshakeCompleted Listener started saying the handshake
was completed, but I still have the same problem: Data is being sent from
one secure socket to the other, but there is no indication it's ever
received by the other socket -- and this happens both ways, from client to
server and from server to client.

So if the handshake is being completed, are there other likely causes for
data to not be received by an SSLSocket?

Any help or links are appreciated. I've tried Googling for info on
SSLSockets in Java with handshaking, but cannot find much that is helpful.

I got lucky and found something on this. It's not in the Javadocs, it's not
anywhere that I'd expect to find it, so it seems to be one of those
unofficially documented "things" that you know if you really know the topic
inside and out, but otherwise, you just have to learn it by spending hours
searching on a hit or miss basis. If I'm wrong and anyone can tell me
where there is official documentation on this, that is where one would
expect to read about problems with SSLSockets, please tell me.

Okay, that said, it's a simple issue: InputStream.available() does not work
if the InputStream is from an SSLSocket. It's that simple, but there is no
place in the docs for InputStream or SSLSocket (please -- someone, prove me
wrong!) that mentions this issue at all.

So how is a self-taught, hard working coder supposed to figure this out?
How would I have found this information while reading up on the sockets in
"normal" research? I only found it when I got lucky with the search terms
I used in the Sun Java forums. I never found it in any official docs.

I'd like, if possible, an idea or two on that because it might help me in
finding problems like this in the future.

Thanks!

Hal

 

[Chris Uppal]

Okay, that said, it's a simple issue: InputStream.available() does not
work if the InputStream is from an SSLSocket. It's that simple, but
there is no place in the docs for InputStream or SSLSocket (please --
someone, prove me wrong!) that mentions this issue at all.

I know very little about SSL (except that I once gave up on reading the spec
;-) But there is almost no valid use for InputStream.available(), and if your
code is using it then I'd guess that it was already broken before you plugged
in the SSL stuff. I.e. SSL is exposing a problem in your code, not causing
one.

I may be wrong of course -- I haven't seen your code -- but that's the way I'd
bet.

-- chris

 

[Hal Vaughan]

I know very little about SSL (except that I once gave up on reading the
spec
;-)  But there is almost no valid use for InputStream.available(), and if
your code is using it then I'd guess that it was already broken before you
plugged
in the SSL stuff.  I.e. SSL is exposing a problem in your code, not
causing one.

I'd use available() to see how much data was waiting, create a byte[] of
that length, read it, then write it out.  I've changed it to loop and set
the byte[] to a set length.  It's a different way to do it, but I don't see
why using available() to set the byte[] length would not be a valid way to
do things.

Still, to have something like that not working and not have it documented is
poor documentation.

Hal

 

[EJP]

I'd use available() to see how much data was waiting, create a byte[] of
that length, read it, then write it out. I've changed it to loop and set
the byte[] to a set length. It's a different way to do it, but I don't see
why using available() to set the byte[] length would not be a valid way to
do things.

As read() already tells you in effect how much was available there's no
*advantage* in using available(), and in the interval between calling
available() and calling read(), more data may have arrived, so you would
be reading less than you could have done; also, this strategy commits
you to using a new byte[] array per read which is otherwise unnecessary.

 

[Chris Uppal]

I'd use available() to see how much data was waiting, create a byte[] of
that length, read it, then write it out.

And how is available() supposed to know how much data is available to be read ?
It can certainly check to see how much data in its input buffers, but if they
are empty, then it has the choice of returning 0 (how do you handle that case?)
or of performing an operation that might block (depending on the stream
implementation). Suppose it does return how much data is in its buffers; why
is that interesting to you ? What's so magical about that figure that you
should want to consume exactly that much data in the next step of your
processing ?

And then, even if the stream you are reading from has a custom implementation
of available() which can determine something more meaningful, why should you
use it ? The answer is not going to be the same (in general) as the number of
bytes that /will/ be available when you come to do the read(). And, what's
more, the cost of the custom implementation of available() may not be trivial
itself (depending on the stream implementation). So you have gained exactly
nothing by using it -- even if it /does/ work as you expect...

Except in exceptional circumstances[*], using available() is simply wrong. The
correct ways to read input include:
-- wrap a BufferedInputStream around the raw input, and consume the data
one byte at a time (the zero-argument form of read()).
-- issue a read() into a sensibly sized buffer (not forgetting, of course,
to use the return value !).

([*] I can't think of any, offhand.)

Still, to have something like that not working and not have it documented
is poor documentation.

I agree that InputStream.available() is badly documented. The JavaDoc entirely
fails to be explicit about how useless this method is. Incidentally, the
documentation for the array forms of InputStream.read() are almost as poor --
why else do so #*&%^ing many programmers not realise that they /have/ to use
the return value !?

-- chris