POSSIBLE SECURITY PROBLEM in Java 5!

T

Twisted

This happened accidentally, but an attacker might use this
intentionally in a hostile applet to perpetrate a denial of service
attack.

I had a Java app compile and run that generated ~200 threads accessing
the network. It suddenly stopped functioning; it hadn't hung -- the UI
still worked -- but the network activity dropped to zero and it
wouldn't do anything. (Picture a Web browser that works, except every
attempt to go to a page produces a timeout or the browser just spins.
Sort of like that: the UI is working but the backend isn't.)

Naturally, suspecting a bug, I exited and went to rerun the app, this
time attaching a debugger process. Failed. Eclipse showed "Launching
(83%)..." and stuck there. Details showed "searching for free socket"
or some such.

At this point I discovered that almost every other network using app on
the system had locked up. Shareaza, MSN messenger, etc. -- Firefox was
the only survivor, and it would not function (see above description of
semi-functional Web browser; this was Firefox after the accident).

Task Manager showed two Java tasks, one with over 100 threads;
presumably the app that was supposed to have exited. I repeatedly
attacked it with "End Process" and "End Process Tree"; no effect.
Eclipse, when exited, likewise remained as a headless zombie: no
visible UI but still an Eclipse process and a Java task with a couple
of threads. Neither could be killed.

After over half an hour, none of the tasks had exited that had been
told in various increasingly imperative ways to terminate, and the
network remained unusable. I concluded I had to reboot.

To top it off, a clean reboot wasn't possible -- the system would close
a couple apps then just sit there. I had to power-cycle the fucking
box! This on an NT kernel OS and with the trigger being Java, the
paragon of security and non-crashingness? What the ****?!

Now I've lost all my open Explorer folders and various other
inconveniences can all be laid at the doorstep of ... whatever the ****
it was that happened.

Can anyone explain this event? Opening a lot of network connections
shouldn't be a problem on XP, and it sure as hell shouldn't render the
system unusable to the point of forcing a cold boot. Even if the Java
app ate all the free network sockets, it should have been terminatable,
and killing it should have released the sockets. Even terminating other
tasks (such as the nonresponsive Shareaza) that used the network didn't
free any up (Shareaza, when end-tasked, disappeared promptly from all
task and process lists, but Firefox didn't magically start working
again).

Annoyingly, the system help also stopped working(!); I was desperate
enough to actually resort to Help and Support Center to find info on
terminating processes and/or freeing sockets, given that the usual
first-resort, Google, was rendered unavailable by the problem.

This Should Not Have Happened.

We have at least 3 separate problems:
* Java can do things that DoS a Win32 box, without doing anything but
open sockets.
Applets can open sockets (though only back to the originating site);
Win32 boxen are
extremely commonplace. You do the math.
* Windows XP SP2, with all the latest updates (including and especially
security updates),
allows a single app to hog all the network sockets, and apps that
want a socket but
can't get one seem usually to hang rather than gracefully report an
error condition to the
user.
* And it does not seem to properly "terminate with prejudice" an
unresponsive process or
free up resources one had when it does terminate it successfully. Is
there no equivalent of
"kill -9", via Task Manager or otherwise, even when you're logged on
as administrator?

It looks like some problems (can't kill processes, processes can kill
parts of the system's overall functionality, killed tasks don't always
have their resources released by the OS) from the bad old days of
Windows 3.1 are still present and accounted for...

Win32 SP2 with all security fixes;
JDK/JRE 1.5.0_06 (latest, as of a couple weeks ago anyway)
 
D

Daniel Dyer

Can anyone explain this event?

Firstly and most importantly, I'd blame Windows for not being in control
of its processes and not being able to kill them.

Beyond that I'd blame Firefox. The Windows version seems very reluctant
to die when things go wrong. If you can't kill Firefox it's not
surprising that you can't kill the Java process that it spawned. It would
be interesting to see if the applet caused such problems under IE or
Opera, or whether it was just Firefox. Furthermore, I'd guess that if you
ran the Java code as a standalone app it wouldn't be nearly so problematic.

I do have to ask though, what are you doing that needs 200 network
connections from a single applet back to the single originating host?

Dan.
 
T

Twisted

My Java code was a standalone app, not an applet inside Firefox. But
because it crashed the box using only the opening of http connections,
an applet could do the same, hence my security concern.
 
C

Chris Uppal

Twisted said:
* Windows XP SP2, with all the latest updates (including and especially
security updates),
allows a single app to hog all the network sockets, and apps that
want a socket but
can't get one seem usually to hang rather than gracefully report an
error condition to the
user.
* And it does not seem to properly "terminate with prejudice" an
unresponsive process or
free up resources one had when it does terminate it successfully. Is
there no equivalent of
"kill -9", via Task Manager or otherwise, even when you're logged on
as administrator?

This is pure speculation, but the symptoms have the smell of a bug in
kernel-level network code. If that's the case then it's probably not easily
reproducible, and is nothing whatever to do with Java.

Speculating further (with even less evidential basis -- if that were possible
;-) a modern, properly patched sp2 box has a lot of fairly new (or fairly
recently activated) code for its firewall stuff, could that be where the
(hypothetical) bug lies ? Alternatively, if you are using some other firewall
product, then that too will be operating (in part) at kernel level, and may be
the source of the problem.

-- chris
 
M

Martin Gregorie

Twisted said:
This happened accidentally, but an attacker might use this
intentionally in a hostile applet to perpetrate a denial of service
attack.
Isn't there a hard-coded limit to the number of threads a 'doze box can
run? I seem to remember that being said about NT a long while ago and
for some reason a limit of 256 threads comes to mind.

The inability to kill things off would certainly fit there being a hard
limit: presumably the task to kill a thread would run as a transient
thread and your box is already on the limit....
 
T

Twisted

Don't think so. I've spawned 500 without a problem from Java before;
and the crash happened with maybe 220 threads created.
 
N

Nigel Wade

Twisted said:
This happened accidentally, but an attacker might use this
intentionally in a hostile applet to perpetrate a denial of service
attack.

I had a Java app compile and run that generated ~200 threads accessing
the network. It suddenly stopped functioning; it hadn't hung -- the UI
still worked -- but the network activity dropped to zero and it
wouldn't do anything. (Picture a Web browser that works, except every
attempt to go to a page produces a timeout or the browser just spins.
Sort of like that: the UI is working but the backend isn't.)

Naturally, suspecting a bug, I exited and went to rerun the app, this
time attaching a debugger process. Failed. Eclipse showed "Launching
(83%)..." and stuck there. Details showed "searching for free socket"
or some such.

At this point I discovered that almost every other network using app on
the system had locked up. Shareaza, MSN messenger, etc. -- Firefox was
the only survivor, and it would not function (see above description of
semi-functional Web browser; this was Firefox after the accident).

Task Manager showed two Java tasks, one with over 100 threads;
presumably the app that was supposed to have exited. I repeatedly
attacked it with "End Process" and "End Process Tree"; no effect.
Eclipse, when exited, likewise remained as a headless zombie: no
visible UI but still an Eclipse process and a Java task with a couple
of threads. Neither could be killed.

After over half an hour, none of the tasks had exited that had been
told in various increasingly imperative ways to terminate, and the
network remained unusable. I concluded I had to reboot.

To top it off, a clean reboot wasn't possible -- the system would close
a couple apps then just sit there. I had to power-cycle the fucking
box! This on an NT kernel OS and with the trigger being Java, the
paragon of security and non-crashingness? What the ****?!

Now I've lost all my open Explorer folders and various other
inconveniences can all be laid at the doorstep of ... whatever the ****
it was that happened.

Can anyone explain this event? Opening a lot of network connections
shouldn't be a problem on XP, and it sure as hell shouldn't render the
system unusable to the point of forcing a cold boot. Even if the Java
app ate all the free network sockets, it should have been terminatable,
and killing it should have released the sockets. Even terminating other
tasks (such as the nonresponsive Shareaza) that used the network didn't
free any up (Shareaza, when end-tasked, disappeared promptly from all
task and process lists, but Firefox didn't magically start working
again).

Annoyingly, the system help also stopped working(!); I was desperate
enough to actually resort to Help and Support Center to find info on
terminating processes and/or freeing sockets, given that the usual
first-resort, Google, was rendered unavailable by the problem.

This Should Not Have Happened.

We have at least 3 separate problems:
* Java can do things that DoS a Win32 box, without doing anything but
open sockets.
Applets can open sockets (though only back to the originating site);
Win32 boxen are
extremely commonplace. You do the math.
* Windows XP SP2, with all the latest updates (including and especially
security updates),
allows a single app to hog all the network sockets, and apps that
want a socket but
can't get one seem usually to hang rather than gracefully report an
error condition to the
user.
* And it does not seem to properly "terminate with prejudice" an
unresponsive process or
free up resources one had when it does terminate it successfully. Is
there no equivalent of
"kill -9", via Task Manager or otherwise, even when you're logged on
as administrator?

It looks like some problems (can't kill processes, processes can kill
parts of the system's overall functionality, killed tasks don't always
have their resources released by the OS) from the bad old days of
Windows 3.1 are still present and accounted for...

Win32 SP2 with all security fixes;
JDK/JRE 1.5.0_06 (latest, as of a couple weeks ago anyway)


I doubt it's a problem in either Java or Windows. I had an applet (which I have
since optimized) which would open over 10,000 sockets in a very rapid sequence.
On Linux this ran fine, when a socket is closed it closes immediately and is
available for re-use. On Windows (both 2k and XP) the applet would stall due to
socket starvation, when a socket is closed it hangs around for several 10s of
seconds and then a few hundred close all at once.

Even when the applet was totally starved of sockets this did not affect (much)
any other applications. I could kill off the browser (at least what parts of IE
Windows allows you to kill), I could kill the JVM. I could do other networking
tasks, but they had to fight for the sockets.

I think you must have something else interfering with the opening/closing of
sockets.

It's not a Java security issue anyway. Presumably any non-Java application on
your system will allow the same DoS attack.
 
S

Steve Horsley

Twisted said:
We have at least 3 separate problems:
* Java can do things that DoS a Win32 box, without doing anything but
open sockets.
Applets can open sockets (though only back to the originating site);
Win32 boxen are
extremely commonplace. You do the math.

What's your point here?

You think that java should be hobbled so it can't open sockets?
Why pick on one language? I've seen the same problem provoked by
both Python scripts and C programs.

Or do you think applets (java, flash, activeX, javascript etc)
should be barred from using sockets?

* Windows XP SP2, with all the latest updates (including and especially
security updates),
allows a single app to hog all the network sockets, and apps that
want a socket but
can't get one seem usually to hang rather than gracefully report an
error condition to the
user.
* And it does not seem to properly "terminate with prejudice" an
unresponsive process or
free up resources one had when it does terminate it successfully. Is
there no equivalent of
"kill -9", via Task Manager or otherwise, even when you're logged on
as administrator?

It looks like some problems (can't kill processes, processes can kill
parts of the system's overall functionality, killed tasks don't always
have their resources released by the OS) from the bad old days of
Windows 3.1 are still present and accounted for...

Win32 SP2 with all security fixes;
JDK/JRE 1.5.0_06 (latest, as of a couple weeks ago anyway)

I think this is a problem with windows. As I said, I have seen it
before, where something opens enough ports that the OS can't do
its normal networking stuff. I have also seen that when stuck
like this, sometimes the OS won't reboot either, and it's time
for the Big Red Switch.

Try and limit the number of ports being used. Or try another OS.
Java runs on lots of OSs.
 
T

Twisted

Wiping everything and installing a whole new OS to avoid a bug THAT BY
DEFINITION SHOULDN'T EVEN BE THERE? Are you FUCKING NUTS?!

<calmer>
Why doesn't the system release the sockets properly, or terminate the
misbehaving apps properly? Why? WHY??
</calmer>

WHY???????????

This is the sort of behavior you expect from Win3.11 for workgroups.
Bah!
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,968
Messages
2,570,154
Members
46,702
Latest member
LukasConde

Latest Threads

Top