Clarification on firewall issues with Java networking APIs

[Lew]

Where finicky here means that any meaningful firewall will block the
incoming connections by default and NAT routers will completely block the
incoming requests unless forwarding is explicitly allowed.

Quite so.

 

[Qu0ll]

Quite so.

So basically I am in a no-win situation. I can have callbacks with RMI but
it doesn't play well with firewalls and I can't have callbacks with any
other method for the same reason. Quite frankly, if callbacks of any kind
can't be made to work with firewalls then they are useless in a practical
sense given that most/all corporations use firewalls. So-called "Comet"
approaches looked promising because the callbacks pass through the standard
HTTP port (or that's my understanding) but it seems that they can't be made
to work with applets. *sigh* Ho hum.

--
And loving it,

-Q
_________________________________________________
(e-mail address removed)
(Replace the "SixFour" with numbers to email me)

 

[Matt Humphrey]

So basically I am in a no-win situation. I can have callbacks with RMI
but it doesn't play well with firewalls and I can't have callbacks with
any other method for the same reason. Quite frankly, if callbacks of any
kind can't be made to work with firewalls then they are useless in a
practical sense given that most/all corporations use firewalls. So-called
"Comet" approaches looked promising because the callbacks pass through the
standard HTTP port (or that's my understanding) but it seems that they
can't be made to work with applets. *sigh* Ho hum.

It's a fundamental characteristic of TCP/IP that listeners cannot not reach
out to callers--they have to wait for the caller to first connect. Network
topology (NATs & firewalls) has grown up around this concept to the extent
that client-to-server connections are easy and common whereas
server-to-client connections require special configuration. If you want
clients to receive timely (non-polled) events, they must maintain an open
connection to the server. It is entirely possible to do so, but you have to
keep in mind that you're trading server performance for client performance.

It's not clear to me why servlets would not work for this, even if the
connection is proxied. The technique I use (with RMI, not HTTP) is that
the client requests "waiting events". This makes the connection which then
stays open so long as there are no waiting events, but only up to a small
time limit, such as a minute--something well below the servlet container's
threshold. When an event arrives it either is sent as the reply directly
and the connection closed or it is queued until the next request. (You could
keep the connection open depending on how long "long" is and whether
proxying is a problem.) The point is that the connection doesn't have to
stay open indefinately. We're simply trying to beat the latency of polling.
(If the container threshold is low enough there will be little difference
between this and polling and you'll get the worst of both worlds.)

The performance limitation of this technique is that all your clients will
have simultaneously pending waits that are all essentially idle. They will
receive a fast response to any arriving event, but the server will not
support very many clients, hence the tradeoff of client for server
performance.

I plan to try this out soon, as soon as I get Eclipse to work with JBoss
again.

Matt Humphrey http://www.iviz.com/

 

[Martin Gregorie]

Try send an HTTP request directly to www.google.com, if it fails then
either Google is down or direct outbound HTTP are blocked

Or maybe it makes more sense to actually try and connect to your
server on the correct port just in case 80 is open but your
port is not.

Arne

Use "traceroute -Tp=80 google.com" to find out where the problem is.

 

[Martin Gregorie]

Try send an HTTP request directly to www.google.com, if it fails then
either Google is down or direct outbound HTTP are blocked

Or maybe it makes more sense to actually try and connect to your
server on the correct port just in case 80 is open but your
port is not.

Arne

Oops - make that "traceroute -Tp 80 google.com" (no = ).

 

[Esmond Pitt]

It is possible.

RMISocketFactory.setSocketFactory(new FixedPortRMISocketFactory());
LocateRegistry.createRegistry(60000);

[snip]

It is much more possible than that. You don't haev to use socket
factories at all. Just call UnicastRemoteObject.exportObject(Remote
remote, int port), or super(port) if you extend URO.

 

[Nigel Wade]

This may sound like a silly question, but by opening sockets on particular
port(s), aren't we back at square one with respect to firewalls and their
related problems? This is the key thing I am trying to work around.

Yes.

If the server wants to callback to the client the client has to be listening on
some port. The server has to be able to open a socket to that port. If it's a
fixed port you only need to poke one hole in the firewall, if it's a random
port you need to poke a very wide hole. If there is any NAT involved then it
becomes even more complicated. The same applies to any callback mechanism,
whether it's based on RMI, servlets or raw sockets.

This is why a blocking call method might be best. But you need to ascertain
whether it can meet other requirements. Each client waiting for a response will
generate a thread in the servlet, with each thread blocked in the service
routine. The constraints on the servlet/servlet container may not allow that.
If you implement a timeout mechanism to reduce the length of time for which the
servlet blocks (effectively delayed polling) you will inevitably increase the
number of sockets being created/destroyed, and that may cause other problems.

Any method to implement bi-directional asynchronous transfer through firewalls
is complicated. Callback mechanisms are typically used within LANs, between
trusted hosts, where any firewalls are under the the same overall
administrative control.

 

[Arne Vajhøj]

"Long" as in "how long is a piece of string" is a relative term... ;-)

Yup. But most web developers have some idea about how long time it
should take to respond to a HTTP requests. Some web servers have
a limit they enforce.

I have tried this specific method and failed. The thread would run perfectly for
anything between 12 to 14 days, and then die without any indication as to why
it had died. There was no output in any of the Tomcat logs, or the servlet
logs, no stack trace, and I was unable to establish any monitor which would
trap the thread's death. It just vanished without a trace. Needless to say, a
thread which runs for 12 to 14 days and then vanishes is rather hard to debug.
True.

In my case the "long" as discussed above is typically of the order of 10s,

10s should be OK. Heavy database queries could take that time. I would
be worried if it was hours or many minutes.

It would be if it can be made to work. Besides keeping the thread alive, you
have the problem of the servlet being able to establish a connection back to
the client through whatever firewalls are in the way.

It could be easier to have the client connect to a port.

Arne

 

[Arne Vajhøj]

Use "traceroute -Tp=80 google.com" to find out where the problem is.

I don't think where matters for the poster.

Arne

 

[Arne Vajhøj]

If I open a socket to the client in a separate thread, how do that
thread know who/where the client is and how does it know when the client
is ready to accept data?

The info comes with the HTTP request and the client start listening
before it sends the HTTP request so it will always be ready when the
server connects.

But for client firewall it may be better to have the clients
connect.

Arne

 

[Martin Gregorie]

I don't think where matters for the poster.

Well.... if traceroute shows its blocking in his host or the firewall
between his LAN and the wider 'net he may be able to fix the cause. If
the blockage is outside his firewall he will have to find a workround
which may mean changes to the application.

I'd want to know who/what is blocking that before I started changing things.

 

[TwelveEighty]

Thanks for the info...


If I open a socket to the client in a separate thread, how do that thread
know who/where the client is and how does it know when the client is ready
to accept data? And how does the client know when to respond to accept the
socket connection and when to receive data? Sorry for the confusion - this
is all very new to me.

--
And loving it,

-Q
_________________________________________________
(e-mail address removed)
(Replace the "SixFour" with numbers to email me)

Folks,

Wouldn't a way better design pattern for the OP to use JMS here? If he
is truly looking for async communication originating from possibly any
"node" on a network going to another node, but firewall friendly...?
The "client" would make a request (either through a web service or JMS
message) and then simply wait for messages to show up.

 

[Richard Maher]

Hi,

Wouldn't a way better design pattern for the OP to use JMS here?

Wouldn't better still be a Multicast Socket setup such as outlined in: -
http://groups.google.com/group/comp.lang.java.programmer/msg/ae117827662e8ea5

Whether an unsigned Applet is permitted to read a Multicast message from the
codebase I still don't know, but if you try it and report back that'd be
great

Cheers Richard Maher

 

[Qu0ll]

Wouldn't better still be a Multicast Socket setup such as outlined in: -
http://groups.google.com/group/comp.lang.java.programmer/msg/ae117827662e8ea5

Whether an unsigned Applet is permitted to read a Multicast message from
the
codebase I still don't know, but if you try it and report back that'd be
great

Well according to the JSE 6 JavaDocs:

"Currently applets are not allowed to use multicast sockets."

so I guess the answer is no. And wouldn't this solution still have the
problem with getting through firewalls?

--
And loving it,

-Q
_________________________________________________
(e-mail address removed)
(Replace the "SixFour" with numbers to email me)

 

[Richard Maher]

Hi Q,

Well according to the JSE 6 JavaDocs:

"Currently applets are not allowed to use multicast sockets."

Thanks for the pointer. I have to say that I'm a bit curious as to what
exactly "Currently" means in this context. It looks like multicast socket
functionality has been with the Java SDK since 1.1 so maybe it's a bit like
"Currently the sun rises in the east ans sets in the west"

What I did find on the web were references to people who successfully
deployed *signed* applets that use multi-cast sockets. (Along with others
who say it can't be done, and yet others who've altered the security manager
behaviour) My impression is that it works with signed-applets but there's
probably only one sure way to find out.

(Personally, I can't see why an unsigned applet would be breaking any
sandbox rules by *reading* a multicast message eminating from the codebase,
but maybe that's just me?)

And wouldn't this solution still have the
problem with getting through firewalls?

No idea; best ask a firewall guy How they are configured for UDP traffic
and, in the absence of a destination port, how you'd designate a multicast
group address as being "open" is a mystery to me.

Sorry not to be of more help.

Cheers Richard Maher