C
Chris Uppal
FamilyNet International Newsgate
From: "Chris Uppal" <[email protected]>
But this is a completely different problem from the one you originally
described, where the worker could ask for classes it needed but didn't have.
The new version, fire-and-forget, is perfectly reasonable, but requires a
different architecture.
For a start, the job fired off to the worker must have /all/ the necessary
information for computation to complete (configuration, input data, classfiles,
....). One natural medium for delivery of all that data would be a JAR file
containing all the needed. Note that in a fire-and-forget environment it is
better to be pessimistic about providing data, in particular the effort of
identifying a truly minimal set of required classes would be a waste of time.
So I would just fire off a JAR file containing all the calculation classes with
every request. There would be a lot of duplication, but so what ? It would
make your life much easier.
If you do want to try to cut down the number of classes sent, then you'll have
to do it on the master machine, or in advance (perhaps by hand). If you want
to generate such a list automatically (and I repeat that I don't think it would
be worthwhile), then a good way to do it would be to scan the bytecodes of the
relevant classes, forming a transitive closure of the class references starting
at the "entry-point". The easy, but coarse, way to track inter-class
references is to look at each classfile's constant pool for references to other
classes. If you are up for a bit of extra effort you could do the same kind of
thing, but at method granularity. In either case, a classfile library like ASM
or BCEL would do the bulk of the work for you. Note that you would have to
find some way of handling reflective references to class (Class.forName()) --
the simplest way of doing that would be to forbid them. Alternatively look for
literal strings which are valid class names (in JNI syntax).
There are tools available which will do this kind of analysis in order to
reduce the size of JAR files. You might be able to apply such a tool directly,
or "borrow" some of its code.
-- chris
FamilyNet <> Internet Gated Mail
http://www.familynet-international.org
From: "Chris Uppal" <[email protected]>
But this is a completely different problem from the one you originally
described, where the worker could ask for classes it needed but didn't have.
The new version, fire-and-forget, is perfectly reasonable, but requires a
different architecture.
For a start, the job fired off to the worker must have /all/ the necessary
information for computation to complete (configuration, input data, classfiles,
....). One natural medium for delivery of all that data would be a JAR file
containing all the needed. Note that in a fire-and-forget environment it is
better to be pessimistic about providing data, in particular the effort of
identifying a truly minimal set of required classes would be a waste of time.
So I would just fire off a JAR file containing all the calculation classes with
every request. There would be a lot of duplication, but so what ? It would
make your life much easier.
If you do want to try to cut down the number of classes sent, then you'll have
to do it on the master machine, or in advance (perhaps by hand). If you want
to generate such a list automatically (and I repeat that I don't think it would
be worthwhile), then a good way to do it would be to scan the bytecodes of the
relevant classes, forming a transitive closure of the class references starting
at the "entry-point". The easy, but coarse, way to track inter-class
references is to look at each classfile's constant pool for references to other
classes. If you are up for a bit of extra effort you could do the same kind of
thing, but at method granularity. In either case, a classfile library like ASM
or BCEL would do the bulk of the work for you. Note that you would have to
find some way of handling reflective references to class (Class.forName()) --
the simplest way of doing that would be to forbid them. Alternatively look for
literal strings which are valid class names (in JNI syntax).
There are tools available which will do this kind of analysis in order to
reduce the size of JAR files. You might be able to apply such a tool directly,
or "borrow" some of its code.
-- chris
FamilyNet <> Internet Gated Mail
http://www.familynet-international.org