dependency-detection in java

[Andreas Leitgeb]

Say, I've got two classes A and B, one of which (A)
contains "static final" fields, the other (B)
references these fields. Now the compiler doesn't
use references, but includes the values from A
directly in B.

Now suppose some of these fields in A are changed.

What is the usual "trick" to make sure that all
"B"-type classes are recompiled, when an "A"-type
class has changed?

Is the only sensible way really to remove *all*
..class files and have *all* files recompiled, or
does there exist some trick to let the compiler
know these dependencies, and do the right thing?

Seemingly, it didn't do it last time I had to
change such an "A"-type class.

Does there perhaps exist a thirdparty-tool, (a la
makedepends & make for C++) that makes sure that
really all neccessary recompilations are performed?

As it seems, the class-file for "B" doesn't even
have any mention of "A" (unless it also uses other
fields or methods of A, of course).

As a programmer I'd rather not have to think about
manually removing .class-files that are no longer
up-to-date.

PS: I'm bound to still work with java 1.4, but if
such a feature exists for newer versions, I'd
still also like to know.

 

[Lew]

Say, I've got two classes A and B, one of which (A)
contains "static final" fields, the other (B)
references these fields. Now the compiler doesn't
use references, but includes the values from A
directly in B.

Now suppose some of these fields in A are changed.

What is the usual "trick" to make sure that all
"B"-type classes are recompiled, when an "A"-type
class has changed?

Compile B, which will cause it to compile A.

Is the only sensible way really to remove *all*
..class files and have *all* files recompiled, or
does there exist some trick to let the compiler
know these dependencies, and do the right thing?
Yes.

Seemingly, it didn't do it last time I had to
change such an "A"-type class.

Because you compiled A, not B.

Does there perhaps exist a thirdparty-tool, (a la
makedepends & make for C++) that makes sure that
really all neccessary recompilations are performed?
"javac".

As it seems, the class-file for "B" doesn't even
have any mention of "A" (unless it also uses other
fields or methods of A, of course).

But the source file does.

As a programmer I'd rather not have to think about
manually removing .class-files that are no longer
up-to-date.
Ant.

PS: I'm bound to still work with java 1.4, but if
such a feature exists for newer versions, I'd
still also like to know.

Use Ant.

 

[Stefan Ram]

What is the usual "trick" to make sure that all
"B"-type classes are recompiled, when an "A"-type
class has changed?

It is assumed that constants do not change.

A change to correct a mistake should happen
rarely and might trigger a manual deletion of
all class files.

If the value might change in time, it might
be better read from a configuration file than
be hard-coded in the source code.

As a work-around, a getter-method might be used.

 

[Andreas Leitgeb]

Compile B, which will cause it to compile A.

That's not the point. Actually, I don't really
know which classes do reference some A's static
final fields (there are usually a couple of such
type "B" classes for each type "A" class).

The only thing I see is, that some .java-file, of which
I don't necessarily know that it happens to be of
type A (that is: contains static final constants)
has been changed (e.g. in some version control system),
and I want to be sure that after next "build" the
whole project is correctly (but not unneccessarily)
recompiled.

Is this really such an absurd wish in the
world of java?

"javac".
Ant.

The combination of these two makes it not work.
As it seems, ant only passes those .java files
to the compiler that are newer than their .class
files (or whose .class-file doesn't exist), which
means it passes only A.java, but not B.java to
the compiler in the given szenario.

If ant were able to know B's dependency on
A, and pass B.java to the compiler as well, my
problem would be solved.

What one needs is some dependency-information
generated by javac when first compiling B.java (of
course it would generate that for all classes,
but only for type B classes it would be non-empty).
Actually that would have to be exactly the list of
other classes, of which static finals were used.

With this information, ant could make a better
guess at what filenames to pass to the compiler.

 

[Andreas Leitgeb]

It is assumed that constants do not change.
A change to correct a mistake should happen
rarely and might trigger a manual deletion of
all class files.

Changes happen. :-/ There can be various
reasons for why constants change "just this time"...

In my case it were strings defined in an .idl file.

If the value might change in time, it might
be better read from a configuration file than
be hard-coded in the source code.

It's supposed to be const, but casually, such
things just do change. The customer wants it
that other way...

 

[Daniel Pitts]

Changes happen. :-/ There can be various
reasons for why constants change "just this time"...

In my case it were strings defined in an .idl file.


It's supposed to be const, but casually, such
things just do change. The customer wants it
that other way...

Generally, if the user wants a constant "that other way", then you do
externalize it into a configuration file.

In any case, I suggest using "ant" (google it). It helps manage builds
and build-related tasks.

 

[Roedy Green]

What is the usual "trick" to make sure that all
"B"-type classes are recompiled, when an "A"-type
class has changed?

see http://mindprod.com/jgloss/javacexe.html#CIRCULAR

 

[Andreas Leitgeb]

Generally, if the user wants a constant "that other way", then you do
externalize it into a configuration file.

Ok, I put it another way: The constant may change a couple of
times during development+testing-phase, but will not change
in the future. If it weren't ever to change, I would have
inlined the value at each place where it is used, directly.

One can think of it that it actually *is* in a separate
configuration file, which happens to be in the format of
java-source (actually, it's in an idl-file, from which the
java-file is generated)...

In any case, I suggest using "ant" (google it). It helps manage builds
and build-related tasks.

Actually, we *use* ant, and as it is now, "ant" is part of the problem.
because it only feeds the changed classes to the compiler, but not the
dependent ones.

 

[Andreas Leitgeb]

see http://mindprod.com/jgloss/javacexe.html#CIRCULAR

Sorry, that isn't it in this case.

First, passing "both" files to javac means pass "all"
files to javac, since I'm not necessarily aware of all
circular references, not even of all non-circular
references.

Second, it's exactly "ant" that triggers the problem,
by reducing the number of files to compile too far.
(it cannot do better, though, because it hasn't got the
necessary dependency-information)

 

[Bent C Dalager]

Actually, we *use* ant, and as it is now, "ant" is part of the problem.
because it only feeds the changed classes to the compiler, but not the
dependent ones.

As far as I can recall:

- final statics are inlined in client classes without any reference to
where they came from and so changes to the class that defines them
will not cause client classes to be recompiled.

- javac doesn't handle this at all and so fails to correctly recompile
code unless you start by deleting all .class files.

- someone once claimed that jikes and/or antmake (can't remember
which) may handle this better, but I haven't actually checked.

- consequently, I always do an "ant clean install" when I do a make.

- an alternative is to write accessor methods for your constants and
always use those. The final static data member can then be private and
changes to it need never propagate to client classes.


As for final statics never changing - well, that's bullshit. They do
not change /during execution/ but that's not the problem here.

Cheers
Bent D

 

[Andreas Leitgeb]

As far as I can recall:
- an alternative is to write accessor methods for your constants and
always use those. The final static data member can then be private and
changes to it need never propagate to client classes.

Not always possible, because at certain places, java demands constants,
and accessor-methods cannot be used there for obvious reasons.

- someone once claimed that jikes and/or antmake (can't remember
which) may handle this better, but I haven't actually checked.

jikes (unforunately) isn't among my options, either.

(The other paragraphs I snipped are those of which I was aware already)

Thanks. Even though it didn't really help technically, I at least
felt understood

 

[Daniel Pitts]

Not always possible, because at certain places, java demands constants,
and accessor-methods cannot be used there for obvious reasons.


jikes (unforunately) isn't among my options, either.

(The other paragraphs I snipped are those of which I was aware already)

Thanks. Even though it didn't really help technically, I at least
felt understood

The only place I can think of that java demands constants is the "case
" branch of a switch statement.
If thats truly what you need to worry about, you might consider using
enums instead.

In any case, it is pretty common to set up your ant build.xml file to
remove all .class files automatically. Specifically, to build into a
separate directory tree, and nuke that whole tree. That would solve
your dependency problems.

 

[Andreas Leitgeb]

The only place I can think of that java demands constants is the "case
" branch of a switch statement.
If thats truly what you need to worry about, you might consider using
enums instead.

I can't use enums, since the project is still bound to java 1.4.
And it's not only about constants. These constants are actually
just one example of "incompatible changes", that would require
recompilation of other classes.

In any case, it is pretty common to set up your ant build.xml file to
remove all .class files automatically. Specifically, to build into a
separate directory tree, and nuke that whole tree. That would solve
your dependency problems.

That's the "rebuild from scratch"-pattern, that I also wished to avoid.
The project is large enough to make complete compilation painfully slow.

Please see also my Posting "dependency-detection in java - Take 2"

 

[Lew]

That's the "rebuild from scratch"-pattern, that I also wished to avoid.
The project is large enough to make complete compilation painfully slow.

How slow is "painfully" slow? I remember when recompiling /one/ module took
three days, when I was first learning to program. Now what does it take your
recompile? Twenty minutes? An hour?

Live with it and do the recompile that you need to do. I've seen two or more
answers that that is what you have to do. If it isn't acceptable, refactor
your global variables not to be so global.

 

[Roedy Green]

Second, it's exactly "ant" that triggers the problem,
by reducing the number of files to compile too far.
(it cannot do better, though, because it hasn't got the
necessary dependency-information)

I would hope there is only as small amount of circularity. Then you
can do a mini ant script to compile just those pieces, then do a big
build afterward.

 

[Daniel Pitts]

Sorry, that isn't it in this case.

First, passing "both" files to javac means pass "all"
files to javac, since I'm not necessarily aware of all
circular references, not even of all non-circular
references.

Second, it's exactly "ant" that triggers the problem,
by reducing the number of files to compile too far.
(it cannot do better, though, because it hasn't got the
necessary dependency-information)

I know that IntelliJ IDEA will let you determine what classes use a
constant, I'd be surprised if other IDEs didn't also. Check it out.

In any case, like I said, the only place that absolutely requires a
constant is a switch statement, perhaps if you refactored that special
case to an if statement, your wouldn't have that slight problem.

Although, you could come up with NoSuchMethod exceptions if you move
methods around too, so its best to recompile.

 

[Andreas Leitgeb]

I would hope there is only as small amount of circularity. Then you
can do a mini ant script to compile just those pieces, then do a big
build afterward.

The point is, I don't really know the amount of circularity and
even further it could easily change with every check in.

 

[Andreas Leitgeb]

How slow is "painfully" slow?

Long enough that it feels painful to me.
Really, I don't want to throw around with numbers, since they don't
really tell anything. Would you judge for me the value per hour?

I remember when recompiling /one/ module took three days, ...

Sure, and if it took a month, our developers would probably
take more care, but on the other hand, our projects would
take perhaps ten times longer to finish (for all the manual
checking for forgotten semicolons), so I'm rather glad about
shorter compile-cycles.

Live with it and do the recompile that you need to do.

I'd like to, but I'd like to avoid any full recompiles
that I don't really need - supported by tools, which either
don't exist yet, or are not known to anyone answering here
so far.

I've seen two or more answers that that is what you have to do.
If it isn't acceptable, ...

Of course I have to accept the current state for now, and of course
I do a full recompile casually (everytime I notice that some change
in a java-file doesn't have the desired runtime-effect.)

I want to trigger a general discussion about reliable dependency-
management in the java-world.
What I didn't originally expect nor intend were numerous similar
response, suggesting to live with current state.

I do appreciate all attempts to help, but I feel sorry for
helpers' time wasted in redundant replies.

 

[Andreas Leitgeb]

I know that IntelliJ IDEA will let you determine what classes
use a constant, ...

"let me determine" ? I'm not sure if I parse this phrase correctly.
Does it mean, it determines it for me, or asks me to determine it myself?

I'd be surprised if other IDEs didn't also. Check it out.

The IDE's I've used so far offer these features (with respect to this context)
- create an ant-script and use ant for building, offering a "build" and
a "build all" button. This is where this Thread's problem is
*not* solved. "Reverse-dependencies" as they seem to be called
are not considered.
- for a certain identifier give me a list of locations where it is used.
This is the right direction, but doesn't go far enough.
I haven't yet seen an IDE, that would use this information
automatically for rebuilding.

Did I miss any other feature that you had in mind?

 

[Lew]

I want to trigger a general discussion about reliable dependency-
management in the java-world.
What I didn't originally expect nor intend were numerous similar
response, suggesting to live with current state.

While javac has trouble with certain dependencies, in particular the one that
is vexing Andreas right now, and despite my previous comments I agree with the
need to resolve it, it is superior to some other compilers in resolving
dependencies at all.

I suggest setting up your Ant targes so that compiling the classes that define
compile-time constants trigger targets that compile classes specifically
affected by the change. This does require a build designer to analyze the
dependencies by hand AFAIK, unless someone can suggest tools to help there.
If a constant is global enough that freaking /every/ class depends on it,
you'll get hit with compile-the-world, but for classes that have a more
limited effect you'll be able to craft more limited rebuilds.

Careful attention to filesets in the source tree and how they ripple out, and
careful crafting of relevant targets should make it possible for the build.xml
to specify the tightest possible rebuilds without risk.

In a truly worst-case scenario you could use a custom Ant task, but it
shouldn't be necessary.