So I stumped you? Classes available to the ClassLoader...

[rwfields]

The nearest thing I can figure is to create a manifest of all classes
available. This solution obviously has its drawbacks, as the list
would have to be maintained by hand.

Why would I do such a thing? The idea I am pursuing is a genetic
programming study. There is a code generator that selects function
calls (from available classes) randomly to create organisms. I cannot
do this unless I know which functions are available. Any suggestions
would be well received. My original post follows:

---

Is there a way to determine which classes are available for loading in
the ClassLoader? I have created a custom class loader that derives
from the system ClassLoader. I have no problem querying which Packages

are available, but cannot seem to find which classes are available.
Any suggestions?

Thanks,
Randall

 

[Oliver Wong]

IMHO, you should have replied to the original message, rather than
starting this new one, and posting the contents in reverse order. It took me
a while to figure out what was going on. I wasn't sure if you were replying
to someone else, or if you were assuming some previous context knowledge,
etc.

The nearest thing I can figure is to create a manifest of all classes
available. This solution obviously has its drawbacks, as the list
would have to be maintained by hand.

Why would I do such a thing? The idea I am pursuing is a genetic
programming study. There is a code generator that selects function
calls (from available classes) randomly to create organisms. I cannot
do this unless I know which functions are available. Any suggestions
would be well received. My original post follows:

---

Is there a way to determine which classes are available for loading in
the ClassLoader? I have created a custom class loader that derives
from the system ClassLoader. I have no problem querying which Packages

are available, but cannot seem to find which classes are available.
Any suggestions?

This is usually NOT how one does "genetic programming". Usually you
would have a well defined problem which you want to solve, and a "fitness"
metric for determining how close each organism is to solving that problem.
You then fine tune parameters to algorithms; you don't generate code which
randomly calls API methods (e.g. file.delete()).

- Oliver

 

[Daniel Dyer]

Are you trying to list methods only from some small subset of custom
classes or from every class on the classpath, including the platform
classes? If it's the former, then it shouldn't be a problem to generate
that list at build time. If it's the latter, I fear you are doomed to
failure. Your search space for the genetic programming algorithm is huge
and most random combinations will be invalid anyway.

Dan.

 

[rwfields]

Daniel,

I am trying to list methods from every class on the classpath,
including the platform classes. The first problem is finding out which
classes are on the classpath. For the system classes, I just did a
"find" on the javadoc. I redirected the results to a file, and coaxed
that file into something that looks like a list of classes (thus, the
manifest). This works fine for the classes that are listed in the
javadoc, however there are many other packages that are not covered by
the javadoc. I suppose I could go find the docs associated with each
package (some of the com.sun stuff) and make manifests for those too.
This procedure seems like a setup for failure though. What happens
when I change Java versions, and the manifest file is no longer valid?
I was just hoping that there was a way to query which classes are
available for loading.

Whether my genetic algorithm is destined for failure or not (probably
it is!) is orthoganol to whether we can discover which classes are
available for loading.

 

[Daniel Dyer]

Daniel,

I am trying to list methods from every class on the classpath,
including the platform classes. The first problem is finding out which
classes are on the classpath. For the system classes, I just did a
"find" on the javadoc. I redirected the results to a file, and coaxed
that file into something that looks like a list of classes (thus, the
manifest). This works fine for the classes that are listed in the
javadoc, however there are many other packages that are not covered by
the javadoc. I suppose I could go find the docs associated with each
package (some of the com.sun stuff) and make manifests for those too.
This procedure seems like a setup for failure though. What happens
when I change Java versions, and the manifest file is no longer valid?
I was just hoping that there was a way to query which classes are
available for loading.

It might be better to scan the jar files (including rt.jar), you can run
"jar -tf myjar.jar" and redirect the output to a file. This gives you the
class names, which is really the only problematic part. From the class
names you can get the method names programatically.

Whether my genetic algorithm is destined for failure or not (probably
it is!) is orthoganol to whether we can discover which classes are
available for loading.

Indeed, but I thought it was the more interesting part

Out of interest, what kind of program are you trying to evolve? What will
it do? Most applications of genetic programming techniques seem to use a
very small set of building blocks, to restrict the evolution runtime to
something less than the remaining life of the universe.

You've chosen a massive set of building blocks, most of which are probably
irrelevant for your chosen problem. The All Classes frame of the Java 5.0
documentation lists 3279 public classes and interfaces. Do you really
need all the methods for XML parsing, CORBA integration, database access,
etc.?

Dan.

 

[Oliver Wong]

Oliver,

Sorry for the breach in ediquette, I'll try to do better.

No problem; did you see the stuff I wrote at the bottom of that post?

- Oliver

 

[rwfields]

Oliver, I saw your notes. Thanks.

Daniel, looking at the rt.jar is an excellent idea, I'll give it a try.
Probably you are right, it will be best to limit the set of functions
used as building blocks, so as not to get overwhelmed.

All of the genetic algorithms I have seen are fixed. That is, the
algorithm itself does not change, only the organisms. In real
genetics, the algorithm is part of the genotype of the organism. I
left genetic algorithms and started toward genetic programming in hopes
of solving this problem. The pie in the sky is a program that is able
to identify its peers and breed with them (if appropriate) according to
its own set of code. Thus, the code itself will be open to the genetic
operations, and could evolve.

I know it is an enormous task, but I like to dream. Just getting a
code generator to generate a simple

public static void main(String args[]) {
System.out.println("Hello world!");
}

becomes non-trivial because we are calling the System class,
referencing the static field "out", then using that field to call the
println method. In order for the a program to select this code, it has
to be able to find it. As I move down this path, I find that I am
writing a compiler, or nearly so. Thank goodness for JavaCC. Each
organism has a symbol table that it keeps for each compilation unit.
Other symbols available for use are discovered through reflection. The
referenced set of symbols (the stuff the organism is actually using) is
one of the mutable parts of the organism itself. The project
JRefactory gave some good insight on how to move things around...

Anyway, that was probably way more that either of you wanted. For now,
my immediate target is to get code that will generate the simple
HelloWorld. From there I'll move onto equations, and hopefully someday
onto code that can generate itself.

Randall

 

[Oliver Wong]

All of the genetic algorithms I have seen are fixed. That is, the
algorithm itself does not change, only the organisms. In real
genetics, the algorithm is part of the genotype of the organism. I
left genetic algorithms and started toward genetic programming in hopes
of solving this problem. The pie in the sky is a program that is able
to identify its peers and breed with them (if appropriate) according to
its own set of code. Thus, the code itself will be open to the genetic
operations, and could evolve.

I know it is an enormous task, but I like to dream. Just getting a
code generator to generate a simple

public static void main(String args[]) {
System.out.println("Hello world!");
}

becomes non-trivial because we are calling the System class,
referencing the static field "out", then using that field to call the
println method. In order for the a program to select this code, it has
to be able to find it. As I move down this path, I find that I am
writing a compiler, or nearly so. Thank goodness for JavaCC. Each
organism has a symbol table that it keeps for each compilation unit.
Other symbols available for use are discovered through reflection. The
referenced set of symbols (the stuff the organism is actually using) is
one of the mutable parts of the organism itself. The project
JRefactory gave some good insight on how to move things around...

I'm not a biochemist, so this is a bit simplification:

The way it works in "real life" is DNA randomly mutates, and the laws of
chemistry and physics causes DNA to interact with other molecules in certain
ways which lead to certain behaviour. The laws of chemistry do not
themselves mutate, for example.

Rather than have your program (written in Java) mutate itself, perhaps
it might be better to design a new programming language, and then write an
interpreter for that language in Java. The interpreter, as part of its
interpretation phase, will randomly mutate the program it is running.

If you want to emulate biological evolution, that'll be closer, and
it'll probably greatly simplify your design.

The problem with having a program mutate itself is that it might mutate
out its ability to mutate itself, and thus stop mutating.

- Oliver

 

[rwfields]

Oliver, you have some great ideas.

My take is that mutation is an artifact of the environment, and not the
organism itself. Therefore the agent that mutates each organism will
remain separate from the mutating code. Thus a program cannot loose
the ability to mutate itself, because that is an ability it never
really possessed. The breeding algorithm is what will be coded in the
mutable organism. Of course, it might loose the ability to breed, but
then it stops adding its bad habits to the population.

This is a divergence from the typical GA literature which tends to
encode mutation rate into each organism. I suppose results speak
volumes, but I still do not agree with the norm.