OutOfMemoryError in java app

[mike]

java.lang.OutOfMemoryError in Cruisecontrol 2.5 (bin)
=====================================================

As far as I know there are 3 ways to increase the heap size when you
get the
java.lang.OutOfMemoryError in cruisecontrol:

- CC_OPTS - incr. memory to build loop
- ANT_OPTS - incr. menory to the actual ant jvm.
- Forked jvm witing ant - (when attribute forked="on" is used) incr.
the memory for the forked process.
This is done with: <jvmarg value="-Xmx756M"
/> inside the <java> task.
This matter has making me very confused a longer period. I have not
been sure what is
making the jvm run out of memory. So decided to make a test. Run the
ant tasks from
a 'shell script' and run exactly the same tasks from 'cruisecontrol'.

This is the setup I use:

Shell script
------------

ANT_OPTS="-Xms512m -Xmx756m"
I use them as jvm options when starting the jvm with java:

$JAVACMD $ANT_OPTS -Xbootclasspath/p:$XALAN:$XERCES_IMPL:$XML_APIS
-classpath "${ANT_HOME}/lib/ant-launcher.jar" "-Dant.home=${ANT_HOME}"
org.apache.tools.ant.launch.Launcher -lib $JUNIT $*


CruiseControl
-------------

In 'cruisecontrol.sh' we use the following:

CC_OPTS="-Xms256m -Xmx512m"

Same here. The CC_OPTS are used as jvm args

EXEC="$JAVA_HOME/bin/java $CC_OPTS -cp $CRUISE_PATH
-Djavax.management.builder.initial=mx4j.server.MX4JMBeanServerBuilder
CruiseControlWithJetty $@ -jmxport 8000"


The 'antscript' pointed out in the config.xml contains the following:

ANT_OPTS="-Xms512m -Xmx756m"

$JAVACMD $ANT_OPTS -classpath "${ANT_HOME}/lib/ant-launcher.jar"
"-Dant.home=${ANT_HOME}" org.apache.tools.ant.launch.Launcher "$@"


When I run the shell script the build is successful and I view all the
results in a browser.

When I run cruisecontrol, with attribute usedebug="true" in ant plug-in
I get the following error:

Error:

[junitreport] Parsing file:
'/vobs/rbs/sw/rbssw_boam/boam_subsys/boam_swb/boam_bldu/ant_build/coverage/TEST-se.ericsson.wcdma.rbs.boam.mao.dmo.auxdevice.tb.sut.AuxDeviceSetMoFilterTest.xml'
[junitreport] Parsing file:
'/vobs/rbs/sw/rbssw_boam/boam_subsys/boam_swb/boam_bldu/ant_build/coverage/TEST-se.ericsson.wcdma.rbs.boam.mao.dmo.auxdevice.tb.sut.AuxDeviceSetMoImplTest.xml'
[junitreport] Parsing file:
'/vobs/rbs/sw/rbssw_boam/boam_subsys/boam_swb/boam_bldu/ant_build/coverage/TEST-se.ericsson.wcdma.rbs.boam.mao.dmo.auxdevice.tb.sut.AuxPlugInUnitTest.xml'
[junitreport] Parsing file:
'/vobs/rbs/sw/rbssw_boam/boam_subsys/boam_swb/boam_bldu/ant_build/coverage/TEST-se.ericsson.wcdma.rbs.boam.mao.dmo.auxdevice.tb.sut.HardwareTest.xml'
[junitreport] Parsing file:
'/vobs/rbs/sw/rbssw_boam/boam_subsys/boam_swb/boam_bldu/ant_build/coverage/TEST-se.ericsson.wcdma.rbs.boam.mao.dmo.tr.tb.sut.TxDeviceGroupMoImplTest.xml'
[junitreport] Parsing file:
'/vobs/rbs/sw/rbssw_boam/boam_subsys/boam_swb/boam_bldu/ant_build/coverage/TEST-se.ericsson.wcdma.rbs.boam.mao.dmo.tr.tb.sut.UbchDeviceSetMoImplTest.xml'
[junitreport] Parsing file:
'/vobs/rbs/sw/rbssw_boam/boam_subsys/boam_swb/boam_bldu/ant_build/coverage/TEST-se.ericsson.wcdma.rbs.boam.mao.emo.bb.tb.sut.UpLinkBaseBandPoolTest.xml'
[junitreport] Parsing file:
'/vobs/rbs/sw/rbssw_boam/boam_subsys/boam_swb/boam_bldu/ant_build/coverage/TEST-se.ericsson.wcdma.rbs.boam.mao.rmo.cell.tb.sut.CarrierMoImplTest.xml'
[junitreport] Parsing file:
'/vobs/rbs/sw/rbssw_boam/boam_subsys/boam_swb/boam_bldu/ant_build/coverage/TEST-se.ericsson.wcdma.rbs.boam.mao.rmo.cell.tb.sut.SectorMoTest.xml'
[junitreport] Parsing file:
'/vobs/rbs/sw/rbssw_boam/boam_subsys/boam_swb/boam_bldu/ant_build/coverage/TEST-se.ericsson.wcdma.rbs.boam.mao.rmo.channel.tb.sut.EDchResourcesPMTest.xml'
[junitreport] Parsing file:
'/vobs/rbs/sw/rbssw_boam/boam_subsys/boam_swb/boam_bldu/ant_build/coverage/TEST-se.ericsson.wcdma.rbs.boam.mao.rmo.channel.tb.sut.HsDschResourcesMoTest.xml'
[junitreport] Parsing file:
'/vobs/rbs/sw/rbssw_boam/boam_subsys/boam_swb/boam_bldu/ant_build/coverage/TEST-se.ericsson.wcdma.rbs.boam.mao.rmo.channel.tb.sut.PccpchMoImplTest.xml'
[junitreport] Parsing file:
'/vobs/rbs/sw/rbssw_boam/boam_subsys/boam_swb/boam_bldu/ant_build/coverage/TEST-se.ericsson.wcdma.rbs.boam.mao.rmo.channel.tb.sut.PrachPMTest.xml'
[junitreport] Parsing file:
'/vobs/rbs/sw/rbssw_boam/boam_subsys/boam_swb/boam_bldu/ant_build/coverage/TEST-se.ericsson.wcdma.rbs.boam.mao.rmo.channel.tb.sut.SccpchMoImplTest.xml'
[junitreport] Parsing file:
'/vobs/rbs/sw/rbssw_boam/boam_subsys/boam_swb/boam_bldu/ant_build/coverage/TEST-se.ericsson.wcdma.rbs.boam.mao.rmo.iub.tb.sut.IubDataStreamMoImplTest.xml'
[junitreport] Parsing file:
'/vobs/rbs/sw/rbssw_boam/boam_subsys/boam_swb/boam_bldu/ant_build/coverage/TEST-se.ericsson.wcdma.rbs.boam.mao.rmo.node.tb.sut.RbsSynchronizationMoImplTest.xml'

BUILD FAILED
java.lang.OutOfMemoryError
at
org.apache.tools.ant.Project.executeSortedTargets(Project.java:1225)
at
org.apache.tools.ant.Project.executeTarget(Project.java:1185)
at
org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:40)
at
org.apache.tools.ant.Project.executeTargets(Project.java:1068)
at org.apache.tools.ant.Main.runBuild(Main.java:668)
at org.apache.tools.ant.Main.startAnt(Main.java:187)
at org.apache.tools.ant.launch.Launcher.run(Launcher.java:246)
at org.apache.tools.ant.launch.Launcher.main(Launcher.java:67)
Caused by: java.lang.OutOfMemoryError
--- Nested Exception ---
java.lang.OutOfMemoryError


So what happens in junitreport in ant?
--------------------------------------

When executing target <junit> an xml file for each suite is generated
in a specified directory.
Name of the file is something like:

TEST-<packagename>.RbsSynchronizationMoImplTest.xml

These files will be merged into another xml file called
TESTS-TestSuites.xml.

This is were things go wrong the merge and write to file is not
properly done within eclipse.

The

<?xml version="1.0" encoding="UTF-8" ?>
<testsuites>
<testsuite errors="0" failures="2" id="0" name="TestItAll"
package="se.ericsson.wcdma.rbs.boam.common.tb.sut" tests="8099"
time="168.631">

<properties>
.......

<testcase
classname="se.ericsson.wcdma.rbs.boam.common.tb.sut.TestCreate_rbs3101"
name="test_3101_0021_underMinCreate_Battery_1"
time="0.0030"></testcase>

<testcase
classname="se.ericsson.wcdma.rbs.boam.common.tb.sut.TestCreate_rbs3101"
name="test_3101_0022_overMaxCreate_Battery_1" time="0.0010"></testcase>

.....

<system-out><![CDATA[

Here it seems like the last part <![CDATA[ is not complete. To be
sure I ran the 'shell script' and
could see that data was missing after CDATA[.

Here is part of the 'TESTS-TestSuites.xml' from the 'shell script'.

........................

<system-out><![CDATA[EMMA: collecting runtime coverage data ...

Boam Test log: info: se.ericsson.cello.CmaMgr, init()[main]


Boam Test log: info: se.ericsson.cello.CmaMgr, initFactory()[main]


Boam Test log: info: se.ericsson.cello.CmaMgr, createRootMo()[main]


So what is really done in ant that consumes so much memory?
-----------------------------------------------------------

I checked the java file in the svn source repository.
http://svn.apache.org/viewvc/ant/co...kdefs/optional/junit/XMLResultAggregator.java

The parsing takes place in here ( just look at the trace from
cruisecontrol above 'Parsing file:')

protected Element createDocument() {
// create the dom tree
DocumentBuilder builder = getDocumentBuilder();
Document doc = builder.newDocument();
Element rootElement = doc.createElement(TESTSUITES);
doc.appendChild(rootElement);

generatedId = 0;

// get all files and add them to the document
File[] files = getFiles();
for (int i = 0; i < files.length; i++) {
File file = files;
try {
log("Parsing file: '" + file + "'",
Project.MSG_VERBOSE);
if(file.length()>0) {
Document testsuiteDoc
=
builder.parse(FileUtils.getFileUtils().toURI(files.getAbsolutePath()));
Element elem = testsuiteDoc.getDocumentElement();
// make sure that this is REALLY a testsuite.
if (TESTSUITE.equals(elem.getNodeName())) {
addTestSuite(rootElement, elem);
generatedId++;
} else {
//wrong root element name
// issue a warning.
log("the file " + file
+ WARNING_INVALID_ROOT_ELEMENT,
Project.MSG_WARN);
}
} else {
log("the file " + file
+ WARNING_EMPTY_FILE,
Project.MSG_WARN);
}
} catch (SAXException e) {
// a testcase might have failed and write a zero-length
document,
// It has already failed, but hey.... mm. just put a
warning
log("The file " + file + WARNING_IS_POSSIBLY_CORRUPTED,
Project.MSG_WARN);
log(StringUtils.getStackTrace(e), Project.MSG_DEBUG);
} catch (IOException e) {
log("Error while accessing file " + file + ": "
+ e.getMessage(), Project.MSG_ERR);
}
}
return rootElement;
}

The next step is a the actual writing to file with the following
code:

/**
* Write the DOM tree to a file.
* @param doc the XML document to dump to disk.
* @param file the filename to write the document to. Should
obviouslly be a .xml file.
* @throws IOException thrown if there is an error while
writing the content.
*/
protected void writeDOMTree(Document doc, File file) throws
IOException {
OutputStream out = null;
PrintWriter wri = null;
try {
out = new BufferedOutputStream(new
FileOutputStream(file));
wri = new PrintWriter(new OutputStreamWriter(out,
"UTF8"));
wri.write("<?xml version=\"1.0\" encoding=\"UTF-8\"
?>\n");
(new
DOMElementWriter()).write(doc.getDocumentElement(), wri, 0, " ");
wri.flush();
// writers do not throw exceptions, so check for them.
if (wri.checkError()) {
throw new IOException("Error while writing DOM
content");
}
} finally {
FileUtils.close(wri);
FileUtils.close(out);
}
}


I cannot see anything obvous with the code. Also it is very
important to note that the shell script works fine while
cruisecontrol gets the OutOfMemoryError!

Any ideas? Your own thoughts and similair experience is interesting
for me.

cheers,

//mikael