Memory Leak Issue in Weblogic, SUN, Apache and Oracle classes

[Amit Jain]

Hi All,

Please find below the description of memory leaks issues.

1. Statistics show major growth in the perm area (static classes).
Flows were ran for 8 hours , Heap dump was taken after 2 hours and at
the end. A growth in Perm area was identified

2. Statistics show from our last run 240MB growth in 6 hour,40mb
growth every hour 2GB heap –can hold ¾ days ,heap will be full in ¾
days
Heap dump show –growth in area as mentioned below

JMS connection/session Area

Apache
org.apache.xml.dtm.DTM[]
org.apache.xml.dtm.ref.ExpandedNameTable$ExtendedType
org.jdom.AttributeList
org.jdom.Content[]
org.jdom.ContentList
org.jdom.Element

SUN
* ConstantPoolCacheKlass
* ConstantPoolKlass
* ConstMethodKlass
* MethodDataKlass
* MethodKlass
* SymbolKlass
byte[]
char[]
com.sun.org.apache.xml.internal.dtm.DTM[]
com.sun.org.apache.xml.internal.dtm.ref.ExtendedType
java.beans.PropertyDescriptor
java.lang.Class
java.lang.Long
java.lang.ref.WeakReference
java.lang.ref.SoftReference
java.lang.String
java.text.Format[]
java.util.concurrent.ConcurrentHashMap$Segment
java.util.LinkedList$Entry

Weblogic
com.bea.console.cvo.ConsoleValueObject$PropertyInfo
com.bea.jsptools.tree.TreeNode
com.bea.netuix.servlets.controls.content.StrutsContent
com.bea.netuix.servlets.controls.layout.FlowLayout
com.bea.netuix.servlets.controls.layout.GridLayout
com.bea.netuix.servlets.controls.layout.Placeholder
com.bea.netuix.servlets.controls.page.Book
com.bea.netuix.servlets.controls.window.Window[]
com.bea.netuix.servlets.controls.window.WindowMode
javax.management.modelmbean.ModelMBeanAttributeInfo
weblogic.apache.xerces.parsers.SecurityConfiguration
weblogic.apache.xerces.util.AugmentationsImpl
weblogic.apache.xerces.util.AugmentationsImpl$SmallContainer
weblogic.apache.xerces.util.SymbolTable$Entry
weblogic.apache.xerces.util.XMLAttributesImpl$Attribute
weblogic.apache.xerces.xni.QName
weblogic.apache.xerces.xni.QName[]
weblogic.ejb.container.cache.CacheKey
weblogic.ejb20.manager.SimpleKey
weblogic.jdbc.common.internal.ConnectionEnv
weblogic.jdbc.common.internal.StatementCacheKey
weblogic.jms.common.Item
weblogic.jms.common.JMSID
weblogic.jms.frontend.FEConnection
weblogic.logging.MessageLogger$1
weblogic.logging.WLLogRecord
weblogic.rjvm.BubblingAbbrever$BubblingAbbreverEntry
weblogic.rjvm.ClassTableEntry
weblogic.rjvm.JVMID
weblogic.rmi.cluster.ClusterableRemoteRef
weblogic.rmi.internal.CollocatedRemoteRef
weblogic.rmi.internal.PhantomRef
weblogic.rmi.spi.ServiceContext[]
weblogic.security.acl.internal.AuthenticatedSubject
weblogic.security.acl.internal.AuthenticatedSubject$SealableSet
weblogic.servlet.internal.ServletRuntimeMBeanImpl
weblogic.transaction.internal.XidImpl
weblogic.utils.collections.ConcurrentHashMap$Entry

Oracle XA Transaction
oracle.jdbc.driver.Binder[]
oracle.jdbc.driver.OracleDatabaseMetaData
oracle.jdbc.driver.T4C7Ocommoncall
oracle.jdbc.driver.T4C7Oversion
oracle.jdbc.driver.T4C8Oall
oracle.jdbc.driver.T4C8Oclose
oracle.jdbc.driver.T4C8TTIBfile
oracle.jdbc.driver.T4C8TTIBlob
oracle.jdbc.driver.T4C8TTIClob
oracle.jdbc.driver.T4C8TTIdty
oracle.jdbc.driver.T4C8TTILobd
oracle.jdbc.driver.T4C8TTIpro
oracle.jdbc.driver.T4C8TTIrxh
oracle.jdbc.driver.T4C8TTIuds
oracle.jdbc.driver.T4CCallableStatement
oracle.jdbc.driver.T4CClobAccessor
oracle.jdbc.driver.T4CConnection
oracle.jdbc.driver.T4CMAREngine
oracle.jdbc.driver.T4CNumberAccessor
oracle.jdbc.driver.T4CPreparedStatement
oracle.jdbc.driver.T4CTTIdcb
oracle.jdbc.driver.T4CTTIk2rpc
oracle.jdbc.driver.T4CTTIoac
oracle.jdbc.driver.T4CTTIoac[]
oracle.jdbc.driver.T4CTTIoauthenticate
oracle.jdbc.driver.T4CTTIokeyval
oracle.jdbc.driver.T4CTTIoscid
oracle.jdbc.driver.T4CTTIoses
oracle.jdbc.driver.T4CTTIOtxen
oracle.jdbc.driver.T4CTTIOtxse
oracle.jdbc.driver.T4CTTIsto
oracle.jdbc.driver.T4CXAConnection
oracle.jdbc.driver.T4CXAResource
oracle.jdbc.oracore.OracleTypeADT[]
oracle.jdbc.xa.OracleXAResource$XidListEntry
oracle.net.ano.Ano
oracle.net.ns.ClientProfile
oracle.net.ns.ClientProfile
oracle.net.ns.NetInputStream
oracle.net.ns.NetOutputStream
oracle.net.ns.SessionAtts
oracle.net.nt.ConnOption
oracle.net.nt.ConnStrategy
oracle.net.resolver.AddrResolution
oracle.sql.CharacterSet1Byte

we are using
Oracle BEA Weblogic 9.2 MP3
JDK 1.5.12
Oracle versoin 10.2.0.4 (for oracle we found one path which is needed
to applied to avoid XA transaction memory leaks).
But we are stuck to resolve SUN, BEA Weblgogic and Apache leaks.

please suggest...

regards, Amit J.

 

[Lew]

Hi All,

Please find below the description of memory leaks issues.

1. Statistics show major growth in the perm area (static classes).
Flows were ran for 8 hours , Heap dump was taken after 2 hours and at
the end. A growth in Perm area was identified

So what's loading classes and not collecting them?

2. Statistics show from our last run 240MB growth in 6 hour,40mb
growth every hour 2GB heap –can hold ¾ days ,heap will be full in ¾
days
Heap dump show –growth in area as mentioned below ....
we are using
Oracle BEA Weblogic 9.2 MP3
JDK 1.5.12
Oracle versoin 10.2.0.4 (for oracle we found one path which is needed
to applied to avoid XA transaction memory leaks).
But we are stuck to resolve SUN, BEA Weblgogic and Apache leaks.

What makes you think those well-established products are responsible for the
leaks and not the application code?

Does that obsolete version of Java support perm-gen sweeping for GC?

 

[Amit Jain]

hi Lew,

What makes you think those well-established products are responsible
for the
leaks and not the application code?
yes you are right. How can we identify fault in our Application
code. Histogram is showing classes of Apache, BEA and Oracle.

Does that obsolete version of Java support perm-gen sweeping for GC?
could you please us more view on this point...

Thanks Lew...

regrds, Amit J.

 

[Lew]

By convention we cite the author of a quoted Usenet post and set the quoted
material off with various depths of consecutive right-angle-bracket ('>')
characters.

yes you are right. How can we identify fault in our Application
code. Histogram is showing classes of Apache, BEA and Oracle.

Histogram from where?

Servers like WebSphere Application Server have options to help track memory
leaks, like Tivoli ITCAM.

We penurious students of the art need less expensive alternatives. The
"-verbose:gc" option to the JVM might help, as might the Java Console.

There are several standard tools for troubleshooting memory usage, starting
with "-verbose:gc", jhat and jmap through JConsole and on to Java VisualVM
<http://java.sun.com/javase/6/docs/technotes/guides/visualvm/index.html>
which last, alas for you, only came out full time with Java 6. You might be
able to use it under Java 6 to debug your Java 5 program, though.

Most of the available tools require deciphering and interpreting log files. See
<http://java.sun.com/javase/6/docs/technotes/tools/index.html#troubleshoot>
and following paragraphs.

could you please us more view on this point...

There is an option to the Sun JVM, "-XX:+CMSClassUnloadingEnabled", that
allows GC to unload unused classes after a while, and another,
"-XX:+CMSPermGenSweepingEnabled", that allows GC to clean up the permanent
generation.

To assist your analysis, "-XXermSize=??m XX:MaxPermSize=???m" let you
control the size of the permanent generation. I run NetBeans with a
"MaxPermSize=200m", and with all it controls, including Tomcat, that seems to
work so far.

I don't know if these options are available with Java 5 either, but you can
run your Java 5 apps on a Java 6 JVM and use these tools to help you diagnose
things.

 

[Amit Jain]

By convention we cite the author of a quoted Usenet post and set the quoted
material off with various depths of consecutive right-angle-bracket ('>')
characters.



Histogram from where?

Servers like WebSphere Application Server have options to help track memory
leaks, like Tivoli ITCAM.

We penurious students of the art need less expensive alternatives.  The
"-verbose:gc" option to the JVM might help, as might the Java Console.

There are several standard tools for troubleshooting memory usage, starting
with "-verbose:gc", jhat and jmap through JConsole and on to Java VisualVM
<http://java.sun.com/javase/6/docs/technotes/guides/visualvm/index.html>
which last, alas for you, only came out full time with Java 6.  You might be
able to use it under Java 6 to debug your Java 5 program, though.

Most of the available tools require deciphering and interpreting log files.  See
<http://java.sun.com/javase/6/docs/technotes/tools/index.html#troubles...>
and following paragraphs.



There is an option to the Sun JVM, "-XX:+CMSClassUnloadingEnabled", that
allows GC to unload unused classes after a while, and another,
"-XX:+CMSPermGenSweepingEnabled", that allows GC to clean up the permanent
generation.

To assist your analysis, "-XXermSize=??m XX:MaxPermSize=???m" let you
control the size of the permanent generation.  I run NetBeans with a
"MaxPermSize=200m", and with all it controls, including Tomcat, that seems to
work so far.

I don't know if these options are available with Java 5 either, but you can
run your Java 5 apps on a Java 6 JVM and use these tools to help you diagnose
things.

Thanks Lew for your valuable inputs.

regards, Amit J.

 

[Volker Borchert]

But we are stuck to resolve SUN, BEA Weblgogic and Apache leaks.

please suggest...

Are you redeploying your applications? I think there was a JDK bug that
caused spurious references to classloaders and classes to be held in such
cases, leading to VM running out of perm gen space.

Check the "bugs fixed" lists on java.sun.com

See also <bc6b36e7-6a5c-450e-ba5d-c0ca07c8a93e@a12g2000pro.googlegroups.com>

 

[Sarkar]

Are you redeploying your applications? I think there was a JDK bug that
caused spurious references to classloaders and classes to be held in such
cases, leading to VM running out of perm gen space.

Check the "bugs fixed" lists on java.sun.com

See also <bc6b36e7-6a5c-450e-ba5d-c0ca07c8a...@a12g2000pro.googlegroups.com>

We are not redeploy we have taken Histogram after running some flow in
different time.

Thanks for your post

 

[Sarkar]

Are you redeploying your applications? I think there was a JDK bug that
caused spurious references to classloaders and classes to be held in such
cases, leading to VM running out of perm gen space.

Check the "bugs fixed" lists on java.sun.com

See also <bc6b36e7-6a5c-450e-ba5d-c0ca07c8a...@a12g2000pro.googlegroups.com>

Hi Volker,

I checked java.sun.com for "bugs" but didn't found any relevant one
for my case.

regards, Amit J.

 

[Volker Borchert]

I checked java.sun.com for "bugs" but didn't found any relevant one
for my case.

Might have been
6636650 java classes_lang (cl) Resurrected ClassLoaders can still have children
fixed in 1.6.0_17 but bug details are no{t, longer} public.

PS please don't quote signatures.

 

[Sarkar]

Might have been
  6636650 java classes_lang (cl) Resurrected ClassLoaders can still have children
fixed in 1.6.0_17 but bug details are no{t, longer} public.

PS please don't quote signatures.

Is there any way to get detail for bug#6636650

regards, Amit J.