Memory leak and SoftReferences

[Joe]

Hi,

I am working on an application that gets large objects from a remote
server and stores them in a Vector for a short period of time. When
the next set of large objects come in the removeAll() method is
called on the Vector object and the next set of large objects are
placed one at a time into the Vector. These large objects stay in the
Vector for a short amount of time (about 5 minutes) before the next
large set comes in.

However, there is a memory leak and I believe it has to do with this
Vector and the objects. I am guessing that for some reason the GC
isn't garbage collecting these large objects after they are removed
from the Vector so I was thinking of using SoftReferences. Now, I
don't know much about SoftReferences but I figure storing Soft-
References would be better than storing the references to the acutal
object.

First, do you think my guess about the GC not collecting unused
references is valid?
Second, would softReferences help reduce these memory leaks?
Third, are there any good tutorials on using Soft/WeakReferences?

Any thoughts or comments would be greatly appreciated.

Thanks,
Joe

 

[steve]

Hi,

I am working on an application that gets large objects from a remote
server and stores them in a Vector for a short period of time. When
the next set of large objects come in the removeAll() method is
called on the Vector object and the next set of large objects are
placed one at a time into the Vector. These large objects stay in the
Vector for a short amount of time (about 5 minutes) before the next
large set comes in.

However, there is a memory leak and I believe it has to do with this
Vector and the objects. I am guessing that for some reason the GC
isn't garbage collecting these large objects after they are removed
from the Vector so I was thinking of using SoftReferences. Now, I
don't know much about SoftReferences but I figure storing Soft-
References would be better than storing the references to the acutal
object.

First, do you think my guess about the GC not collecting unused
references is valid?
Second, would softReferences help reduce these memory leaks?
Third, are there any good tutorials on using Soft/WeakReferences?

Any thoughts or comments would be greatly appreciated.

Thanks,
Joe

check the sun news groups there is an excellent article on soft references.

But you will need to down load the soft compiler version , for your JVM.

Steve

 

[Patricia Shanahan]

First, do you think my guess about the GC not collecting unused
references is valid?

Are you actually running out of memory, or just seeing the memory usage
increase?

Patricia

 

[Joe]

I am getting an OutOfMemoryException and my application will stop
working. The application has an upper limit of 1.2 gig.

BTW, I am using JDK 1.4.2.

-- Joe

 

[Joe]

BTW, I am using the JDK 1.4.2.

I tried to tried to implement SoftReferences just now but it didn't
seem to work. However, I don't think I did it right. I created a
SoftReference and put it into the Vector and then set the large
object that I create the SoftReference for to null. I still got an
OutOfMemoryException.

What do you mean I "will need to download the soft compiler version?

Thanks,
Joe

 

[Wibble]

BTW, I am using the JDK 1.4.2.

I tried to tried to implement SoftReferences just now but it didn't
seem to work. However, I don't think I did it right. I created a
SoftReference and put it into the Vector and then set the large
object that I create the SoftReference for to null. I still got an
OutOfMemoryException.

What do you mean I "will need to download the soft compiler version?

Thanks,
Joe

You should get a profiler like optimizeIt or JProbe and find out
what your leak really is. Its probably not what you think.

You can get a free 30 days of either.

 

[Wibble]

BTW, I am using the JDK 1.4.2.

I tried to tried to implement SoftReferences just now but it didn't
seem to work. However, I don't think I did it right. I created a
SoftReference and put it into the Vector and then set the large
object that I create the SoftReference for to null. I still got an
OutOfMemoryException.

What do you mean I "will need to download the soft compiler version?

Thanks,
Joe

Also, check your process size. Sometimes you get OOM errors
when you exceed stack size or allocate buffers with size of
Integer.MAX_VALUE or something. Run it in jdb and catch OOM Error.

 

[Patricia Shanahan]

Hi,

I am working on an application that gets large objects from a remote
server and stores them in a Vector for a short period of time. When
the next set of large objects come in the removeAll() method is
called on the Vector object and the next set of large objects are
placed one at a time into the Vector.

Are you really using removeAll(), rather than clear() or
removeAllElements()?

removeAll() takes another collection as argument, and only removes
elements in that collection. clear() and removeAllElements() are
equivalent, and each leaves you with a zero size Vector.

Patricia

 

[steve]

BTW, I am using the JDK 1.4.2.

I tried to tried to implement Soft-References just now but it didn't
seem to work. However, I don't think I did it right. I created a
Soft-Reference and put it into the Vector and then set the large
object that I create the Soft-Reference for to null. I still got an
OutOfMemoryException.

What do you mean I "will need to download the soft compiler version?

Thanks,
Joe

sorry could not resist.

instead of using the removeAll(), just redefine the vector using the same
name or set it to null when you have finished.

It works, and so will the GC, unless you did something really silly.

Another trick to use , is instead of sticking your objects into a vector,
write the objects out to temp files , then store the file names in a hash
map (just do it in a background thread), I use this system for displaying
scanned document pages, and it really is quite fast.

be clear what a soft reference is, it can be just as dangerous as a "hard"
ref, when used wrongly and will not always guarantee that you will get rid
of the OutOfMemoryException.

seve

 

[Larry Barowski]

You should get a profiler like optimizeIt or JProbe and find out
what your leak really is. Its probably not what you think.

You can get a free 30 days of either.

Or get Java 1.6 and use jmap and jhat.

 

[Joe]

Actually, I believe it is removeAllElements().

-- Joe

 

[steve]

Actually, I believe it is removeAllElements().

-- Joe

if that were true, then all my apps would be leaking memory as well.
I still extensively use removeAllElements() mainly in my routines for
pulling data from oracle, and have not seen extensive memory leaks in that
area.


int ColumnCount = rsmd.getColumnCount();
// Loop through ResultSet rows
int loop = 0;
newmapping_list.removeAllElements();
// empty the supplier list
while (rset.next()) {
String[] record1 = new String[ColumnCount];
for (int i = 0; i < record1.length; i++) {
record1 = (String)rset.getString(i + 1);
// COPY THE DATA TO A LOCAL ARRAY OF THE RIGHT
SIZE
}
loop++;
newmapping_list.addElement(record1);
}
} catch (Exception e) {
Error_stuff.handleError(e, Error_stuff.EXEPTION_ERROR, -1);
}
//never allow a bad array out
finally {
closeports(cstmt, rset);
if (newmapping_list.size() == 0) {
String[] record1 = new String[2];
record1[0] = EMPTYBADTABLE;
record1[1] = EMPTYBADTABLE;
newmapping_list.addElement(record1);
}

}
return newmapping_list;
}