java.net.SocketException + too many open files

[bond]

Hi,

I am using URLConnecton(wrapped with HttpURLConnection) class to
download images and store into file system from a HTTP source.
If the number of images exceeds 10K, i get java.net.SocketException
and "too many open files" error.
I do close the inputstream and diconnect the connection after i read
each image.
Note: this happens only on Linux

Can any one give me any fix for this?

thanks in advance,

 

[Daniel Pitts]

Hi,

I am using URLConnecton(wrapped with HttpURLConnection) class to
download images and store into file system from a HTTP source.
If the number of images exceeds 10K, i get java.net.SocketException
and "too many open files" error.
I do close the inputstream and diconnect the connection after i read
each image.
Note: this happens only on Linux

Can any one give me any fix for this?

thanks in advance,

It might be useful to do a heap dump to see what objects are still lying
around. You might find that you aren't actually closing all streams/files.

You might also want to change your ulimits, although I'm not sure
that'll help in this case.

 

[Gordon Beaton]

I am using URLConnecton(wrapped with HttpURLConnection) class to
download images and store into file system from a HTTP source. If
the number of images exceeds 10K, i get java.net.SocketException and
"too many open files" error.
I do close the inputstream and diconnect the connection after i read
each image.
Note: this happens only on Linux

There are other places your application might be opening files than
just URLConnection... for example every Runtime.exec() costs 3 open
files.

To check the limit, use "ulimit -a" or "ulimit -n" in the shell.

This isn't a solution, but you can increase the limit by running
"ulimit -n someNumber" in the shell before running the application, or
put the command in your .bash_profile or similar place so it runs
every time you login.

The solution is to find out what files your application isn't closing,
and make sure you close them. You can do that by doing "ls -l" in
/proc/NN/fd, or running lsof on the process.

/gordon

--

 

[dg2]

CLOSE_WAIT and java.net.SocketException too many open files issue

You can find the solution of this "java.net.SocketException + too many open files" and CLOSE_WAIT issue at :
sportswithin.com/generalTopics.jsp