File.deleteOnExit() is evil

Javadoc for File.deleteOnExit() says : 

“Requests that the file or directory denoted by this abstract 

pathname be deleted when the virtual machine terminates.”

Every time we use “ImageIO.write(img, “png”, outStream)” a temporary file is created, and the “deleteOnExit()” function is called on that temporary file. Every “deleteOnExit()” allocates a “struct dlEntry” in the shared “io_util.c” native source file. This “struct dlEntry” is about 1K in size, and will never be freed until the system exits, so every time we do an ImageIO.write(), we take up another 1K of memory! No package should use “deleteOnExit()” dynamically - it guarantees a memory leak.

One workaround is to disallow the use of cache files by passing false to the static ImageIO.setUseCache() method. This avoids the deleteOnExit() problems and should have little adverse effect on performance, especially in the case of server-side applications working with relatively small images. This memory leak is a major reliability issue for developers using ImageIO in their server side applications.

The worst part: the data structure is kept in native memory.If it only was in Java heap, you might be able to diagnose the problem much quicker. Since it is in native memory rather than Java heap, profiler tools will not be able to find this memory leak.

There has been a bug filed on Sun.

Coming to the most important part, we did not use deleteOnExit() directly. Rather, ImageIO.jar we used in our application uses it. Looking into javax.imageio.ImageIO class yields some very interesting results :-

It uses a cached file in memory which it marks as deleteOnExit(). deleteOnExit() means that VM will clean it up when VM shuts down cleanly. However, it also does a file.delete() later. This should clear up the deleteOnExit status for this file. But it does not. This means there is a memory leak in deleteOnExit + delete combination that it does delete the physical file but does not clear the data-structures associated with it in native memory. Since, these are not on heap memory but on native memory, this leak will not show up on OptimizeIt profiling.

Finally, this has resolved the OOME which we had been trying hard to resolve

Performance tuning and OutOfMemoryError(OOME)

For the past few days, I have been trying to find out the cause for OOME occuring in our application. In the rigorous web-searching that followed, I have found lot many sites/blogs/bug-databases which were helpful in deciding our next approach to be taken to resolve OOME.

It is after we started to see lot many OOMEs that we started worrying seriously about nailing down the application to find memory leaks. The profiler tool we used did not provide any lead either… It was then that I started to search web to see if someone was in our shoes earlier. Fortunately, there have been many and some have suggested workarounds for such problems. Below are some of the URLs that are worth browsing. Hope these enlighten u into tuning ur application to improve its availability.

Configuring heap, internal workings of the Garbage Collector and JVM are a must when you are deploying a critical application that requires a high degree of availability and a good performance. All the very best in finetuning ur app… So get set go….

http://cretesoft.com/archive/archive.jsp

http://www.javaperformancetuning.com/newsletter.shtml

http://www.javaperformancetuning.com/tools/hpjtune/

http://www.javaworld.com/javaworld/jw-11-2000/jw-1117-performance_p.html

http://www.opensourcetutorials.com/tutorials/Server-Side-Coding/Java/optimize-java-app/page8.html

http://www.petefreitag.com/articles/gctuning/

http://www.unixville.com/~moazam/categories/java/

http://www.unixville.com/~moazam/stories/2004/05/18/debuggingHangsInTheJvm.html

http://www.lambdacs.com/debugger/UserManual/ODBUserManual.html

http://www-128.ibm.com/developerworks/java/library/j-dyn0429/

http://www-128.ibm.com/developerworks/java/library/j-threads3.html

http://www-128.ibm.com/developerworks/java/library/j-perf06304/

http://www-128.ibm.com/developerworks/java/library/j-dyn0429/

Info abt GC

http://java.sun.com/docs/hotspot/gc1.4.2/

http://java.sun.com/developer/technicalArticles/Programming/GCPortal/

Blogs…

http://blog.arendsen.net/?p=14

http://rifers.org/blogs/gbevin/2005/6/8/outofmemoryerror_with_free_mem

http://rifers.org/blogs/gbevin/java?offset=5

http://www.szegedi.org/articles/memleak.html

http://www.szegedi.org/articles/memleak2.html

http://www.szegedi.org/articles/memleak3.html

http://www.patrickpeak.com/page/patrick ( Read under the section : Your Web App is leaking memory during restarts - Here’s why…)

http://www.javaworld.com/javaworld/jw-10-1996/jw-10-indepth.html

BUGS filed

http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4405807
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6232010
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4957990
http://bugs.sun.com/bugdatabase/view_bug.do;:YfiG?bug_id=4697804
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4705373