File.deleteOnExit() is evil
Javadoc for File.deleteOnExit() says :
"Requests that the file or directory denoted by this abstract
pathname be deleted when the virtual machine terminates."
Every time we use “ImageIO.write(img, “png”, outStream)” a temporary file is created, and the “deleteOnExit()” function is called on that temporary file. Every “deleteOnExit()” allocates a “struct dlEntry” in the shared “io_util.c” native source file. This “struct dlEntry” is about 1K in size, and will never be freed until the system exits, so every time we do an ImageIO.write(), we take up another 1K of memory! No package should use “deleteOnExit()” dynamically – it guarantees a memory leak.
One workaround is to disallow the use of cache files by passing false to the static ImageIO.setUseCache() method. This avoids the deleteOnExit() problems and should have little adverse effect on performance, especially in the case of server-side applications working with relatively small images. This memory leak is a major reliability issue for developers using ImageIO in their server side applications.
The worst part: the data structure is kept in native memory.If it only was in Java heap, you might be able to diagnose the problem much quicker. Since it is in native memory rather than Java heap, profiler tools will not be able to find this memory leak.
There has been a bug filed on Sun.
Coming to the most important part, we did not use deleteOnExit() directly. Rather, ImageIO.jar we used in our application uses it. Looking into javax.imageio.ImageIO class yields some very interesting results :-
It uses a cached file in memory which it marks as deleteOnExit(). deleteOnExit() means that VM will clean it up when VM shuts down cleanly. However, it also does a file.delete() later. This should clear up the deleteOnExit status for this file. But it does not. This means there is a memory leak in deleteOnExit + delete combination that it does delete the physical file but does not clear the data-structures associated with it in native memory. Since, these are not on heap memory but on native memory, this leak will not show up on OptimizeIt profiling.
Finally, this has resolved the OOME which we had been trying hard to resolve :-)