Message5862

woo hoo! After much debugging with visualvm and friends I have finally tracked down all of the sources of classloader leaks in jython, and after making a couple of changes to the jython source I successfully built a servlet that calls into the jython interpreter and whose classloader is garbage collected when the servlet is reloaded.

Here are the issues I found, along with the fixes I applied. If someone could apply changes along these lines to jython I would really appreciate it and I'd consider this bug closed:

(1) Use of thread-local storage: PySystemState objects are stored in TLS, and since my servlet has jython in its WEB-INF/lib directory, that means that all jython classes are loaded by the servlet's classloader. In Tomcat, threads are long-lived, so putting something in TLS causes it to hang around for a long time. Consequently, references to the servlet classloader remained in TLS long after the servlet was reloaded.

To fix this, I added a method to the Py class to explicitly clear out the PySystemState in thread-local storage. I am calling this in a finally block after my interpreter runs; it seems to me that it would be nice to add this call to PythonInterpreter.cleanup() so that cleanup() can be the one method that users call to fully cleanup.

Changes:
  (a) added the following to the class org.python.core.Py:

    public static void clearThreadLocalState()
    {
        threadStateMapping.remove();
    }

    (b) added the remove() method to org.python.core.ThreadStateMapping:

    public void remove()
    {
        cachedThreadState.remove();
    }

(2) PyFile's use of JVM shutdown hooks: PyFile adds a PyFileCloser instance to the JVM's list of shutdown hooks on line 584 of PyFile.java. This causes a classloader leak because the instance added has the Servlet's classloader, and the list of shutdown hooks is never cleared out until the JVM shuts down.

I do not think this is necessary - the JVM will close all open file handles when it exits, so this shutdown hook seems to serve no purpose. I fixed this by commenting out all references to the Closers and the shutdown hook.

After I made both of the changes above, my servlet began unloading correctly. As I said above, I really hope you apply these changes and close this bug so that I can start using an official build of jython (rather than my hacked up local copy!)