Issue1746

There is a PermGen memory leak caused by the combination of
codecs.java and the CPython Lib's encodings/__init__.py.  The code in
question is:

    # Register the search_function in the Python codec registry
    codecs.register(search_function)

which adds the search_function PyFunction to
org.python.core.codecs.searchPath which is a static PyList.  Each of
these PyFunctions has a func_globals StringMap containing a key
'codecs', whose value is a PyModule whose __dict__ is a StringMap
containing a key 'sys' (because CPython Lib's codecs.py imports
'sys'), whose value is a PySystemState instance.  So for every Jython
thread which imports encodings, that thread's PySystemState and many
related objects will never be garbage collected.  After enough Jython
threads (only around 20 in our case), PermGen is exhausted resulting
in an OutOfMemoryError.

Our workaround has been to modify codecs.java to check whether
search_function is already in the PyList and if not, avoid adding it
again, but I'm not sure whether this is optimal.  Maybe a better
solution would be to ensure that it is removed from the PyList during
thread cleanup - but only if this could be done without using
finalizers, which according to Joshua Bloch and others should be
avoided if at all possible.

We have been testing out Jython 2.7 and this can also be fixed by making the codecs attached to the system state rather than being static. So we created a CodecState class that basically implements all the same methods as codecs and redirected the methods in codecs back to the current PySystemState's CodecState. This way the codecs are cleaned up along with the system state and the codecs do not have to be synchronized (to avoid race conditions when populating the search function)

The problem with doing as stated in the initial description is that the search function from the first interpreter is going to be the only function loaded and would probably be the cause of another leak (the initial PySystemState would hang around since the search_function PyFunction object hangs around in the static codecs class...though this is unconfirmed)

Brandon: any chance of getting your solution as a patch? Even better if you have tests (but not a requirement)?

Submitted as a pull request here https://bitbucket.org/jython/jython/pull-request/5/issue1746-fix-issue-with-static-codecs

Sorry I don't have any tests...if you need more info on how to reproduce let me know...the basic test is just to keep creating and destroying jython interpreters and within each one import the encodings module and watch the permgen memory increase, though it might take a while unless you import lots of other stuff to fill up the PermGen space more quickly.

Thanks Brandon!

Setting to urgent to poke my future self.

Fixed in 7126:7ff23feeb526