Message5810

Author matt_brinkley
Recipients adam.spiers, bpedman, matt_brinkley, pjenvey, rkurin
Date 2010-06-15.22:20:04
SpamBayes Score 1.66394e-05
Marked as misclassified No
Message-id <1276640408.03.0.563732398771.issue1522@psf.upfronthosting.co.za>
In-reply-to
Content
I looked into this a bit, since I'm also having lots of problems with permgen space, classloader leaks etc.

I found that this is related to python classes that implement java interfaces - the root cause in this example is the "import os" call - the os module has a python class called PythonPosixHandler that implements a java interface (org.python.posix.PosixHandler)

I found that each run of the while loop below causes the size of PyType.class_to_type to increase by one: an additional entry for PythonPOSIXHandler is added, with a monotonically increasing number appended (this is added in MakeProxies.makeProxy - you can see that there is a static int that is incremented and appended) ... it seems to me that this is the root of (one of the) problem(s): the hash map lookups in class_to_type always result in a cache miss because each time the proxy is generated it has a unique name.

class_to_type seems very problematic in dynamic classloader environments (where classloaders can come and go, e.g. tomcat servlet reloading or OSGi) - classes are added to this hash map but never cleared out, and the hash map is global across all interpreter instances, so there is no way to clean it out or otherwise prevent it from growing without bound. The code here is complex enough that I can't figure out a good way to deal with this (other than adding an explicit "clearCache()" method, which would be fine by me) but maybe someone can help explain what class_to_type is used for, why the proxy names have a monotonically increasing int appended to them, etc.
History
Date User Action Args
2010-06-15 22:20:08matt_brinkleysetmessageid: <1276640408.03.0.563732398771.issue1522@psf.upfronthosting.co.za>
2010-06-15 22:20:08matt_brinkleysetrecipients: + matt_brinkley, pjenvey, bpedman, rkurin, adam.spiers
2010-06-15 22:20:07matt_brinkleylinkissue1522 messages
2010-06-15 22:20:04matt_brinkleycreate