Message7935

Author janvh
Recipients janvh
Date 2013-03-15.17:22:50
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1363368170.61.0.725380496269.issue2026@psf.upfronthosting.co.za>
In-reply-to
Content
We executing Jython scripts using a thread pool. And have noticed a memory leak when instantiating multiple instances of the PyScriptEngine in the same thread.

Versions tested: 2.5.3 / 2.5.4-rc1 / 2.7-b1 
All have the same issue.
The problem did not occur on 2.5.1. It is linked with the introduction of a ThreadLocal object in the PythonInterpreter.

The following code can be used to reproduce it:

public static void testScriptingLeakDetail() throws InterruptedException {
    ExecutorService pool = Executors.newSingleThreadExecutor();

    int i = 0;
    while (true) {
        i++;
        if (i % 10 == 0) {
            System.out.println("Iteration " + i);
        }

        pool.submit(new Runnable() {
            @Override
            public void run() {
                PyScriptEngineFactory factory = new PyScriptEngineFactory();
                PyScriptEngine engine = (PyScriptEngine) factory.getScriptEngine();
                try {
                    engine.eval("s = 'hello' + ' world'");
                } catch (ScriptException e) {
                }
            }
        });
        Thread.sleep(10);
    }
}

The leak seems to be linked to the factory.getScriptEngine() instantiating the PythonInterpreter using ThreadLocal objects. 
The ThreadLocalMaps of the threads in our Thread pool seems to be filling up with PyScriptEngineScope objects.
This would go unnoticed if the Threads would be created as required and disposed of afterwards, but because we are using a thread pool, the threads (and their thread local map) get recycled.

I have tried getting to the bottom of the problem, but so far have not found a solution.
The only fix I can suggest is making the protected ThreadLocal<PyObject> threadLocals static.
History
Date User Action Args
2013-03-15 17:22:50janvhsetmessageid: <1363368170.61.0.725380496269.issue2026@psf.upfronthosting.co.za>
2013-03-15 17:22:50janvhsetrecipients: + janvh
2013-03-15 17:22:50janvhlinkissue2026 messages
2013-03-15 17:22:50janvhcreate