Issue1482

classification
Title: Use of String.intern fills up the Permgen Space and causes OutOfMemoryError
Type: crash Severity: critical
Components: Core Versions: Jython 2.1, 2.2.2, 2.5.1
process
Status: closed Resolution: works for me
Dependencies: Superseder:
Assigned To: Nosy List: gsqfs, zyasoft
Priority: urgent Keywords: patch

Created on 2009-09-30.15:12:17 by gsqfs, last changed 2010-09-04.01:05:24 by zyasoft.

Files
File name Uploaded Description Edit Remove
t_intern.py zyasoft, 2010-09-04.01:00:49 Intern test program
canonical-map-string-literal.patch zyasoft, 2010-09-04.01:02:02 Patch to PyString, PyUnicode to remove intern use
Messages
msg5208 (view) Author: Gregor Schmid (gsqfs) Date: 2009-09-30.15:12:16
The problem shows when scripts are compiled repeatedly on the fly.
Slowly but surely the PermGen space will fill up with no way to recover.

I can provide a patch against Jython 2.2.1 that uses a Weak Map instead.
From what I googled this is even supposed to be more efficient than
String.intern.
msg6033 (view) Author: Jim Baker (zyasoft) Date: 2010-09-03.16:34:17
This resource leak -- which should be applicable to 2.5, I don't believe anything has changed in the use of intern usage in String -- has escaped attention. We need to fix it before 2.5.2 beta 2 too. (2.5.2b2 is really trying to address all serious resource leaks like this one.)

Fortunately it should be an easy fix, and it's even something that will clean up our code in a bit in PyString.
msg6036 (view) Author: Jim Baker (zyasoft) Date: 2010-09-04.01:00:49
I was unable to reproduce this problem on trunk. Using a canonical map (using weak keys) instead of intern seems to make sense, but monitoring with JMX doesn't show any difference.

Attached is a patch to use a canonical map that passes regrtest; and my test program that should intern everything, but also will GC since nothing is held - the globals map is clean each time, that is we do

exec stmt in {}
msg6037 (view) Author: Jim Baker (zyasoft) Date: 2010-09-04.01:05:23
I can see this patch being useful in 2.6, but without seeing benefit here, it seems too late to apply for 2.5. Most importantly, there are still other uses of intern in our codebase, and usage of the API might also assume that.

Also see the irc discussion on #jython, http://www.extreme.st/jython.extreme.st/irclogs/index.php/?date=2010-09-04

Please reopen if you can submit a reproducible test case - it's important we get this right.
History
Date User Action Args
2010-09-04 01:05:24zyasoftsetstatus: open -> closed
resolution: accepted -> works for me
messages: + msg6037
2010-09-04 01:02:04zyasoftsetfiles: + canonical-map-string-literal.patch
keywords: + patch
2010-09-04 01:00:50zyasoftsetfiles: + t_intern.py
messages: + msg6036
2010-09-03 16:34:18zyasoftsetpriority: urgent
resolution: accepted
messages: + msg6033
nosy: + zyasoft
2009-09-30 15:12:17gsqfscreate