Issue1098

classification
Title: PyStringMap doesn't deserialize correctly on 2.2
Type: crash Severity: critical
Components: Core Versions: 2.2.2
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: colinhevans, nriley
Priority: Keywords:

Created on 2008-08-05.03:51:12 by colinhevans, last changed 2008-10-14.17:41:55 by zyasoft.

Files
File name Uploaded Description Edit Remove
stringmaptest.py colinhevans, 2008-08-05.03:51:11
Messages
msg3400 (view) Author: Colin Evans (colinhevans) Date: 2008-08-05.03:51:11
PyStringMap has a nasty bug where if it has a size exactly equal to one
of the numbers in PyStringMap.primes and is serialized and deserialized,
all get caught in an infinite loop.

A test case is attached.  Results and a stck dump are below.  Note that
the main thread is stuck in a busy wait in PyStringMap.__finditem__:

---
trying size 0
0
None
trying size 1
1
None
trying size 2
2
None
trying size 3
3
None
trying size 4
4
None
trying size 5
5
None
trying size 6
6
None
trying size 7
7
None
trying size 8
8
None
trying size 9
9
None
trying size 10
10
None
trying size 11
11
None
trying size 12
12
None
trying size 13
13
^\Full thread dump Java HotSpot(TM) Client VM (1.5.0_13-119 mixed mode,
sharing):

"Low Memory Detector" daemon prio=5 tid=0x01008fe0 nid=0x81d000 runnable
[0x00000000..0x00000000]

"CompilerThread0" daemon prio=9 tid=0x01008510 nid=0x81c200 waiting on
condition [0x00000000..0xb0b077d8]

"Signal Dispatcher" daemon prio=9 tid=0x010080a0 nid=0x81b400 waiting on
condition [0x00000000..0x00000000]

"Finalizer" daemon prio=8 tid=0x010078e0 nid=0x817c00 in Object.wait()
[0xb0a05000..0xb0a05d90]
        at java.lang.Object.wait(Native Method)
        - waiting on <0x25a7ee30> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
        - locked <0x25a7ee30> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
        at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)

"Reference Handler" daemon prio=10 tid=0x01007500 nid=0x816400 in
Object.wait() [0xb0984000..0xb0984d90]
        at java.lang.Object.wait(Native Method)
        - waiting on <0x25a7eeb0> (a java.lang.ref.Reference$Lock)
        at java.lang.Object.wait(Object.java:474)
        at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
        - locked <0x25a7eeb0> (a java.lang.ref.Reference$Lock)

"main" prio=5 tid=0x010013b0 nid=0xb0801000 runnable
[0xb07ff000..0xb0800188]
        at org.python.core.PyStringMap.__finditem__(Unknown Source)
        - waiting to lock <0x25610028> (a org.python.core.PyStringMap)
        at org.python.core.PyStringMap.__finditem__(Unknown Source)
        at org.python.core.PyStringMap.get(Unknown Source)
        at org.python.core.PyStringMap.get(Unknown Source)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:585)
        at org.python.core.PyReflectedFunction.__call__(Unknown Source)
        at org.python.core.PyMethod.__call__(Unknown Source)
        at org.python.core.PyObject.__call__(Unknown Source)
        at org.python.core.PyObject.invoke(Unknown Source)
        at
org.python.pycode._pyx0.f$0(../../colin/jython/stringmaptest.py:20)
        at
org.python.pycode._pyx0.call_function(../../colin/jython/stringmaptest.py)
        at org.python.core.PyTableCode.call(Unknown Source)
        at org.python.core.PyCode.call(Unknown Source)
        at org.python.core.Py.runCode(Unknown Source)
        at org.python.core.__builtin__.execfile_flags(Unknown Source)
        at org.python.util.PythonInterpreter.execfile(Unknown Source)
        at org.python.util.jython.main(Unknown Source)

"VM Thread" prio=9 tid=0x01006c50 nid=0x815600 runnable 

"VM Periodic Task Thread" prio=9 tid=0x0100a790 nid=0x805800 waiting on
condition 

"Exception Catcher Thread" prio=10 tid=0x01001600 nid=0x809800 runnable
msg3401 (view) Author: Colin Evans (colinhevans) Date: 2008-08-05.03:55:24
The bug is caused because PyStringMap.readObject() doesn't ensure that
there is a null entry, which causes PyStringMap.__finditem__() to loop
when trying to retrieve keys that are not in the map.  The easy fix is
to add 1 to the required capacity in PyStringMap.readObject().
msg3402 (view) Author: Nicholas Riley (nriley) Date: 2008-08-05.05:42:45
This bug doesn't exist in 2.5 because it uses a ConcurrentHashMap as the 
underlying storage mechanism, instead of key/value arrays.

(Certainly worth fixing in 2.2.2, of course.)
History
Date User Action Args
2008-10-14 17:41:55zyasoftsettitle: PyStringMap doesn't deserialize correctly -> PyStringMap doesn't deserialize correctly on 2.2
2008-08-11 07:57:20nrileysetversions: + 2.2.2
2008-08-05 05:42:46nrileysetnosy: + nriley
messages: + msg3402
2008-08-05 03:55:25colinhevanssetmessages: + msg3401
2008-08-05 03:51:12colinhevanscreate