Title: defaultdict get behavior causes inconsistent map entries leading to potential memory leaks
Type: behaviour Severity: critical
Components: Core Versions: Jython 2.7
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: zyasoft Nosy List: jayv, zyasoft
Priority: Keywords:

Created on 2014-04-30.21:26:24 by jayv, last changed 2014-05-21.20:35:17 by zyasoft.

msg8315 (view) Author: Jo Voordecekers (jayv) Date: 2014-04-30.21:26:24
Expected behavior:
defaultdict.get(key, default) should not create entries in the map when no key is found and a default is specified.


>>> from collections import defaultdict
>>> d = defaultdict(set)
>>> d['one'] = [1]
>>> d['two']
>>> print(d)
defaultdict(<type 'set'>, {'two': set([]), 'one': [1]})
>>> d.get('three', [])
>>> print(d)
defaultdict(<type 'set'>, {'two': set([]), 'one': [1], 'three': set([])})

The map now contains an entry 'three', this is a nasty source of memory-leaks (our system had maps with millions of entries caused by this subtle bug).
msg8316 (view) Author: Jim Baker (zyasoft) Date: 2014-04-30.22:35:58
Can be fixed by overriding with PyDefaultDict#get(PyObject key, PyObject defaultObj), which in turn uses Cache#getIfPresent. Assumes this fix: by upgrading to using LoadingCache.

Should be readily backportable to 2.5
msg8377 (view) Author: Jim Baker (zyasoft) Date: 2014-05-11.04:32:39
Fixed in 7240:3059c41e3838
Date User Action Args
2014-05-21 20:35:17zyasoftsetstatus: pending -> closed
2014-05-11 04:32:46zyasoftsetversions: + Jython 2.7, - Jython 2.5
2014-05-11 04:32:39zyasoftsetstatus: open -> pending
assignee: zyasoft
resolution: accepted -> fixed
messages: + msg8377
2014-04-30 22:35:58zyasoftsetresolution: accepted
messages: + msg8316
nosy: + zyasoft
2014-04-30 21:26:24jayvcreate