Issue2842

classification
Title: Jython converts None to null when calling put(), but not when setting item with Python syntax
Type: behaviour Severity: major
Components: Core Versions: Jython 2.7.1
Milestone: Jython 2.7.2
process
Status: pending Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: doublep, jeff.allen, zyasoft
Priority: normal Keywords:

Created on 2019-11-29.15:58:50 by doublep, last changed 2020-02-01.08:36:10 by jeff.allen.

Messages
msg12812 (view) Author: (doublep) Date: 2019-11-29.15:58:50
I'm not 100% sure it is a bug, but:

m = HashMap()
m.put (1, None) # => {1=null}

m = HashMap()
m[1] = None # => {1=None}

The latter of course causes exceptions when Java code bumps into a Jython object where it doesn't expect it.

Or is it intended that None is converted to null only when invoking a Java method?
msg12814 (view) Author: Jim Baker (zyasoft) Date: 2019-11-30.00:53:35
This violates our existing semantic equivalence, where we try to translate Python objects to equivalent Java objects in crossing these boundaries. Should be a straightforward fix.
msg12816 (view) Author: (doublep) Date: 2019-12-02.11:04:04
Then I would appreciate a quick fix.

I made a few more tests with HashMap (m) and ArrayList (l):

None translated to null: m.put, a.add, a[...] =, a[...:...] =, a.add

None _preserved as-is_, i.e. a bug: m[...] =, m.setdefault, l.append, l.extend.
msg12818 (view) Author: (doublep) Date: 2019-12-02.11:09:40
By the way, this is a regression compared with 2.5.1 in those cases where methods were available.  E.g.

m = HashMap()
m[1] = None

gives {1=null} on 2.5.1.
msg12870 (view) Author: Jeff Allen (jeff.allen) Date: 2019-12-22.07:55:20
See also #2838. I've a horrid feeling we are trying to satisfy two irreconcilable demands.

ISTR making some changes around the handling of null to prevent the problems that arise with containers that can't hold null (but might hold None?), and containers that can hold but whose Java clients don't know what to do with a Py.None.

Yes, here I am making java.util.Map more like dict: https://hg.python.org/jython/rev/65c4b774a715 . But then, of course, it makes it less like java.util.Map.
msg12948 (view) Author: Jeff Allen (jeff.allen) Date: 2020-01-31.20:13:15
Version 2.7.2 is more consistent now:
>>> m[0] = None
>>> m
{0: None, u'a': u'hello', <type 'int'>: 42}
>>> m.put(1, None)
>>> m
{0: None, u'a': u'hello', 1: None, <type 'int'>: 42}

However, I think this is more consistently the opposite of what Jim recommends. I'll see what I can do about that.

I believe I did this to make the wrapped containers more like Python ones (pass more of the tests extended to these wrapped types). Some java containers disallow null keys or values or both (Hashtable, for example), so if we convert None to null, these will raise an exception. But that may be the price of the equivalence.
msg12950 (view) Author: (doublep) Date: 2020-01-31.20:22:03
FWIW, in my usecase it's all about integration with Java. I don't actually create maps in Python (Jython), I just modify (put, remove, etc.) existing ones created in Java. Also, Java code doesn't know and for purity reasons must not care how its HashMaps are used and of course has no idea about existence of "None".

I.e. in my usecase it is Python scripts that are wrapped around Java code and are the higher level. Scripts can and should adapt to changes in the Java-written core as needed, but Java part should never be bothered with peculiarities of whatever uses it.
msg12954 (view) Author: Jeff Allen (jeff.allen) Date: 2020-02-01.08:36:09
On closer inspection, we're good on this already (in 2.7.2). The REPL uses repr(), so you see None, but Java toString() tells a different story.

>>> m.toString()
u'{0=null, a=hello, 1=null, class org.python.core.PyInteger=42}'

It was probably fixed in the course of https://hg.python.org/jython/rev/65c4b774a715 . I'll check to see this works everywhere it should. JavaProxyMap and test_dict_jy, look fairly comprehensive, and the test seems to know which types don't support null. Ok, I found a typo ... .

JavaProxySet has not had the same scrub (e.g. improved repr), but doesn't appear to suffer this problem (e.g add and update are consistent in storing a null).
History
Date User Action Args
2020-02-01 08:36:10jeff.allensetstatus: open -> pending
resolution: accepted -> fixed
messages: + msg12954
2020-01-31 20:22:03doublepsetmessages: + msg12950
2020-01-31 20:13:16jeff.allensetmessages: + msg12948
2019-12-22 07:55:20jeff.allensetnosy: + jeff.allen
messages: + msg12870
2019-12-13 07:56:22jeff.allensetpriority: normal
resolution: accepted
milestone: Jython 2.7.2
2019-12-02 11:09:40doublepsetmessages: + msg12818
2019-12-02 11:04:04doublepsetmessages: + msg12816
severity: normal -> major
2019-11-30 00:53:35zyasoftsetmessages: + msg12814
2019-11-29 22:33:53zyasoftsetnosy: + zyasoft
2019-11-29 15:58:50doublepcreate