Issue2838

Benoit Cantin makes the following observation on 2.7.2b2:
"""
from java.util import ArrayList
test = ArrayList()
test.remove("aaa")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: aaa is not in list

The same code returns False with Jython 2.7b3 which is correct according to the documentation: https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/ArrayList.html#remove(java.lang.Object)

If I remember correctly, a teammate also found this bug in version 2.7.1.
"""

This (I suppose) is because we have given ArrayList some behaviour that is Python-like. We have noted in #2645 the  behaviour of LinkedList.pop(), which is surprising (to some) in that it removes from the opposite end from Python list.pop().

The idea that we dress Java containers as their Python near-equivalents is attractive because we will often be able to pass an object returned from Java directly to Python code expecting (say) a list. Unfortunately, it creates these surprises. If the receiving Python code expects pop() to work as advertised, it will get the wrong result. We have perhaps wrapped remove() in order that Python code doesn't malfunction that calls remove() expecting this behaviour of list.remove.

I think that an object claiming to have Java type should work as that type, and where we want it to behave as a Python list, it should be wrapped to do so, explicit being better than implicit here.

It's an interesting problem.

In general, we have had this implicit magic of semantic equivalence of Python and Java for a while, going back to the beginning of the project. It usually works well, until we further fix things by making the semantic equivalence ore regular, as we see in this bug report.

(See https://hg.python.org/jython/rev/ae2a1efe4192 for when this regularization fix was introduced.)

However, Jython also implements a number of hooks to control such magic more precisely. So for example, we have the ability to register the java2py adapter (use a custom one based on ExtensiblePyObjectAdapter per sys instance instead of ClassicPyObjectAdapter, or use its pre/post adapter scheme), manage ClassDict initialization, and in jythonlib, the ability to directly use a constructed Java ConcurrentMap for initializing a PyDictionary.

I can see the possibility of using the object adapter support to restore older usage. But it might be complicated in practice - now we have this per PySystemState setup, etc.

One thought I had is to make *explicit* (sorry!!!!) when we don't want the implicit magic, similar to what was done with jythonlib (see its usage in Lib/weakref.py, Lib/threading.py, and a few other libraries, 
https://bugs.jython.org/issue1631).

So this is the current behavior, which we can continue to support:

from java.util import LinkedList

x = LinkedList([1, 2, 3, 4, 5])
assert x.pop() == 1 

However we could instead do the following with an alternative wrapper class:

from jythonlib import RawJavaObject

y = RawJavaObject(LinkedList([1, 2, 3, 4, 5]))
assert y.pop() == 5

The RawJavaObject support would expose an alternative implementation of the underlying Java type with minimal wrapping, and would instead pass through all calls to the underlying object. Also, much like some other jythonlib functionality, there could be a factory function to control exactly what is done. Example: what happens when comparing the types of x and y?

One last thought: my proposal is probably best for Jython 3, although I suppose it could be readily backported to 2.7. I don't think it's a lot of code to prototype (probably similar to what we have seen in jythonlib), but given the longterm cost of any new feature, it seems best to raise on jython-users to determine value and the actual semantics.

My preference (and I'm thinking of Jython 3 too) would be that if an object claims as the name of its Python type, the name of a Java type, then it should behave exactly as the Java documentation describes.

It is certainly useful to be able to pass a list returned from a Java method into a Python function and have it treated as a list, but then inside the Python function I think it should behave exactly as a Python list. Otherwise, the result may not be according to the contract of the Python code.

This involves some kind of wrapper that proxies the Java List, and ideally, claims the type 'list'. I'm not sure this is possible in 2.7, but I have an idea to allow multiple implementations of a single Python type. The wrapper could go on:

1. On return from the Java function. (Not keen as we lose Java added value )
2. Implicitly as it becomes a call argument. (Precludes use as Java type in function.)
3. Explicitly with some kind of object.as_list() method. (Other syntax available.)

Tentatively tagging for 2.7.2, meaning let's find and clarify the least surprising behaviour. Overall, I think we may be trying to reconcile the irreconcilable (that such objects should behave exactly like both types).

It's clear from the change set Jim links (https://hg.python.org/jython/rev/ae2a1efe4192) that the behaviour of remove is a feature not a bug.

No change for 2.7.2, then.

Conversions at the boundary between Java and Python are indeed an interesting problem. The simplicity of the early examples is seductive but doesn't continue.