Issue2670

I noticed this in the investigation of #2662 and I'm spinning it off as a separate issue as I'm not really sure how to fix it. The obvious choice results in new failure in test_dict2java. So if that is the correct change, then the test is  wrong. I think it will be easier to see what to do after #2662 is fixed, which is more pressing.

I'm parking this so I don't have to fix it right away, but here are my observations.

There's a bit of PyJavaType.init() that goes:

        /*
         * PyReflected* can't call or access anything from non-public classes that aren't in
         * org.python.core
         */
        if (!Modifier.isPublic(forClass.getModifiers()) && !name.startsWith("org.python.core")
                && Options.respectJavaAccessibility) {
            handleSuperMethodArgCollisions(forClass);
            return;
        }

The problem with this is that for classes in core with names beginning Py, name is not set to the Java name but to a Python-like equivalent. Thus org.python.core.PyMapEntrySet has the name mapentryset by this point. The startsWith test returns false in these cases, so the if-body is executed even though the class is accessible to PyReflected* classes. Note also the return statement, which prevents subsequent processing in PyJavaType.init(), in particular it prevents the processing that would exposes the class as a Python-like collection.

The fix for this, so it works as apparently intended is to replace:
    name.startsWith("org.python.core")
with:
    forClass.getName().startsWith("org.python.core")

In that case, non-public classes from org.python.core (PyMapEntrySet and its like) will be subject to the processing in init() that applies to public classes from that Java package. Unfortunately, this causes a test failure in test.test_dict2java.py:

PS jython-jvm9> dist\bin\jython -m test.regrtest -v test_dict2java
== 2.7.2a1+ (default:623eaa3d7834+, Apr 17 2018, 06:52:37)
== [Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)]
== platform: java1.7.0_60
== encodings: stdin=ms936, stdout=ms936, FS=utf-8
== locale: default=('en_GB', 'windows-1252'), actual=(None, None)
test_dict2java
test_basic_map_operations (test.test_dict2java.JythonMapInJavaTest) ... ok
test_entryset (test.test_dict2java.JythonMapInJavaTest) ... FAIL
test_keyset (test.test_dict2java.JythonMapInJavaTest) ... FAIL
test_values (test.test_dict2java.JythonMapInJavaTest) ... FAIL

======================================================================
FAIL: test_entryset (test.test_dict2java.JythonMapInJavaTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\Jeff\Documents\Eclipse-O\jython-jvm9\dist\Lib\test\test_dict2java.py", line 87, in test_entryset
    self.failUnless(set.remove(eentry))
AssertionError: None is not true

======================================================================
FAIL: test_keyset (test.test_dict2java.JythonMapInJavaTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\Jeff\Documents\Eclipse-O\jython-jvm9\dist\Lib\test\test_dict2java.py", line 124, in test_keyset
    self.failUnless(keyset.remove(None))
AssertionError: None is not true

======================================================================
FAIL: test_values (test.test_dict2java.JythonMapInJavaTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\Jeff\Documents\Eclipse-O\jython-jvm9\dist\Lib\test\test_dict2java.py", line 147, in test_values
    self.failUnless(values.remove(None))
AssertionError: None is not true

----------------------------------------------------------------------
Ran 4 tests in 0.047s

I believe this happens because because the classes that are return types from methods in javatests.Dict2JavaTest have a remove method that no longer returns a boolean result, because of the wrapping applied in PyJavaType.addCollectionProxies(Class<?>), making the Java API remove that returns boolean, look like the Python one, which returns None. See JavaProxySet.removeOverrideProxy

The dict of PyJavaType(PyMapEntrySet) is:
    {'__init__': <java constructor mapentryset 0x2>, 
     '__deepcopy__': <method '__deepcopy__' of 'mapentryset' objects>,
     'remove': <method 'remove' of 'mapentryset' objects>}

This is a much shorter list than results when the class is handled by the (possibly inappropriate) handleSuperMethodArgCollisions, and even the type of remove is different:

   {'getClass': <java function getClass 0x2>,
    'wait': <java function wait 0x3>, 
    'notifyAll': <java function notifyAll 0x4>, 
    'notify': <java function notify 0x5>, 
    'remove': <java function remove 0x6>, 
    'removeAll': <java function removeAll 0x7>, 
    'iterator': <java function iterator 0x8>, 
    'stream': <java function stream 0x9>, 
    'hashCode': <java function hashCode 0xa>, 
    'toArray': <java function toArray 0xb>, 
    'parallelStream': <java function parallelStream 0xc>, 
    'add': <java function add 0xd>, 
    'spliterator': <java function spliterator 0xe>, 
    'forEach': <java function forEach 0xf>, 
    'containsAll': <java function containsAll 0x10>, 
    'isEmpty': <java function isEmpty 0x11>, 
    'clear': <java function clear 0x12>, 
    'removeIf': <java function removeIf 0x13>, 
    'contains': <java function contains 0x14>, 
    'size': <java function size 0x15>, 
    'addAll': <java function addAll 0x16>, 
    'equals': <java function equals 0x17>, 
    'toString': <java function toString 0x18>, 
    'retainAll': <java function retainAll 0x19>}

https://hg.python.org/jython/rev/07959ff1d8e9 is the change at which test_dict2java would have failed, were it not for the name.startsWith("org.python.core") bug.

I suppose this also means that non-public implementations of collections do not get Python collection methods added (or Comparable __lt__ etc.).