Title: Non-public Java classes implementing collection types
Type: behaviour Severity: normal
Components: Core Versions: Jython 2.7
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: jeff.allen, zyasoft
Priority: normal Keywords: test failure causes

Created on 2018-04-17.19:23:57 by jeff.allen, last changed 2018-04-18.06:09:56 by jeff.allen.

msg11912 (view) Author: Jeff Allen (jeff.allen) Date: 2018-04-17.19:23:55
I noticed this in the investigation of #2662 and I'm spinning it off as a separate issue as I'm not really sure how to fix it. The obvious choice results in new failure in test_dict2java. So if that is the correct change, then the test is  wrong. I think it will be easier to see what to do after #2662 is fixed, which is more pressing.

I'm parking this so I don't have to fix it right away, but here are my observations.

There's a bit of PyJavaType.init() that goes:

         * PyReflected* can't call or access anything from non-public classes that aren't in
         * org.python.core
        if (!Modifier.isPublic(forClass.getModifiers()) && !name.startsWith("org.python.core")
                && Options.respectJavaAccessibility) {

The problem with this is that for classes in core with names beginning Py, name is not set to the Java name but to a Python-like equivalent. Thus org.python.core.PyMapEntrySet has the name mapentryset by this point. The startsWith test returns false in these cases, so the if-body is executed even though the class is accessible to PyReflected* classes. Note also the return statement, which prevents subsequent processing in PyJavaType.init(), in particular it prevents the processing that would exposes the class as a Python-like collection.

The fix for this, so it works as apparently intended is to replace:

In that case, non-public classes from org.python.core (PyMapEntrySet and its like) will be subject to the processing in init() that applies to public classes from that Java package. Unfortunately, this causes a test failure in

PS jython-jvm9> dist\bin\jython -m test.regrtest -v test_dict2java
== 2.7.2a1+ (default:623eaa3d7834+, Apr 17 2018, 06:52:37)
== [Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)]
== platform: java1.7.0_60
== encodings: stdin=ms936, stdout=ms936, FS=utf-8
== locale: default=('en_GB', 'windows-1252'), actual=(None, None)
test_basic_map_operations (test.test_dict2java.JythonMapInJavaTest) ... ok
test_entryset (test.test_dict2java.JythonMapInJavaTest) ... FAIL
test_keyset (test.test_dict2java.JythonMapInJavaTest) ... FAIL
test_values (test.test_dict2java.JythonMapInJavaTest) ... FAIL

FAIL: test_entryset (test.test_dict2java.JythonMapInJavaTest)
Traceback (most recent call last):
  File "C:\Users\Jeff\Documents\Eclipse-O\jython-jvm9\dist\Lib\test\", line 87, in test_entryset
AssertionError: None is not true

FAIL: test_keyset (test.test_dict2java.JythonMapInJavaTest)
Traceback (most recent call last):
  File "C:\Users\Jeff\Documents\Eclipse-O\jython-jvm9\dist\Lib\test\", line 124, in test_keyset
AssertionError: None is not true

FAIL: test_values (test.test_dict2java.JythonMapInJavaTest)
Traceback (most recent call last):
  File "C:\Users\Jeff\Documents\Eclipse-O\jython-jvm9\dist\Lib\test\", line 147, in test_values
AssertionError: None is not true

Ran 4 tests in 0.047s

I believe this happens because because the classes that are return types from methods in javatests.Dict2JavaTest have a remove method that no longer returns a boolean result, because of the wrapping applied in PyJavaType.addCollectionProxies(Class<?>), making the Java API remove that returns boolean, look like the Python one, which returns None. See JavaProxySet.removeOverrideProxy

The dict of PyJavaType(PyMapEntrySet) is:
    {'__init__': <java constructor mapentryset 0x2>, 
     '__deepcopy__': <method '__deepcopy__' of 'mapentryset' objects>,
     'remove': <method 'remove' of 'mapentryset' objects>}

This is a much shorter list than results when the class is handled by the (possibly inappropriate) handleSuperMethodArgCollisions, and even the type of remove is different:

   {'getClass': <java function getClass 0x2>,
    'wait': <java function wait 0x3>, 
    'notifyAll': <java function notifyAll 0x4>, 
    'notify': <java function notify 0x5>, 
    'remove': <java function remove 0x6>, 
    'removeAll': <java function removeAll 0x7>, 
    'iterator': <java function iterator 0x8>, 
    'stream': <java function stream 0x9>, 
    'hashCode': <java function hashCode 0xa>, 
    'toArray': <java function toArray 0xb>, 
    'parallelStream': <java function parallelStream 0xc>, 
    'add': <java function add 0xd>, 
    'spliterator': <java function spliterator 0xe>, 
    'forEach': <java function forEach 0xf>, 
    'containsAll': <java function containsAll 0x10>, 
    'isEmpty': <java function isEmpty 0x11>, 
    'clear': <java function clear 0x12>, 
    'removeIf': <java function removeIf 0x13>, 
    'contains': <java function contains 0x14>, 
    'size': <java function size 0x15>, 
    'addAll': <java function addAll 0x16>, 
    'equals': <java function equals 0x17>, 
    'toString': <java function toString 0x18>, 
    'retainAll': <java function retainAll 0x19>}
msg11913 (view) Author: Jeff Allen (jeff.allen) Date: 2018-04-18.06:09:55 is the change at which test_dict2java would have failed, were it not for the name.startsWith("org.python.core") bug.

I suppose this also means that non-public implementations of collections do not get Python collection methods added (or Comparable __lt__ etc.).
Date User Action Args
2018-04-18 06:09:56jeff.allensetmessages: + msg11913
2018-04-17 19:23:57jeff.allencreate