Issue2455

[DESCRIPTION]

The following problem is a regression in Jython 2.7.0 with respect to Jython 2.5.2.

Download the attached jar file and unpack it -alternatively, add it to the classpath, or by means of sys.path.append("examples.jar")-.

Here are the contents:

herl55:~> jar tvf examples.jar
     0 Fri Jan 22 09:06:28 CET 2016 META-INF/
    68 Fri Jan 22 09:06:28 CET 2016 META-INF/MANIFEST.MF
     0 Fri Jan 22 08:55:26 CET 2016 example1/
    47 Fri Jan 22 08:55:20 CET 2016 example1/SomeClass.java
   201 Fri Jan 22 08:55:26 CET 2016 example1/SomeClass.class
     0 Fri Jan 22 08:59:52 CET 2016 example2/
  2014 Fri Jan 22 09:01:54 CET 2016 example2/__init__$py.class
     0 Thu Jan 21 18:47:10 CET 2016 example2/__init__.py
   203 Fri Jan 22 08:56:18 CET 2016 example2/OtherClass.class
    48 Fri Jan 22 08:56:10 CET 2016 example2/OtherClass.java

Basically, it has two classes, in different packages, one of which has also a __init__.py file.

With Jython 2.5.2, the following works:

herl55:~> jython2.5
Jython 2.5.2 (Release_2_5_2:7206, Mar 2 2011, 23:12:06) 
[Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)] on java1.8.0_60
Type "help", "copyright", "credits" or "license" for more information.
>>> import example1          
>>> import example2          
>>> print example1.SomeClass 
<type 'example1.SomeClass'>
>>> print example2.OtherClass
<type 'example2.OtherClass'>

The second print fails, however, with Jython 2.7.0:

herl55:~> jython2.7
Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11) 
[Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)] on java1.8.0_60
Type "help", "copyright", "credits" or "license" for more information.
>>> import example1
>>> import example2
>>> print example1.SomeClass
<type 'example1.SomeClass'>
>>> print example2.OtherClass
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'OtherClass'


[DIAGNOSIS]

While example1 is represented in Jython by a PyJavaPackage, example2 is loaded as a PyModule (I suppose because of the __init__.py file). This is the case in both Jython versions.

The last element of the path (OtherClass) is tried to be determined
as an attribute of example2, by calling __findattr_ex__ on it.

The method PyObject.__findattr_ex__ looks up the name in a local dictionary of attributes. "OtherClass" is not found there.

In Jython 2.5.2, PyModule overrides __findattr_ex__ to call impAttr if super.__findattr_ex__ returns null. This is when it finds OtherClass by importing it.

However, in Jython 2.7.0 this overriding version of __findattr_ex__ has been deleted from PyModule. Then PyObject.__findattr_ex__ is directly called, so OtherClass is not found.


[SOLUTION]

Recover the deleted method PyModule.__findattr_ex__, as it was in version 2.5.2:

    @Override
    public PyObject __findattr_ex__(String name) {
        PyObject attr=super.__findattr_ex__(name);
        if (attr!=null)
            return attr;
        return impAttr(name);
    }

Jaime, assuming no regrtest failures with this proposed change - can you write a test so we can include this fix in 2.7.1? We are planning to put out beta 3 momentarily, but this can and should be part of the RC from what I can tell.

The biggest issue is that we do not accept such test jars to be included in Jython, but test code that generates a jar is fine. You can see an example of a simple such setup in test_java_integration with test_proxy_serialization - it just uses the jar command (assumed available, fine for when testing) to build such a jar. Similarly code can be compiled as well.

As of today: we now have CI on PRs against https://github.com/jythontools/jython, so I recommend doing a PR accordingly. It will be the process we adopt for 2.7.2 FWIW.

OK, there is no need to create a jar file on the fly: plain directories and files would be enough.

Would just need to call javac.

See the compile_java_source function in https://github.com/jythontools/jython/blob/master/Lib/test/test_java_integration.py#L639

I recommend adding your new test to test_java_integration; it's a big file now, but I rather break it up systematically.

The attached file TestBug2455.java contains a unit test for this issue.

I've never committed a change in bugs.jython.org; it would be easier if anyone used to do it could upload it instead.

Let's get this into the RC

http://bugs.jython.org/issue1811 is where the line was removed. Need to see if returning this code does not make that issue reappear.

Still passes with the patch.

I've made some changes to the test to have it run on the "local" compiler and some other cleanups.

Thanks for the report Jaime, the test and the fix!

https://hg.python.org/jython/rev/ac01360b4252

Unfortunately this change leads to incorrect behaviour w.r.t. pure Python imports, see #2654.

The Jython Book describes mixed Java/Python packages as "disatrous":
http://www.jython.org/jythonbook/en/1.0/ModulesPackages.html#sys-path
so I question whether we should (or can correctly) support it.

Evidently we don't have strong enough regression tests for the behaviour intended in the design of Jython.