Title: array type prevents __radd__ fallback
Type: Severity: major
Components: Versions: 2.5.1
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: pjenvey Nosy List: akong, babelmania, doublep, pjenvey
Priority: Keywords:

Created on 2010-06-22.12:59:57 by babelmania, last changed 2010-07-28.07:18:49 by babelmania.

msg5830 (view) Author: Jorgo Bakker (babelmania) Date: 2010-06-22.12:59:55
Given a java class that can work with PyArrays within the following methods:
class MyJava {
    int[] getArray();
    PyObject __add__(PyObject);
    PyObject __radd__(PyObject);

Then the commutative property is lost:

   b+a # ERROR

In Jython-2.1 this worked, but in Jython-2.5 an error is raised because only PyArray types can be considered in the hard-wired implementation of PyArray?

I tried to add a new PyBuiltinMethod for PyArray, but that road seems also to be blocked becuse it is considered a builtin type.

Possible workaround (other than hacking the Jython code) would also be appreciated.
msg5845 (view) Author: Anthony Kong (akong) Date: 2010-06-27.21:23:57
What is the exact exception/stacktrace?

Better still, please provide the implementation of your class MyJava.
msg5846 (view) Author: Jorgo Bakker (babelmania) Date: 2010-06-28.06:51:29
 * On request: BOGUS class to reflect the issue
 * @author jbakker
public class MyJava21 {

    public MyJava21() {
    public PyObject __add__(PyObject rhs) {
        System.out.println("this + "+rhs);
        return Py.java2py(this);

    public PyObject __radd__(PyObject lhs) {
        System.out.println(lhs+" + this");
        return Py.java2py(this);
msg5847 (view) Author: Jorgo Bakker (babelmania) Date: 2010-06-28.06:51:52
Jython 2.5.1 (Release_2_5_1:6813, Sep 26 2009, 13:47:54) 
[Java HotSpot(TM) 64-Bit Server VM (Apple Inc.)] on java1.6.0_20
Type "help", "copyright", "credits" or "license" for more information.
>>> from jarray import array
>>> from issue1622 import MyJava21
>>> x=array([1,1],'i')
>>> y=MyJava21()
>>> x+y
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: can only append another array to an array
>>> y+x
this + array('i', [1, 1])
msg5848 (view) Author: Jorgo Bakker (babelmania) Date: 2010-06-28.06:52:35
Jython 2.1 on java1.6.0_20 (JIT: null)
Type "copyright", "credits" or "license" for more information.
>>> from jarray import array
from issue1622 import MyJava21
>>> >>> >>> >>> org.python.core.PyArray@b24044e + this
>>> this + org.python.core.PyArray@b24044e
msg5849 (view) Author: Jorgo Bakker (babelmania) Date: 2010-06-28.06:53:46
Above a bogus java class and its behavior under Jython 2.1 and Jython 2.5.1

As you can see, commutative property is lost
msg5851 (view) Author: Anthony Kong (akong) Date: 2010-06-28.21:06:21
I am not sure if "commutative property is lost" is a right description here.

The exception is thrown because org.jython.core.PyArray has performed some check in __add__(). One of them is 

    final PyObject array___add__(PyObject other) {
        PyArray otherArr = null;
        if(!(other instanceof PyArray)) {
            throw Py.TypeError("can only append another array to an array");

So from PyArray point of view, it cannot operate on your class MyJava21

I think it is a right behavior. 

In C python, you cannot just add anything to a list. For example, "1 + [2, 3]" will throw an example.

The appearance of __add__ is commutative between your class and list  probably arises because 

1) no such check is enforced in jython 2.1, and
2) your code did not check if the operand can be added to MyJava21
msg5854 (view) Author: Jorgo Bakker (babelmania) Date: 2010-06-29.07:27:16
Allow me to point out that overriding of __radd__ within my class has the desired behavior on other classes such as PyList

In the example below 'MyJava' is actually an integer array of rank 1 (called Int1d).
It allows, like in any scripting language to do operations like:
   x+1 == 1+x == x+y == y+x == x+z == z+x
where e.g. x=Int1d([1,2]), y=array([1,1],'i'), z=[1,1]

Suddenly blocking this mechanism on only a fraction (PyArray) of your classes alone does not make sense, and I consider that a bug.
msg5861 (view) Author: (doublep) Date: 2010-06-29.10:17:10
I agree that it's a bug.  The whole point of __r*__ methods is to add support for operations in reversed situation where first class doesn't know about the second.  If works perfectly fine in CPython:

class Foo (object):
    def __add__(self, that):
        print self, that
    def __radd__(self, that):
        print self, that

[] + Foo ()
Foo () + []

This never throws TypeError.

The correct behavior for PyArray is to _return_ NotImplemented, not raise a TypeError.  E.g. consider the following example:

class Foo (object):
    def __add__(self, that):
        return NotImplemented
    def __radd__(self, that):
        return NotImplemented

class Bar (object):
    def __add__(self, that):
        print self, that
    def __radd__(self, that):
        print self, that

Foo () + Bar ()
Bar () + Foo ()
Foo () + 1

Class Foo is declared to never allow any addition whatsoever.  However, Bar can override this with __radd__ method and it works fine.  But the last statement fails, because neither Foo.__add__(int) nor int.__radd__(Foo) produces a result:

Traceback (most recent call last):
  File "/home/developer/test/", line 15, in <module>
    Foo () + 1
TypeError: unsupported operand type(s) for +: 'Foo' and 'int'
msg5865 (view) Author: Anthony Kong (akong) Date: 2010-07-01.01:20:59
Thanks babelmania and doublep for the explanation of __r*__.
msg5928 (view) Author: Philip Jenvey (pjenvey) Date: 2010-07-27.23:46:27
Yes, PyArray array___add__ and array__iadd__ were throwing TypeErrors preventing the __radd__ from being tried. Instead they need to return NotImplemented (actually null from Java methods exposed as MethodType.BINARY)

fixed in 7082
msg5930 (view) Author: Jorgo Bakker (babelmania) Date: 2010-07-28.07:18:49
Thank you!
Date User Action Args
2010-07-28 07:18:49babelmaniasetmessages: + msg5930
2010-07-27 23:46:29pjenveysetstatus: open -> closed
resolution: fixed
messages: + msg5928
title: Operators lost their commutative property when dealing with builtin types??? -> array type prevents __radd__ fallback
2010-07-27 23:07:11pjenveysetassignee: pjenvey
nosy: + pjenvey
2010-07-01 01:20:59akongsetmessages: + msg5865
2010-06-29 10:17:11doublepsetnosy: + doublep
messages: + msg5861
2010-06-29 07:27:17babelmaniasetmessages: + msg5854
2010-06-28 21:06:22akongsetmessages: + msg5851
2010-06-28 06:53:46babelmaniasetmessages: + msg5849
2010-06-28 06:52:35babelmaniasetmessages: + msg5848
2010-06-28 06:51:52babelmaniasetmessages: + msg5847
2010-06-28 06:51:30babelmaniasetmessages: + msg5846
2010-06-27 21:23:57akongsetnosy: + akong
messages: + msg5845
2010-06-22 12:59:57babelmaniacreate