Title: List set slice fix, and CollectionProxy behavior
Type: behaviour Severity: normal
Components: Core Versions: 2.5alpha3
Status: closed Resolution: accepted
Dependencies: Superseder:
Assigned To: fwierzbicki Nosy List: MrMeanie, cgroves, fwierzbicki, zyasoft
Priority: Keywords:

Created on 2008-10-11.13:02:11 by MrMeanie, last changed 2008-12-02.10:58:57 by cgroves.

File name Uploaded Description Edit Remove
list_fixes_Geoffrey_French_20081011.patch MrMeanie, 2008-10-11.13:02:11 Patch to latest SVN to fix list behavior
list_fixes_20081011_v2.patch MrMeanie, 2008-10-11.21:51:19
list_fixes_issue1148_20081122_v3.patch MrMeanie, 2008-11-22.13:12:23 Patch to latest SVN
list_fix__and_proxy_slices_20081123.patch MrMeanie, 2008-11-23.18:51:05 Patch to newstyle-java-types branch
msg3657 (view) Author: Geoffrey French (MrMeanie) Date: 2008-10-11.13:02:10
List behavior

The behavior of lists in Jython differes from CPython when setting
slices in some cases.


>>> x = range(0,10)
>>> x[-9:-12:-1] = range(0,2)
>>> x
[1, 0, 2, 3, 4, 5, 6, 7, 8, 9]

Jython 2.5a3 and latests SVN:
>>> x = range(0,10)
>>> x[-9:-12:-1] = range(0,2)
>>> x
[0, 1, 2, 3, 4, 5, 6, 7, 1, 0]

This is fixed in the attached patch, which modifies org.python.core.PyList.

CollectionProxy behavior

The ListProxy class maps python subscript operators to underlying Java
List. However, it only supports positive integer indices.

The attached patch also contains changes to ListProxy which allow
negative integer indices, and the use of slices.

This is tested by the '' file which is included.
msg3667 (view) Author: Geoffrey French (MrMeanie) Date: 2008-10-11.21:51:18
New patch (supercedes old one)

Apologies for this second patch.
The CollectionProxy modifications needed an update; no py->java
conversion was performed on objects coming from Python. This is fixed now.

Still to fix:

1) CollectionProxy.findCollection() checks the object to see if it is a
List instance, before checking for a Vector instance. This means that
Vectors will be given the default ListProxy, not a VectorProxy.

2) The modifications that I have made to ListProxy have not been made to
VectorProxy. In order that Vector objects can use negative indices and
slices too, the changes could either be copied to VectorProxy, or
VectorProxy could subclass ListProxy.

3) When wrapping a java Collection instance, PyInstance directs calls to
py-list methods (__getitem__ etc) to the CollectionProxy system. This is
fine, unless you:
   a) Create a class in java which implements List and provides a
__getitem__ method (for example)
     - or -
   b) Create a class in python which does the same.
In both cases, the method call is directed to the CollectionProxy,
instead of using the objects own implementation.
PyInstance could check the objects capabilities first, then try the
CollectionProxy, rather than the other way round.
msg3802 (view) Author: Geoffrey French (MrMeanie) Date: 2008-11-22.13:12:22
New Patch (supercedes previous 2)

This patch fixes the remaining issues.

1) CollectionProxy.findCollection() will use VectorProxy if to wrap
java.util.Vector objects.
2) VectorProxy now derives from ListProxy.
3) PyInstance checks methods from underlying Java/Python object before
trying the collection proxy. This means that a Java or Python object
that implements __len__, __iter__, __getitem__, __setitem__, or
__delitem__, and derives from a Java collection class, will be able to
control the behavior of these methods. Previously, the collection proxy
intervened and translated these calls to the underlying Java collection
methods (get/set/put/etc). This also affected Python objects that
derived from java.util.List (for example) in the same way.

#3 Was accomplished by:
a) A new set of invoke methods were implemented for collection proxy
work; iinvoke_collectionProxy(). These return a special error value
rather than throwing the AttributeError, to indicate the no method could
be found. This allows the __finditem__/__setitem__/etc to try the
collection proxy should it be available, before raising this exception.
b) The __finditem__/__setitem__/__delitem__ were modified to try the
invoke path before trying the collection proxy.

'' has been updated to test this functionality.
msg3809 (view) Author: Charlie Groves (cgroves) Date: 2008-11-22.23:53:13
Unfortunately CollectionProxy is going to go away in the
newstyle-java-type branch.  Rather than creating a CollectionProxy for
each wrapped Java object as necessary, PyJavaType fills in __len__,
__get__ and so on in its type dict with individual proxy methods as
appropriate.  All of the Proxy classes towards the end of PyJavaType are
bridges for various collections methods.  It's not complete yet, but it
gets the bulk of what CollectionProxy currently does.

That handles a couple of the issues you solved here: not picking up the
correct exposing method for List vs Vector and not using a Python
implementation of __get__ or so on if its first in the lookup order
before the Java version.  Since the individual proxy methods are just
part of the type, they go through the normal mro lookup.

However, all of the nice new features you've added aren't in the new
proxy methods.  I definitely want to pick all of those up, so we have a
couple ways to do that:

1. You could check out the branch and modify the methods in PyJavaType
to have your collections modification.  That would be a nice sanity
check of my copying of the original CollectionProxy.  I will warn you
that the branch isn't close to completion, so it could blow up in all
sorts of interesting ways as we depend heavily on Java integration.  I
think it's stable enough now that you could work on it though.

2.  If you don't feel like adapting this or walking through the
minefield of a halfway completed branch, I'll go through your patches
and update the new proxy methods.

Feel free to respond here or to jump on #jython on for
more discussion.  I'm groves in there.
msg3836 (view) Author: Frank Wierzbicki (fwierzbicki) Date: 2008-11-23.14:45:39
By the way, as the local legal pedant -- would you mind signing a PSF
Contributor agreement?
since this is a big contribution -- that helps us keep the legal crap
msg3838 (view) Author: Geoffrey French (MrMeanie) Date: 2008-11-23.15:43:23

I have made a start in porting my changes to PyJavaType in your branch.
msg3839 (view) Author: Geoffrey French (MrMeanie) Date: 2008-11-23.18:51:05
Here is a patch that:

1) Fixes the PyList slice issue
2) Adds slice support to the java collection proxy (PyJaveType now)

This patch is for the newstyle-java-types branch, instead of trunk.

The same test as in previous patches is included.
msg3864 (view) Author: Charlie Groves (cgroves) Date: 2008-12-02.10:57:36
Committed in r5676 on the branch.

I ended up not using any of the Java part of the patch, and instead
rewrote it like we discussing using a delegate.  Index calculation is
shared between PySequence and PyJavaType.

I also changed getslice to return instances of the list being sliced
rather than PyList, as I imagine users of this want to use slices of the
list in place of the original.  You should probably check that I didn't
screw anything up with that.
Date User Action Args
2008-12-02 10:58:57cgrovessetstatus: open -> closed
resolution: accepted
2008-12-02 10:57:37cgrovessetmessages: + msg3864
2008-11-23 18:51:06MrMeaniesetfiles: + list_fix__and_proxy_slices_20081123.patch
messages: + msg3839
2008-11-23 15:43:23MrMeaniesetmessages: + msg3838
2008-11-23 14:45:39fwierzbickisetmessages: + msg3836
2008-11-22 23:53:14cgrovessetnosy: + cgroves
messages: + msg3809
2008-11-22 18:45:15fwierzbickisetassignee: fwierzbicki
2008-11-22 13:12:23MrMeaniesetfiles: + list_fixes_issue1148_20081122_v3.patch
messages: + msg3802
2008-10-20 16:50:48zyasoftsetnosy: + zyasoft
2008-10-11 21:51:19MrMeaniesetfiles: + list_fixes_20081011_v2.patch
messages: + msg3667
2008-10-11 13:14:03fwierzbickisetnosy: + fwierzbicki
2008-10-11 13:02:11MrMeaniecreate