Title: broken iterator propagation
Type: behaviour Severity: critical
Components: Core Versions: Jython 2.7
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: zyasoft Nosy List: fwierzbicki, zyasoft, zzzeek
Priority: high Keywords:

Created on 2012-11-19.17:47:25 by zzzeek, last changed 2014-06-25.16:05:44 by zyasoft.

msg7530 (view) Author: mike bayer (zzzeek) Date: 2012-11-19.17:47:25
This might be limited to the itertools library, as I was not able to reproduce this using freeform iterator objects...nonetheless this is pretty core.

Propagation of iterator objects, at least in the case of itertools.chain(), appears to be broken:

from itertools import chain

class MyClass(object):
    def __iter__(self):
        return chain([1, 2, 3])

my_obj = MyClass()

print list(my_obj)  # python prints [1,2,3], jython prints []

chain_obj = my_obj.__iter__()

assert list(chain_obj) == list(my_obj)

Above, the "chain()" object returned by my_obj.__iter__() works fine and yields [1,2,3], however when my_obj itself is called in an iterator context, the iterator yields nothing.  The assertion at the end fails on Jython and passes on cPython.
msg7531 (view) Author: Frank Wierzbicki (fwierzbicki) Date: 2012-11-19.17:53:02
zzzeek: thanks for the ongoing beat down on 2.7 - I'll keep an eye on these!
msg8555 (view) Author: Jim Baker (zyasoft) Date: 2014-05-22.02:22:40
Still broken in 2.7 trunk

Target beta 4
msg8567 (view) Author: Jim Baker (zyasoft) Date: 2014-05-22.19:02:43
It is interesting that iterator identity is broken, but since I gave a talk on this subject (iterator algebra) at a previous pycon, I will definitely get this fixed :)
msg8654 (view) Author: Jim Baker (zyasoft) Date: 2014-06-17.17:25:40
Apparently the problem is that itertools.chain (much like some of the other itertools classes that were defined for 2.7) does not extend PyIterator. We also want that to be the case not just for Mike's test case, but also for Java integration since PyIterator also implements the Iterable interface.

A patch like the following is almost complete for Mike's test case; we will need to do the same for most of itertools classes:

 @ExposedType(name = "itertools.chain", base = PyObject.class)
-public class chain extends PyObject {
+public class chain extends PyIterator {

     public static final PyType TYPE = PyType.fromClass(chain.class);
     private itertools.ItertoolsIterator iter;
@@ -84,14 +84,8 @@

-    @ExposedMethod
-    public PyObject __iter__() {
-        return iter;
-    }
-    @ExposedMethod
-    public PyObject next() {
-        return;
+    public PyObject __iternext__() {
+        return iter.__iternext__();


To make this complete, PyIterator should also expose a next method.
msg8658 (view) Author: Jim Baker (zyasoft) Date: 2014-06-17.22:43:46
Fixed in
Date User Action Args
2014-06-25 16:05:44zyasoftsetstatus: pending -> closed
2014-06-17 22:43:46zyasoftsetstatus: open -> pending
resolution: accepted -> fixed
messages: + msg8658
2014-06-17 17:25:40zyasoftsetmessages: + msg8654
2014-05-22 19:02:43zyasoftsetmessages: + msg8567
2014-05-22 18:24:20zyasoftsetassignee: fwierzbicki -> zyasoft
2014-05-22 02:22:40zyasoftsetresolution: accepted
messages: + msg8555
nosy: + zyasoft
2013-02-20 00:20:09fwierzbickisetpriority: high
assignee: fwierzbicki
versions: + Jython 2.7, - 2.7a2
2012-11-19 17:53:03fwierzbickisetnosy: + fwierzbicki
messages: + msg7531
2012-11-19 17:47:25zzzeekcreate