Issue2323

classification
Title: Calling object.__reduce__ in a subclass results in infinite recursion
Type: Severity: normal
Components: Versions: Jython 2.7
Milestone: Jython 2.7.0
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: zyasoft Nosy List: jmadden, zyasoft
Priority: urgent Keywords: patch

Created on 2015-04-13.18:20:08 by jmadden, last changed 2015-04-27.01:53:49 by zyasoft.

Files
File name Uploaded Description Edit Remove
factor-out-common-reduce-to-prevent-infinite-recursion.patch jmadden, 2015-04-16.13:25:19
factor-out-common-reduce-to-prevent-infinite-recursion2.patch jmadden, 2015-04-16.13:52:52
Messages
msg9809 (view) Author: Jason Madden (jmadden) Date: 2015-04-13.18:23:09
This code recurses infinitely on Jython2.7RC2:

>>> class X(object):
...     def __reduce__(self): return object.__reduce__(self)
...
>>> X().__reduce__()
Traceback (most recent call last):
 ...
RuntimeError: maximum recursion depth exceeded (Java StackOverflowError)

But works fine on CPython 2.7 and PyPy 2.5:

>>> class X(object):
...    def __reduce__(self): return object.__reduce__(self)
...
>>> X().__reduce__()
(<function _reconstructor at 0x1028912a8>, (<class '__main__.X'>, <type 'object'>, None))

Interestingly, using `super(X,self).__reduce__()` works correctly.

I suspect this may be a known issue because the tests for it were disabled in https://hg.python.org/jython/rev/c82067c0766f.
msg9820 (view) Author: Jim Baker (zyasoft) Date: 2015-04-13.21:57:48
Those tests were disabled per this megabug, http://bugs.jython.org/issue1861, just to get Jython running, but then not re-addressed.

Any packages this is specifically impacting, so we can determine prioritization?

I suspect this is an unstopped super call, but will investigate when we have a chance.
msg9821 (view) Author: Jason Madden (jmadden) Date: 2015-04-13.22:02:26
I encountered this while making the `zodbpickle` package (used, obviously, by ZODB) work on Jython. It has a copy of the CPython pickle test cases so that exact failing test was still there to fail; I changed to the `super` call there as a temporary workaround. So far I haven't encountered it anywhere else.
msg9871 (view) Author: Jason Madden (jmadden) Date: 2015-04-16.13:25:19
The attached patch solves this issue by taking the same approach that CPython does (in typeobject.c) and not having object.__reduce__ call object.__reduce_ex__; instead they both call a common helper method. It re-enables the failing test in test_cpickle.py which now passes.
msg9872 (view) Author: Jason Madden (jmadden) Date: 2015-04-16.13:52:52
The second version of the patch plucks a couple more low-hanging fruit by mapping the exceptions from load/loads and enabling their tests.
msg9886 (view) Author: Jim Baker (zyasoft) Date: 2015-04-19.15:16:43
Fixed as of https://hg.python.org/jython/rev/74811340dfe3
History
Date User Action Args
2015-04-27 01:53:49zyasoftsetstatus: pending -> closed
2015-04-19 15:16:58zyasoftsetpriority: urgent
milestone: Jython 2.7.0
2015-04-19 15:16:43zyasoftsetstatus: open -> pending
assignee: zyasoft
resolution: fixed
messages: + msg9886
2015-04-16 13:52:53jmaddensetfiles: + factor-out-common-reduce-to-prevent-infinite-recursion2.patch
messages: + msg9872
2015-04-16 13:25:20jmaddensetfiles: + factor-out-common-reduce-to-prevent-infinite-recursion.patch
keywords: + patch
messages: + msg9871
2015-04-13 22:02:26jmaddensetmessages: + msg9821
2015-04-13 21:57:49zyasoftsetnosy: + zyasoft
messages: + msg9820
2015-04-13 18:23:09jmaddensetmessages: + msg9809
2015-04-13 18:20:08jmaddencreate