Issue1561793

classification
Title: Jython 2.2 cannot pickle types again (with patch)
Type: Severity: normal
Components: Core Versions:
Milestone:
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: cgroves Nosy List: cgroves, rabidechidna
Priority: normal Keywords:

Created on 2006-09-19.22:50:43 by rabidechidna, last changed 2006-09-23.19:38:07 by cgroves.

Files
File name Uploaded Description Edit Remove
new_pickle_class_patch.diff rabidechidna, 2006-09-21.07:46:35 More extensive patch for cPickle
Messages
msg1234 (view) Author: D Morley (rabidechidna) Date: 2006-09-19.22:50:43
I notice that [1530257] "Jython 2.2 cannot pickle types" has been 
closed with the comment "This is fixed in trunk", however when I 
compile the SVN head (revision 2942) I still get errors:

Test file: picklebug.py
================
import cPickle
import traceback

class SimpleObj(object):
    pass

def ptest(x):
    try:
        print "PICKLING", x
        cPickle.dumps(x)
    except:
        traceback.print_exc()
        result = None
        
ptest(int)
ptest(SimpleObj())
ptest(SimpleObj)
================

When I run this  test, it seems that cPickle is mistakenly attempting to 
call the __reduce__ method on the class object:

~% ~/jython/jythonHead picklebug.py 
PICKLING <type 'int'>
Traceback (most recent call last):
  File "picklebug.py", line 10, in ptest
    cPickle.dumps(x)
TypeError: descriptor '__reduce__' of 'int' object needs an argument
PICKLING <__main__.SimpleObj object 1>
Traceback (most recent call last):
  File "picklebug.py", line 10, in ptest
    cPickle.dumps(x)
TypeError: descriptor '__reduce__' of 'object' object needs an argument
PICKLING <class '__main__.SimpleObj'>
Traceback (most recent call last):
  File "picklebug.py", line 10, in ptest
    cPickle.dumps(x)
TypeError: descriptor '__reduce__' of 'object' object needs an argument


The attached patch fixes the problem.

msg1235 (view) Author: Charlie Groves (cgroves) Date: 2006-09-20.13:42:55
Logged In: YES 
user_id=1174327

I closed 1530257 because it had more to do with pickling in
general being broken.  Your tests pass if you import pickle
instead of cPickle.  I figured leaving 1291509 open to
indicate the cPickle brokenness was enough.

That said, your patch fixes 15 out of the 37 broken tests in
cPickle.  I'll apply it whenever I get a chance to see
what's causing the remaining 15 to fail.  If you want to run
test_cPickle.py in Lib/test yourself and take out the
remaining 15 with an updated patch(or explain why they can't
be fixed), I'll apply it immediately(and be forever
grateful).  It'll probably be another week or two until I
get to look at cPickle.  Leave a comment here if you're
working on this, and I'll check in with you when I'm ready
to start.
msg1236 (view) Author: D Morley (rabidechidna) Date: 2006-09-21.07:46:35
Logged In: YES 
user_id=1034565

I have uploaded an improved patch which additionally fixes the test_maxint64 
test case and implements a (vacuous) setter for the undocumented "fast" 
attribute of cPickle.Pickler (when this attribute is set to True in the C version 
of cPickle, the memoization is turned off - which is okay, so long as you have 
no circular references). 

This leaves 4 failures and 14 errors.

The 4 failures are from cPickle *not* raising exceptions when dumping values 
with circular references when the fast attribute is set. This is to be expected, 
since setting the fast attribute in the patched version does nothing.

4 errors (test_metaclass) come from metaclasses not working yet - the tests 
fail creating the objects to pickle, not while pickling them.

The 10 remaining NullPointerException errors are from the 
cPickleListPicklerTests test suite. This test suite tries to pass the dump 
method's bin parameter in as the Pickler constructor's file parameter. I think 
the bug is in the test suite, not in Jython.
msg1237 (view) Author: Charlie Groves (cgroves) Date: 2006-09-21.14:25:39
Logged In: YES 
user_id=1174327

Sounds great!  

I should have some time this weekend to apply the patch.  If
everything looks kosher, I'll commit it and a jython
specific version of test_cPickle without the
cPickleListPicklerTests and the circularity tests for fast.
 The 4 metaclass errors appear in test_pickle as well so
that's something we'll actually have to dig in and fix.  
msg1238 (view) Author: Charlie Groves (cgroves) Date: 2006-09-23.19:38:07
Logged In: YES 
user_id=1174327

Patch applied in r2945.  Closing this since 1291509 still
stands as a reminder of cPickle's remaining brokenness.
History
Date User Action Args
2006-09-19 22:50:43rabidechidnacreate