Issue1506749

when assigning to the __dict__ attribute of an object 
instance, the namespace dictionary (stringmap) is not
replaced, but a '__dict__' key is created, and the new
stringmap is assigned as a value to this key 
_in the existing namespace_. This happens both when using
direct instance.__dict__=xyz assignment and when using
the __setattr__() method of the class, of the 'object'
class, or of the instance. The following code
demonstrates the problem:

############### snip ###########################
# let's define a simple class
class C(object):
  pass

c=C()
c.attr1=42

print c.__dict__, type(c.__dict__)
# we know Jython uses stringmaps as namespace 
# dicts, let's create a new stringmap object
# (perhaps there is a simpler way)
smap_type=type(c.__dict__)
sm2=smap_type.__new__(smap_type)
sm2['attr2']=0

# our first attempt to assign new stringmap to c.__dict__
c.__dict__=sm2
print '#1:', c.__dict__

#this obviously does not work, let's try a different method
c.__setattr__('__dict__', sm2)
print '#2:', c.__dict__
############### snip ###########################

CPython2.4 output:
{'attr1': 42} <type 'dict'>
#1: {'attr2': 0}
#2: {'attr2': 0}


Jython2.2a1 output:
{'attr1': 42} <type 'stringmap'>
#1: {'__dict__': {'attr2': 0}, 'attr1': 42}
#2: {'__dict__': {'attr2': 0}, 'attr1': 42}


This bug hampers many advanced uses of Python's
meta-object protocol, such as the _threading_local.py
module from CPython2.4 which allows to have "global"
(syntactically, they indeed are) variables which 
in fact are stored on a per-thread basis. Which in 
turn is very useful when you have thread-specific
"background state" information, such as the Request
object in web programming. 

Logged In: YES 
user_id=18288


newType() in PyType sets up __dict__ descriptors with null
setters. Changing the setter to use a setDict method, and
adding setDict to the various *Derived's (I guess via the
code generation defined in src/templates) might make this
work. Guess i'll give a try, though if someone has a better
idea of how this should work i'd be happy to hear it.

- kz

Small patch for trunk using the previously described approach attached. Allows both PyStringMap and PyDictionary dicts to be assigned. After applying, need to regenerate the *Derived's from the templates. 

Not entirely confident its right, so holding back on commit.

- kz

Looks right to me if that gives you any confidence :)    

Well, I take that back.  It's almost right.  It gets most of the stuff in the setdict method in test_descr to pass.  It fails on 'del a.__dict__' where a is an instance of a subclass of object.  Maybe PyGetSetDescr.java for __dict__ on PyType needs to grow a __delete__ method like PySlot.java did?

Thanks for testing. Checking cpython, you can del an instance dict, but then go on assigning into it as the instance grows a fresh, empty dict. The implementation seems to accomplish this by setting the dict to null on del (explicit __set__ to null, not __delete__ as far as i can tell), and then allocating a new dict on the subsequent lookup.

For jython, the suggestion of adding __delete__ to the dict descriptor, and having it assign a fresh PyStringMap for the __dict__ seems to provide the equivalent behaviour (without the lazy dict allocation). I'm attaching a patch that adds the ability to specify a __delete__ method on PyGetSetDescr, and updates the previous patch to use that. Seems to work in a bit of testing. But now the templating needs to grow an optional extra parameter for expose_getset to be used in type.expose, as the patch includes a hand-edited PyType.java (and omits type.expose for now). 

Hope i'm getting warmer ... more clues welcome!

- kz

Looks like progress to me.  setdict in test_descr.py still fails because it expects __dict__ on a class to be unassignable.  That's the final line in the test though, so once that one passes test_descr will be happy.  

Hmm, works for me. Maybe i ommitted something in the patch i posted. Will review.

- kz

It looks like I need __delete__ on PyGetSetDescr.java for bug 1604252.  Have you made any progress on exposing it from the templates?

Yes, i have an update to gexpose.py and gexpose-defs that gets the code generator working with the extra arg. Maybe i can get to committing it tonight.

- kz

I've gone ahead and committed the templating changes for __delete__ followed by the updates for assignable instance dicts in r3009, r3010, & r3011 to trunk. Will close this ticket after a bit if no alarms come up. 

I've been sometimes craving a buildbot for jython as a sanity check against my own testing, especially if we could get coverage for mulitiple jvm versions and os platforms.

- kz

Looks good.  The commit got the last of the setdict failures.

I'd love it if we got a buildbot too.  

Closed.

- kz