Issue1618

classification
Title: Default globals dict should be a regular Python dict, not a stringmap
Type: behaviour Severity: normal
Components: Core Versions: Deferred
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: zyasoft Nosy List: akong, fwierzbicki, njoyce, thijs, zyasoft
Priority: normal Keywords: patch

Created on 2010-06-07.18:53:18 by thijs, last changed 2013-02-06.19:17:17 by fwierzbicki.

Files
File name Uploaded Description Edit Remove
issue1618-type-stringmap-should-return-dict.patch zyasoft, 2010-06-19.03:58:19
Messages
msg5798 (view) Author: Thijs Triemstra (thijs) Date: 2010-06-07.18:53:16
I'm running into the following error that doesn't show when using CPython:

Jython 2.5.1+ (trunk:7063, Jun 7 2010, 20:39:19) 
[Java HotSpot(TM) 64-Bit Server VM (Apple Inc.)] on java1.6.0_20
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyamf
>>> import optparse
>>> pyamf.register_package(globals(), package='flex.messaging.io')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pyamf/__init__.py", line 1790, in register_package
    classes = filter(check_attr, [get(x) for x in keys])
  File "pyamf/__init__.py", line 1755, in <lambda>
    get = lambda x: getattr(module, x)
AttributeError: 'stringmap' object has no attribute 'pyamf'
>>> globals()
{'pyamf': <module 'pyamf' from 'pyamf/__init__$py.class'>, '__doc__': None, 'optparse': <module 'optparse' from '/Users/thijstriemstra/Sites/projects/opensource/jython-trunk/dist/Lib/optparse$py.class'>, '__name__': '__main__'}


Python 2.6.5 (r265:79063, Mar 23 2010, 18:27:10) 
[GCC 4.2.1 (Apple Inc. build 5646) (dot 1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyamf
>>> import optparse
>>> pyamf.register_package(globals(), package='flex.messaging.io')
{}
>>> globals()
{'__builtins__': <module '__builtin__' (built-in)>, 'pyamf': <module 'pyamf' from 'pyamf/__init__.pyc'>, '__package__': None, 'optparse': <module 'optparse' from '/usr/local/lib/python2.6/optparse.pyc'>, '__name__': '__main__', '__doc__': None}
msg5818 (view) Author: Anthony Kong (akong) Date: 2010-06-16.21:00:31
In the pyamf code, they have this check:

1780        if type(module) is dict:
1781	        has = lambda x: x in module.keys()
1782	        get = module.__getitem__
1783	    elif type(module) is list:
1784	        has = lambda x: x in module
1785	        get = module.__getitem__
1786	        strict = False
1787	    else:
1788	        has = lambda x: hasattr(module, x)
1789	        get = lambda x: getattr(module, x)

Since type(globals()) is 'stringmap', it fails

You can try 

pyamf.register_package(dict(globals()), package='flex.messaging.io')

instead
msg5819 (view) Author: Thijs Triemstra (thijs) Date: 2010-06-17.07:55:50
Thanks for the reply akong. When looking at the jython documentation [1] it seems that globals() should always return a dictionary, and there's no need to wrap it in a dict(). Unless PyAMF somehow assigned a string to the globals() but that somehow seems impossible. So it this a Jython bug or should PyAMF revisit it's type check?

[1] http://jythonpodcast.hostjava.net/jythonbook/en/1.0/appendixC.html#globals
msg5820 (view) Author: Thijs Triemstra (thijs) Date: 2010-06-17.08:05:49
Found this ticket that describes the same problem and includes a workaround: http://psf.upfronthosting.co.za/roundup/jython/msg2933
msg5821 (view) Author: Anthony Kong (akong) Date: 2010-06-17.09:56:20
(in response to msg5819)

I can see your point, and I have expected it to be dict() too. All I can find about 'stringmap' is this old documentation here (http://www.jython.org/archive/21/docs/differences.html)

Let me quote the relevant passage here:

The dictionaries used by classes, instances, and modules in Jython are not the same as the dictionaries created by {}. They are StringMap's which require all of their keys to be strings. After "class c: pass", c.__dict__[1] = 2 will work in CPython, but will raise a "TypeError: keys in namespace must be strings" error in Jython. Both behaviors are acceptable -- CPython might adopt Jython's approach in the future for the performance gains it can provide.
msg5826 (view) Author: Jim Baker (zyasoft) Date: 2010-06-18.18:18:50
PyStringMap should duck type the same as PyDictionary for user Python code. The name remains from an unfortunate optimization from pre Jython 2.5, which these obsolete docs from 2.1 refer to.

When we switched to a java.util.ConcurrentHashMap as the backing store, we relaxed its semantics so PyObjects (if hashable) could be used as keys. However, we also retained the earlier interface that supports Java strings as keys.

CPython 2.5 specifies that globals must be a dict, however, it does not apparently enforce this requirement. Neither does Jython. In any event, it should be straightforward to change the default globals such that it uses PyDictionary instead, since it's perfectly reasonable for a user to pass their own dict - or in fact dict like object! - instead for the globals map.

I have changed the bug title accordingly.
msg5827 (view) Author: Jim Baker (zyasoft) Date: 2010-06-18.18:48:37
Some more study of the code suggests that this will not be so easy to fix.  Also 2.6 presents a more robust solution through the use of ABCs. Deferring to 2.6.
msg5828 (view) Author: Jim Baker (zyasoft) Date: 2010-06-19.03:58:18
Oops, didn't mean to close the bug too.

However, it looks like there's an alternate solution - we can pretend it's a dict. This means special casing PyType#type___new__ such that it returns dict instead of stringmap. I have attached a small patch that does this, and it passes our regrtest.

However, I'm pretty certain that PyStringMap does not support the complete dict interface, including some (undocumented) support for concurrency. So I need to add the appropriate methods and tests before committing this patch.
msg5893 (view) Author: Jim Baker (zyasoft) Date: 2010-07-14.16:22:53
FYI - my patch got applied as part of r7072, so it's in 2.5.2 beta 1. I still need to add the missing methods from dict, as well as tests, so I'm leaving it open. However, it's likely to now work for the OP, so please test.
msg5895 (view) Author: Thijs Triemstra (thijs) Date: 2010-07-14.17:10:37
Awesome, this seems to be fixed:

$ jython
Jython 2.5.2b1 (trunk:7078, Jul 14 2010, 18:47:14) 
[Java HotSpot(TM) 64-Bit Server VM (Apple Inc.)] on java1.6.0_20
Type "help", "copyright", "credits" or "license" for more information.
>>> impor pyamf
  File "<stdin>", line 1
    impor pyamf
         ^
SyntaxError: mismatched input 'pyamf' expecting NEWLINE
>>> import pyamf
>>> import optparse
>>> pyamf.register_package(globals(), package='flex.messaging.io')
{}
>>> globals()
{'pyamf': <module 'pyamf' from 'pyamf/__init__$py.class'>, '__doc__': None, 'optparse': <module 'optparse' from '/Volumes/Users/thijstriemstra/Sites/projects/opensource/jython-trunk/dist/Lib/optparse$py.class'>, '__name__': '__main__'}
>>>
msg7374 (view) Author: Frank Wierzbicki (fwierzbicki) Date: 2012-08-10.20:44:54
Jim: should this still be open?
msg7630 (view) Author: Frank Wierzbicki (fwierzbicki) Date: 2013-02-06.19:17:17
Since stringmap behaves better now, we can close this.
History
Date User Action Args
2013-02-06 19:17:17fwierzbickisetstatus: open -> closed
resolution: accepted -> fixed
messages: + msg7630
2012-08-10 20:44:54fwierzbickisetnosy: + fwierzbicki
messages: + msg7374
2010-09-21 04:32:25zyasoftsetpriority: high -> normal
2010-08-22 22:42:46zyasoftsetpriority: high
2010-07-14 17:10:37thijssetmessages: + msg5895
2010-07-14 16:22:54zyasoftsetmessages: + msg5893
2010-06-19 03:58:21zyasoftsetstatus: closed -> open
files: + issue1618-type-stringmap-should-return-dict.patch
resolution: postponed -> accepted
messages: + msg5828
keywords: + patch
2010-06-18 18:48:38zyasoftsetstatus: open -> closed
resolution: postponed
messages: + msg5827
components: + Core, - Any
versions: + Deferred, - 2.5.1
2010-06-18 18:18:52zyasoftsetassignee: zyasoft
messages: + msg5826
nosy: + zyasoft
title: Issue with globals() and 'stringmap' object -> Default globals dict should be a regular Python dict, not a stringmap
2010-06-17 09:56:21akongsetmessages: + msg5821
2010-06-17 08:05:50thijssetmessages: + msg5820
2010-06-17 07:55:51thijssetmessages: + msg5819
2010-06-16 22:20:38thijssetnosy: + njoyce
2010-06-16 21:00:32akongsetnosy: + akong
messages: + msg5818
2010-06-07 18:53:18thijscreate