Issue2642

I have an application which creates multiple PyScriptEngine instances on different threads. If multiple threads concurrently try to import I often get errors like:

Traceback (most recent call last):
  File ".\main.py", line 34, in <module>
    import threading
  File ".\lib\jython-standalone-2.7.1.jar\Lib\threading.py", line 88, in <module>
AttributeError: type object 'java.lang.Thread' has no attribute 'State'

and

Traceback (most recent call last):
  File ".\main.py", line 44, in <module>
    import gzip
  File ".\lib\jython-standalone-2.7.1.jar\Lib\gzip.py", line 10, in <module>
  File ".\lib\jython-standalone-2.7.1.jar\Lib\io.py", line 69, in <module>
  File ".\lib\jython-standalone-2.7.1.jar\Lib\_io.py", line 50, in <module>
ImportError: cannot import name DEFAULT_BUFFER_SIZE

Whilst I haven't been able to verify it, it appears to only be when importing Java classes.

By experimentation, if only a single thread is importing at a time the errors do not occur. As a workaround for my application I'm using a global import lock rather than an import lock per PySystemState, which seems to have resolved the issue. It's possible this is overkill and only part of the import process needs to be protected with a global lock.

Things not having attributes when you know they do, and concurrency is in play, is reminiscent of #2609. A good thing to check in 2.7.2 beta, when it appears (or build your own alpha).

Bot promising ofr 2.7.2 as I think concurrent interpreters is a topic needing careful thought (and testing). However, it is still possible #2609 has fixed it in this case.

Just dropping in to report that contrary to Jeff's hope, #2609 did not fix this.

I've checked with a 2.7.2a1+ built from hg master at the time of posting, and it is still necessary to prevent Python imports from coinciding on different JVM threads -- otherwise we observe "impossible" errors such as the Python code "from collections import deque" failing with an ImportError "cannot import name deque".

The underlying cause is, as you might guess, that the first thread to request an import starts initializing the new object, but the second thread receives a reference to the same object while the first thread is still only halfway through the classDictInit method, requests an attribute that has not been added yet, and dies.

So, a global lock it is, for now.  But we're not quite confident we're locking the right things - we've stopped seeing crashes, but maybe we've just made them much rarer? - and it would be much better if whatever locking is actually required were done by Jython itself.