Message11280

So having spent a bit more time on this, I think Stefan is right with the deadlocks being caused by skipped finally blocks. IntelliJ's debugger seems to have no issues pausing on StackOverflowError and watching the execution. So if you want to see what happens I recommend trying IntelliJ.

So what happens is, at some point a StackOverflowError is thrown, some finally blocks are executed but some are skipped when another StackOverflowError is thrown during the finally, anyway eventually this is caught in Jython code, line 168 in PyTableCode which calls Py.JavaError(t), which then explicitly handles the StackOverflowError returning a Py.RuntimeError("maximum recursion depth exceeded (Java StackOverflowError)"). The issue occurs because this is exactly what these recursion test are expecting to happen. So they see the RuntimeError and that means the test passes and normal execution resumes (e.g line 108 in /Lib/json/tests/test_recursion.py - This is a good test to run always reproduces this issue) So the good thing is this works as expected I guess, except the part where finally blocks were skipped. So if there is a way to handle this in Java (i.e somehow ensure all the locks are unlocked) then we could do that by hooking the Py.JavaError(Throwable t) method which handles OutOfMemoryError and StackOverflowError, explicitly.

The thing I don't understand is how increasing Xss has any effect on this (although I observe it does), even with Xss5m the StackOverflowError still happens, as this is a infinite recursion test. So you would think that the unlocking issue would still happen but it doesn't? I can only assume the stack is so deep, by the time you make it back to a lock the finally will be executed successfully.

Maybe this is useful, if nothing else I enjoyed seeing how this happens.