Issue1883

classification
Title: infinte recusrion causes java.lang.NoClassDefFoundError: Could not initialize class org.python.core.PyTraceback
Type: crash Severity: major
Components: Core Versions: Jython 2.5
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: fwierzbicki Nosy List: amak, darjus, fwierzbicki, hnaderi
Priority: high Keywords:

Created on 2012-05-11.03:44:20 by hnaderi, last changed 2013-02-25.18:15:12 by fwierzbicki.

Files
File name Uploaded Description Edit Remove
test-scheduler.zip hnaderi, 2012-05-20.22:51:42 A samle test program which reproduces the issue
Messages
msg7084 (view) Author: Habib Naderi (hnaderi) Date: 2012-05-11.04:13:30
Hi,

I hope it's a right place for putting description.
when I run a python script which contains infinite or deep recursion, sometimes I get "maximum recusrion depth exceeded", sometimes I get the following exception. The problem is that after this exception happens, all run-time errors generate the same exception until I restart the application. Another thing is it happens when I run pythonInterpreter in a java thread. If I call it in the main, it works fine. reducing recersion depth with sys.setrecursionlimit does not resolve the issue.

Exception in thread "JythonThread[0]" java.lang.NoClassDefFoundError: Could not initialize class org.python.core.PyTraceback
	at org.python.core.PyException.tracebackHere(PyException.java:158)
	at org.python.core.PyException.tracebackHere(PyException.java:145)
	at org.python.core.PyTableCode.call(PyTableCode.java:169)
	at org.python.core.PyCode.call(PyCode.java:18)
	at org.python.core.Py.runCode(Py.java:1275)
	at org.python.core.Py.exec(Py.java:1319)
	at org.python.util.PythonInterpreter.exec(PythonInterpreter.java:215)
	at Demo.JythonThread1.runScript(jython_test.java:104)
	at Demo.JythonThread1.run(jython_test.java:129)
msg7109 (view) Author: Darjus Loktevic (darjus) Date: 2012-05-20.18:43:25
Hey Habib,

Can you submit a test that reproduces the issue?

Thanks!
Darjus
msg7119 (view) Author: Habib Naderi (hnaderi) Date: 2012-05-20.22:51:42
Hi,

This is a sample program which reprodeuces the issue. It creates a thread and run a python scripts inside the thread. This script,pfile25.py, is an infinite recursion. I always get that exception with this program. You can increase the number of threads but only one thread is enough to reproduce the issue.
msg7123 (view) Author: Frank Wierzbicki (fwierzbicki) Date: 2012-05-21.18:23:18
Hi Darjus, I'm responding here to your question for IRC:

sys.setrecursionlimit does not seem to have effect because what it seems is happening is that we get Java's StackOverflow (the code to test that is def recurse(): recurse() )
Should we be checking the recursion limit with systemstate.getrecursionlimit() somewhere and we don't? Or that's expected?

You are right that we don't want to see Java's stack overflow when we are dealing in pure Python code and so you are probably right that we need to better check our recursion limit.
msg7125 (view) Author: Darjus Loktevic (darjus) Date: 2012-05-21.23:59:47
Hey Frank,

I did a little experiment in PyFunction. Wrapping the __call__ with recursion_depth check makes it obey the recursion limits, and it fixes Habib's problem (i guess because there's enough stack for classloader not to go mad). Now, this is a bit more code that every function call will use and will probably impact performance a bit (IMO, correctness first but I'll need to come up with a good benchmark to validate the performance difference). My biggest worry though is that stuff that worked before might not work. Without this code my system is able to recurse ~1600 times, but with the new check, it would do only 1000 times. Maybe we add it to 2.7 only and not 2.5...

One more thing. If someone sets a smaller java stack size, we would still hit this obscurish problem. Maybe we should change the exception message a bit so that it's at least a little more descriptive?

Let me know what you think?

    @Override
    public PyObject __call__(ThreadState state, PyObject arg0, PyObject arg1) {
        if (state.recursion_depth++ > state.systemState.getrecursionlimit())
            throw Py.RuntimeError("maximum recursion depth exceeded");
        try {
            return __code__.call(state, arg0, arg1, __globals__, func_defaults, func_closure);
        } finally {
            --state.recursion_depth;
        }
    }
msg7126 (view) Author: Darjus Loktevic (darjus) Date: 2012-05-22.04:22:39
Hey Frank,

I did some simple tests with recursionlimit checks on PyFunction vs without vs CPython. There does not seem to be any noticeable difference. Let me know if you want me to prepare a patch (have not written any tests for it yet).

Darjus
msg7128 (view) Author: Frank Wierzbicki (fwierzbicki) Date: 2012-05-22.14:40:42
Darjus: I agree with correctness first here. We really don't want to see java specific exceptions in pure Python code.
msg7476 (view) Author: Darjus Loktevic (darjus) Date: 2012-10-16.05:52:45
Hey Frank,

Here's a pull request that should hopefully fix this one:
https://bitbucket.org/python_mirrors/jython/pull-request/7/respect-recursion-limit-in-functions-1883/

Thanks!
Darjus
History
Date User Action Args
2013-02-25 18:15:12fwierzbickisetpriority: high
assignee: fwierzbicki
versions: + Jython 2.5, - 2.5.2
2013-02-25 18:04:01amaksetnosy: + amak
2012-10-16 05:52:46darjussetmessages: + msg7476
2012-05-22 14:40:43fwierzbickisetmessages: + msg7128
2012-05-22 04:22:39darjussetmessages: + msg7126
2012-05-21 23:59:48darjussetmessages: + msg7125
2012-05-21 18:23:18fwierzbickisetmessages: + msg7123
2012-05-21 18:17:32fwierzbickisetnosy: + fwierzbicki
2012-05-20 22:51:42hnaderisetfiles: + test-scheduler.zip
messages: + msg7119
2012-05-20 18:43:25darjussetnosy: + darjus
messages: + msg7109
2012-05-11 04:13:30hnaderisetmessages: + msg7084
2012-05-11 03:48:58hnaderisetseverity: normal -> major
2012-05-11 03:44:20hnadericreate