Issue1883

Hi,

I hope it's a right place for putting description.
when I run a python script which contains infinite or deep recursion, sometimes I get "maximum recusrion depth exceeded", sometimes I get the following exception. The problem is that after this exception happens, all run-time errors generate the same exception until I restart the application. Another thing is it happens when I run pythonInterpreter in a java thread. If I call it in the main, it works fine. reducing recersion depth with sys.setrecursionlimit does not resolve the issue.

Exception in thread "JythonThread[0]" java.lang.NoClassDefFoundError: Could not initialize class org.python.core.PyTraceback
	at org.python.core.PyException.tracebackHere(PyException.java:158)
	at org.python.core.PyException.tracebackHere(PyException.java:145)
	at org.python.core.PyTableCode.call(PyTableCode.java:169)
	at org.python.core.PyCode.call(PyCode.java:18)
	at org.python.core.Py.runCode(Py.java:1275)
	at org.python.core.Py.exec(Py.java:1319)
	at org.python.util.PythonInterpreter.exec(PythonInterpreter.java:215)
	at Demo.JythonThread1.runScript(jython_test.java:104)
	at Demo.JythonThread1.run(jython_test.java:129)

Hey Habib,

Can you submit a test that reproduces the issue?

Thanks!
Darjus

Hi,

This is a sample program which reprodeuces the issue. It creates a thread and run a python scripts inside the thread. This script,pfile25.py, is an infinite recursion. I always get that exception with this program. You can increase the number of threads but only one thread is enough to reproduce the issue.

Hi Darjus, I'm responding here to your question for IRC:

sys.setrecursionlimit does not seem to have effect because what it seems is happening is that we get Java's StackOverflow (the code to test that is def recurse(): recurse() )
Should we be checking the recursion limit with systemstate.getrecursionlimit() somewhere and we don't? Or that's expected?

You are right that we don't want to see Java's stack overflow when we are dealing in pure Python code and so you are probably right that we need to better check our recursion limit.

Hey Frank,

I did a little experiment in PyFunction. Wrapping the __call__ with recursion_depth check makes it obey the recursion limits, and it fixes Habib's problem (i guess because there's enough stack for classloader not to go mad). Now, this is a bit more code that every function call will use and will probably impact performance a bit (IMO, correctness first but I'll need to come up with a good benchmark to validate the performance difference). My biggest worry though is that stuff that worked before might not work. Without this code my system is able to recurse ~1600 times, but with the new check, it would do only 1000 times. Maybe we add it to 2.7 only and not 2.5...

One more thing. If someone sets a smaller java stack size, we would still hit this obscurish problem. Maybe we should change the exception message a bit so that it's at least a little more descriptive?

Let me know what you think?

    @Override
    public PyObject __call__(ThreadState state, PyObject arg0, PyObject arg1) {
        if (state.recursion_depth++ > state.systemState.getrecursionlimit())
            throw Py.RuntimeError("maximum recursion depth exceeded");
        try {
            return __code__.call(state, arg0, arg1, __globals__, func_defaults, func_closure);
        } finally {
            --state.recursion_depth;
        }
    }

Hey Frank,

I did some simple tests with recursionlimit checks on PyFunction vs without vs CPython. There does not seem to be any noticeable difference. Let me know if you want me to prepare a patch (have not written any tests for it yet).

Darjus

Darjus: I agree with correctness first here. We really don't want to see java specific exceptions in pure Python code.

Hey Frank,

Here's a pull request that should hopefully fix this one:
https://bitbucket.org/python_mirrors/jython/pull-request/7/respect-recursion-limit-in-functions-1883/

Thanks!
Darjus

This may have been fixed by http://hg.python.org/jython/rev/5688f9c2b743

The example script is not a pattern we can readily support - it's basically exhausting Java resources because of the use of eval of strings. Please compile your functions and run them as desired.