Message12088

I've been trying to port CPython Modules/main.c::main() into util/jython.java, using 2.7.15 as my reference, as the best way to sort out our logic. Along the way I've written an options scanner based on CPython Python/getopt.c that has potential for wider use (if we ever needed another command-line tool).

My main aim is to make our choices about interactivity and buffering the same as CPython's. This doesn't directly remove our dependence on isatty(), but I have noticed that Py_FdIsInteractive does more than just call isatty(), which may be a clue how we handle not being able to use it under Java 9 without being told-off. This, and the handling of buffering, are still the trickiest aspects to handle AFAIK, even after looking at CPython.

There's an interesting watershed in the middle of CPython main() where Py_Initialize() is called to bring up the type system and create the first interpreter. Before this point, CPython is careful (in 2.7) not to use any PyObjects, and after this point PyObjects and the interpreter may be handled reliably. It is also careful to have main() deal with settings that are command-line only issues, and initialize() deal with those from environment variables that also affect embedded Python. Not all the landmark structures (sys.argv, sys.path) have their proper values until main() is ready to run whatever code has been indicated.

We do not achieve this clean switch (which itself follows a controlled sequence in CPython): some actions are performed using methods in Py and PySystemState before PySystemState.initialize() is called, so the static initialisation that entails is what brings up at least the Python type system. This might be my fault. But even if these calls did not occur, the very fact that it is PySystemState.initialize() we call, causes the static initialisation of that class before initialize() runs. When working on PyType I noticed creation was triggered from different places for embedded and "jython" start-up.

It would stabilise this situation if we were to defer all static initialisation of PySystemState until invoked by initialize(), and to create a bolt-hole for utility methods, that is not Py, if they might be needed before the watershed. I believe I can see the shreds of this design principle in PySystemState, but if it was documented anywhere, then I missed it.

C doesn't have this static initialisation proprty, but it is interesting to note that Py_Initialize() and its relatives are in pythonrun.c not the sys module.

We have a couple of extra things to deal with: the Jython registry and running Python code from a JAR, but I think CPython shows us how to incorporate these by analogy with environment variables and the runnable zip.

---
A couple of interesting references:
https://www.python.org/dev/peps/pep-0432/ PEP 432 Restructuring the CPython startup sequence (only applies directly to 3.x).
https://wiki.python.org/moin/CPythonInterpreterInitialization Python wiki contains an interesting analysis of the sequence of events during startup, based on 3.x before PEP 432, but close to what's observable in 2.7.15.