Message12065

One of the joys of Jython (USPs if you like) is that it leaves reference counting to the JVM. Another is that it runs on the JVM, which has certain security/safety features.

When jnr-posix etc. reach past the JVM with an integer file handle or a FILE* it breaks this, yet we seem to have to do that because Python libraries permit this direct engagement with C libraries behind CPython, for those who want to do low-level things. I have some discomfort about this.

Java wants to protect us from this. Or is that rather protect its JVM from us? You can see this in one of the things blocking Java 9: improved modularisation stops us getting an integer file handle. It never worked on Windows and now it doesn't on Unix unless you go to some lengths to disable the safety features.

If you're running C, I guess you already circumvented the safety features, and the convenience of not reference-counting since you must sometimes have to lock objects down. We must be careful not to create (I mean we must recover from) the situation in which Jython *only* runs with the safety turned off. I think this leads us to one or more runtime options that say "it's ok, turn off protection X", with which certain low-level features are enabled. But also it means we echew these unsafe features elsewhere in Jython (and test that we have). That's if future versions of the JVM give us the option at all.

I'm slightly foxed by the stated motivation for PyFile_IncUseCount. If I open a file in one thread and use it, meanwhile another thread might close it, don't I deserve the mess I get into? :)

In CPython 2.7 source, I only see PyFile_(Inc|Dec)UseCount used as brackets around a place where the GIL is released, with no hint of threads racing to close a file. I expecetd it to work as a deferred close, but actually it just seems to raise an error if you get it wrong. I notice this feature has disappeared by Python 3.6, the library relying instead on C stdio (https://www.commandlinux.com/man-page/man3/flockfile.3.html) to make a thread wait, or a no-op where that's not available.

In Jython, the GIL is permanently released. I feel I haven't entirely grasped the implications of that and may not be alone.

Would it really affect C extensions if these were no-ops?
Does it actually protect against anything (in a sensible application) given the way threads work in Jython?