Created on 2020-02-12.13:57:04 by r_walter, last changed 2020-03-01.15:10:09 by jeff.allen.
|msg12986 (view)||Author: Roland Walter (r_walter)||Date: 2020-02-12.13:57:03|
OS: RHEL 7, openSUSE 15.1 Java 1.8.0 I created a snapshot installer build on 2020-02-05. I used that to install jython in /usr/local/jython2.7.2b3-rwalter20200205. Calling jython as root works. Calling jython as user roland fails with Exception in thread "main" Exception: org.python.core.PyException thrown from the UncaughtExceptionHandler in thread "main" A workaround is to remove all *$py.class files from the Lib directory of the jython installation. This error was not there in snapshot builds before 2020-01-20.
|msg12988 (view)||Author: Roland Walter (r_walter)||Date: 2020-02-12.14:00:54|
Calling with jython -vvv gives in a German environment: org.python FEIN initializer: 'META-INF/services/org.python.core.JythonInitializer' not found on sun.misc.Launcher$AppClassLoader@33909752 org.python FEIN initializer: 'META-INF/services/org.python.core.JythonInitializer' not found on sun.misc.Launcher$AppClassLoader@33909752 org.python.package FEIN reading cache of '/usr/local/jython2.7.2b3-rwalter20200205/jython.jar' org.python.package FEIN reading cache of '/usr/lib64/jvm/java-1.8.0-openjdk-1.8.0/jre/lib/resources.jar' org.python.package FEIN reading cache of '/usr/lib64/jvm/java-1.8.0-openjdk-1.8.0/jre/lib/rt.jar' org.python.package FEIN reading cache of '/usr/lib64/jvm/java-1.8.0-openjdk-1.8.0/jre/lib/jsse.jar' org.python.package FEIN reading cache of '/usr/lib64/jvm/java-1.8.0-openjdk-1.8.0/jre/lib/jce.jar' org.python.package FEIN reading cache of '/usr/lib64/jvm/java-1.8.0-openjdk-1.8.0/jre/lib/charsets.jar' org.python.package FEIN reading cache of '/usr/lib64/jvm/java-1.8.0-openjdk-1.8.0/jre/lib/ext/jfxrt.jar' org.python.package FEIN reading cache of '/usr/lib64/jvm/java-1.8.0-openjdk-1.8.0/jre/lib/ext/sunec.jar' org.python.package FEIN reading cache of '/usr/lib64/jvm/java-1.8.0-openjdk-1.8.0/jre/lib/ext/nashorn.jar' org.python.package FEIN reading cache of '/usr/lib64/jvm/java-1.8.0-openjdk-1.8.0/jre/lib/ext/zipfs.jar' org.python.package FEIN reading cache of '/usr/lib64/jvm/java-1.8.0-openjdk-1.8.0/jre/lib/ext/jaccess.jar' org.python.package FEIN reading cache of '/usr/lib64/jvm/java-1.8.0-openjdk-1.8.0/jre/lib/ext/sunjce_provider.jar' org.python.package FEIN reading cache of '/usr/lib64/jvm/java-1.8.0-openjdk-1.8.0/jre/lib/ext/sunpkcs11.jar' org.python.package FEIN reading cache of '/usr/lib64/jvm/java-1.8.0-openjdk-1.8.0/jre/lib/ext/dnsns.jar' org.python.package FEIN reading cache of '/usr/lib64/jvm/java-1.8.0-openjdk-1.8.0/jre/lib/ext/icedtea-sound.jar' org.python.package FEIN reading cache of '/usr/lib64/jvm/java-1.8.0-openjdk-1.8.0/jre/lib/ext/cldrdata.jar' org.python.package FEIN reading cache of '/usr/lib64/jvm/java-1.8.0-openjdk-1.8.0/jre/lib/ext/localedata.jar' org.python FEIN threadstate: no current system state org.python.core KONFIGURATION sys module instance created org.python.import FEIN # trying org.python.core.exceptions as builtin module in SysPathJavaLoader org.python.import KONFIGURATION import exceptions # builtin org.python.core.exceptions org.python.import FEIN # trying precompiled /usr/local/jython2.7.2b3-rwalter20200205/Lib/site$py.class Exception in thread "main" org.python.import FEIN # trying precompiled /usr/local/jython2.7.2b3-rwalter20200205/Lib/encodings/__init__$py.class Exception: org.python.core.PyException thrown from the UncaughtExceptionHandler in thread "main"
|msg12990 (view)||Author: Roland Walter (r_walter)||Date: 2020-02-12.14:09:49|
The real workaround is to set read rights for group and other: find Lib -type f -name "*py.class" | xargs chmod 644 This was the case for builds on Windows 10 and Linux.
|msg12992 (view)||Author: Jeff Allen (jeff.allen)||Date: 2020-02-15.07:34:21|
Thanks for trying this. This is surely a side-effect of #2044. It was always a dubious design to cache compiled files by default into the installation directory. Compilation to there during installation is reasonable, and that the files should then be world-read, but caches specific to the application ought to be private to the user (and local or in TEMP, I think).
|msg12994 (view)||Author: Jeff Allen (jeff.allen)||Date: 2020-02-20.18:29:18|
When we compile, since #2044, we make class files private to the current user. When we install, the files come out of a JAR. Since the process does not preserve permissions, the files have the default permissions for the installing user, and end up world-readable (fortuitously, almost). However, we also run an installation script for pip, and at that point it seems we recompile a proportion (not all) of the library. These class files become private to the installing user. I believe this happens because either : 1. the source appears newer because we touch it somehow after the class is created. 2. the source time in the class file does not match the one rounded by the JAR format. This is a Bad Thing generally, and not just because it has this effect. It will makes Jython try to ignore (always try to rewrite) compiled version we supply. Good discovery. At the moment, I'm looking at the rounding theory.
|msg12996 (view)||Author: Jeff Allen (jeff.allen)||Date: 2020-02-20.22:41:12|
It seems I have a choice forming the JAR to round up or down. Let's choose down, so that the source via the JAR not older than the mtime in the $py.class. If I log this difference, it does indeed seem to be spread from 0 to 2000 ms: runpy$py.class time is 1,161 ms later than source imp$py.class time is 1,599 ms later than source os$py.class time is 1,755 ms later than source ntpath$py.class time is 958 ms later than source stat$py.class time is 1,271 ms later than source genericpath$py.class time is 83 ms later than source warnings$py.class time is 677 ms later than source linecache$py.class time is 849 ms later than source types$py.class time is 661 ms later than source UserDict$py.class time is 1,864 ms later than source _abcoll$py.class time is 1,255 ms later than source abc$py.class time is 1,911 ms later than source _weakrefset$py.class time is 1,317 ms later than source jythonlib$py.class time is 1,661 ms later than source pkgutil$py.class time is 1,786 ms later than source ensurepip\__init__$py.class time is 1,552 ms later than source __future__$py.class time is 1,880 ms later than source shutil$py.class time is 1,208 ms later than source fnmatch$py.class time is 36 ms later than source ... Tolerating this, we should be able to avoid the recompilation that affects the permissions, without reverting the fix of the CVE. A dry run on Linux confirms this, and I installed as root (with sudo) and can run it (sort of) as an ordinary user. pip and setuptools installed by ensurepip, which surely involves some compilation, do not result in private files, so probably the installation of packages takes care of permissions. The fault now is that Jython is trying to place the JAR cache in the installation directory.
|msg12998 (view)||Author: Jeff Allen (jeff.allen)||Date: 2020-02-23.15:26:04|
Here we deal with the spurious recompilation that occurs when root runs Jython to install pip: https://hg.python.org/jython/rev/2a77fdc4417c and here relocate the cache directory to user space, so it works again in shared installations, after #2044: https://hg.python.org/jython/rev/33ea06302061 I thought a longer name was necessary, now the cache it is out of the Jython installation context. I considered placing it in the user's home directory. This would have the advantage that Jython run from different directories by the same user would not experience the start-up delay of rebuilding the cache. However, the down-side is a potential for interaction when different versions of Java or different versions of the JARs are in play. This seemed the safer choice to make, this close to a release.
|2020-03-01 15:10:09||jeff.allen||set||status: pending -> closed|
|2020-02-23 15:26:04||jeff.allen||set||status: open -> pending|
resolution: accepted -> fixed
messages: + msg12998
versions: - Jython 2.7.3
|2020-02-20 22:41:12||jeff.allen||set||assignee: jeff.allen|
messages: + msg12996
|2020-02-20 18:29:18||jeff.allen||set||messages: + msg12994|
|2020-02-15 07:34:21||jeff.allen||set||priority: normal|
messages: + msg12992
nosy: + jeff.allen
milestone: Jython 2.7.2
|2020-02-12 14:20:30||r_walter||set||components: + Installer|
|2020-02-12 14:09:49||r_walter||set||messages: + msg12990|
|2020-02-12 14:00:54||r_walter||set||messages: + msg12988|