Title: "from distutils import sysconfig" fails if Jython was started via java
Type: Severity: normal
Components: Library Versions: Jython 2.7
Milestone: Jython 2.7.1
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: stefan.richthofer Nosy List: stefan.richthofer, zyasoft
Priority: Keywords:

Created on 2015-08-30.12:40:19 by stefan.richthofer, last changed 2015-09-05.01:31:35 by stefan.richthofer.

msg10182 (view) Author: Stefan Richthofer (stefan.richthofer) Date: 2015-08-30.12:40:18
The problem originates in the fact that sys.executable is None if
Jython was started via java -cp jython.jar org.python.util.jython

This makes the line
project_base = os.path.dirname(os.path.realpath(sys.executable))
in fail.

While it might be intended behavior that sys.executable is None
in this case, to indicate that no Jython-launcher was used, I
think importing sysconfig from distutils should still work.
So I see two ways to improve this situation
(maybe we should even apply both):

1) Set sys.executable to some sane value even if no launcher was used.
Any suggestions what this should be? Path to java-executable?

2) Fix the line
project_base = os.path.dirname(os.path.realpath(sys.executable))
such that it has a fallback in case that sys.executable is None.
E.g. it could parse System.getProperty('java.class.path') to find
the location of jython.jar and infer the launcher-directory from
that path, i.e. by adding "/bin" to the base path of jython.jar.
msg10183 (view) Author: Stefan Richthofer (stefan.richthofer) Date: 2015-08-30.13:26:57
I just worked out and tested variant 2):

if not sys.executable is None:
    project_base = os.path.dirname(os.path.realpath(sys.executable))
    from java.lang import System
    from import File as jFile
    cpstr = System.getProperty('java.class.path')
    cps = cpstr.split(jFile.pathSeparator)
    for fl in cps:
        file = jFile(fl)
        if (file.isFile()):
            fn = file.getName()
            if (fn.startswith("jython") and fn.endswith(".jar")):
                project_base = file.getParentFile().getAbsolutePath()+jFile.separator+"bin"

It appears to be working fine, at least if jython.jar was not renamed to something cumbersome. Still, this should avoid the problem in 99.9% of the cases. If there are no objections, I would merge this into master.

msg10184 (view) Author: Stefan Richthofer (stefan.richthofer) Date: 2015-08-30.15:01:03
Okay, while this solution solves the project_base-problem, importing distutils.sysconfig fails in an other place:

distutils/command/, line 118 also needs sys.executable.

So, solution 1) is needed I suppose. But how to init sys.executable?

1a) let it point to java(.exe)
1b) just pretend it was launched via jython, i.e. create project_base like described before, then set sys.executable=project_base+"/jython"

Any opinion? While 1b) is not accurate, I suppose it would work better, as this stuff is usually used to find the jython-directory rather than the java-directory. I guess the best idea would be to apply 1b), but only in Also, I would suggest to put a utility-function somewhere that creates and returns project_base as described above for every potential use-case. Suggestions, where this should go? Maybe into Or into some lib-python code?
msg10196 (view) Author: Jim Baker (zyasoft) Date: 2015-09-01.22:54:30
You should be able to set sys.executable via the python.executable flag, which is how the launcher does it:

$ jython27 --print
java -Xmx512m -Xss1024k -classpath /Users/jbaker/jython2.7.0/jython.jar:. -Dpython.home=/Users/jbaker/jython2.7.0 -Dpython.executable=/Users/jbaker/jython2.7.0/bin/jython -Dpython.launcher.uname=darwin -Dpython.launcher.tty=true -Dfile.encoding=UTF-8 org.python.util.jython

Regardless, solution #2 is reasonable, although it would not work for the uber jar (aka single jar) case. We can make it stronger by following the approach taken by PySystemState, via a currently private method:

$ java -Xmx512m -Xss1024k -classpath /Users/jbaker/jython2.jython.jar:. org.python.util.jython
Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11)
[Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)] on java1.8.0_45
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.getJarFileName()

Ordinarily we would just promote to be public, because access modifiers can just get in the way, but that does mean this method is now in the sys module namespace. So perhaps just the logic itself should be extracted into sysconfig; or put in another location like org.python.core.Py.
msg10198 (view) Author: Stefan Richthofer (stefan.richthofer) Date: 2015-09-02.00:29:29
Can we move getJarFileName() as public method to or There might be more use-cases for it and I feel that this wheel is frequently reinvented by several projects.
msg10199 (view) Author: Jim Baker (zyasoft) Date: 2015-09-02.00:44:00
+1 on the change. Let's move getJarFileName() to org.python.core.Py
msg10201 (view) Author: Stefan Richthofer (stefan.richthofer) Date: 2015-09-02.14:07:50
Were you aware that getJarFileName() produces a malformed format under windows?

1) It prepends a "/" before the drive-letter
2) It does not use system specific seperator-char

I am going to fix this and hope that there is no code depending on this bug. Also note that using the python-launcher sys.executable is set to "blah\\", which is fine. But I suspect it would do the same if launched with jython.exe. Scanning the source of I see nothing to care for this (but maybe I overlooked it).

Unfortunately I could not test this directly using jython.exe, because jython.exe crashes on my system (win 7 64 bit). But that's a separat issue (coming soon).
msg10202 (view) Author: Jim Baker (zyasoft) Date: 2015-09-02.22:19:29
Stefan, I was not aware it was not working on Windows. Sounds like we are not capturing in a unit test! Please make the fixes you mention - one reason for betas is to determine any backwards compatibility issues.

re Windows 7 - perhaps a 32 bit vs 64 bit issue?
msg10207 (view) Author: Stefan Richthofer (stefan.richthofer) Date: 2015-09-02.22:53:34
Looking at tests/java/org/python/core/
it appears that ignoring system-specific separator-chars is intended behavior. Maybe this was fine because it was internal API. However the leading "/" before the windows drive letter is not considered by the test and would let it fail on my system.

My suggestion: Make this method protected in and leave its behavior as it is for internal use. Maybe fix the leading "/" in windows case, I'm not sure - would be good to hear someone who is more confident about this.
Then I would create a slightly renamed public version that replaces "/" by system-specific separators.
Date User Action Args
2015-09-05 01:31:35stefan.richthofersetstatus: open -> closed
resolution: accepted -> fixed
2015-09-02 22:53:35stefan.richthofersetmessages: + msg10207
2015-09-02 22:19:30zyasoftsetmessages: + msg10202
2015-09-02 14:07:51stefan.richthofersetmessages: + msg10201
2015-09-02 00:44:12zyasoftsetresolution: accepted
milestone: Jython 2.7.1
2015-09-02 00:44:01zyasoftsetmessages: + msg10199
2015-09-02 00:29:30stefan.richthofersetmessages: + msg10198
milestone: Jython 2.7.1 -> (no value)
2015-09-01 22:54:31zyasoftsetnosy: + zyasoft
messages: + msg10196
milestone: Jython 2.7.1
2015-08-30 15:01:04stefan.richthofersetmessages: + msg10184
2015-08-30 13:26:57stefan.richthofersetassignee: stefan.richthofer
messages: + msg10183
2015-08-30 12:40:19stefan.richthofercreate