Title: virtualenv broken on installed Jython
Type: Severity: normal
Components: Core Versions: 2.5b0
Status: closed Resolution: fixed
Dependencies: JYTHON_OPTS broken in jython.bat
View: 1187
Assigned To: pjenvey Nosy List: otmarhumbel, pjenvey
Priority: urgent Keywords:

Created on 2008-11-21.19:35:47 by pjenvey, last changed 2008-12-10.21:06:49 by pjenvey.

File name Uploaded Description Edit Remove
site.diff otmarhumbel, 2008-11-27.16:10:40
msg3799 (view) Author: Philip Jenvey (pjenvey) Date: 2008-11-21.19:35:46
The installer currently hardcodes JYTHON_HOME in the script runner. This 
makes the jython executable portable to other paths (which is the point) 
-- but this breaks virtualenv

We need to work like CPython in that the executable needs to detect what 
looks like a surrounding JYTHON_HOME and use that instead. virtualenv 
takes advantage of this by copying sys.executable to an entirely 
different HOME

We probably need the installer to hardcode something like a 
JYTHON_INSTALLED_HOME -- to be used as a last resort if the CPython like 
search fails

The responsible code in CPython seems to be Modules/getpath::calculate_path
msg3801 (view) Author: Oti Humbel (otmarhumbel) Date: 2008-11-22.12:43:36
Is this the right track:

jekyll:virtualenv oti$ ~/stuff/jython/jython5593/jython 
*sys-package-mgr* <snip>
New jython executable in ./ENVs/first_try/bin/jython
ERROR: The executable ./ENVs/first_try/bin/jython is not functioning
ERROR: It thinks sys.prefix is '/Users/oti/stuff/jython/jython5593' 
(should be '/Users/oti/stuff/jython/virtualenv/ENVs/first_try')
ERROR: virtualenv is not compatible with this system or executable
jekyll:virtualenv oti$
msg3804 (view) Author: Philip Jenvey (pjenvey) Date: 2008-11-22.21:22:58
yea, you've reproduced it
msg3806 (view) Author: Oti Humbel (otmarhumbel) Date: 2008-11-22.22:14:10
by not hardcoding JYTHON_HOME, and with a little guessing i got:

jekyll:virtualenv oti$ ~/stuff/jython/jython5593/jython 
New jython executable in ./ENVs/first_try/bin/jython
Installing setuptools...........................done.
jekyll:virtualenv oti$ 

I think this is an improvement.
As soon as it passes the installer verifications, i'll check that in
msg3807 (view) Author: Oti Humbel (otmarhumbel) Date: 2008-11-22.22:17:27
it looks like below (not the most elegant shell script code, but 
and of course the windows version needs to be updated, too

# try to dynamically determine jython home
# (this script can reside in jython home, or in the /bin subdirectory)
if [ -z "$JYTHON_HOME" ] ; then
  if [ "$PRG" = "./jython" ] ; then
    # current dir is the script dir
    # current dir is not the script dir
    JYTHON_HOME_1=`dirname "$PRG"`
  if [ -f "$JYTHON_HOME_1"/jython-complete.jar ] ; then
  elif [ -f "$JYTHON_HOME_1"/jython.jar ] ; then
    JYTHON_HOME=`dirname "$JYTHON_HOME_1"`
msg3847 (view) Author: Oti Humbel (otmarhumbel) Date: 2008-11-25.10:08:15
JYTHON_HOME determination for Windows:

rem try to dynamically determine jython home
rem this script usually resides in the home directory, or in the /bin
pushd "%~dp0%"
if exist %_JYTHON_HOME%\jython.jar goto gotHome
if exist %_JYTHON_HOME%\jython-complete.jar goto gotHome
pushd "%~dp0%\.."
if exist %_JYTHON_HOME%\jython.jar goto gotHome
if exist %_JYTHON_HOME%\jython-complete.jar goto gotHome
rem jython home fallback (if all else fails)
rem if present, %JYTHON_HOME_FALLBACK% is already quoted
msg3848 (view) Author: Oti Humbel (otmarhumbel) Date: 2008-11-25.21:37:26 
leads to:

jekyll:virtualenv oti$ /tmp/jython5639/jython 
*sys-package-mgr*: <snip>
Jython 2.5b0+ (trunk:5639M, Nov 25 2008, 15:46:00) 
[Java HotSpot(TM) Client VM (Apple Inc.)] on java1.5.0_16
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys; sys.exit()

jekyll:virtualenv oti$ /tmp/jython5639/jython 
New jython executable in ./ENVs/2nd_try/bin/jython
Installing setuptools...........................done.

jekyll:virtualenv oti$ ./ENVs/2nd_try/bin/jython 
Jython 2.5b0+ (trunk:5639M, Nov 25 2008, 15:46:00) 
[Java HotSpot(TM) Client VM (Apple Inc.)] on java1.5.0_16
Type "help", "copyright", "credits" or "license" for more information.
msg3859 (view) Author: Oti Humbel (otmarhumbel) Date: 2008-11-27.10:17:16
Tried to verify virtualenv on Windows, but got

Jython 2.5b0+ (trunk:5639M, Nov 25 2008, 15:46:00)
[Java HotSpot(TM) Client VM (Sun Microsystems Inc.)] on java1.6.0_07
error importing site
Traceback (most recent call last):
  File "C:\stuff\jython\virtualenv\ENVs\2nd_try\Lib\", line 589,
in <module>
  File "C:\stuff\jython\virtualenv\ENVs\2nd_try\Lib\", line 567,
in main
    ENABLE_USER_SITE = check_enableusersite()
  File "C:\stuff\jython\virtualenv\ENVs\2nd_try\Lib\", line 265,
in check_enableusersite
    if os.getegid() != os.getgid():
  File "C:\stuff\jython\virtualenv\ENVs\2nd_try\Lib\", line 913, in
    return _posix.getegid()
  File "C:\stuff\jython\virtualenv\ENVs\2nd_try\Lib\", line 107, in
    raise NotImplementedError(method_name)
NotImplementedError: egid
msg3860 (view) Author: Oti Humbel (otmarhumbel) Date: 2008-11-27.16:10:39 installs a modified in /Lib, which acesses posix
group id functions, which in turn are not implemented on windows.
The new is encoded as base64 inside

The unified diff is attached as site.diff

I am not sure how to proceed now.
msg3861 (view) Author: Oti Humbel (otmarhumbel) Date: 2008-11-27.16:57:04
Commenting out the offending lines in, i got:

C:\stuff\jython\virtualenv>..\jython5639\jython.bat -v
Testing executable with ENVs\2nd_try\bin\jython.bat -c "import sys;
print sys.prefix"
*sys-package-mgr*: processing new jar,
Usage: [OPTIONS] DEST_DIR error: no such option: -c
ERROR: The executable ENVs\2nd_try\bin\jython.bat is not functioning

It looks like the main() method of is reentrant.
Definitely out of ideas now.

BUT, manually testing the just created jython.bat proves that sys.prefix
is correct:

C:\stuff\jython\virtualenv>ENVs\2nd_try\bin\jython.bat -c "import sys;
print sys.prefix"

So i think that the .bat part of the story is solved.
msg3881 (view) Author: Philip Jenvey (pjenvey) Date: 2008-12-08.07:19:44
We'll have to manually disable more os functions when os._name is nt (or 
maybe only enable them when it's posix). I previously did this for a few 
of them

I'll have a look at virtualenv on Windows
msg3882 (view) Author: Oti Humbel (otmarhumbel) Date: 2008-12-08.07:28:53
Thanks for taking over!

If i remember correctly, i had to comment out the last two if statements 
in method check_enableusersite() - in the zipped/encoded

They looked like:
  if hasattr(os, "getuid") and hasattr(os, "geteuid"):
  if hasattr(os, "getgid") and hasattr(os, "getegid"):

commenting out those two brought me a step further
msg3883 (view) Author: Philip Jenvey (pjenvey) Date: 2008-12-08.07:31:35
thanks for taking care of the bigger first part, Oti =]
msg3904 (view) Author: Oti Humbel (otmarhumbel) Date: 2008-12-10.10:19:16
runs for me with trunk revision 5731
a short log is here:
msg3908 (view) Author: Philip Jenvey (pjenvey) Date: 2008-12-10.21:06:48
thanks again Oti, closing out
Date User Action Args
2008-12-10 21:06:49pjenveysetstatus: open -> closed
resolution: fixed
messages: + msg3908
2008-12-10 10:19:16otmarhumbelsetmessages: + msg3904
2008-12-08 07:31:35pjenveysetmessages: + msg3883
2008-12-08 07:28:54otmarhumbelsetmessages: + msg3882
2008-12-08 07:19:44pjenveysetassignee: otmarhumbel -> pjenvey
messages: + msg3881
2008-12-01 14:54:37otmarhumbelsetdependencies: + JYTHON_OPTS broken in jython.bat
2008-11-27 16:57:05otmarhumbelsetmessages: + msg3861
2008-11-27 16:10:41otmarhumbelsetfiles: + site.diff
messages: + msg3860
2008-11-27 10:17:18otmarhumbelsetmessages: + msg3859
2008-11-25 21:37:27otmarhumbelsetmessages: + msg3848
2008-11-25 10:08:16otmarhumbelsetmessages: + msg3847
2008-11-22 22:17:27otmarhumbelsetmessages: + msg3807
2008-11-22 22:14:10otmarhumbelsetmessages: + msg3806
2008-11-22 21:22:58pjenveysetmessages: + msg3804
2008-11-22 12:43:37otmarhumbelsetassignee: otmarhumbel
messages: + msg3801
nosy: + otmarhumbel
2008-11-21 19:35:47pjenveycreate