Issue1180

classification
Title: virtualenv broken on installed Jython
Type: Severity: normal
Components: Core Versions: 2.5b0
process
Status: closed Resolution: fixed
Dependencies: JYTHON_OPTS broken in jython.bat
View: 1187
Superseder:
Assigned To: pjenvey Nosy List: otmarhumbel, pjenvey
Priority: urgent Keywords:

Created on 2008-11-21.19:35:47 by pjenvey, last changed 2008-12-10.21:06:49 by pjenvey.

Files
File name Uploaded Description Edit Remove
site.diff otmarhumbel, 2008-11-27.16:10:40
Messages
msg3799 (view) Author: Philip Jenvey (pjenvey) Date: 2008-11-21.19:35:46
The installer currently hardcodes JYTHON_HOME in the script runner. This 
makes the jython executable portable to other paths (which is the point) 
-- but this breaks virtualenv

We need to work like CPython in that the executable needs to detect what 
looks like a surrounding JYTHON_HOME and use that instead. virtualenv 
takes advantage of this by copying sys.executable to an entirely 
different HOME

We probably need the installer to hardcode something like a 
JYTHON_INSTALLED_HOME -- to be used as a last resort if the CPython like 
search fails

The responsible code in CPython seems to be Modules/getpath::calculate_path
msg3801 (view) Author: Oti Humbel (otmarhumbel) Date: 2008-11-22.12:43:36
Is this the right track:

jekyll:virtualenv oti$ ~/stuff/jython/jython5593/jython virtualenv.py 
./ENVs/first_try/
*sys-package-mgr* <snip>
New jython executable in ./ENVs/first_try/bin/jython
ERROR: The executable ./ENVs/first_try/bin/jython is not functioning
ERROR: It thinks sys.prefix is '/Users/oti/stuff/jython/jython5593' 
(should be '/Users/oti/stuff/jython/virtualenv/ENVs/first_try')
ERROR: virtualenv is not compatible with this system or executable
jekyll:virtualenv oti$
msg3804 (view) Author: Philip Jenvey (pjenvey) Date: 2008-11-22.21:22:58
yea, you've reproduced it
msg3806 (view) Author: Oti Humbel (otmarhumbel) Date: 2008-11-22.22:14:10
by not hardcoding JYTHON_HOME, and with a little guessing i got:

jekyll:virtualenv oti$ ~/stuff/jython/jython5593/jython virtualenv.py 
./ENVs/first_try
New jython executable in ./ENVs/first_try/bin/jython
Installing setuptools...........................done.
jekyll:virtualenv oti$ 

I think this is an improvement.
As soon as it passes the installer verifications, i'll check that in
msg3807 (view) Author: Oti Humbel (otmarhumbel) Date: 2008-11-22.22:17:27
it looks like below (not the most elegant shell script code, but 
working);
and of course the windows version needs to be updated, too

# try to dynamically determine jython home
# (this script can reside in jython home, or in the /bin subdirectory)
if [ -z "$JYTHON_HOME" ] ; then
  if [ "$PRG" = "./jython" ] ; then
    # current dir is the script dir
    JYTHON_HOME_1=`pwd`
  else
    # current dir is not the script dir
    JYTHON_HOME_1=`dirname "$PRG"`
  fi
  if [ -f "$JYTHON_HOME_1"/jython-complete.jar ] ; then
    JYTHON_HOME=$JYTHON_HOME_1
  elif [ -f "$JYTHON_HOME_1"/jython.jar ] ; then
    JYTHON_HOME=$JYTHON_HOME_1
  else
    JYTHON_HOME=`dirname "$JYTHON_HOME_1"`
  fi
fi
msg3847 (view) Author: Oti Humbel (otmarhumbel) Date: 2008-11-25.10:08:15
JYTHON_HOME determination for Windows:

rem try to dynamically determine jython home
rem this script usually resides in the home directory, or in the /bin
subdirectory
pushd "%~dp0%"
set _JYTHON_HOME="%CD%"
popd
if exist %_JYTHON_HOME%\jython.jar goto gotHome
if exist %_JYTHON_HOME%\jython-complete.jar goto gotHome
pushd "%~dp0%\.."
set _JYTHON_HOME="%CD%"
popd
if exist %_JYTHON_HOME%\jython.jar goto gotHome
if exist %_JYTHON_HOME%\jython-complete.jar goto gotHome
rem jython home fallback (if all else fails)
rem if present, %JYTHON_HOME_FALLBACK% is already quoted
set _JYTHON_HOME=%JYTHON_HOME_FALLBACK%
msg3848 (view) Author: Oti Humbel (otmarhumbel) Date: 2008-11-25.21:37:26
http://jython.svn.sourceforge.net/viewvc/jython?view=rev&revision=5639 
leads to:

jekyll:virtualenv oti$ /tmp/jython5639/jython 
*sys-package-mgr*: <snip>
Jython 2.5b0+ (trunk:5639M, Nov 25 2008, 15:46:00) 
[Java HotSpot(TM) Client VM (Apple Inc.)] on java1.5.0_16
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys; sys.exit()

jekyll:virtualenv oti$ /tmp/jython5639/jython virtualenv.py 
./ENVs/2nd_try
New jython executable in ./ENVs/2nd_try/bin/jython
Installing setuptools...........................done.

jekyll:virtualenv oti$ ./ENVs/2nd_try/bin/jython 
Jython 2.5b0+ (trunk:5639M, Nov 25 2008, 15:46:00) 
[Java HotSpot(TM) Client VM (Apple Inc.)] on java1.5.0_16
Type "help", "copyright", "credits" or "license" for more information.
>>>
msg3859 (view) Author: Oti Humbel (otmarhumbel) Date: 2008-11-27.10:17:16
Tried to verify virtualenv on Windows, but got

C:\stuff\jython\virtualenv\ENVs\2nd_try\bin>jython.bat
Jython 2.5b0+ (trunk:5639M, Nov 25 2008, 15:46:00)
[Java HotSpot(TM) Client VM (Sun Microsystems Inc.)] on java1.6.0_07
error importing site
Traceback (most recent call last):
  File "C:\stuff\jython\virtualenv\ENVs\2nd_try\Lib\site.py", line 589,
in <module>
    main()
  File "C:\stuff\jython\virtualenv\ENVs\2nd_try\Lib\site.py", line 567,
in main
    ENABLE_USER_SITE = check_enableusersite()
  File "C:\stuff\jython\virtualenv\ENVs\2nd_try\Lib\site.py", line 265,
in check_enableusersite
    if os.getegid() != os.getgid():
  File "C:\stuff\jython\virtualenv\ENVs\2nd_try\Lib\os.py", line 913, in
getegid
    return _posix.getegid()
  File "C:\stuff\jython\virtualenv\ENVs\2nd_try\Lib\os.py", line 107, in
unimplementedError
    raise NotImplementedError(method_name)
NotImplementedError: egid
msg3860 (view) Author: Oti Humbel (otmarhumbel) Date: 2008-11-27.16:10:39
virtualenv.py installs a modified site.py in /Lib, which acesses posix
group id functions, which in turn are not implemented on windows.
The new site.py is encoded as base64 inside virtualenv.py.

The unified diff is attached as site.diff

I am not sure how to proceed now.
msg3861 (view) Author: Oti Humbel (otmarhumbel) Date: 2008-11-27.16:57:04
Commenting out the offending lines in site.py, i got:

C:\stuff\jython\virtualenv>..\jython5639\jython.bat virtualenv.py -v
ENVs\2nd_try
<snip>
Testing executable with ENVs\2nd_try\bin\jython.bat -c "import sys;
print sys.prefix"
*sys-package-mgr*: processing new jar,
'C:\stuff\jython\virtualenv\ENVs\2nd_try\jython-complete.jar'
Usage: virtualenv.py [OPTIONS] DEST_DIR

virtualenv.py: error: no such option: -c
ERROR: The executable ENVs\2nd_try\bin\jython.bat is not functioning


It looks like the main() method of virtualenv.py is reentrant.
Definitely out of ideas now.

BUT, manually testing the just created jython.bat proves that sys.prefix
is correct:

C:\stuff\jython\virtualenv>ENVs\2nd_try\bin\jython.bat -c "import sys;
print sys.prefix"
C:\stuff\jython\virtualenv\ENVs\2nd_try

So i think that the .bat part of the story is solved.
msg3881 (view) Author: Philip Jenvey (pjenvey) Date: 2008-12-08.07:19:44
We'll have to manually disable more os functions when os._name is nt (or 
maybe only enable them when it's posix). I previously did this for a few 
of them

I'll have a look at virtualenv on Windows
msg3882 (view) Author: Oti Humbel (otmarhumbel) Date: 2008-12-08.07:28:53
Thanks for taking over!

If i remember correctly, i had to comment out the last two if statements 
in method check_enableusersite() - in the zipped/encoded site.py

They looked like:
  if hasattr(os, "getuid") and hasattr(os, "geteuid"):
    ...
  if hasattr(os, "getgid") and hasattr(os, "getegid"):
    ...

commenting out those two brought me a step further
msg3883 (view) Author: Philip Jenvey (pjenvey) Date: 2008-12-08.07:31:35
thanks for taking care of the bigger first part, Oti =]
msg3904 (view) Author: Oti Humbel (otmarhumbel) Date: 2008-12-10.10:19:16
runs for me with trunk revision 5731
a short log is here: http://pylonshq.com/pasties/1027
msg3908 (view) Author: Philip Jenvey (pjenvey) Date: 2008-12-10.21:06:48
thanks again Oti, closing out
History
Date User Action Args
2008-12-10 21:06:49pjenveysetstatus: open -> closed
resolution: fixed
messages: + msg3908
2008-12-10 10:19:16otmarhumbelsetmessages: + msg3904
2008-12-08 07:31:35pjenveysetmessages: + msg3883
2008-12-08 07:28:54otmarhumbelsetmessages: + msg3882
2008-12-08 07:19:44pjenveysetassignee: otmarhumbel -> pjenvey
messages: + msg3881
2008-12-01 14:54:37otmarhumbelsetdependencies: + JYTHON_OPTS broken in jython.bat
2008-11-27 16:57:05otmarhumbelsetmessages: + msg3861
2008-11-27 16:10:41otmarhumbelsetfiles: + site.diff
messages: + msg3860
2008-11-27 10:17:18otmarhumbelsetmessages: + msg3859
2008-11-25 21:37:27otmarhumbelsetmessages: + msg3848
2008-11-25 10:08:16otmarhumbelsetmessages: + msg3847
2008-11-22 22:17:27otmarhumbelsetmessages: + msg3807
2008-11-22 22:14:10otmarhumbelsetmessages: + msg3806
2008-11-22 21:22:58pjenveysetmessages: + msg3804
2008-11-22 12:43:37otmarhumbelsetassignee: otmarhumbel
messages: + msg3801
nosy: + otmarhumbel
2008-11-21 19:35:47pjenveycreate