Issue1518

classification
Title: os.getpid() is missing
Type: behaviour Severity: major
Components: Core Versions: 2.5.1
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: pjenvey Nosy List: glyph, pjenvey, thijs, wmeissner
Priority: Keywords:

Created on 2009-12-07.21:09:29 by thijs, last changed 2010-02-15.18:15:05 by thijs.

Messages
msg5357 (view) Author: Thijs Triemstra (thijs) Date: 2009-12-07.21:09:28
Our Jython 2.5.1+ (r6954) buildslave for Twisted is reporting the following error:

Traceback (most recent call last):
  File "./bin/trial", line 22, in <module>
    run()
  File "/home/buildbot/Buildslaves/twisted/jython2.5-thijs-
ubuntu/Twisted/twisted/scripts/trial.py", line 363, in run
    test_result = trialRunner.run(suite)
  File "/home/buildbot/Buildslaves/twisted/jython2.5-thijs-
ubuntu/Twisted/twisted/trial/runner.py", line 833, in run
    return self._runWithoutDecoration(test)
  File "/home/buildbot/Buildslaves/twisted/jython2.5-thijs-
ubuntu/Twisted/twisted/trial/runner.py", line 859, in _runWithoutDecoration
    oldDir = self._setUpTestdir()
  File "/home/buildbot/Buildslaves/twisted/jython2.5-thijs-
ubuntu/Twisted/twisted/trial/runner.py", line 749, in _setUpTestdir
    if self._testDirLock.lock():
  File "/home/buildbot/Buildslaves/twisted/jython2.5-thijs-
ubuntu/Twisted/twisted/python/lockfile.py", line 125, in lock
    symlink(str(os.getpid()), self.name)
  File "/home/buildbot/Buildslaves/twisted/jython2.5-thijs-
ubuntu/Twisted/twisted/python/lockfile.py", line 125, in lock
    symlink(str(os.getpid()), self.name)
AttributeError: 'module' object has no attribute 'getpid'

(Full error log can be found on http://buildbot.twistedmatrix.com/builders/ubuntu64-jython2.5-
select/builds/1/steps/trial/logs/stdio)

When comparing CPython 2.5/2.6 with Jython, it seems getpid() is missing, as the session below 
shows:

thijs@saturn:/usr/local/src/jython-trunk$ jython
Jython 2.5.1+ (trunk:6954, Dec 7 2009, 11:16:37) 
[Java HotSpot(TM) 64-Bit Server VM (Sun Microsystems Inc.)] on java1.5.0_06
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.getpid()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'getpid'
>>> 
thijs@saturn:/usr/local/src/jython-trunk$ python2.6
Python 2.6.4 (r264:75706, Nov  9 2009, 18:11:10) 
[GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> 
thijs@saturn:/usr/local/src/jython-trunk$ python2.5
impoPython 2.5.4 (r254:67916, Dec 25 2008, 15:04:44) 
[GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
import os
>>> os.getpid()
22374
>>>
msg5359 (view) Author: Philip Jenvey (pjenvey) Date: 2009-12-08.03:10:28
As we talked on IRC the missing getpid is due to this platform not using 
the native posix libraries. I don't know why the native posix lib 
wouldn't load on Linux. As of r6955 Jython will emit information about 
it loading when -vv is specified on the command line, e.g.:

$ dist/bin/jython -vv
initializer: 'META-INF/services/org.python.core.JythonInitializer' not 
found on sun.misc.Launcher$AppClassLoader@7e26e431
initializer: 'META-INF/services/org.python.core.JythonInitializer' not 
found on sun.misc.Launcher$AppClassLoader@7e26e431
Successfully loaded native POSIX impl.
<etc>
msg5449 (view) Author: Philip Jenvey (pjenvey) Date: 2010-01-22.02:53:58
Is this still an issue Thijs? If so, what does jython -vv report?
msg5511 (view) Author: Thijs Triemstra (thijs) Date: 2010-02-09.16:32:38
Philip, it's still an issue, a test run from feb 8 shows the same error, see http://buildbot.twistedmatrix.com/builders/ubuntu64-jython2.5-select/builds/69/steps/trial/logs/stdio

This is using r6976. We are using the following command to invoke Jython/trial: jython -W all ./bin/trial --reporter=bwverbose --unclean-warnings twisted

I'll check what happens with 'jython -vv'.
msg5512 (view) Author: Thijs Triemstra (thijs) Date: 2010-02-09.16:35:10
Alright, -vv reports an error! :)

$ jython -vv
initializer: 'META-INF/services/org.python.core.JythonInitializer' not found on sun.misc.Launcher$AppClassLoader@2f2498b5
initializer: 'META-INF/services/org.python.core.JythonInitializer' not found on sun.misc.Launcher$AppClassLoader@2f2498b5
Failed to load native POSIX impl; falling back on Java impl. Stacktrace follows.
java.lang.UnsatisfiedLinkError: /tmp/jffi25093.tmp: /lib/libc.so.6: version `GLIBC_2.4' not found (required by /tmp/jffi25093.tmp)
	at java.lang.ClassLoader$NativeLibrary.load(Native Method)
	at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1751)
	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1647)
	at java.lang.Runtime.load0(Runtime.java:769)
	at java.lang.System.load(System.java:967)
	at com.kenai.jffi.Init.loadFromJar(Init.java:127)
	at com.kenai.jffi.Init.load(Init.java:64)
	at com.kenai.jffi.Init.<clinit>(Init.java:46)
	at com.kenai.jffi.Foreign.<init>(Foreign.java:38)
	at com.kenai.jffi.Foreign.<init>(Foreign.java:28)
	at com.kenai.jffi.Foreign$SingletonHolder.<clinit>(Foreign.java:30)
	at com.kenai.jffi.Foreign.getInstance(Foreign.java:34)
	at com.kenai.jffi.Library.dlopen(Library.java:72)
	at com.kenai.jffi.Library.openLibrary(Library.java:135)
	at com.kenai.jffi.Library.getCachedInstance(Library.java:109)
	at com.kenai.jaffl.provider.jffi.Library.loadNativeLibraries(Library.java:66)
	at com.kenai.jaffl.provider.jffi.Library.getNativeLibraries(Library.java:56)
	at com.kenai.jaffl.provider.jffi.Library.getSymbolAddress(Library.java:35)
	at com.kenai.jaffl.provider.jffi.Library.findSymbolAddress(Library.java:45)
	at com.kenai.jaffl.provider.jffi.AsmLibraryLoader.generateInterfaceImpl(AsmLibraryLoader.java:193)
	at com.kenai.jaffl.provider.jffi.AsmLibraryLoader.loadLibrary(AsmLibraryLoader.java:121)
	at com.kenai.jaffl.provider.jffi.Provider.loadLibrary(Provider.java:30)
	at com.kenai.jaffl.provider.jffi.Provider.loadLibrary(Provider.java:24)
	at com.kenai.jaffl.Library.loadLibrary(Library.java:73)
	at org.jruby.ext.posix.POSIXFactory$LinuxLibCProvider$SingletonHolder.<clinit>(POSIXFactory.java:103)
	at org.jruby.ext.posix.POSIXFactory$LinuxLibCProvider.getLibC(POSIXFactory.java:107)
	at org.jruby.ext.posix.BaseNativePOSIX.<init>(BaseNativePOSIX.java:28)
	at org.jruby.ext.posix.LinuxPOSIX.<init>(LinuxPOSIX.java:18)
	at org.jruby.ext.posix.POSIXFactory.loadLinuxPOSIX(POSIXFactory.java:65)
	at org.jruby.ext.posix.POSIXFactory.getPOSIX(POSIXFactory.java:24)
	at org.python.modules.posix.PosixModule.<clinit>(PosixModule.java:62)
	at org.python.modules.Setup.<clinit>(Setup.java:29)
	at org.python.core.PySystemState.initBuiltins(PySystemState.java:1000)
	at org.python.core.PySystemState.doInitialize(PySystemState.java:837)
	at org.python.core.PySystemState.initialize(PySystemState.java:756)
	at org.python.core.PySystemState.initialize(PySystemState.java:706)
	at org.python.core.PySystemState.initialize(PySystemState.java:699)
	at org.python.util.jython.run(jython.java:150)
	at org.python.util.jython.main(jython.java:129)
msg5513 (view) Author: Thijs Triemstra (thijs) Date: 2010-02-09.16:42:03
I'm running an older version of Ubuntu (6.1) and seems like it doesn't contain a version of GLIB that Jython is happy with:

$ file /lib/libc-2.3.6.so
/lib/libc-2.3.6.so: ELF 64-bit LSB shared object, AMD x86-64, version 1 (SYSV), for GNU/Linux 2.6.0, stripped

Is there a way to work around this (without installing a newer version of glib)? And if there isn't, can Jython throw a more visible error instead of reporting that getpid() doesn't exist?
msg5514 (view) Author: Thijs Triemstra (thijs) Date: 2010-02-09.16:46:54
So turns out glibc 2.3.6 on Dapper is from November 2005, so I would understand if you don't want to support such an ancient version. I will upgrade to the latest (2.11.1) and see if that helps.
msg5515 (view) Author: Thijs Triemstra (thijs) Date: 2010-02-09.17:01:03
So upgrading glibc seems to have big consequences for the stability of the system if I have to believe the warning message that their 'configure' throws:

*** On GNU/Linux systems the GNU C Library should not be installed into
*** /usr/local since this might make your system totally unusable.
*** We strongly advise to use a different prefix.  For details read the FAQ.
*** If you really mean to do this, run configure again using the extra
*** parameter `--disable-sanity-checks'.

Their FAQ contradicts this though, saying '/usr/local/' is safe:

"Like all other GNU packages GNU libc is designed to use a base
directory and install all files relative to this.  The default is
/usr/local, because this is safe (it will not damage the system if installed
there).  If you wish to install GNU libc as the primary C library on your
system, set the base directory to /usr (i.e. run configure --prefix=/usr
<other_options>).  Note that this can damage your system; see question 2.3 for
details."

So this is confusing, sigh... suggestions?
msg5516 (view) Author: Thijs Triemstra (thijs) Date: 2010-02-09.17:14:49
Alright according to the documentation Ubuntu 6.06 is no longer supported,  8.04 is the current LTS version. This means I'll probably have to upgrade my system which will give me the required glibc 2.4. I'm worried this won't be a painless process so I'll have to stick to 6.06 for now and hope there's an alternative solution for this jython problem.
msg5521 (view) Author: Wayne Meissner (wmeissner) Date: 2010-02-12.22:21:57
There is a jffi stub built for x86_64-Linux on ubuntu 6.06 available at http://github.com/wmeissner/jffi/blob/0.6/archive/x86_64-Linux.jar

You need to copy that over extlibs/jffi-x86_64-Linux.jar in jython trunk and rebuild jython

I haven't tested that it works beyond the smoke tests in jffi, so beware.
msg5522 (view) Author: Philip Jenvey (pjenvey) Date: 2010-02-14.00:08:35
Is that an older, 0.6 release of jffi Wayne?
msg5524 (view) Author: Wayne Meissner (wmeissner) Date: 2010-02-14.00:48:59
Yes, thats from the 0.6 branch, which is the same as is currently in jython trunk (and JRuby 1.4).  All the 0.6 stubs are binary compatible with all 0.6.x versions of jffi.jar
msg5525 (view) Author: Philip Jenvey (pjenvey) Date: 2010-02-14.03:12:04
Thanks Wayne

I committed this stub in r6979. I'll assume this fixes the problem, otherwise please reopen the ticket Thijs
msg5527 (view) Author: Philip Jenvey (pjenvey) Date: 2010-02-14.19:33:39
actually r6980
msg5531 (view) Author: Thijs Triemstra (thijs) Date: 2010-02-15.18:15:05
Thanks guys, that fixed it:

Jython 2.5.1+ (trunk:6980, Feb 15 2010, 19:03:47) 
[Java HotSpot(TM) 64-Bit Server VM (Sun Microsystems Inc.)] on java1.5.0_06
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.getpid()
17210
>>>
History
Date User Action Args
2010-02-15 18:15:05thijssetmessages: + msg5531
2010-02-14 19:33:39pjenveysetmessages: + msg5527
2010-02-14 03:12:04pjenveysetstatus: open -> closed
resolution: fixed
messages: + msg5525
2010-02-14 00:48:59wmeissnersetmessages: + msg5524
2010-02-14 00:08:36pjenveysetmessages: + msg5522
2010-02-12 22:21:57wmeissnersetnosy: + wmeissner
messages: + msg5521
2010-02-09 17:14:49thijssetmessages: + msg5516
2010-02-09 17:01:04thijssetmessages: + msg5515
2010-02-09 16:46:54thijssetmessages: + msg5514
2010-02-09 16:42:03thijssetmessages: + msg5513
2010-02-09 16:35:11thijssetmessages: + msg5512
2010-02-09 16:32:39thijssetmessages: + msg5511
2010-01-22 02:53:59pjenveysetmessages: + msg5449
2009-12-08 03:10:29pjenveysetassignee: pjenvey
messages: + msg5359
nosy: + pjenvey
2009-12-07 21:09:30thijscreate