Issue2405

This appears to be a critical regression vs jython 2.5.2, the last version I used.

Jython 2.5.2 (Release_2_5_2:7206, Mar 2 2011, 23:12:06)
[OpenJDK 64-Bit Server VM (Oracle Corporation)] on java1.8.0-internal
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.getpid()
16076


Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11)
[OpenJDK 64-Bit Server VM (Oracle Corporation)] on java1.8.0-internal
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.getpid()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'getpid'
>>>


See also http://bugs.jython.org/issue1518

On OS X 10.10.5

$ jython27
Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11)
[Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)] on java1.8.0_45
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.getpid()
36949

On Windows 8.1:

c:\jython2.7.0>bin\jython.exe
Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11)
[Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)] on java1.7.0_55
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.getpid()
5512

It's possible you have an implicit security manager policy running. Thi function, like other similar Posix functions, is not available if the underlying Posix support is missing.  Using the python_home.policy found in our tests (see usage in test_java_integration.py), which may be more restrictive than what you might be using (namely the readFileDescriptor failure):

grant codeBase "file:${python.home}/-" {
  permission java.io.FilePermission "<<ALL FILES>>", "read,write";
  permission java.lang.RuntimePermission "accessDeclaredMembers";
  permission java.lang.RuntimePermission "createClassLoader";
  permission java.lang.RuntimePermission "getProtectionDomain";
  permission java.util.PropertyPermission "*", "read,write";
};

$ jython27 -J-Djava.security.manager -J-Djava.security.policy=python_home.policy
console: Failed to install 'org.python.util.JLineConsole': java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "readFileDescriptor").
Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11)
[Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)] on java1.8.0_45
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.getpid()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'getpid'

As I noted, this works fine under 2.5.2.

To be clear, same host, same OS, same java.  The *only* difference is the version of the standalone jar I'm putting into my library path.

With the 2.5.2 standalone jar, the os.* functions are there.  With 2.7.0 and 2.7.1 beta, a few are there, but the majority are not.

if it was a security issue as you state, then I would not have different behavior between 2.5.2 and 2.7.x.  This is very clearly a bug with the 2.7.x standalone jar.

I would note that this is on Linux, not mac OR windows.

I would also note that at least one other person is reporting a similar issue with the standalone jar on the mailing list.

s/security issue/java security policy/

Point being, if it was a java security policy, it would equally affect 2.5.2 and 2.7.x.  Instead, the behavior differs depending on the jython standalone jar version.

Works correctly in 2.5.4 RC1
Works correctly in 2.7.0 BETA 1
Works correctly in 2.7.0 BETA 2
Works correctly in 2.7.0 BETA 3

BROKEN in 2.7.0 BETA4 and later

Please reopen.  Clearly the standalone jar is busted for linux

Sounds reasonable to look at standalone specifically

Thanks!  I'd really like to get off of jython 2.5.2, and this is a blocker for me. ;)  Can you remove the "resolved/WFM" resolution? ;)

I'd be happy to test any fixes in my environment.

Hey Quanah,

Can you show the actual command you're running?

➜  jython  java -jar dist/jython-standalone.jar
Jython 2.7.1 (default:39fa8f188c32, Oct 7 2015, 16:00:49)
[Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)] on java1.8.0_51
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.getpid()
47458

Also, you're running OpenJDK. Can you try with Oracle version?

zimbra@q1:~$ java -jar /tmp/jython-2.7.0.jar
Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11)
[OpenJDK 64-Bit Server VM (Oracle Corporation)] on java1.8.0-internal
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.getpid()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'getpid'
>>> quit()

zimbra@q1:~$ java -jar /tmp/jython-2.7.0b3.jar
Jython 2.7b3 (default:e81256215fb0, Aug 4 2014, 02:39:51)
[OpenJDK 64-Bit Server VM (Oracle Corporation)] on java1.8.0-internal
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.getpid()
25298
>>> quit()

I fail to see what Oracle's Java build vs OpenJDK has to do with it.  It clearly works fine with OpenJDK in 2.7.0 beta3, and not in any later Jython standalone build.  I'll certainly give it a shot however.

Identical behavior using Oracle's JDK:

zimbra@q1:~$ /tmp/jdk1.8.0_60/bin/java -jar /tmp/jython-2.7.0.jar
Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11)
[Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)] on java1.8.0_60
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.getpid()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'getpid'
>>> quit()
zimbra@q1:~$ /tmp/jdk1.8.0_60/bin/java -jar /tmp/jython-2.7.0b3.jar
Jython 2.7b3 (default:e81256215fb0, Aug 4 2014, 02:39:51)
[Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)] on java1.8.0_60
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.getpid()
1385
>>>

Further testing shows this problem is specific to Ubuntu14.

Ubuntu12 box: >>> os.getpid()
26243
Linux zre-ldap002 3.13.0-48-generic #80~precise1-Ubuntu SMP Thu Mar 12 19:30:15 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

RHEL6 box: >>> os.getpid()
29072
Linux zre-ldap003.eng.zimbra.com 2.6.32-504.3.3.el6.x86_64 #1 SMP Wed Dec 17 01:55:02 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

My Ubuntu14 box is where it fails (kernel 3.13.0-37)

Testing updating it to the latest Ubuntu14 kernel

I'm not sure what to think now... On my VMware based VM that I run, it works fine:

On my VMware based VM:
build@zre-ubuntu14-64:~$ /tmp/jdk1.8.0_60/bin/java -jar /tmp/jython-2.7.0.jar
Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11)
[Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)] on java1.8.0_60
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.getpid()
1769
>>> quit()
build@zre-ubuntu14-64:~$ uname -a
Linux zre-ubuntu14-64 3.13.0-65-generic #106-Ubuntu SMP Fri Oct 2 22:08:27 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
VERSION="14.04.3 LTS, Trusty Tahr"


On the KVM based VM I have hosted in Digital Ocean:
zimbra@q1:~$ java -jar /tmp/jython-2.7.0.jar
Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11)
[OpenJDK 64-Bit Server VM (Oracle Corporation)] on java1.8.0-internal
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.getpid()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'getpid'
>>> quit()
zimbra@q1:~$ uname -a
Linux q1.aon.zimbraview.com 3.13.0-65-generic #106-Ubuntu SMP Fri Oct 2 22:08:27 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
VERSION="14.04.3 LTS, Trusty Tahr"

Difference with posix not being present in > 2.7.0 beta 3, but only on this KVM host.  ???

zimbra@q1:~$ java -jar /tmp/jython-2.7.0b3.jar
Jython 2.7b3 (default:e81256215fb0, Aug 4 2014, 02:39:51)
[OpenJDK 64-Bit Server VM (Oracle Corporation)] on java1.8.0-internal
Type "help", "copyright", "credits" or "license" for more information.
>>> import posix
>>> posix.getpid()
13266
>>> quit()

zimbra@q1:~$ java -jar /tmp/jython-2.7.0b4.jar
Jython 2.7b4 (default:3672e624962a, Feb 13 2015, 04:59:14)
[OpenJDK 64-Bit Server VM (Oracle Corporation)] on java1.8.0-internal
Type "help", "copyright", "credits" or "license" for more information.
>>> import posix
>>> posix.getpid()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: type object 'posix' has no attribute 'getpid'
>>> quit()

Another interesting note -- Using the Ubuntu provided OpenJDK 1.7 build, things work fine:

root@q1:/tmp# /usr/bin/java -jar /tmp/jython-2.7.0.jar
Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11)
[OpenJDK 64-Bit Server VM (Oracle Corporation)] on java1.7.0_79
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.getpid()
21237
>>> quit()

VS Oracle's 1.8 update 60 JDK:
root@q1:/tmp# /tmp/jdk1.8.0_60/bin/java -jar /tmp/jython-2.7.0.jar
Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11)
[Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)] on java1.8.0_60
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.getpid()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'getpid'
>>> quit()

Ran:

$ java -cp jython-2.7.0.jar org.python.util.jython -vv <<< "import os; os.getpid()"

Relevant message:

import: trying org.python.modules.posix.PosixModule as builtin modules in SysPathJavaLoader
import: 'posix' as org.python.modules.posix.PosixModule in builtin modules
Failed to load native POSIX impl; falling back on Java impl. Stacktrace follows.
java.lang.NoClassDefFoundError: Could not initialize class jnr.posix.POSIXFactory$DefaultLibCProvider$SingletonHolder

Looking at the jar that doesn't work:

$ unzip -l jython-2.7.0.jar | grep POSIXFactory
     1022  2014-12-12 09:23   jnr/posix/POSIXFactory$1.class
      894  2014-12-12 09:23   jnr/posix/POSIXFactory$DefaultLibCProvider$SingletonHolder.class
      774  2014-12-12 09:23   jnr/posix/POSIXFactory$DefaultLibCProvider.class
     6341  2014-12-12 09:23   jnr/posix/POSIXFactory.class

And the last one that did:

$ unzip -l jython-2.7.0b3.jar | grep POSIXFactory
      807  2012-12-02 22:32   jnr/posix/POSIXFactory$1.class
      862  2012-12-02 22:32   jnr/posix/POSIXFactory$DefaultLibCProvider$SingletonHolder.class
      833  2012-12-02 22:32   jnr/posix/POSIXFactory$DefaultLibCProvider.class
      863  2012-12-02 22:32   jnr/posix/POSIXFactory$LinuxLibCProvider$SingletonHolder.class
      825  2012-12-02 22:32   jnr/posix/POSIXFactory$LinuxLibCProvider.class
      869  2012-12-02 22:32   jnr/posix/POSIXFactory$SolarisLibCProvider$SingletonHolder.class
      833  2012-12-02 22:32   jnr/posix/POSIXFactory$SolarisLibCProvider.class
      887  2012-12-02 22:32   jnr/posix/POSIXFactory$WindowsLibCProvider$SingletonHolder.class
     1522  2012-12-02 22:32   jnr/posix/POSIXFactory$WindowsLibCProvider.class
     3660  2012-12-02 22:32   jnr/posix/POSIXFactory.class

Pushing those classes into the new jar allows jython to load the native posix implementation again:

$ cp jython-2.7.0.jar jython-2.7.0-berg.jar
$ mkdir jython-2.7.0b3
$ cd jython-2.7.0b3
$ unzip ../jython-2.7.0b3.jar
$ jar uf ../jython-2.7.0-berg.jar jnr/posix/*
$ cd ..
$ java -cp jython-2.7.0-berg.jar org.python.util.jython <<< "import os; os.getpid()"                           
15152

Interestingly, this happens with Java 8 (both OpenJDK and Oracle) but not with an old Java 7 install on the box.

$ java -version
openjdk version "1.8.0-internal"
OpenJDK Runtime Environment (build 1.8.0-internal-build_2015_03_04_15_08-b00)
OpenJDK 64-Bit Server VM (build 25.40-b25, mixed mode)
$ java -cp jython-2.7.0.jar org.python.util.jython <<< "import os; os.getpid()"                          
Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11) 
[OpenJDK 64-Bit Server VM (Oracle Corporation)] on java1.8.0-internal
Type "help", "copyright", "credits" or "license" for more information.
>>> import os; os.getpid()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'getpid'

$ /usr/bin/java -version
java version "1.7.0_79"
OpenJDK Runtime Environment (IcedTea 2.5.6) (7u79-2.5.6-0ubuntu1.14.04.1)
OpenJDK 64-Bit Server VM (build 24.79-b02, mixed mode)
$ /usr/bin/java -cp jython-2.7.0.jar org.python.util.jython <<< "import os; os.getpid()"
16639

Guys thanks for the detailed writeup guys. Most peculiar and i guess related to this: http://bugs.jython.org/issue2110

Prepared a new version of standalone from trunk with much newer jni-posix-3.0.17.jar and friends. It's a bit of a shot in the dark, but please try and let me know the outcomes: https://s3-eu-west-1.amazonaws.com/darjus/jython/jython-standalone-Issue2405.jar

That fixed it:

root@q1:/tmp# /opt/zimbra/java/bin/java -jar jython-standalone-Issue2405.jar
Jython 2.7.1 (default:9adfcf913dff+, Oct 8 2015, 22:13:44)
[OpenJDK 64-Bit Server VM (Oracle Corporation)] on java1.8.0-internal
Type "help", "copyright", "credits" or "license" for more information.
>>> import posix
>>> posix.getpid()
1759
>>> import os
>>> os.getpid()
1759
>>> quit()


Is there any chance that there will be a re-release of the broken 2.7.0 standalone jar?

Good to know this addresses the problem. Let's update in trunk so this fix can be part of the forthcoming release of 2.7.1, due in November.

I also need to send a notice re 2.7.1 beta 2 - the original schedule I posted was for today, but that's obviously slipped. I expect we will do it in 2 weeks.

Glad to hear this solved the problem, should be released for 2.7.1.