Issue2610

As of now we are using 2.5.3 version, there is not issues with that but when we migrate it to 2.7.0 we are getting the following error on confluence macro, please help us to resolve the issue.

Error rendering macro 'jython' : Unexpected program error: ImportError: Cannot import site module and its dependencies: No module named site Determine if the following attributes are correct: * sys.path: ['__classpath__', '__pyclasspath__/'] This attribute might be including the wrong directories, such as from CPython * sys.prefix: null This attribute is set by the system property python.home, although it can be often automatically determined by the location of the Jython jar file You can use the -S option or python.import.site=false to not import the site module

Thanks for upgrading! There's a 2.7.1 out now, which you should also consider.

As the message says, sys.prefix is null and (probably as a consequence) sys.path does not contain the library. The question is why, but there's not enough here to answer that. When I run jython at the prompt, the values I see are:

> jython
Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11)
[Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)] on java1.8.0_131
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.prefix
'C:\\Jython\\2.7.0'
>>> sys.path
['', 'C:\\Jython\\2.7.0\\Lib', '__classpath__', '__pyclasspath__/', 'C:\\Jython\\2.7.0\\Lib\\site-packages']

Possibly something in the environment (e.g. JYTHON_HOME) is pointing Jython at an invalid location, where it does not find its JAR file, etc., and therefore won't believe is its home location.

When you say "confluence macro", are we talking about: https://bobswift.atlassian.net/wiki/display/SCRP/Jython+Macro ? I didn't know you could do that! It should be possible to investigate the environment seen by Jython from there, and compare it with what you get at the user prompt, and what I show above as normal.

Tell us how you get on.

Thanks for the response. We will check and get back to you.

We ran the commands you have suggested and the below is the response for the same. Can you check is there anything missing?

$ jython
Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11) 
[Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)] on java1.8.0_45
Type "help", "copyright", "credits" or "license" for more information.
>>> 
>>> import sys
>>> sys.prefix
'/Applications/atlassian/jython'
>>> sys.path
['', '/Applications/atlassian/jython/bin', '/Applications/atlassian/jython/Lib', '__classpath__', '__pyclasspath__/', '/Applications/atlassian/jython/Lib/site-packages']

Yes we are talking about https://bobswift.atlassian.net/wiki/display/SCRP/Jython+Macro and found that we are getting this exception while initializing PythonInterpreter object. But this is not the case with 2.5.3, there is no environment changes/configuration changes after this. If I revert back to 2.5.3 it is just working fine.
PythonInterpreter interpreter = interpreter = new PythonInterpreter();;

RK: I'm glad you were able to run that at the interactive prompt. The output is what I would expect and this suggests the installation is basically good.

However, you'll notice that interactively you didn't get a message about failing to import site, that you quote in your first post, and that there sys.prefix and sys.path are different. The messaage is from this bit of code: https://hg.python.org/jython/file/412a8f9445f7/src/org/python/core/Py.java#l1563

The difference is therefore down to the environment in which Jython is launched by Confluence. Something about that environment causes Jython not to find its home directory. Jython uses this code to find home: https://hg.python.org/jython/file/412a8f9445f7/src/org/python/core/PySystemState.java#l709

Is it possible to define the propery python.home in the JVM that runs Confluence?

I appreciate you haven't changed the environment from 2.5.3, but there's quite an elapsed period between the two, and Jython evolves. It looks like Bob Swift's scripting plug-in has only advanced to Jython 2.5.3. It would be good to see it move to 2.7.1. Do you represent the developer, or are you just a user of it?

Jeff Allen: I am a developer and Thanks for the response,

Jython 2.7.1 seems to be not in the maven repo, however we were able to set 2.7.1 by using the instructions mentioned at https://wiki.python.org/jython/InstallationInstructions#jython-2-7-1. 

After installing we executed the same command again and the following is the result.
$ echo $PYTHON_HOME
/Applications/atlassian/jython
$ jython --version
Jython 2.7.1
$ jython
[appfire@firebird-confluence Lib]$ echo $PYTHON_HOME
/Applications/atlassian/jython
[appfire@firebird-confluence Lib]$ jython --version
Jython 2.7.1
[appfire@firebird-confluence Lib]$ jython
Jython 2.7.1 (default:0df7adb1b397, Jun 30 2017, 19:02:43) 
[Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)] on java1.8.0_45
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.prefix
'/Applications/atlassian/jython'
>>> sys.path
['', '/Applications/atlassian/jython/bin', '/Applications/atlassian/jython/Lib', '__classpath__', '__pyclasspath__/', '/Applications/atlassian/jython/Lib/site-packages']

We are using PythonInterpreter class to execute the code in our macro. And as I said earlier after moving to 2.7.0 we are getting an error while initializing PythonInterpreter object.Not sure what is going wrong from our end, can you please check the above details and suggest.

It's a good sanity check that jython at the prompt gets the right sys.path.

The real question is why your PythonInterpreter gets an incomplete sys.path, *in your Confluence plug-in*.

Here (https://github.com/jythontools/jython/issues/78#issuecomment-309899489) is a simple example in which a Java application creates a PythonInterpreter, and you can see that it gets a working sys.path. (Don't worry about the rest of yiranxijie's issue.)

In your plug-in, or Confluence, I suggest something has prepared the JVM environment incorrectly for the current version of Jython. We've addressed a lot of minor problems to do with launching Jython since 2.5.3 so it would not be surprising to find that your expectations or work-arounds built on 2.5.3 are incorrect for 2.7. Initialisation is driven largely by the contents of the Java properties and some environment variables, so looking at the contents of System.getProperties(), and System.getenv() just before you create the interpreter ought to help you.

Although jython launched at the prompt *is not the place to look for your problem*, it does have some things to help you. One is the --print option in which you can see the java command that would be issued to run Jython proper, and the other is --help that tells you what system properties and environment variables Jython looks at. And finally you can inspect environemnt and properties at the prompt. Compare that with what you get from within your application, just before you (fail to) create the interpreter.

Sorry for the delay in response. 

I tried the sample example given and it is working fine for both 2.5.3 and 2.7.0, but when it comes to add-on it is failing at the instantiation stage.

Did the console output with System.getProperties() and System.getenv() before initializing an object 
and compared both the outputs for 2.5.3 and 2.7.0 and found both the output's are same no change. 

--print option gives the following. 
$ jython --print
/sde/jdk/jdk1.8.0_45/bin/java -Xmx512m -Xss2560k -classpath ../../Applications/atlassian/jython/jython.jar:. -Dpython.home=../../Applications/atlassian/jython -Dpython.executable=../../Applications/atlassian/jython/bin/jython -Dpython.launcher.uname=linux -Dpython.launcher.tty=true org.python.util.jython

are we missing anything still, please check. As per your suggestion seems to be our confluence add-on problem as local is working fine, but not sure where are we missing it and why it is working for 2.5.3 and not for 2.7.0. still a confusion.

What is in System.getProperties(), just before you create the PythonInterpreter, *in your Confluence plug-in*?

From what you write, I am not sure this is what you have looked at.

You can see from the output of jython --print that we define some properties (in the -D... switches). In 2.7.x we use these properties to tell Jython where it is installed. In 2.5.3 Jython tries to work it out, but it wasn't reliable for everyone.

My guess is that you need to set some of the same values up for your plug-in. You can do this either in the java command that launches Confluence, or later in the application/plug-in.

As per your query here is the System.getProperties() output just before initializing PythonInterpreter in our confluence add-on.

{java.vendor=Oracle Corporation, sun.java.launcher=SUN_STANDARD, catalina.base=/Applications/atlassian/confluence-5.10.2, sun.management.compiler=HotSpot 64-Bit Tiered Compilers, catalina.useNaming=true, os.name=Linux, sun.boot.class.path=/sde/jdk/jdk1.8.0_45/jre/lib/resources.jar:/sde/jdk/jdk1.8.0_45/jre/lib/rt.jar:/sde/jdk/jdk1.8.0_45/jre/lib/sunrsasign.jar:/sde/jdk/jdk1.8.0_45/jre/lib/jsse.jar:/sde/jdk/jdk1.8.0_45/jre/lib/jce.jar:/sde/jdk/jdk1.8.0_45/jre/lib/charsets.jar:/sde/jdk/jdk1.8.0_45/jre/lib/jfr.jar:/sde/jdk/jdk1.8.0_45/jre/classes, java.util.logging.config.file=/Applications/atlassian/confluence-5.10.2/conf/logging.properties, java.vm.specification.vendor=Oracle Corporation, atlassian.plugins.enable.wait=300, java.runtime.version=1.8.0_45-b14, user.name=appfire, tomcat.util.scan.StandardJarScanFilter.jarsToScan=log4j-core*.jar,log4j-taglib*.jar,log4javascript*.jar,slf4j-taglib*.jar, shared.loader=, tomcat.util.buf.StringCache.byte.enabled=true, python.console.encoding=UTF-8, user.language=en, java.naming.factory.initial=org.apache.naming.java.javaURLContextFactory, sun.boot.library.path=/sde/jdk/jdk1.8.0_45/jre/lib/amd64, atlassian.enable.spring.strong.cache.bean.metadata=true, jdk.tls.ephemeralDHKeySize=2048, java.version=1.8.0_45, java.util.logging.manager=org.apache.juli.ClassLoaderLogManager, python.path=/Applications/atlassian/jython/bin, user.timezone=America/New_York, sun.arch.data.model=64, atlassian.enable.spring.strong.cache.bean.metadata.flush=true, java.endorsed.dirs=/Applications/atlassian/confluence-5.10.2/endorsed, sun.cpu.isalist=, sun.jnu.encoding=UTF-8, file.encoding.pkg=sun.io, package.access=sun.,org.apache.catalina.,org.apache.coyote.,org.apache.jasper.,org.apache.tomcat., file.separator=/, java.specification.name=Java Platform API Specification, java.class.version=52.0, user.country=US, atlassian.org.osgi.framework.bootdelegation.extra=org.apache.lucene.*, java.home=/sde/jdk/jdk1.8.0_45/jre, java.vm.info=mixed mode, os.version=3.10.0-229.el7.x86_64, sun.font.fontmanager=sun.awt.X11FontManager, path.separator=:, java.vm.version=25.45-b02, java.awt.printerjob=sun.print.PSPrinterJob, sun.io.unicode.encoding=UnicodeLittle, sun.java2d.opengl=true, awt.toolkit=sun.awt.X11.XToolkit, package.definition=sun.,java.,org.apache.catalina.,org.apache.coyote.,org.apache.jasper.,org.apache.naming.,org.apache.tomcat., java.naming.factory.url.pkgs=org.apache.naming, user.home=/home/appfire, java.specification.vendor=Oracle Corporation, tomcat.util.scan.StandardJarScanFilter.jarsToSkip=bootstrap.jar,commons-daemon.jar,tomcat-juli.jar,annotations-api.jar,el-api.jar,jsp-api.jar,servlet-api.jar,websocket-api.jar,catalina.jar,catalina-ant.jar,catalina-ha.jar,catalina-storeconfig.jar,catalina-tribes.jar,jasper.jar,jasper-el.jar,ecj-*.jar,tomcat-api.jar,tomcat-util.jar,tomcat-util-scan.jar,tomcat-coyote.jar,tomcat-dbcp.jar,tomcat-jni.jar,tomcat-websocket.jar,tomcat-i18n-en.jar,tomcat-i18n-es.jar,tomcat-i18n-fr.jar,tomcat-i18n-ja.jar,tomcat-juli-adapters.jar,catalina-jmx-remote.jar,catalina-ws.jar,tomcat-jdbc.jar,tools.jar,commons-beanutils*.jar,commons-codec*.jar,commons-collections*.jar,commons-dbcp*.jar,commons-digester*.jar,commons-fileupload*.jar,commons-httpclient*.jar,commons-io*.jar,commons-lang*.jar,commons-logging*.jar,commons-math*.jar,commons-pool*.jar,jstl.jar,taglibs-standard-spec-*.jar,geronimo-spec-jaxrpc*.jar,wsdl4j*.jar,ant.jar,ant-junit*.jar,aspectj*.jar,jmx.jar,h2*.jar,hibernate*.jar,httpclient*.jar,jmx-tools.jar,jta*.jar,log4j*.jar,mail*.jar,slf4j*.jar,xercesImpl.jar,xmlParserAPIs.jar,xml-apis.jar,junit.jar,junit-*.jar,ant-launcher.jar,cobertura-*.jar,asm-*.jar,dom4j-*.jar,icu4j-*.jar,jaxen-*.jar,jdom-*.jar,jetty-*.jar,oro-*.jar,servlet-api-*.jar,tagsoup-*.jar,xmlParserAPIs-*.jar,xom-*.jar, java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib, java.vendor.url=http://java.oracle.com/, java.vm.vendor=Oracle Corporation, common.loader="${catalina.base}/lib","${catalina.base}/lib/*.jar","${catalina.home}/lib","${catalina.home}/lib/*.jar", java.runtime.name=Java(TM) SE Runtime Environment, sun.java.command=org.apache.catalina.startup.Bootstrap start, java.class.path=/Applications/atlassian/confluence-5.10.2/bin/bootstrap.jar:/Applications/atlassian/confluence-5.10.2/bin/tomcat-juli.jar, java.vm.specification.name=Java Virtual Machine Specification, java.vm.specification.version=1.8, catalina.home=/Applications/atlassian/confluence-5.10.2, sun.cpu.endian=little, sun.os.patch.level=unknown, java.awt.headless=true, java.io.tmpdir=/Applications/atlassian/confluence-5.10.2/temp, java.vendor.url.bug=http://bugreport.sun.com/bugreport/, server.loader=, os.arch=amd64, java.awt.graphicsenv=sun.awt.X11GraphicsEnvironment, java.ext.dirs=/sde/jdk/jdk1.8.0_45/jre/lib/ext:/usr/java/packages/lib/ext, user.dir=/Applications/atlassian/confluence-5.10.2/bin, line.separator=
, java.vm.name=Java HotSpot(TM) 64-Bit Server VM, file.encoding=UTF-8, java.specification.version=1.8, plugin.webresource.javascript.try.catch.wrapping=true}

Excellent. The parts of this we care about for now are:
 'python.console.encoding': 'UTF-8',
 'python.path': '/Applications/atlassian/jython/bin',

You will see from --print that our launcher sets up a few other values:

python.home=/Applications/atlassian/jython
python.executable=/Applications/atlassian/jython/bin/jython
python.launcher.uname=linux

I suggest you set those in system properties for your application, in the java command or elsewhere. You probably don't need to set python.launcher.tty, or should set it to "false" as you don't have an interactive console.

This should allow it to get sys.prefix right, sys.path right and then I hope everything else will follow. Let us know if that solves it. If not, we'll think again.

We tried to the values before setting them, but the values are already in synch with what you are suggested us to update. Please find below commands we tried. Let me know are we doing anything wrong here. 

$ pwd
/Applications/atlassian/jython

$ bin/jython --print
/sde/jdk/jdk1.8.0_45/bin/java -Xmx512m -Xss2560k -classpath ./jython.jar:. -Dpython.home=. -Dpython.executable=bin/jython -Dpython.launcher.uname=linux -Dpython.launcher.tty=true org.python.util.jython

Thanks
RK

After the changes you made, what are the values in System.getProperties(), for:

python.home
python.executable
python.launcher.uname

just before you create the PythonInterpreter, *in your Confluence plug-in*?

RK:

I notice from support guides like this one: https://confluence.atlassian.com/confkb/how-to-point-to-a-different-java-installation-598213326.html, that one can easily set command-line arguments to java for the launch of Confluence. This might be the easiest place to set (using the -D option):
python.home
python.executable
python.launcher.uname

In the released product, you would want working default values for these to be set up on installation.

Closing this as it is not an issue in Jython, and a decent interval has elspased since our last attempt to help.