Issue2225

classification
Title: Jython+django-jython - no module named site
Type: behaviour Severity: normal
Components: Library Versions: Jython 2.7
Milestone:
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: amak Nosy List: DHELIO, alex.gronholm, amak, strider, valtron
Priority: Keywords:

Created on 2014-10-30.15:51:32 by valtron, last changed 2015-01-21.10:14:40 by strider.

Files
File name Uploaded Description Edit Remove
jython.exception.txt DHELIO, 2014-11-07.19:08:15 Java Exceptions after the workaround for the ImportError
Messages
msg9187 (view) Author: (valtron) Date: 2014-10-30.15:51:31
Same problem as http://bugs.jython.org/issue1629, but has resurfaced from a different cause. In this case, I think it's due to this commit: https://hg.python.org/jython/rev/51b28cc2c43d.

What happens:
- ModjyJServlet.init: creates a new interpreter, and then calls setupEnvironment
- creating a new interpreter calls imp.load("site") because of Options.importSite
- this throws "ImportError: No module named site"
- from the fix to #1629, it looks like processPythonLib has to be called before the "site" package can be imported...but that method needs an interpreter, whose instantiation tries to load "site", so it's a kind of circular problem

Maybe I need to set the importSite option to false; but I'm not sure how to do this in a war file, especially since that option isn't read from the registry.

Here's the traceback I get (running the war file in tomcat):

ImportError: No module named site

	at org.python.core.Py.ImportError(Py.java:300)
	at org.python.core.imp.import_first(imp.java:760)
	at org.python.core.imp.load(imp.java:633)
	at org.python.util.PythonInterpreter.<init>(PythonInterpreter.java:126)
	at org.python.util.PythonInterpreter.<init>(PythonInterpreter.java:104)
	at com.xhaus.modjy.ModjyJServlet.init(ModjyJServlet.java:108)
	at javax.servlet.GenericServlet.init(GenericServlet.java:158)
	at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1284)
	at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1197)
	at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1087)
	at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5231)
	at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5518)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1575)
	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1565)
	at java.util.concurrent.FutureTask.run(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)
msg9191 (view) Author: David (DHELIO) Date: 2014-11-07.19:08:14
I've got the same problem using tomcat and by just trying the modjy_webapp example.  I managed to solve the "ImportError: No module named site" by copying all the jython/Lib directory into modjy_webapp/WEB-INF/lib-python. Whatever combination I tried to change on the web.xml defining python.home was not successful.

Once the ImportError was gone then I got Java Exception, and these I don't really understand.  The traceback I've got is attached.
msg9197 (view) Author: Alan Kennedy (amak) Date: 2014-11-09.16:51:12
I can easily reproduce this issue. You were right to identify the root cause as this checkin

https://hg.python.org/jython/rev/51b28cc2c43d

I will checking in a fix shortly.
msg9198 (view) Author: Alan Kennedy (amak) Date: 2014-11-09.18:04:16
Fix checked in at

https://hg.python.org/jython/rev/fcd4dfc05813
msg9199 (view) Author: Alan Kennedy (amak) Date: 2014-11-09.18:05:01
Closing this issue: please re-open if you experience issues with the fix.

Thanks for reporting this issue.
msg9206 (view) Author: David (DHELIO) Date: 2014-11-12.19:23:10
Alan, I've downloaded and compiled the latest version that includes your changes from the mercurial repository and still have the same issue. Let me know if there's some log file that would be helpful to solve the problem.
msg9207 (view) Author: Alex Grönholm (alex.gronholm) Date: 2014-11-14.16:42:12
Reopening since DHELIO experienced issues.
msg9208 (view) Author: Alan Kennedy (amak) Date: 2014-11-14.17:14:49
Hi David,

Yes, I would need more details in order to determine what is going on.

What version of Java are you using?
What revision number of Jython?
What servlet container are you using?

The log file containing further information will vary by container.
msg9211 (view) Author: David (DHELIO) Date: 2014-11-17.11:07:33
Hi Alan,

> What version of Java are you using?

$ java -version
java version "1.7.0_65"
OpenJDK Runtime Environment (fedora-2.5.2.5.fc20-x86_64 u65-b17)
OpenJDK 64-Bit Server VM (build 24.65-b04, mixed mode)

> What revision number of Jython?

I compiled from your last contribution: 7412:fcd4dfc05813
using (as suggested in the IRC): 
$ ant jar-complete

> What servlet container are you using?

Apache Tomcat 7.0.57

The steps to replicate the problem is:
untar tomcat, copy the demo modjy_webapp from jython to the webapps directory, and copy the jython.jar and python libraries (Lib) in the respective directories getting the following:

tomcat/webapps/modjy_webapp/
├ demo_app.py
└── WEB-INF
    ├── lib
    │   ├── jython.jar
    │   └── readme.txt
    ├── lib-python
    │   ├── _abcoll.py
    │   ├── ...
    │   ├── site.py
    │   ├── ...
    │   └── zlib.py
    └ web.xml

> The log file containing further information will vary by container.

The traceback I get when browsing to the example is the following:

exception

javax.servlet.ServletException: Exception creating modjy servlet: ImportError: No module named site

	com.xhaus.modjy.ModjyJServlet.init(ModjyJServlet.java:127)
	javax.servlet.GenericServlet.init(GenericServlet.java:158)
	org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:503)
 	org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
	org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
	org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421)
	org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1070)
	org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611)
	org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
	java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	java.lang.Thread.run(Thread.java:745)

root cause

ImportError: No module named site

	org.python.core.Py.ImportError(Py.java:315)
	org.python.core.imp.import_first(imp.java:760)
	org.python.core.imp.load(imp.java:633)
	org.python.util.PythonInterpreter.<init>(PythonInterpreter.java:126)
	org.python.util.PythonInterpreter.<init>(PythonInterpreter.java:104)
	com.xhaus.modjy.ModjyJServlet.init(ModjyJServlet.java:111)
	javax.servlet.GenericServlet.init(GenericServlet.java:158)
	org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:503)
	org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
	org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
	org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421)
	org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1070)
	org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611)
	org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
	java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	java.lang.Thread.run(Thread.java:745)


I've tried to look in all the logs files I've got and I've seen also the following in logs/localhost.2014-11-17.log:


Nov 17, 2014 12:37:53 PM org.apache.catalina.core.StandardContext loadOnStartup
SEVERE: Servlet /modjy_webapp threw load() exception
ImportError: No module named site

        at org.python.core.Py.ImportError(Py.java:315)
        at org.python.core.imp.import_first(imp.java:760)
        at org.python.core.imp.load(imp.java:633)
        at org.python.util.PythonInterpreter.<init>(PythonInterpreter.java:126)
        at org.python.util.PythonInterpreter.<init>(PythonInterpreter.java:104)
        at com.xhaus.modjy.ModjyJServlet.init(ModjyJServlet.java:111)
        at javax.servlet.GenericServlet.init(GenericServlet.java:158)
        at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1284)
        at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1197)
        at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1087)
        at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5231)
        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5518)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
        at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:901)
        at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:877)
        at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:649)
        at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1245)
        at org.apache.catalina.startup.HostConfig$DeployDirectory.run(HostConfig.java:1895)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)

Nov 17, 2014 12:38:06 PM org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: Allocate exception for servlet modjy
ImportError: No module named site

        at org.python.core.Py.ImportError(Py.java:315)
        at org.python.core.imp.import_first(imp.java:760)
        at org.python.core.imp.load(imp.java:633)
        at org.python.util.PythonInterpreter.<init>(PythonInterpreter.java:126)
        at org.python.util.PythonInterpreter.<init>(PythonInterpreter.java:104)
        at com.xhaus.modjy.ModjyJServlet.init(ModjyJServlet.java:111)
        at javax.servlet.GenericServlet.init(GenericServlet.java:158)
        at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1284)
        at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1197)
        at org.apache.catalina.core.StandardWrapper.allocate(StandardWrapper.java:864)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:134)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:503)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
        at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421)
        at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1070)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611)
        at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:745)


I think that's all I've got.  Thank you!! and let me know if I can run some other test (I'm also at the IRC channel).
msg9213 (view) Author: Alan Kennedy (amak) Date: 2014-11-19.21:56:13
Sorry for the delay in reply: I appear to not be receiving email updates for this issue.

Last thing I need to know is content of your web.xml.

Particularly, I am interesting in knowing

1. What is your setting for python.home? Does it point to a jython installation?

2. What is your setting for load_site_packages?
msg9216 (view) Author: David (DHELIO) Date: 2014-11-21.14:51:08
Alan!!! it works!!!!! :)

Sorry for the last reply, but I hadn't had a minute till now to play with it.  Your question make me think in detail what I could be doing wrong, re-did the whole thing from scratch, taking special care in all the steps - probably this is to obvious for all of you, but just in case someone gets here with the same problem.

1.- build jython.jar with: ant jar-complete
2.- build the installer: ant -f maven/build.xml
3.- copy the installer to the server machine and install it; it took me a while to realize the bundle was not the installer, and I had to unzip it first (otherwise I was getting "no main manifest attribute, in jython-installer-2.7.0-SNAPSHOT-bundle.jar")
4.- copy the demo modjy_webapp to the webapps directory in tomcat.
5.- follow the instructions in the modjy_webapp/readme.txt
   - copy from the installation: ~/jython/jythor.jar to webapps/modjy_webapp/WEB-INF/lib
   - copy also the python libraries: ~/jython/Lib/* to webapps/modjy_webapp/WEB-INF/lib-python  # I think this is not strictly needed
6.- change in modjy_webapp/WEB-INF/web.xml the following variable:
   - python.home -> /home/username/jython # Do not put ~/jython as this won't work!!!

And then it should work.

Allan, thank you again, and I'm really sorry that all this time it was my mistake.
msg9218 (view) Author: Alan Kennedy (amak) Date: 2014-11-21.17:29:06
Glad you got it sorted.

Closing this issue as fixed.
msg9428 (view) Author: Dmitry Bazhal (strider) Date: 2015-01-20.14:09:19
Good day good sires.
I'm new to jython(experienced only with cpython), and am trying to launch simple django project.
At the moment I'm having trouble with mentioned error
"ImportError: No module named site"
when I try to set up application to tomcat.

Here is my info:
Apache Tomcat/7.0.52 (Ubuntu)
JVM: SUN 1.8.0_25-b17
Sys arch: 3.13.0-24-generic	amd64
Jython v2.7b4+ (compiled by DHELIO step-by-step instruction(thanks for instructions, mate) from latest version - 7542:88209128cde8)

At the end i have next app structure:
.
├── WEB-INF
│   ├── lib
│   │   ├── jruby-extras-fileservlet.jar
│   │   ├── jython.jar
│   │   ├── mysql-connector-java-5.1.34-bin.jar
│   │   └── README
│   ├── lib-python
│   │   ├── _abcoll.py
│   │   ├── _abcoll$py.class
│   │   ├── abc.py
│   │   ├── abc$py.class
│   │   ├── aifc.py
│   │   ├── aifc$py.class
│   │   ├── anydbm.py
│   │   ├── anydbm$py.class
│   │   ├── application_settings.py
│   │   └── <here more Lib/ contents, site-packages, my app, etc.>
│   └── web.xml
└── wsgi.py

Inside web.xml I've default contents made by 
jython manage.py buildwar --include-java-libs=<jdbc>:<jython> --base-dir=<app_dir>
except lines you've mentioned:
        <init-param>
            <param-name>python.home</param-name>
            <param-value>/usr/share/tomcat7/jython</param-value>
        </init-param>
and inside that dir jython is installed with it's installer.

I'll keep trying debug it myself, but I'm not strong at it with such environment(java, tomcat, etc.), so I ask for your help. What I could've done wrong? Any suggestions will be appreciated.
msg9434 (view) Author: Dmitry Bazhal (strider) Date: 2015-01-21.10:14:40
UPD:
If I set "load_site_packages" to "0", I get next exception:
SEVERE: Servlet /test_project threw load() exception
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/var/lib/tomcat7/webapps/test_project/WEB-INF/lib-python/modjy/modjy.py", line 30, in <module>
    from modjy_exceptions import *
  File "/var/lib/tomcat7/webapps/test_project/WEB-INF/lib-python/modjy/modjy_exceptions.py", line 23, in <module>
    import traceback
  File "/var/lib/tomcat7/webapps/test_project/WEB-INF/lib-python/traceback.py", line 3, in <module>
    import linecache
  File "/var/lib/tomcat7/webapps/test_project/WEB-INF/lib-python/linecache.py", line 9, in <module>
    import os
  File "/var/lib/tomcat7/webapps/test_project/WEB-INF/lib-python/os.py", line 725, in <module>
    from subprocess import _os_system as system
  File "/var/lib/tomcat7/webapps/test_project/WEB-INF/lib-python/subprocess.py", line 398, in <module>
    import signal
  File "/var/lib/tomcat7/webapps/test_project/WEB-INF/lib-python/signal.py", line 41, in <module>
    import threading
  File "/var/lib/tomcat7/webapps/test_project/WEB-INF/lib-python/threading.py", line 13, in <module>
    from traceback import print_exc as _print_exc
ImportError: cannot import name print_exc

        at org.python.core.Py.ImportError(Py.java:326)
History
Date User Action Args
2015-01-21 10:14:40stridersetmessages: + msg9434
2015-01-20 14:09:20stridersetnosy: + strider
messages: + msg9428
2014-11-21 17:29:06amaksetstatus: open -> closed
resolution: fixed
messages: + msg9218
2014-11-21 14:51:09DHELIOsetmessages: + msg9216
2014-11-19 21:56:14amaksetmessages: + msg9213
2014-11-17 11:07:35DHELIOsetmessages: + msg9211
2014-11-14 17:14:49amaksetmessages: + msg9208
2014-11-14 16:42:13alex.gronholmsetstatus: closed -> open
resolution: fixed -> (no value)
messages: + msg9207
nosy: + alex.gronholm
2014-11-12 19:23:10DHELIOsetmessages: + msg9206
2014-11-09 18:05:01amaksetstatus: open -> closed
resolution: fixed
messages: + msg9199
2014-11-09 18:04:16amaksetmessages: + msg9198
2014-11-09 16:51:12amaksetmessages: + msg9197
2014-11-07 19:08:16DHELIOsetfiles: + jython.exception.txt
nosy: + DHELIO
messages: + msg9191
2014-11-01 18:13:53amaksetassignee: amak
nosy: + amak
2014-10-30 15:51:32valtroncreate