Issue2721

Here is the stack-trace when running the below Java program to invoke simple python code:
************
Exception in thread "main" Traceback (most recent call last):
  File "/Users/<user>/Downloads/jython-standalone-2.7.1.jar/Lib/site.py", line 68, in <module>
  File "/Users/<user>/Downloads/jython-standalone-2.7.1.jar/Lib/os.py", line 426, in <module>
java.lang.IllegalAccessError: tried to access method org.python.core.PyList.getList()Ljava/util/List; from class org.python.core.PyShadowString
	at org.python.core.PyShadowString.isTarget(PyShadowString.java:108)
	at org.python.core.PyShadowString.__eq__(PyShadowString.java:170)
	at org.python.core.PyObject._eq(PyObject.java:1525)
	at org.python.core.PyObject.equals(PyObject.java:318)
	at org.python.core.PyObject.object___contains__(PyObject.java:1769)
	at org.python.core.PyObject.__contains__(PyObject.java:1764)
	at org.python.core.PyObject._in(PyObject.java:1744)
	at os$py.f$0(/Users/<user>/Downloads/jython-standalone 2.7.1.jar/Lib/os.py:726)
	at os$py.call_function(/Users/<user>/Downloads/jython-standalone-2.7.1.jar/Lib/os.py)
	at org.python.core.PyTableCode.call(PyTableCode.java:167)
	at org.python.core.PyCode.call(PyCode.java:18)
	at org.python.core.imp.createFromCode(imp.java:436)
	at org.python.core.util.importer.importer_load_module(importer.java:109)
	at org.python.modules.zipimport.zipimporter.zipimporter_load_module(zipimporter.java:163)
	at org.python.modules.zipimport.zipimporter$zipimporter_load_module_exposer.__call__(Unknown Source)
	at org.python.core.PyBuiltinMethodNarrow.__call__(PyBuiltinMethodNarrow.java:46)
	at org.python.core.imp.loadFromLoader(imp.java:587)
	at org.python.core.imp.find_module(imp.java:537)
	at org.python.core.imp.import_next(imp.java:840)
	at org.python.core.imp.import_module_level(imp.java:959)
	at org.python.core.imp.importName(imp.java:1062)
	at org.python.core.ImportFunction.__call__(__builtin__.java:1280)
	at org.python.core.PyObject.__call__(PyObject.java:431)
	at org.python.core.__builtin__.__import__(__builtin__.java:1232)
	at org.python.core.imp.importOne(imp.java:1081)
	at site$py.f$0(/Users/<user>/Downloads/jython-standalone-2.7.1.jar/Lib/site.py:637)
	at site$py.call_function(/Users/<user>/Downloads/jython-standalone-2.7.1.jar/Lib/site.py)
	at org.python.core.PyTableCode.call(PyTableCode.java:167)
	at org.python.core.PyCode.call(PyCode.java:18)
	at org.python.core.imp.createFromCode(imp.java:436)
	at org.python.core.util.importer.importer_load_module(importer.java:109)
	at org.python.modules.zipimport.zipimporter.zipimporter_load_module(zipimporter.java:163)
	at org.python.modules.zipimport.zipimporter$zipimporter_load_module_exposer.__call__(Unknown Source)
	at org.python.core.PyBuiltinMethodNarrow.__call__(PyBuiltinMethodNarrow.java:46)
	at org.python.core.imp.loadFromLoader(imp.java:587)
	at org.python.core.imp.find_module(imp.java:537)
	at org.python.core.imp.import_next(imp.java:840)
	at org.python.core.imp.import_first(imp.java:861)
	at org.python.core.imp.load(imp.java:716)
	at org.python.core.Py.importSiteIfSelected(Py.java:1558)
	at org.python.util.PythonInterpreter.<init>(PythonInterpreter.java:116)
	at org.python.util.PythonInterpreter.<init>(PythonInterpreter.java:94)
	at org.python.util.PythonInterpreter.<init>(PythonInterpreter.java:71)
	at PythonInJava.main(PythonInJava.java:17)
java.lang.IllegalAccessError: java.lang.IllegalAccessError: tried to access method org.python.core.PyList.getList()Ljava/util/List; from class org.python.core.PyShadowString

*********** JAVA ************
import java.util.Properties;
import org.python.util.PythonInterpreter;

public class PythonInJava {

        @SuppressWarnings("resource")
        public static void main(final String[] args) throws Exception {
                System.setProperty("python.cachedir.skip", "true");

                Properties p = new Properties();
                p.setProperty("python.path","/Users/<user>/Downloads/jython-standalone-2.7.1.jar");
                p.setProperty("python.home","/Users/<user>/Downloads/jython-standalone-2.7.1.jar");
                p.setProperty("python.prefix","/Users/<user>/Downloads/jython-standalone-2.7.1.jar");
                PythonInterpreter.initialize(System.getProperties(), p, new String[] {});
                PythonInterpreter interpreter = new PythonInterpreter();
                interpreter.exec("import sys");
                interpreter.exec("sys.path.append(\"/Library/Python/2.7/site-packages\")");
                interpreter.execfile("/Users/<user>/test.py");
        }
}

Are you using Java 9 or higher?
Jython 2.7.1 was released before Java 9 and is not designed to respect the new stricter access rules.

See #2656, #2582, #2680 for discussions about this.

Some work went into Java 9+ support since Jython 2.7.1 release. It would be good if you could check if this solved the problem. This has not been released yet, it would require building from source. Also checking with Java 8 would give helpful insight. If the problem occurs there too we would have to revisit this issue.

What you can try with Java 9: Run with --permit-illegal-access as mentioned in #2582.

I don't think this is the Java 9 / Jigsaw problem. The message is not the same.

This will cause trouble:

                interpreter.exec("sys.path.append(\"/Library/Python/2.7/site-packages\")");

because CPython and Jython just don't mix. And if I count correctly, this is the place the stack dump ends. I'm stumped, though. How can there be any problem with access to PyList.getList() from PyShadowString in the same package, via the same classloader (from the same JAR)?

I believe you need a full installation of Jython and then use the pip it gives you to install the packages you need (which have to be pure Python) in the site-packages of your Jython library.

Thanks Jeff, Stefan for your response.

When you say full Jython installation, guess you are referring to http://www.jython.org/downloads.html where I grab the installer to install 
Jython 2.7.0 and other one is standalone*.jar
Is there anything besides that, I have those on my local.

Then how can I use pip to install above Jython*.jar to site-packages.

Apologies if am asking naive qns due to deep exposure in Java however 2 months of exposure to Python. Thanks again.

@Jeff The java.lang.IllegalAccessError looked so Java 9-ish to me that I wanted to clarify the situation before continuing.

@Navneeth
> Jython 2.7.0 and other one is standalone*.jar
Are you using Jython 2.7.0 installer?
jython.org is outdated and will be replaced by https://jython.github.io.
The downloads section of the new page contains an actual Jython 2.7.1 installer. Use that one. An additional standalone jar is not required. The standalone version is for bundling Jython with your application etc.

@Stefan: somewhat similar, I agree, but have become a close acquaintance :)

@Navneeth: this is not quite the right question:

> Then how can I use pip to install above Jython*.jar to site-packages.

Jython is an implementation of Python in Java, entirely separate (as far as the user is concerned) from CPython, which is what most people mean by "Python". Installing Jython gives you a lay-down of directories much like installing CPython, so it has a Lib/site-packages of its own. You should find pip already installed (in bin and in site-packages). It's a slightly tweaked pip, so don't let it update itself.

Here's mine (on Windows):

PS Jeff> ls C:\Jython\2.7.1\bin
-a----       07/05/2018     16:29          98161 easy_install-2.7.exe
-a----       07/05/2018     16:29          98161 easy_install.exe
-a----       30/06/2017     19:03         800491 jython.exe
-a----       30/06/2017     19:03            282 jython_regrtest.bat
-a----       07/05/2018     16:29          98133 pip.exe
-a----       07/05/2018     16:29          98133 pip2.7.exe
-a----       07/05/2018     16:29          98133 pip2.exe
-a----       30/06/2017     19:03        3407872 python27.dll
-a----       07/05/2018     16:43          98140 virtualenv.exe

PS Jeff> tree C:\Jython\2.7.1\Lib\site-packages
C:\JYTHON\2.7.1\LIB\SITE-PACKAGES
├─pip
│  ├─commands
│  ├─compat
│  ├─models
│  ├─operations
│  ├─req
│  ├─utils
│  ├─vcs
│  └─_vendor
│      ├─cachecontrol
:      :
│      └─webencodings
├─pip-9.0.1.dist-info
├─pkg_resources
│  ├─extern
│  └─_vendor
│      └─packaging
├─setuptools
│  ├─command
│  └─extern
├─setuptools-28.8.0.dist-info
├─virtualenv-15.2.0.dist-info
└─virtualenv_support

You can see I've installed virtualenv (with "pip install virtualenv").

I mentioned site-packages because I could see in your Java that you were adding CPython's  to your sys.path. I deduce this is because you already pip-installed some packages for CPython and you want to use them from Jython. But you can't do that: you have to install them again into your copy of Jython. And they have to include no compiled C (sorry).

For that to go smoothly, you need the Jython bin directory on your path ahead of any competing tools of the same name. A check is:

PS Jeff> pip -V
pip 9.0.1 from C:\Jython\2.7.1\Lib\site-packages (python 2.7)

You can also launch pip as a module:
PS Jeff> jython -m pip list
...
pip (9.0.1)
setuptools (28.8.0)
virtualenv (15.2.0)
You are using pip version 9.0.1, however version 18.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
PS Jeff> 

Remember don't upgrade! (It breaks.)