Message12296

Compilation of large modules (and modules with large methods) operates by spawning a CPython complation command. This works correctly, but no longer in the context of pip install.

Large module and method compilation works only if a path to CPython 2.7 is given via a Java system property:

PS> del large_module.pyc
PS> jython -J-Dcpython_cmd=C:\Python\2.7.15\python.exe large_module.py
PS> dir large_module.*
...
-a----       07/01/2019     20:21        1028129 large_module.py
-a----       08/01/2019     06:55         791318 large_module.pyc

Without the property, this is the result, even with CPython on the PATH:

PS> del large_module.pyc
PS> jython large_module.py
java.lang.RuntimeException:
Encountered too large method code in
large_module.py

Please provide a CPython 2.7 bytecode file (.pyc) to proceed, e.g. run
python -m py_compile large_module.py
and try again.

Alternatively provide proper CPython 2.7 execute command via
cpython_cmd property, e.g. call
    jython -J-Dcpython_cmd=python
or if running pip on Jython:
    pip install --global-option="-J-Dcpython_cmd=python" <package>
...

(On my machine, this advice comes out twice.) That this does not find the CPython available on the PATH may be a defect. But the main point of the issue I'm raising is that advice given with respect to pip no longer works.

PS> pip install --global-option=-J-Dcpython_cmd=C:\Python\2.7.15\python.exe mpmath
...
    error: option -J not recognized

This is the mpmath setup.py replying that it doesn't understand the Java property definition passed to it through --global-option. (See https://pip.pypa.io/en/stable/reference/pip/#build-options.)

Previously -J options given anywhere on the jython command line would (I think) be collected in our command line processing, and fronted, but this was technically incorrect and disappeared with #2686. The options after the executable file argument (-c, -m or a *.py) belongs to that program. The other "obvious" possibility:

PS> jython -Dcpython_cmd=C:\Python\2.7.15\python.exe -m pip install mpmath

fails because it is not the top-level Jython that needs to find CPython, but a subprocess created by pip, and Java properties are not inherited.

Solution ideas include:
1. Restore the non-standard processing of -J (or -J-D) options.
2. Find CPython another way (like PYTHONHOME or PATH).
3. Use a compiler we provide in a reliable location (e.g. a script in ./bin).
4. Advise defining cpython_cmd via JAVA_OPTS or JYTHON_OPTS or the registry.