Issue1891

I tried easy_install Pygments (which is a syntax highlighting library) it occurs "Method code too large!" error.


$ ../../.tox/jython/bin/easy_install Pygments
Searching for Pygments
Reading http://pypi.python.org/simple/Pygments/
Reading http://pygments.org/
Reading http://pygments.pocoo.org/
Best match: Pygments 1.5
Downloading http://pypi.python.org/packages/2.7/P/Pygments/Pygments-1.5-py2.7.egg#md5=8ea02c2eafb2580336b7637be629f7ce
Processing Pygments-1.5-py2.7.egg
creating /.../jython/Lib/site-packages/Pygments-1.5-py2.7.egg
Extracting Pygments-1.5-py2.7.egg to /.../jython/Lib/site-packages
Traceback (most recent call last):
  File ".../jython/bin/easy_install", line 8, in <module>
    sys.exit(
  File "/.../jython/Lib/site-packages/distribute-0.6.24-py2.7.egg/setuptools/command/easy_install.py", line 1883, in main
    with_ei_usage(lambda:
  File "/.../jython/Lib/site-packages/distribute-0.6.24-py2.7.egg/setuptools/command/easy_install.py", line 1864, in with_ei_usage
    return f()
  File "/.../jython/Lib/site-packages/distribute-0.6.24-py2.7.egg/setuptools/command/easy_install.py", line 1883, in <lambda>
    with_ei_usage(lambda:
  File "/usr/local/jython2.7a1/Lib/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/usr/local/jython2.7a1/Lib/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/usr/local/jython2.7a1/Lib/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/usr/local/jython2.7a1/Lib/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/.../jython/Lib/site-packages/distribute-0.6.24-py2.7.egg/setuptools/command/easy_install.py", line 349, in run
    self.easy_install(spec, not self.no_deps)
  File "/.../jython/Lib/site-packages/distribute-0.6.24-py2.7.egg/setuptools/command/easy_install.py", line 589, in easy_install
    return self.install_item(spec, dist.location, tmpdir, deps)
  File "/.../jython/Lib/site-packages/distribute-0.6.24-py2.7.egg/setuptools/command/easy_install.py", line 619, in install_item
    dists = self.install_eggs(spec, download, tmpdir)
  File "/.../jython/Lib/site-packages/distribute-0.6.24-py2.7.egg/setuptools/command/easy_install.py", line 773, in install_eggs
    return [self.install_egg(dist_filename, tmpdir)]
  File "/.../jython/Lib/site-packages/distribute-0.6.24-py2.7.egg/setuptools/command/easy_install.py", line 845, in install_egg
    self.execute(f, (egg_path, destination),
  File "/usr/local/jython2.7a1/Lib/distutils/cmd.py", line 349, in execute
    util.execute(func, args, msg, dry_run=self.dry_run)
  File "/usr/local/jython2.7a1/Lib/distutils/util.py", line 409, in execute
    func(*args)
  File "/.../jython/Lib/site-packages/distribute-0.6.24-py2.7.egg/setuptools/command/easy_install.py", line 1153, in unpack_and_compile
    self.byte_compile(to_compile)
  File "/.../jython/Lib/site-packages/distribute-0.6.24-py2.7.egg/setuptools/command/easy_install.py", line 1169, in byte_compile
    byte_compile(to_compile, optimize=0, force=1, dry_run=self.dry_run)
  File "/usr/local/jython2.7a1/Lib/distutils/util.py", line 567, in byte_compile
    compile(file, cfile, dfile)
  File "/usr/local/jython2.7a1/Lib/py_compile.py", line 96, in compile
    _py_compile.compile(file, cfile, dfile)
  File "/usr/local/jython2.7a1/Lib/py_compile.py", line 96, in compile
    _py_compile.compile(file, cfile, dfile)
	at org.python.objectweb.asm.MethodWriter.a(Unknown Source)
	at org.python.objectweb.asm.ClassWriter.toByteArray(Unknown Source)
	at org.python.compiler.ClassFile.write(ClassFile.java:124)
	at org.python.compiler.Module.write(Module.java:618)
	at org.python.compiler.Module.compile(Module.java:674)
	at org.python.core.imp.compileSource(imp.java:330)
	at org.python.core.imp.compileSource(imp.java:252)
	at org.python.modules._py_compile.compile(_py_compile.java:28)
	at sun.reflect.GeneratedMethodAccessor33.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:616)

java.lang.RuntimeException: java.lang.RuntimeException: Method code too large!

I manually tried setup.py install and got the same error, too:

$ jython setup.py install
running install
running bdist_egg
running egg_info
creating Pygments.egg-info
writing Pygments.egg-info/PKG-INFO
writing entry points to Pygments.egg-info/entry_points.txt
writing top-level names to Pygments.egg-info/top_level.txt
writing dependency_links to Pygments.egg-info/dependency_links.txt
writing manifest file 'Pygments.egg-info/SOURCES.txt'
reading manifest file 'Pygments.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'Pygments.egg-info/SOURCES.txt'
installing library code to build/bdist.java1.6.0_24/egg
running install_lib
running build_py
creating build/bdist.java1.6.0_24
creating build/bdist.java1.6.0_24/egg
creating build/bdist.java1.6.0_24/egg/pygments
copying build/lib/pygments/scanner.py -> build/bdist.java1.6.0_24/egg/pygments
creating build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/_phpbuiltins.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/asm.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/dotnet.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/text.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/templates.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/__init__.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/_clbuiltins.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/shell.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/special.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/other.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/sql.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/_vimbuiltins.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/functional.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/compiled.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/_luabuiltins.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/hdl.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/parsers.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/_postgres_builtins.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/jvm.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/math.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/web.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/_asybuiltins.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/_mapping.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/_scilab_builtins.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/agile.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/__init__.py -> build/bdist.java1.6.0_24/egg/pygments
copying build/lib/pygments/filter.py -> build/bdist.java1.6.0_24/egg/pygments
copying build/lib/pygments/util.py -> build/bdist.java1.6.0_24/egg/pygments
copying build/lib/pygments/unistring.py -> build/bdist.java1.6.0_24/egg/pygments
copying build/lib/pygments/plugin.py -> build/bdist.java1.6.0_24/egg/pygments
creating build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/rrt.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/friendly.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/trac.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/__init__.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/autumn.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/borland.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/monokai.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/fruity.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/default.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/vs.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/vim.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/murphy.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/manni.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/bw.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/native.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/emacs.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/tango.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/perldoc.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/pastie.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/colorful.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/token.py -> build/bdist.java1.6.0_24/egg/pygments
copying build/lib/pygments/console.py -> build/bdist.java1.6.0_24/egg/pygments
copying build/lib/pygments/style.py -> build/bdist.java1.6.0_24/egg/pygments
creating build/bdist.java1.6.0_24/egg/pygments/filters
copying build/lib/pygments/filters/__init__.py -> build/bdist.java1.6.0_24/egg/pygments/filters
copying build/lib/pygments/cmdline.py -> build/bdist.java1.6.0_24/egg/pygments
creating build/bdist.java1.6.0_24/egg/pygments/formatters
copying build/lib/pygments/formatters/__init__.py -> build/bdist.java1.6.0_24/egg/pygments/formatters
copying build/lib/pygments/formatters/terminal256.py -> build/bdist.java1.6.0_24/egg/pygments/formatters
copying build/lib/pygments/formatters/rtf.py -> build/bdist.java1.6.0_24/egg/pygments/formatters
copying build/lib/pygments/formatters/other.py -> build/bdist.java1.6.0_24/egg/pygments/formatters
copying build/lib/pygments/formatters/img.py -> build/bdist.java1.6.0_24/egg/pygments/formatters
copying build/lib/pygments/formatters/latex.py -> build/bdist.java1.6.0_24/egg/pygments/formatters
copying build/lib/pygments/formatters/terminal.py -> build/bdist.java1.6.0_24/egg/pygments/formatters
copying build/lib/pygments/formatters/svg.py -> build/bdist.java1.6.0_24/egg/pygments/formatters
copying build/lib/pygments/formatters/_mapping.py -> build/bdist.java1.6.0_24/egg/pygments/formatters
copying build/lib/pygments/formatters/bbcode.py -> build/bdist.java1.6.0_24/egg/pygments/formatters
copying build/lib/pygments/formatters/html.py -> build/bdist.java1.6.0_24/egg/pygments/formatters
copying build/lib/pygments/lexer.py -> build/bdist.java1.6.0_24/egg/pygments
copying build/lib/pygments/formatter.py -> build/bdist.java1.6.0_24/egg/pygments
byte-compiling build/bdist.java1.6.0_24/egg/pygments/scanner.py to scanner$py.class
byte-compiling build/bdist.java1.6.0_24/egg/pygments/lexers/_phpbuiltins.py to _phpbuiltins$py.class
byte-compiling build/bdist.java1.6.0_24/egg/pygments/lexers/asm.py to asm$py.class
byte-compiling build/bdist.java1.6.0_24/egg/pygments/lexers/dotnet.py to dotnet$py.class
byte-compiling build/bdist.java1.6.0_24/egg/pygments/lexers/text.py to text$py.class
byte-compiling build/bdist.java1.6.0_24/egg/pygments/lexers/templates.py to templates$py.class
byte-compiling build/bdist.java1.6.0_24/egg/pygments/lexers/__init__.py to __init__$py.class
byte-compiling build/bdist.java1.6.0_24/egg/pygments/lexers/_clbuiltins.py to _clbuiltins$py.class
byte-compiling build/bdist.java1.6.0_24/egg/pygments/lexers/shell.py to shell$py.class
byte-compiling build/bdist.java1.6.0_24/egg/pygments/lexers/special.py to special$py.class
byte-compiling build/bdist.java1.6.0_24/egg/pygments/lexers/other.py to other$py.class
byte-compiling build/bdist.java1.6.0_24/egg/pygments/lexers/sql.py to sql$py.class
byte-compiling build/bdist.java1.6.0_24/egg/pygments/lexers/_vimbuiltins.py to _vimbuiltins$py.class
Traceback (most recent call last):
  File "setup.py", line 61, in <module>
    setup(
  File "/usr/local/jython2.7a1/Lib/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/usr/local/jython2.7a1/Lib/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/usr/local/jython2.7a1/Lib/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/usr/local/jython2.7a1/Lib/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/.../jython/Lib/site-packages/distribute-0.6.24-py2.7.egg/setuptools/command/install.py", line 73, in run
    self.do_egg_install()
  File "/.../jython/Lib/site-packages/distribute-0.6.24-py2.7.egg/setuptools/command/install.py", line 93, in do_egg_install
    self.run_command('bdist_egg')
  File "/usr/local/jython2.7a1/Lib/distutils/cmd.py", line 326, in run_command
    self.distribution.run_command(command)
  File "/usr/local/jython2.7a1/Lib/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/.../jython/Lib/site-packages/distribute-0.6.24-py2.7.egg/setuptools/command/bdist_egg.py", line 179, in run
    cmd = self.call_command('install_lib', warn_dir=0)
  File "/.../jython/Lib/site-packages/distribute-0.6.24-py2.7.egg/setuptools/command/bdist_egg.py", line 166, in call_command
    self.run_command(cmdname)
  File "/usr/local/jython2.7a1/Lib/distutils/cmd.py", line 326, in run_command
    self.distribution.run_command(command)
  File "/usr/local/jython2.7a1/Lib/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/.../jython/Lib/site-packages/distribute-0.6.24-py2.7.egg/setuptools/command/install_lib.py", line 24, in run
    self.byte_compile(outfiles)
  File "/usr/local/jython2.7a1/Lib/distutils/command/install_lib.py", line 136, in byte_compile
    byte_compile(files, optimize=0,
  File "/usr/local/jython2.7a1/Lib/distutils/util.py", line 567, in byte_compile
    compile(file, cfile, dfile)
  File "/usr/local/jython2.7a1/Lib/py_compile.py", line 96, in compile
    _py_compile.compile(file, cfile, dfile)
  File "/usr/local/jython2.7a1/Lib/py_compile.py", line 96, in compile
    _py_compile.compile(file, cfile, dfile)
	at org.python.objectweb.asm.MethodWriter.a(Unknown Source)
	at org.python.objectweb.asm.ClassWriter.toByteArray(Unknown Source)
	at org.python.compiler.ClassFile.write(ClassFile.java:124)
	at org.python.compiler.Module.write(Module.java:618)
	at org.python.compiler.Module.compile(Module.java:674)
	at org.python.core.imp.compileSource(imp.java:330)
	at org.python.core.imp.compileSource(imp.java:252)
	at org.python.modules._py_compile.compile(_py_compile.java:28)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:616)

java.lang.RuntimeException: java.lang.RuntimeException: Method code too large!


You can find pygments/lexers/_vimbuiltins.py file from https://bitbucket.org/birkenfeld/pygments-main/src/1.5/pygments/lexers/_vimbuiltins.py

Generally this happens when there is a very large function definition in a .py file - the JVM currently has an upper limit to the size of methods (I saw some talk about removing these in JDK 8 but I don't know if that will really happen) For now this is very difficult to fix from the Jython side - if you find the method and break it up into smaller methods that is a workaround (I know that's not good news). I'll keep this open for discussion - maybe if we knew exactly what part of pygments has grown so large (I know pygments worked before).

Actually pygments/lexers/_vimbuiltins.py file which exactly occurs the error has no function/method definitions.  Instead, it has three variables that are huge dictionary and list.  These are written in three lines.  Can the error be happened when some line of the source code has very long columns?

You can find the exact source code from the following url:
https://bitbucket.org/birkenfeld/pygments-main/src/1.5/pygments/lexers/_vimbuiltins.py

If this error is currently really hard to fix, I want to send a patch to Pygments.  So I want to know how to workaround it now.  :-)

Yep the body of a module creates a single function. Breaking that module up into separate modules will fix things (I know that is an ugly solution)

$ jython2.5 -m py_compile _vimbuiltins.py
$ jython2.7 -m py_compile _vimbuiltins.py
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/zart/opt/jython-2.7/Lib/runpy.py", line 175, in run_module
    return _run_module_code(code, init_globals, run_name,
  File "/home/zart/opt/jython-2.7/Lib/runpy.py", line 81, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/home/zart/opt/jython-2.7/Lib/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/home/zart/opt/jython-2.7/Lib/py_compile.py", line 128, in <module>
    sys.exit(main())
  File "/home/zart/opt/jython-2.7/Lib/py_compile.py", line 120, in main
    compile(filename, doraise=True)
  File "/home/zart/opt/jython-2.7/Lib/py_compile.py", line 120, in main
    compile(filename, doraise=True)
  File "/home/zart/opt/jython-2.7/Lib/py_compile.py", line 96, in compile
    _py_compile.compile(file, cfile, dfile)
  File "/home/zart/opt/jython-2.7/Lib/py_compile.py", line 96, in compile
    _py_compile.compile(file, cfile, dfile)
        at org.python.objectweb.asm.MethodWriter.a(Unknown Source)
        at org.python.objectweb.asm.ClassWriter.toByteArray(Unknown Source)
        at org.python.compiler.ClassFile.write(ClassFile.java:124)
        at org.python.compiler.Module.write(Module.java:618)
        at org.python.compiler.Module.compile(Module.java:674)
        at org.python.core.imp.compileSource(imp.java:330)
        at org.python.core.imp.compileSource(imp.java:252)
        at org.python.modules._py_compile.compile(_py_compile.java:28)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:616)

java.lang.RuntimeException: java.lang.RuntimeException: Method code too large!



Using smaller version of _vimbuiltins.py to generate .class and checking it with 'javap -c' shows that both 2.5 and 2.7 generate same bytecode. But 2.7 chokes on larger input somehow. So this looks like regression on jython side.

I have tried to load several standard modules (simplejson, requests) and all fail with this same issue at some point. I have also verified that it is a regression bug and that 2.5.3 does not have it. 

This makes 2.7 unusable with many standard python modules.

Can I suggest/request that the exception include the name of the method causing the problem (or something similar like the filename and line number) which would help when trying to fix this in end user code?

Or should that be a new issue?

Here's my exception. I tried with JDK7 and JDK8 (in case there had been improvements there as hinted at), but got the same error. I increased the memory and stack sizes to 2GB each and reduced GC frequency, but still getting the problem on both. The closest to a workaround I found is to use pycimport, which hasn't been updated for a long time AFAICT.

This is jython running SetupTools trying to install PyXB. Any hints much appreciated.

Reading https://pypi.python.org/simple/pyxb/
Best match: PyXB 1.2.3
Downloading https://pypi.python.org/packages/source/P/PyXB/PyXB-1.2.3.tar.gz#md5=e7e15abb99f686939ef4c4bead4f05e3
Processing PyXB-1.2.3.tar.gz
'pyxb' already exists in /home/ellis/FOMextractorLocal/target/jython-plugins-tmp/build; build directory /tmp/easy_install-IsyY5A will not be kept
Writing /tmp/easy_install-IsyY5A/PyXB-1.2.3/setup.cfg
Running PyXB-1.2.3/setup.py -q bdist_egg --dist-dir /tmp/easy_install-IsyY5A/PyXB-1.2.3/egg-dist-tmp-abDmq0
Found bundle in /tmp/easy_install-IsyY5A/PyXB-1.2.3/pyxb/bundles/dc
Found bundle in /tmp/easy_install-IsyY5A/PyXB-1.2.3/pyxb/bundles/common
Found bundle in /tmp/easy_install-IsyY5A/PyXB-1.2.3/pyxb/bundles/wssplat
Found bundle in /tmp/easy_install-IsyY5A/PyXB-1.2.3/pyxb/bundles/saml20
warning: no files found matching 'MANIFEST'
no previously-included directories found matching 'pyxb/bundles/core/schemas'
no previously-included directories found matching 'pyxb/bundles/core/remote'
no previously-included directories found matching 'pyxb/bundles/wssplat/schemas'
no previously-included directories found matching 'pyxb/bundles/wssplat/remote'
no previously-included directories found matching 'pyxb/bundles/opengis/schemas'
no previously-included directories found matching 'pyxb/bundles/opengis/remote'
no previously-included directories found matching 'doc/_build'
warning: no previously-included files found matching 'doc/*.eap'
no previously-included directories found matching 'doc/W3C'
warning: no previously-included files matching '*~' found anywhere in distribution
Traceback (most recent call last):
  File "Lib/site-packages/setuptools-3.5.1-py2.7.egg/easy_install.py", line 5, in <module>
    main()
  File "/home/ellis/FOMextractorLocal/target/jython-plugins-tmp/Lib/site-packages/setuptools-3.5.1-py2.7.egg/setuptools/command/easy_install.py", line 1920, in main
    with_ei_usage(lambda:
  File "/home/ellis/FOMextractorLocal/target/jython-plugins-tmp/Lib/site-packages/setuptools-3.5.1-py2.7.egg/setuptools/command/easy_install.py", line 1907, in with_ei_usage
    return f()
  File "/home/ellis/FOMextractorLocal/target/jython-plugins-tmp/Lib/site-packages/setuptools-3.5.1-py2.7.egg/setuptools/command/easy_install.py", line 1920, in <lambda>
    with_ei_usage(lambda:
  File "/home/ellis/workspace/jython-trunk/dist/Lib/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/home/ellis/workspace/jython-trunk/dist/Lib/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/home/ellis/workspace/jython-trunk/dist/Lib/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/home/ellis/workspace/jython-trunk/dist/Lib/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/home/ellis/FOMextractorLocal/target/jython-plugins-tmp/Lib/site-packages/setuptools-3.5.1-py2.7.egg/setuptools/command/easy_install.py", line 360, in run
    self.easy_install(spec, not self.no_deps)
  File "/home/ellis/FOMextractorLocal/target/jython-plugins-tmp/Lib/site-packages/setuptools-3.5.1-py2.7.egg/setuptools/command/easy_install.py", line 595, in easy_install
    return self.install_item(spec, dist.location, tmpdir, deps)
  File "/home/ellis/FOMextractorLocal/target/jython-plugins-tmp/Lib/site-packages/setuptools-3.5.1-py2.7.egg/setuptools/command/easy_install.py", line 625, in install_item
    dists = self.install_eggs(spec, download, tmpdir)
  File "/home/ellis/FOMextractorLocal/target/jython-plugins-tmp/Lib/site-packages/setuptools-3.5.1-py2.7.egg/setuptools/command/easy_install.py", line 813, in install_eggs
    return self.build_and_install(setup_script, setup_base)
  File "/home/ellis/FOMextractorLocal/target/jython-plugins-tmp/Lib/site-packages/setuptools-3.5.1-py2.7.egg/setuptools/command/easy_install.py", line 1019, in build_and_install
    self.run_setup(setup_script, setup_base, args)
  File "/home/ellis/FOMextractorLocal/target/jython-plugins-tmp/Lib/site-packages/setuptools-3.5.1-py2.7.egg/setuptools/command/easy_install.py", line 1004, in run_setup
    run_setup(setup_script, args)
  File "/home/ellis/FOMextractorLocal/target/jython-plugins-tmp/Lib/site-packages/setuptools-3.5.1-py2.7.egg/setuptools/command/easy_install.py", line 1004, in run_setup
    run_setup(setup_script, args)
  File "/home/ellis/FOMextractorLocal/target/jython-plugins-tmp/Lib/site-packages/setuptools-3.5.1-py2.7.egg/setuptools/sandbox.py", line 50, in run_setup
    lambda: execfile(
  File "/home/ellis/FOMextractorLocal/target/jython-plugins-tmp/Lib/site-packages/setuptools-3.5.1-py2.7.egg/setuptools/sandbox.py", line 50, in run_setup
    lambda: execfile(
  File "/home/ellis/FOMextractorLocal/target/jython-plugins-tmp/Lib/site-packages/setuptools-3.5.1-py2.7.egg/setuptools/sandbox.py", line 100, in run
    return func()
  File "/home/ellis/FOMextractorLocal/target/jython-plugins-tmp/Lib/site-packages/setuptools-3.5.1-py2.7.egg/setuptools/sandbox.py", line 50, in <lambda>
    lambda: execfile(
  File "setup.py", line 220, in <module>
  File "/home/ellis/workspace/jython-trunk/dist/Lib/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/home/ellis/workspace/jython-trunk/dist/Lib/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/home/ellis/workspace/jython-trunk/dist/Lib/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/home/ellis/workspace/jython-trunk/dist/Lib/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/home/ellis/FOMextractorLocal/target/jython-plugins-tmp/Lib/site-packages/setuptools-3.5.1-py2.7.egg/setuptools/command/bdist_egg.py", line 157, in run
    cmd = self.call_command('install_lib', warn_dir=0)
  File "/home/ellis/FOMextractorLocal/target/jython-plugins-tmp/Lib/site-packages/setuptools-3.5.1-py2.7.egg/setuptools/command/bdist_egg.py", line 143, in call_command
    self.run_command(cmdname)
  File "/home/ellis/workspace/jython-trunk/dist/Lib/distutils/cmd.py", line 326, in run_command
    self.distribution.run_command(command)
  File "/home/ellis/workspace/jython-trunk/dist/Lib/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/home/ellis/FOMextractorLocal/target/jython-plugins-tmp/Lib/site-packages/setuptools-3.5.1-py2.7.egg/setuptools/command/install_lib.py", line 12, in run
    self.byte_compile(outfiles)
  File "/home/ellis/workspace/jython-trunk/dist/Lib/distutils/command/install_lib.py", line 136, in byte_compile
    byte_compile(files, optimize=0,
  File "/home/ellis/workspace/jython-trunk/dist/Lib/distutils/util.py", line 567, in byte_compile
    compile(file, cfile, dfile)
  File "/home/ellis/workspace/jython-trunk/dist/Lib/py_compile.py", line 96, in compile
    _py_compile.compile(file, cfile, dfile)
  File "/home/ellis/workspace/jython-trunk/dist/Lib/py_compile.py", line 96, in compile
    _py_compile.compile(file, cfile, dfile)
	at org.objectweb.asm.ClassWriter.toByteArray(Unknown Source)
	at org.python.compiler.ClassFile.write(ClassFile.java:212)
	at org.python.compiler.Module.write(Module.java:605)
	at org.python.compiler.Module.compile(Module.java:661)
	at org.python.core.imp.compileSource(imp.java:330)
	at org.python.core.imp.compileSource(imp.java:252)
	at org.python.modules._py_compile.compile(_py_compile.java:28)
	at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)

java.lang.RuntimeException: java.lang.RuntimeException: Class file too large!

So there is not a general solution yet, but I can report the Pygments and Requests packages here now install against Jython 2.7 trunk; we expect a beta 3 release of this work in the next day or so.

This works because for large collection literals (dict, list, set, tuple) of keys and values that are only primitives (int|float|str|unicode), we now break the construction of these literals into smaller methods.

I sort of doubt PyXB is going to be useful on Jython until the necessary XML support is made more efficient.

Did anyone else notice that this problem seems worse on Jython 2.7b2 than it was on Jython 2.7b1? e.g. http://lists.open-bio.org/pipermail/biopython-dev/2014-May/011376.html

Peter, I did take a look at org.python.compiler.CodeCompiler, which should be the one place where we would expect any changes in generated Java method sizes since beta 1. The only fix since beta 1 is the one I mention, in 7135:845d319f5b89 - Added support for large collection literals, by setting in separate helper methods

biopython itself installs fine from github, but presumably the problem is in code that is subsequently generated. Could you post such generated code somewhere, such as on gist.github.com?

Found this. It might help.

http://stackoverflow.com/questions/3192896/how-to-circumvent-the-method-too-large-error-in-java-compilation

Any fix will have to be at the Jython end, because the other option is to re-compile the JVM from source and change the limit on the method size. I agree. Jython is not really usable until this issue is resolved and this should be made to clear anyone who wants to use it.

I don't think we want to be building modified JVMs.

As a workaround until Python bytecode compilation is directly incorporated into a future Jython (at the earliest, 2.7.1), I could readily write a PyPI package as part of jythontools (https://github.com/jythontools/) that uses CPython 2.7 as a subprocess to compile Python source to Python bytecode via the compile() builtin and compileall. Such Python bytecode can be loaded via standard Python tooling (marshal.Unmarshaller) or pycimport, a metaimporter.

Python bytecode does not have such size limits and using it has other potential advantages, such as reducing permgen pressure, Jython startup time, Android support, and optimizations for invokedynamic. Not only is it used by CPython, it is also used by PyPy, and we have support it in Jython via org.python.core.PyBytecode, a direct port of ceval.c from CPython. Realizing all of these advantages will of course require getting Python bytecode compilation in Jython itself.

In the meantime, such a package would also be useful in testing Jython support in the regrtest of PyBytecode. We can also look at changing the unmarshalling such that it canonicalizes float literals to avoid regrtest errors around floating point representation.

Thoughts, especially on API, as we support this transition to native support? At a minimum, I would expect this package to be a direct substitute for compile and compileall, including the possibility of monkeypatching.

(Java bytecode of course would remain the usual way we support compiling Python source on Jython.)

Created a placeholder PyPI package https://pypi.python.org/pypi/veneer

This is a duplicate of #527524

For the specific case of Pygments, Jython 2.7 trunk now compiles this just fine. This is almost certainly because of a fix made some months ago to support large collection literals, which is the usual reason for such large methods (http://hg.python.org/jython/rev/845d319f5b89)

I ran nosetests and we are seeing 3 failing tests out of 1130, due to regex issues:

======================================================================
ERROR: test_examplefiles.test_example_files(<pygments.lexers.RagelCppLexer>, '/Users/jbaker/opensource/pygments-main/tests/examplefiles/ragel-cpp_rlscan', '/Users/jbaker/opensource/pygments-main/tests/examplefiles/output/ragel-cpp_rlscan')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/jbaker/jythondev/jython27/dist/Lib/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/Users/jbaker/opensource/pygments-main/tests/test_examplefiles.py", line 78, in check_lexer
    for type, val in lx.get_tokens(text):
  File "/Users/jbaker/jythondev/jython27/dist/Lib/site-packages/Pygments-2.0predev_20140717-py2.7.egg/pygments/lexer.py", line 188, in streamer
    for i, t, v in self.get_tokens_unprocessed(text):
  File "/Users/jbaker/jythondev/jython27/dist/Lib/site-packages/Pygments-2.0predev_20140717-py2.7.egg/pygments/lexer.py", line 226, in get_tokens_unprocessed
    for i, t, v in self.language_lexer.get_tokens_unprocessed(text):
  File "/Users/jbaker/jythondev/jython27/dist/Lib/site-packages/Pygments-2.0predev_20140717-py2.7.egg/pygments/lexer.py", line 597, in get_tokens_unprocessed
    m = rexmatch(text, pos)
RuntimeError: maximum recursion depth exceeded (Java StackOverflowError)

----------------------------------------------------------------------
Ran 1130 tests in 51.004s

FAILED (errors=3)

But this is a separate issue in our implementation of sre.

Hi Jim,

We have gone through the ticket http://bugs.jython.org/issue1891

Which is similar to the case specified in current ticket.

As per the ticket 1891, Splitting the larger methods (Inside .py files) in to small peices will work, But this is just a work around.
Even we followed the same procedure of splitting the content of .py file, And in fact it's working in our customer server (O2 UK).

And also it was specified in the ticket (1891), Issue will get solved if the jython version is 2.7 without applying work around(Splitting of method content).

Based on the above can we suggest our customer to upgrade from jython 2.5.2 to jython 2.7.

Regards,
Megha