Issue1891

I tried easy_install Pygments (which is a syntax highlighting library) it occurs "Method code too large!" error.


$ ../../.tox/jython/bin/easy_install Pygments
Searching for Pygments
Reading http://pypi.python.org/simple/Pygments/
Reading http://pygments.org/
Reading http://pygments.pocoo.org/
Best match: Pygments 1.5
Downloading http://pypi.python.org/packages/2.7/P/Pygments/Pygments-1.5-py2.7.egg#md5=8ea02c2eafb2580336b7637be629f7ce
Processing Pygments-1.5-py2.7.egg
creating /.../jython/Lib/site-packages/Pygments-1.5-py2.7.egg
Extracting Pygments-1.5-py2.7.egg to /.../jython/Lib/site-packages
Traceback (most recent call last):
  File ".../jython/bin/easy_install", line 8, in <module>
    sys.exit(
  File "/.../jython/Lib/site-packages/distribute-0.6.24-py2.7.egg/setuptools/command/easy_install.py", line 1883, in main
    with_ei_usage(lambda:
  File "/.../jython/Lib/site-packages/distribute-0.6.24-py2.7.egg/setuptools/command/easy_install.py", line 1864, in with_ei_usage
    return f()
  File "/.../jython/Lib/site-packages/distribute-0.6.24-py2.7.egg/setuptools/command/easy_install.py", line 1883, in <lambda>
    with_ei_usage(lambda:
  File "/usr/local/jython2.7a1/Lib/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/usr/local/jython2.7a1/Lib/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/usr/local/jython2.7a1/Lib/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/usr/local/jython2.7a1/Lib/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/.../jython/Lib/site-packages/distribute-0.6.24-py2.7.egg/setuptools/command/easy_install.py", line 349, in run
    self.easy_install(spec, not self.no_deps)
  File "/.../jython/Lib/site-packages/distribute-0.6.24-py2.7.egg/setuptools/command/easy_install.py", line 589, in easy_install
    return self.install_item(spec, dist.location, tmpdir, deps)
  File "/.../jython/Lib/site-packages/distribute-0.6.24-py2.7.egg/setuptools/command/easy_install.py", line 619, in install_item
    dists = self.install_eggs(spec, download, tmpdir)
  File "/.../jython/Lib/site-packages/distribute-0.6.24-py2.7.egg/setuptools/command/easy_install.py", line 773, in install_eggs
    return [self.install_egg(dist_filename, tmpdir)]
  File "/.../jython/Lib/site-packages/distribute-0.6.24-py2.7.egg/setuptools/command/easy_install.py", line 845, in install_egg
    self.execute(f, (egg_path, destination),
  File "/usr/local/jython2.7a1/Lib/distutils/cmd.py", line 349, in execute
    util.execute(func, args, msg, dry_run=self.dry_run)
  File "/usr/local/jython2.7a1/Lib/distutils/util.py", line 409, in execute
    func(*args)
  File "/.../jython/Lib/site-packages/distribute-0.6.24-py2.7.egg/setuptools/command/easy_install.py", line 1153, in unpack_and_compile
    self.byte_compile(to_compile)
  File "/.../jython/Lib/site-packages/distribute-0.6.24-py2.7.egg/setuptools/command/easy_install.py", line 1169, in byte_compile
    byte_compile(to_compile, optimize=0, force=1, dry_run=self.dry_run)
  File "/usr/local/jython2.7a1/Lib/distutils/util.py", line 567, in byte_compile
    compile(file, cfile, dfile)
  File "/usr/local/jython2.7a1/Lib/py_compile.py", line 96, in compile
    _py_compile.compile(file, cfile, dfile)
  File "/usr/local/jython2.7a1/Lib/py_compile.py", line 96, in compile
    _py_compile.compile(file, cfile, dfile)
	at org.python.objectweb.asm.MethodWriter.a(Unknown Source)
	at org.python.objectweb.asm.ClassWriter.toByteArray(Unknown Source)
	at org.python.compiler.ClassFile.write(ClassFile.java:124)
	at org.python.compiler.Module.write(Module.java:618)
	at org.python.compiler.Module.compile(Module.java:674)
	at org.python.core.imp.compileSource(imp.java:330)
	at org.python.core.imp.compileSource(imp.java:252)
	at org.python.modules._py_compile.compile(_py_compile.java:28)
	at sun.reflect.GeneratedMethodAccessor33.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:616)

java.lang.RuntimeException: java.lang.RuntimeException: Method code too large!

I manually tried setup.py install and got the same error, too:

$ jython setup.py install
running install
running bdist_egg
running egg_info
creating Pygments.egg-info
writing Pygments.egg-info/PKG-INFO
writing entry points to Pygments.egg-info/entry_points.txt
writing top-level names to Pygments.egg-info/top_level.txt
writing dependency_links to Pygments.egg-info/dependency_links.txt
writing manifest file 'Pygments.egg-info/SOURCES.txt'
reading manifest file 'Pygments.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'Pygments.egg-info/SOURCES.txt'
installing library code to build/bdist.java1.6.0_24/egg
running install_lib
running build_py
creating build/bdist.java1.6.0_24
creating build/bdist.java1.6.0_24/egg
creating build/bdist.java1.6.0_24/egg/pygments
copying build/lib/pygments/scanner.py -> build/bdist.java1.6.0_24/egg/pygments
creating build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/_phpbuiltins.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/asm.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/dotnet.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/text.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/templates.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/__init__.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/_clbuiltins.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/shell.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/special.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/other.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/sql.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/_vimbuiltins.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/functional.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/compiled.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/_luabuiltins.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/hdl.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/parsers.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/_postgres_builtins.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/jvm.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/math.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/web.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/_asybuiltins.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/_mapping.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/_scilab_builtins.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/lexers/agile.py -> build/bdist.java1.6.0_24/egg/pygments/lexers
copying build/lib/pygments/__init__.py -> build/bdist.java1.6.0_24/egg/pygments
copying build/lib/pygments/filter.py -> build/bdist.java1.6.0_24/egg/pygments
copying build/lib/pygments/util.py -> build/bdist.java1.6.0_24/egg/pygments
copying build/lib/pygments/unistring.py -> build/bdist.java1.6.0_24/egg/pygments
copying build/lib/pygments/plugin.py -> build/bdist.java1.6.0_24/egg/pygments
creating build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/rrt.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/friendly.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/trac.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/__init__.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/autumn.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/borland.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/monokai.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/fruity.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/default.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/vs.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/vim.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/murphy.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/manni.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/bw.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/native.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/emacs.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/tango.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/perldoc.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/pastie.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/styles/colorful.py -> build/bdist.java1.6.0_24/egg/pygments/styles
copying build/lib/pygments/token.py -> build/bdist.java1.6.0_24/egg/pygments
copying build/lib/pygments/console.py -> build/bdist.java1.6.0_24/egg/pygments
copying build/lib/pygments/style.py -> build/bdist.java1.6.0_24/egg/pygments
creating build/bdist.java1.6.0_24/egg/pygments/filters
copying build/lib/pygments/filters/__init__.py -> build/bdist.java1.6.0_24/egg/pygments/filters
copying build/lib/pygments/cmdline.py -> build/bdist.java1.6.0_24/egg/pygments
creating build/bdist.java1.6.0_24/egg/pygments/formatters
copying build/lib/pygments/formatters/__init__.py -> build/bdist.java1.6.0_24/egg/pygments/formatters
copying build/lib/pygments/formatters/terminal256.py -> build/bdist.java1.6.0_24/egg/pygments/formatters
copying build/lib/pygments/formatters/rtf.py -> build/bdist.java1.6.0_24/egg/pygments/formatters
copying build/lib/pygments/formatters/other.py -> build/bdist.java1.6.0_24/egg/pygments/formatters
copying build/lib/pygments/formatters/img.py -> build/bdist.java1.6.0_24/egg/pygments/formatters
copying build/lib/pygments/formatters/latex.py -> build/bdist.java1.6.0_24/egg/pygments/formatters
copying build/lib/pygments/formatters/terminal.py -> build/bdist.java1.6.0_24/egg/pygments/formatters
copying build/lib/pygments/formatters/svg.py -> build/bdist.java1.6.0_24/egg/pygments/formatters
copying build/lib/pygments/formatters/_mapping.py -> build/bdist.java1.6.0_24/egg/pygments/formatters
copying build/lib/pygments/formatters/bbcode.py -> build/bdist.java1.6.0_24/egg/pygments/formatters
copying build/lib/pygments/formatters/html.py -> build/bdist.java1.6.0_24/egg/pygments/formatters
copying build/lib/pygments/lexer.py -> build/bdist.java1.6.0_24/egg/pygments
copying build/lib/pygments/formatter.py -> build/bdist.java1.6.0_24/egg/pygments
byte-compiling build/bdist.java1.6.0_24/egg/pygments/scanner.py to scanner$py.class
byte-compiling build/bdist.java1.6.0_24/egg/pygments/lexers/_phpbuiltins.py to _phpbuiltins$py.class
byte-compiling build/bdist.java1.6.0_24/egg/pygments/lexers/asm.py to asm$py.class
byte-compiling build/bdist.java1.6.0_24/egg/pygments/lexers/dotnet.py to dotnet$py.class
byte-compiling build/bdist.java1.6.0_24/egg/pygments/lexers/text.py to text$py.class
byte-compiling build/bdist.java1.6.0_24/egg/pygments/lexers/templates.py to templates$py.class
byte-compiling build/bdist.java1.6.0_24/egg/pygments/lexers/__init__.py to __init__$py.class
byte-compiling build/bdist.java1.6.0_24/egg/pygments/lexers/_clbuiltins.py to _clbuiltins$py.class
byte-compiling build/bdist.java1.6.0_24/egg/pygments/lexers/shell.py to shell$py.class
byte-compiling build/bdist.java1.6.0_24/egg/pygments/lexers/special.py to special$py.class
byte-compiling build/bdist.java1.6.0_24/egg/pygments/lexers/other.py to other$py.class
byte-compiling build/bdist.java1.6.0_24/egg/pygments/lexers/sql.py to sql$py.class
byte-compiling build/bdist.java1.6.0_24/egg/pygments/lexers/_vimbuiltins.py to _vimbuiltins$py.class
Traceback (most recent call last):
  File "setup.py", line 61, in <module>
    setup(
  File "/usr/local/jython2.7a1/Lib/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/usr/local/jython2.7a1/Lib/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/usr/local/jython2.7a1/Lib/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/usr/local/jython2.7a1/Lib/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/.../jython/Lib/site-packages/distribute-0.6.24-py2.7.egg/setuptools/command/install.py", line 73, in run
    self.do_egg_install()
  File "/.../jython/Lib/site-packages/distribute-0.6.24-py2.7.egg/setuptools/command/install.py", line 93, in do_egg_install
    self.run_command('bdist_egg')
  File "/usr/local/jython2.7a1/Lib/distutils/cmd.py", line 326, in run_command
    self.distribution.run_command(command)
  File "/usr/local/jython2.7a1/Lib/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/.../jython/Lib/site-packages/distribute-0.6.24-py2.7.egg/setuptools/command/bdist_egg.py", line 179, in run
    cmd = self.call_command('install_lib', warn_dir=0)
  File "/.../jython/Lib/site-packages/distribute-0.6.24-py2.7.egg/setuptools/command/bdist_egg.py", line 166, in call_command
    self.run_command(cmdname)
  File "/usr/local/jython2.7a1/Lib/distutils/cmd.py", line 326, in run_command
    self.distribution.run_command(command)
  File "/usr/local/jython2.7a1/Lib/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/.../jython/Lib/site-packages/distribute-0.6.24-py2.7.egg/setuptools/command/install_lib.py", line 24, in run
    self.byte_compile(outfiles)
  File "/usr/local/jython2.7a1/Lib/distutils/command/install_lib.py", line 136, in byte_compile
    byte_compile(files, optimize=0,
  File "/usr/local/jython2.7a1/Lib/distutils/util.py", line 567, in byte_compile
    compile(file, cfile, dfile)
  File "/usr/local/jython2.7a1/Lib/py_compile.py", line 96, in compile
    _py_compile.compile(file, cfile, dfile)
  File "/usr/local/jython2.7a1/Lib/py_compile.py", line 96, in compile
    _py_compile.compile(file, cfile, dfile)
	at org.python.objectweb.asm.MethodWriter.a(Unknown Source)
	at org.python.objectweb.asm.ClassWriter.toByteArray(Unknown Source)
	at org.python.compiler.ClassFile.write(ClassFile.java:124)
	at org.python.compiler.Module.write(Module.java:618)
	at org.python.compiler.Module.compile(Module.java:674)
	at org.python.core.imp.compileSource(imp.java:330)
	at org.python.core.imp.compileSource(imp.java:252)
	at org.python.modules._py_compile.compile(_py_compile.java:28)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:616)

java.lang.RuntimeException: java.lang.RuntimeException: Method code too large!


You can find pygments/lexers/_vimbuiltins.py file from https://bitbucket.org/birkenfeld/pygments-main/src/1.5/pygments/lexers/_vimbuiltins.py

Generally this happens when there is a very large function definition in a .py file - the JVM currently has an upper limit to the size of methods (I saw some talk about removing these in JDK 8 but I don't know if that will really happen) For now this is very difficult to fix from the Jython side - if you find the method and break it up into smaller methods that is a workaround (I know that's not good news). I'll keep this open for discussion - maybe if we knew exactly what part of pygments has grown so large (I know pygments worked before).

Actually pygments/lexers/_vimbuiltins.py file which exactly occurs the error has no function/method definitions.  Instead, it has three variables that are huge dictionary and list.  These are written in three lines.  Can the error be happened when some line of the source code has very long columns?

You can find the exact source code from the following url:
https://bitbucket.org/birkenfeld/pygments-main/src/1.5/pygments/lexers/_vimbuiltins.py

If this error is currently really hard to fix, I want to send a patch to Pygments.  So I want to know how to workaround it now.  :-)

Yep the body of a module creates a single function. Breaking that module up into separate modules will fix things (I know that is an ugly solution)

$ jython2.5 -m py_compile _vimbuiltins.py
$ jython2.7 -m py_compile _vimbuiltins.py
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/zart/opt/jython-2.7/Lib/runpy.py", line 175, in run_module
    return _run_module_code(code, init_globals, run_name,
  File "/home/zart/opt/jython-2.7/Lib/runpy.py", line 81, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/home/zart/opt/jython-2.7/Lib/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/home/zart/opt/jython-2.7/Lib/py_compile.py", line 128, in <module>
    sys.exit(main())
  File "/home/zart/opt/jython-2.7/Lib/py_compile.py", line 120, in main
    compile(filename, doraise=True)
  File "/home/zart/opt/jython-2.7/Lib/py_compile.py", line 120, in main
    compile(filename, doraise=True)
  File "/home/zart/opt/jython-2.7/Lib/py_compile.py", line 96, in compile
    _py_compile.compile(file, cfile, dfile)
  File "/home/zart/opt/jython-2.7/Lib/py_compile.py", line 96, in compile
    _py_compile.compile(file, cfile, dfile)
        at org.python.objectweb.asm.MethodWriter.a(Unknown Source)
        at org.python.objectweb.asm.ClassWriter.toByteArray(Unknown Source)
        at org.python.compiler.ClassFile.write(ClassFile.java:124)
        at org.python.compiler.Module.write(Module.java:618)
        at org.python.compiler.Module.compile(Module.java:674)
        at org.python.core.imp.compileSource(imp.java:330)
        at org.python.core.imp.compileSource(imp.java:252)
        at org.python.modules._py_compile.compile(_py_compile.java:28)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:616)

java.lang.RuntimeException: java.lang.RuntimeException: Method code too large!



Using smaller version of _vimbuiltins.py to generate .class and checking it with 'javap -c' shows that both 2.5 and 2.7 generate same bytecode. But 2.7 chokes on larger input somehow. So this looks like regression on jython side.