Issue2040

I have some files that contain a pickle, that was compressed with bz2, and then base64 encoded. These files were generated under linux using python.

When I open these programs with my program with python 2.7, I can open them without a problem. The coding is something like that:

l=f.readline()
while len(l):
    b64=base64.b64decode(l)
    arr.append(comp.decompress(b64))
    l=f.readline()
zip = ''.join(arr)
data=cPickle.loads(zip)

However, if I try to use jython to execute the pyhton program I get the following exception:

Traceback (most recent call last):
  File "c:\Test\src\PerfTrace.py", line 115, in <module>
    main()
  File "c:\Test\src\PerfTrace.py", line 110, in main
    data = traceloader.load_trace("c:/temp/Performance.tpt")
  File "c:\Test\src\PerfTrace.py", line 94, in load_trace
    arr.append(comp.decompress(b64))
java.lang.ArrayIndexOutOfBoundsException


If I call the same program from a java program I get the same exception,
but with more details, and the problem seems to be the bz2 decompressor

Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 12198
	at org.python.modules.bz2.PyBZ2Decompressor.BZ2Decompressor_decompress(PyBZ2Decompressor.java:72)
	at org.python.modules.bz2.PyBZ2Decompressor$BZ2Decompressor_decompress_exposer.__call__(Unknown Source)
	at org.python.core.PyObject.__call__(PyObject.java:403)
	at org.python.core.PyObject.__call__(PyObject.java:407)
	at PerfTrace$py.load_trace$23(__pyclasspath__/PerfTrace.py:105)
	at PerfTrace$py.call_function(__pyclasspath__/PerfTrace.py)
	at org.python.core.PyTableCode.call(PyTableCode.java:165)
	at org.python.core.PyBaseCode.call(PyBaseCode.java:301)
	at org.python.core.PyBaseCode.call(PyBaseCode.java:194)
	at org.python.core.PyFunction.__call__(PyFunction.java:417)
	at org.python.core.PyMethod.instancemethod___call__(PyMethod.java:232)
	at org.python.core.PyMethod.__call__(PyMethod.java:223)
	at org.python.core.PyMethod.__call__(PyMethod.java:213)
	at org.python.core.PyMethod.__call__(PyMethod.java:208)
	at org.python.core.PyObject._jcallexc(PyObject.java:3555)
	at org.python.core.PyObject._jcall(PyObject.java:3587)
	at org.python.proxies.PerfTrace$TraceLoader$0.load_trace(Unknown Source)
	at test.Test.main(Test.java:18)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)


Are there any known incompatabilities between the bz2 implementations of pyhton vs. jython or known limitations? 

Regards,
Andreas

Hi Andreas,

The bz2 implementation is brand new and currently causes some test failures in test_tarfile.py. You have likely found a bug in the new code. Thanks for reporting!

A possible fix as a pull request here - https://bitbucket.org/jython/jython/pull-request/24/the-pybz2decompressor-should-check-that/diff

Probably fixed in beta 3; could akoenig please retry since we don't have the data files you were using?

Also this should be pending, given that our tests now pass

I don't beleive that the changes in pullrequest have been merged so I don't think this has been fixed.

Moving back to open, need to take a look at Indra's PR

Target beta 4

Fixed as of http://hg.python.org/jython/rev/02a31eb57af2