Issue2040

classification
Title: Uncompressing file with bz2 fails with ArrayIndexOutOfBoundsException
Type: crash Severity: normal
Components: Library Versions: Jython 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: akoenig, fwierzbicki, indra, zyasoft
Priority: Keywords:

Created on 2013-04-12.14:55:02 by akoenig, last changed 2014-07-01.04:04:40 by zyasoft.

Messages
msg7991 (view) Author: Andreas Koenig (akoenig) Date: 2013-04-12.14:55:01
I have some files that contain a pickle, that was compressed with bz2, and then base64 encoded. These files were generated under linux using python.

When I open these programs with my program with python 2.7, I can open them without a problem. The coding is something like that:

l=f.readline()
while len(l):
    b64=base64.b64decode(l)
    arr.append(comp.decompress(b64))
    l=f.readline()
zip = ''.join(arr)
data=cPickle.loads(zip)

However, if I try to use jython to execute the pyhton program I get the following exception:

Traceback (most recent call last):
  File "c:\Test\src\PerfTrace.py", line 115, in <module>
    main()
  File "c:\Test\src\PerfTrace.py", line 110, in main
    data = traceloader.load_trace("c:/temp/Performance.tpt")
  File "c:\Test\src\PerfTrace.py", line 94, in load_trace
    arr.append(comp.decompress(b64))
java.lang.ArrayIndexOutOfBoundsException


If I call the same program from a java program I get the same exception,
but with more details, and the problem seems to be the bz2 decompressor

Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 12198
	at org.python.modules.bz2.PyBZ2Decompressor.BZ2Decompressor_decompress(PyBZ2Decompressor.java:72)
	at org.python.modules.bz2.PyBZ2Decompressor$BZ2Decompressor_decompress_exposer.__call__(Unknown Source)
	at org.python.core.PyObject.__call__(PyObject.java:403)
	at org.python.core.PyObject.__call__(PyObject.java:407)
	at PerfTrace$py.load_trace$23(__pyclasspath__/PerfTrace.py:105)
	at PerfTrace$py.call_function(__pyclasspath__/PerfTrace.py)
	at org.python.core.PyTableCode.call(PyTableCode.java:165)
	at org.python.core.PyBaseCode.call(PyBaseCode.java:301)
	at org.python.core.PyBaseCode.call(PyBaseCode.java:194)
	at org.python.core.PyFunction.__call__(PyFunction.java:417)
	at org.python.core.PyMethod.instancemethod___call__(PyMethod.java:232)
	at org.python.core.PyMethod.__call__(PyMethod.java:223)
	at org.python.core.PyMethod.__call__(PyMethod.java:213)
	at org.python.core.PyMethod.__call__(PyMethod.java:208)
	at org.python.core.PyObject._jcallexc(PyObject.java:3555)
	at org.python.core.PyObject._jcall(PyObject.java:3587)
	at org.python.proxies.PerfTrace$TraceLoader$0.load_trace(Unknown Source)
	at test.Test.main(Test.java:18)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)


Are there any known incompatabilities between the bz2 implementations of pyhton vs. jython or known limitations? 

Regards,
Andreas
msg7997 (view) Author: Frank Wierzbicki (fwierzbicki) Date: 2013-04-19.23:14:13
Hi Andreas,

The bz2 implementation is brand new and currently causes some test failures in test_tarfile.py. You have likely found a bug in the new code. Thanks for reporting!
msg8231 (view) Author: Indra Talip (indra) Date: 2014-01-16.10:31:20
A possible fix as a pull request here - https://bitbucket.org/jython/jython/pull-request/24/the-pybz2decompressor-should-check-that/diff
msg8481 (view) Author: Jim Baker (zyasoft) Date: 2014-05-21.21:34:26
Probably fixed in beta 3; could akoenig please retry since we don't have the data files you were using?
msg8557 (view) Author: Jim Baker (zyasoft) Date: 2014-05-22.02:26:42
Also this should be pending, given that our tests now pass
msg8575 (view) Author: Indra Talip (indra) Date: 2014-05-25.10:01:05
I don't beleive that the changes in pullrequest have been merged so I don't think this has been fixed.
msg8578 (view) Author: Jim Baker (zyasoft) Date: 2014-05-27.16:41:20
Moving back to open, need to take a look at Indra's PR
msg8579 (view) Author: Jim Baker (zyasoft) Date: 2014-05-27.16:41:33
Target beta 4
msg8782 (view) Author: Jim Baker (zyasoft) Date: 2014-06-22.06:04:25
Fixed as of http://hg.python.org/jython/rev/02a31eb57af2
History
Date User Action Args
2014-07-01 04:04:40zyasoftsetstatus: pending -> closed
2014-06-22 06:04:25zyasoftsetstatus: open -> pending
resolution: remind -> fixed
messages: + msg8782
2014-05-27 16:41:33zyasoftsetmessages: + msg8579
2014-05-27 16:41:21zyasoftsetstatus: pending -> open
messages: + msg8578
2014-05-25 10:01:06indrasetmessages: + msg8575
2014-05-22 02:26:42zyasoftsetstatus: open -> pending
messages: + msg8557
2014-05-21 21:34:26zyasoftsetresolution: remind
messages: + msg8481
nosy: + zyasoft
2014-01-16 10:31:20indrasetnosy: + indra
messages: + msg8231
2013-04-19 23:14:14fwierzbickisetnosy: + fwierzbicki
messages: + msg7997
2013-04-12 14:55:02akoenigcreate