Issue1445

Jython does not include the bz2 module.

I'm looking into this one. I'll update this ticket if for some reason I
get in too deep, but I'm trying to follow the
PortingPythonModulesToJython wiki page.

Hey Peter,

I'm just curious what need you had for bz2. Is another library using it?
The reason I ask is because Philip Jenvey mentioned that the hard part
is handling streaming, which the Java library from Apache doesn't support.

I'm not sure if the current test cases test for streaming or not, I'm
going to check into it. If they don't, maybe your need for bz2 can help
me think of a use case beyond "stream random data into/out of bz2".

Although if I have to create those use cases I can do that, too.

Mercurial uses bz2 for bundles and archives. See "hg help bundle" and
"hg help archive".

The module bz2 is still missing, and opening tar.bz2 files fails with:
"file could not be opened successfully"

It seems there are classes for bzip2 support in ant. Would it be possible to import them in the jython tree?

http://www.kohsuke.org/bzip2/

Here's a jython-dev thread with a little more detail

http://markmail.org/message/fwpt4nlvgiskgfur

Basically we can implement bz2 with the Apache library but it does not support streaming of decompression (which Python's bz2.BZ2Decompressor does). It should support encoding via a stream however (bz2.BZ2Compressor).

This would be better than nothing as I think most uses of the bz2 module are via bz2.BZ2File, which doesn't stream

Another alternative is a former Jython contributor tried implementing a port of the libbz2 C code to Java for his Jythonx project. I don't know what state this code is in though

Furthermore here's my attempt at writing a bz2 module with the Apache lib from a couple years ago. I think I updated it slightly to work with the Apache commons bz2 compressor, which is pretty much the same as http://www.kohsuke.org/bzip2/ under a different name

It'd be a good starting point

Attaching the other jythonx bz2 implemention

Please have a look at the following patch, which enables jython to read tar.bz2 files:
http://freehackers.org/~tnagy/bz2.diff

* is it permitted to import the apache classes like this?
* is a PyFile wrapper ok or is it more recommendable to use a class (in a python file or in a java file?)
* are there gotchas with using PyFile (memory leaks, file remaining open on win32, ...) ?

Note: this bug tracker does not allow attachments. It would be fine if the error message did not eat the message (not nice!)

ita - 

I was hoping the bz2.tar.bz2 I posted could be used as a starting point. It actually has a lot of BZ2File implemented and at least some of work for the stream objects (I think BZ2Compressor is done). Though it's in Java code which may not be as easy to deal with. Have you looked at it?

There's a test_bz2 that we need to be passing at least most of before we ship a bz2 module

Here's an updated version of the code that compiles against the current trunk. There's no BZ2Decompressor code included, fully supporting it will be the tricky bit

It might be worth while to simply port another implementation from scratch. This parallel implementation is supposed to be very fast: http://compression.ca/pbzip2/

It consists of 3000 lines of code.

Would pyflate be a possible base? http://www.paul.sladen.org/projects/pyflate/

How I got here: I’d love to have Mercurial support in Jython, because then we could ship hg infocalypse (and Mercurial) in freenet and use it as a decentral and anonymous wikiengine. 

All the code is in place (called fniki), except for integrating Mercurial in Java via Jython (the freenet maintainer doesn’t want non-java dependencies, so we have to get Mercurial running on Java).

There is now a new java implementation of bzip2:
http://code.google.com/p/jbzip2/

The interface of this implementation looks better than the one of Apache ant, but it is java 6 and so only for Jython 2.6 suitable.

What in bz2_v2 in the class BZ2File is missing, is the seek method. This is essential for the support of the tarfile module. My simple implementation idea is, for reading mode and the whence SEEK_SET: Do nothing when position given is the actual, when argument is greater than actual position do so many dummy reads as the difference, when the argument is smaller, reopen the underlying java stream and do dummy reads as often as the argument says.

This is similar for SEEK_CUR.

SEEK_END is in my opinion not supportable.

Another essential point for the tarfile support is, do not throw exceptions in the constructor, when the arguments are correct. You must defer that to the I/O operations like read() and seek(). When they are called raise an IOError("invalid data stream"). This is needed for the auto detection of the used compression algorithm when you give only "r" as mode to the tarfile.open().

Hi guys. 

I have implemented this module. I have gotten all tests to pass. I will prepare a patch this weekend and upload it here.

Nice - looking forward to it!

Were you able to get the patch working?

Added Julian Kennedy's version that supports all the necessary functionality, including streaming (de)compression.