Issue2434

Originally found while investigating unexpected Jython behaviour in urllib3 issues #743[0] and #747[1].

On CPython and PyPy, when using a zlib.decompressobj, each call to decompress passes Z_SYNC_FLUSH to zlib. This means that, in practice, the call to flush() is a no-op on CPython and PyPy when decompressing zlib compressed streams. This has gone so far that the CPython devs have considered deprecating decompressobj.flush[2].

However, this behaviour is not matched by Jython. Jython uses the standard JDK java.util.zip package to wrap zlib. Specifically, Jython calls the inflate() method with arity 3 as shown here[3], or with arity 1 as shown here[4]. The problem is that these specific overrides of deflate() set the flush mode to Z_NO_FLUSH, as shown in the documentation for JDK 8 here[5] and here[6]. This causes the zlib library to behave very differently for CPython and PyPy.

Sadly, I don't know zlib well enough to know if this matters: it may be that the net effect is totally identical across Jython and CPython and PyPy, but without passing the same flush parameter in all three cases it's hard to be sure.

[0]: https://github.com/shazow/urllib3/issues/743
[1]: https://github.com/shazow/urllib3/pull/747
[2]: https://bugs.python.org/issue23200
[3]: https://bitbucket.org/jython/jython/src/6855fa289d48cf741a3daed2a0bb9c9c87cdf07f/Lib/zlib.py?at=default&fileviewer=file-view-default#zlib.py-200
[4]: https://bitbucket.org/jython/jython/src/6855fa289d48cf741a3daed2a0bb9c9c87cdf07f/Lib/zlib.py?at=default&fileviewer=file-view-default#zlib.py-202
[5]: https://docs.oracle.com/javase/8/docs/api/java/util/zip/Deflater.html#deflate-byte:A-int-int-
[6]: https://docs.oracle.com/javase/8/docs/api/java/util/zip/Deflater.html#deflate-byte:A-

Most likely this is blocking recent changes to pip/setuptools. But at least this should be easy.

Urgent to block beta 3.

Fixed as of https://hg.python.org/jython/rev/767d5206e120

I also ran the urllib3 test suite (using master) with nose. All tests now pass, excluding the tornado-derived tests, which rely on fcntl and may be problematic for being too tightly tied to posix (we will see with any work on #1943) and coverage tests (relies on #1638).

I also opened a bug to address Tornado and indirectly the corresponding failing tests: #2458