Title: Update standard library in lib-python/2.7 to 2.7.latest
Type: Severity: normal
Components: Library Versions: Jython 2.7
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: ajdavis, jeff.allen, pdina, stefan.richthofer, zyasoft
Priority: normal Keywords:

Created on 2015-01-12.18:11:49 by zyasoft, last changed 2018-03-17.13:56:12 by stefan.richthofer.

msg9377 (view) Author: Jim Baker (zyasoft) Date: 2015-01-12.18:11:48
Except for possibly ensurepip, defer to 2.7.1 (or possibly later, but this seems like it's best done in 2.7.1, given how late we are in this process)

The relevant branch is

hg clone
hg update 2.7

then copy over to lib-python/2.7

Failing tests, after removing changes to json and sre_*:

     [exec] 26 fails unexpected:
     [exec]     test_base64 test_csv test_dictviews test_doctest test_email
     [exec]     test_email_renamed test_ensurepip test_fileinput test_grammar
     [exec]     test_gzip test_imghdr test_inspect test_ntpath test_pep263
     [exec]     test_robotparser test_select test_select_new test_socket test_ssl
     [exec]     test_tarfile test_traceback test_urllib2_localnet test_urllib2net
     [exec]     test_urllibnet test_userstring test_xdrlib

I didn't look closely at every single failing test, but I did go through a number:

Various SSL certificate issues. Jython trunk has supported for a while what CPython 2.7.9 just introduced - SSL certificate validation by default, using the underlying OS trusted certs. But we still haven't implemented these changes in the ssl module
Need to update our patches - many failures here
json/test/test_scanstring in test_surrogates contains surrogates, which stops our compiler, since we don't support literal surrogates. I'm going to add a more informative message in the jycompile task to note which file causes it to fail, since otherwise it's an unknown mess.
_hashlib does not contain openssl specific hash algorithms, which hashlib now attempts to import
Some updates to sre which cause issues. I didn't diagnose further, but reverting sre_*.py to the previous version made this go away this causes issues in test_doctest2 and related re dependent tests. (We should probably fix our SRE support, but that's a lot of work.)
test_csv, test_userstring, and probably many other tests are more stringent about the exception messages
various tests like test_robotparser now insist on 

Interesting leak of underlying Java seen here, this should be fixed:

FAILED (errors=1)
Traceback (most recent call last):
  File "dist/Lib/test/", line 120, in <module>
  File "dist/Lib/test/", line 117, in test_main
  File "/Users/jbaker/jythondev/jython27/dist/Lib/test/", line 1274, in run_unittest
  File "/Users/jbaker/jythondev/jython27/dist/Lib/test/", line 1257, in _run_suite
    raise TestFailed(err)
test.test_support.TestFailed: Traceback (most recent call last):
  File "dist/Lib/test/", line 77, in test_bad_args
  File "/Users/jbaker/jythondev/jython27/dist/Lib/", line 21, in what
    res = tf(h, f)
  File "/Users/jbaker/jythondev/jython27/dist/Lib/", line 37, in test_jpeg
    if h[6:10] == 'JFIF':
TypeError: 'java.nio.HeapByteBuffer' object is unsubscriptable

This internals leakage is easy enough to reproduce:

>>> open("NEWS", "rb").fileno().read(8)
java.nio.HeapByteBuffer[pos=0 lim=8 cap=8]
msg9380 (view) Author: Jim Baker (zyasoft) Date: 2015-01-13.19:25:18
Initial ensurepip support was added as of
msg11810 (view) Author: Jeff Allen (jeff.allen) Date: 2018-03-16.20:48:11
Surely the most sensible time to do this (whatever the target version) is just after we release? Not in the run-up, anyway. We can then work off the breakage.
msg11824 (view) Author: Stefan Richthofer (stefan.richthofer) Date: 2018-03-17.13:56:12
About a year ago in thephase of releasing Jython 2.7.1 I experimented with this update and it causes *plenty* of failures (more than 50 or so).
We can check this for a recent Jython, but I suspect it will be not significantly better.
I'd recommend this a won't fix for 2.7 and to rather focus on Jython 3 std lib support.
Date User Action Args
2018-03-17 13:56:12stefan.richthofersetnosy: + stefan.richthofer
messages: + msg11824
2018-03-16 20:48:12jeff.allensetnosy: + jeff.allen
messages: + msg11810
components: + Library
title: Update standard library in lib-python/2.7 to 2.7.9 -> Update standard library in lib-python/2.7 to 2.7.latest
2015-04-13 16:58:58ajdavissetnosy: + ajdavis
2015-01-13 19:25:18zyasoftsetmessages: + msg9380
2015-01-13 07:49:23pdinasetnosy: + pdina
2015-01-12 18:11:49zyasoftcreate