Issue2808

classification
Title: lib2to3 test failures on Windows JDK 11
Type: behaviour Severity: normal
Components: Library Versions: Jython 2.7
Milestone: Jython 2.7.2
process
Status: pending Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: adamburke, jeff.allen
Priority: normal Keywords: test failure causes

Created on 2019-09-30.09:59:16 by adamburke, last changed 2020-02-02.07:33:50 by jeff.allen.

Messages
msg12664 (view) Author: Adam Burke (adamburke) Date: 2019-09-30.09:59:16
ib2to3 failure on windows is caused by diffs between CR and CRLF under Java 11


C:\Users\Adam\jython\jython2>dist\bin\jython.exe -m test.regrtest -e test_lib2to3
test_lib2to3
test test_lib2to3 failed -- Traceback (most recent call last):
  File "C:\Users\Adam\jython\jython2\dist\Lib\lib2to3\tests\test_parser.py", line 175, in test_all_project_files
    self.fail("Idempotency failed: {} using {} encoding\n{}".
AssertionError: Idempotency failed: C:\Users\Adam\jython\jython2\dist\Lib\lib2to3\btm_matcher.py using utf-8 encoding
original line 1:
"""A bottom-up tree matching algorithm implementation meant to speed

 differs from:
"""A bottom-up tree matching algorithm implementation meant to speed

Lines differ at char 68: 0d vs 0a (of 70 vs 69)

1 test failed:
    test_lib2to3
1 fail unexpected:
    test_lib2to3


With extra diff debug in Lib/lib2to3/tests/test_parser.py

def diff(fn, result, encoding):
    "A diff the result and original file content independent of OS."
    r = iter(result.encode(encoding).splitlines(True))
    lineNumber = 0
    with open(fn, "rb") as f:
        for line in f:
            lineNumber += 1
            rline = next(r)
            if rline != line:
                return "original line {}:\n{}\n differs from:\n{}\n{}". \
                       format(lineNumber,line,rline,diffLine(line,rline) )
    return False

def diffLine(orig,result):
    charNumber = 0
    for c in orig:
        if c != result[charNumber]:
            return "Lines differ at char {}: {} vs {} (of {} vs {})".format(
                    charNumber,
                    binascii.hexlify(c),
                    binascii.hexlify(result[charNumber]),
                    len(orig),
                    len(result) )
        charNumber += 1

Have a candidate fix but checking a few things. The test code that fails is Jython specific. Python 2.x doesn't run this on Windows, Python 3.x relies on improved string equality (no need for unicode() function).
msg12666 (view) Author: Adam Burke (adamburke) Date: 2019-10-02.02:29:21
PR https://github.com/jythontools/jython/pull/154
msg12966 (view) Author: Jeff Allen (jeff.allen) Date: 2020-02-01.14:59:08
I have pulled the patch from GitHub and it looks fine to me.
msg12968 (view) Author: Jeff Allen (jeff.allen) Date: 2020-02-02.07:33:50
Seems like this would not be specific to Java 11. Anyway, now in at https://hg.python.org/jython/rev/e81878891a4b. Thanks Adam.
History
Date User Action Args
2020-02-02 07:33:50jeff.allensetstatus: open -> pending
resolution: accepted -> fixed
messages: + msg12968
components: + Library, - Core
2020-02-01 14:59:08jeff.allensettype: behaviour
nosy: + jeff.allen
messages: + msg12966
priority: normal
milestone: Jython 2.7.2
keywords: + test failure causes
resolution: accepted
2019-10-02 02:29:21adamburkesetmessages: + msg12666
2019-09-30 09:59:16adamburkecreate