Issue2808

classification
Title: lib2to3 test failures on Windows JDK 11
Type: Severity: normal
Components: Core Versions: Jython 2.7
Milestone:
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: adamburke
Priority: Keywords:

Created on 2019-09-30.09:59:16 by adamburke, last changed 2019-10-02.02:29:21 by adamburke.

Messages
msg12664 (view) Author: Adam Burke (adamburke) Date: 2019-09-30.09:59:16
ib2to3 failure on windows is caused by diffs between CR and CRLF under Java 11


C:\Users\Adam\jython\jython2>dist\bin\jython.exe -m test.regrtest -e test_lib2to3
test_lib2to3
test test_lib2to3 failed -- Traceback (most recent call last):
  File "C:\Users\Adam\jython\jython2\dist\Lib\lib2to3\tests\test_parser.py", line 175, in test_all_project_files
    self.fail("Idempotency failed: {} using {} encoding\n{}".
AssertionError: Idempotency failed: C:\Users\Adam\jython\jython2\dist\Lib\lib2to3\btm_matcher.py using utf-8 encoding
original line 1:
"""A bottom-up tree matching algorithm implementation meant to speed

 differs from:
"""A bottom-up tree matching algorithm implementation meant to speed

Lines differ at char 68: 0d vs 0a (of 70 vs 69)

1 test failed:
    test_lib2to3
1 fail unexpected:
    test_lib2to3


With extra diff debug in Lib/lib2to3/tests/test_parser.py

def diff(fn, result, encoding):
    "A diff the result and original file content independent of OS."
    r = iter(result.encode(encoding).splitlines(True))
    lineNumber = 0
    with open(fn, "rb") as f:
        for line in f:
            lineNumber += 1
            rline = next(r)
            if rline != line:
                return "original line {}:\n{}\n differs from:\n{}\n{}". \
                       format(lineNumber,line,rline,diffLine(line,rline) )
    return False

def diffLine(orig,result):
    charNumber = 0
    for c in orig:
        if c != result[charNumber]:
            return "Lines differ at char {}: {} vs {} (of {} vs {})".format(
                    charNumber,
                    binascii.hexlify(c),
                    binascii.hexlify(result[charNumber]),
                    len(orig),
                    len(result) )
        charNumber += 1

Have a candidate fix but checking a few things. The test code that fails is Jython specific. Python 2.x doesn't run this on Windows, Python 3.x relies on improved string equality (no need for unicode() function).
msg12666 (view) Author: Adam Burke (adamburke) Date: 2019-10-02.02:29:21
PR https://github.com/jythontools/jython/pull/154
History
Date User Action Args
2019-10-02 02:29:21adamburkesetmessages: + msg12666
2019-09-30 09:59:16adamburkecreate