Message12664

ib2to3 failure on windows is caused by diffs between CR and CRLF under Java 11


C:\Users\Adam\jython\jython2>dist\bin\jython.exe -m test.regrtest -e test_lib2to3
test_lib2to3
test test_lib2to3 failed -- Traceback (most recent call last):
  File "C:\Users\Adam\jython\jython2\dist\Lib\lib2to3\tests\test_parser.py", line 175, in test_all_project_files
    self.fail("Idempotency failed: {} using {} encoding\n{}".
AssertionError: Idempotency failed: C:\Users\Adam\jython\jython2\dist\Lib\lib2to3\btm_matcher.py using utf-8 encoding
original line 1:
"""A bottom-up tree matching algorithm implementation meant to speed

 differs from:
"""A bottom-up tree matching algorithm implementation meant to speed

Lines differ at char 68: 0d vs 0a (of 70 vs 69)

1 test failed:
    test_lib2to3
1 fail unexpected:
    test_lib2to3


With extra diff debug in Lib/lib2to3/tests/test_parser.py

def diff(fn, result, encoding):
    "A diff the result and original file content independent of OS."
    r = iter(result.encode(encoding).splitlines(True))
    lineNumber = 0
    with open(fn, "rb") as f:
        for line in f:
            lineNumber += 1
            rline = next(r)
            if rline != line:
                return "original line {}:\n{}\n differs from:\n{}\n{}". \
                       format(lineNumber,line,rline,diffLine(line,rline) )
    return False

def diffLine(orig,result):
    charNumber = 0
    for c in orig:
        if c != result[charNumber]:
            return "Lines differ at char {}: {} vs {} (of {} vs {})".format(
                    charNumber,
                    binascii.hexlify(c),
                    binascii.hexlify(result[charNumber]),
                    len(orig),
                    len(result) )
        charNumber += 1

Have a candidate fix but checking a few things. The test code that fails is Jython specific. Python 2.x doesn't run this on Windows, Python 3.x relies on improved string equality (no need for unicode() function).