Issue2808

ib2to3 failure on windows is caused by diffs between CR and CRLF under Java 11


C:\Users\Adam\jython\jython2>dist\bin\jython.exe -m test.regrtest -e test_lib2to3
test_lib2to3
test test_lib2to3 failed -- Traceback (most recent call last):
  File "C:\Users\Adam\jython\jython2\dist\Lib\lib2to3\tests\test_parser.py", line 175, in test_all_project_files
    self.fail("Idempotency failed: {} using {} encoding\n{}".
AssertionError: Idempotency failed: C:\Users\Adam\jython\jython2\dist\Lib\lib2to3\btm_matcher.py using utf-8 encoding
original line 1:
"""A bottom-up tree matching algorithm implementation meant to speed

 differs from:
"""A bottom-up tree matching algorithm implementation meant to speed

Lines differ at char 68: 0d vs 0a (of 70 vs 69)

1 test failed:
    test_lib2to3
1 fail unexpected:
    test_lib2to3


With extra diff debug in Lib/lib2to3/tests/test_parser.py

def diff(fn, result, encoding):
    "A diff the result and original file content independent of OS."
    r = iter(result.encode(encoding).splitlines(True))
    lineNumber = 0
    with open(fn, "rb") as f:
        for line in f:
            lineNumber += 1
            rline = next(r)
            if rline != line:
                return "original line {}:\n{}\n differs from:\n{}\n{}". \
                       format(lineNumber,line,rline,diffLine(line,rline) )
    return False

def diffLine(orig,result):
    charNumber = 0
    for c in orig:
        if c != result[charNumber]:
            return "Lines differ at char {}: {} vs {} (of {} vs {})".format(
                    charNumber,
                    binascii.hexlify(c),
                    binascii.hexlify(result[charNumber]),
                    len(orig),
                    len(result) )
        charNumber += 1

Have a candidate fix but checking a few things. The test code that fails is Jython specific. Python 2.x doesn't run this on Windows, Python 3.x relies on improved string equality (no need for unicode() function).

PR https://github.com/jythontools/jython/pull/154

I have pulled the patch from GitHub and it looks fine to me.

Seems like this would not be specific to Java 11. Anyway, now in at https://hg.python.org/jython/rev/e81878891a4b. Thanks Adam.