Title: lib2to3 test failures on Windows JDK 11
Type: behaviour Severity: normal
Components: Library Versions: Jython 2.7
Milestone: Jython 2.7.2
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: adamburke, jeff.allen
Priority: normal Keywords: test failure causes

Created on 2019-09-30.09:59:16 by adamburke, last changed 2020-02-23.21:52:40 by jeff.allen.

msg12664 (view) Author: Adam Burke (adamburke) Date: 2019-09-30.09:59:16
ib2to3 failure on windows is caused by diffs between CR and CRLF under Java 11

C:\Users\Adam\jython\jython2>dist\bin\jython.exe -m test.regrtest -e test_lib2to3
test test_lib2to3 failed -- Traceback (most recent call last):
  File "C:\Users\Adam\jython\jython2\dist\Lib\lib2to3\tests\", line 175, in test_all_project_files"Idempotency failed: {} using {} encoding\n{}".
AssertionError: Idempotency failed: C:\Users\Adam\jython\jython2\dist\Lib\lib2to3\ using utf-8 encoding
original line 1:
"""A bottom-up tree matching algorithm implementation meant to speed

 differs from:
"""A bottom-up tree matching algorithm implementation meant to speed

Lines differ at char 68: 0d vs 0a (of 70 vs 69)

1 test failed:
1 fail unexpected:

With extra diff debug in Lib/lib2to3/tests/

def diff(fn, result, encoding):
    "A diff the result and original file content independent of OS."
    r = iter(result.encode(encoding).splitlines(True))
    lineNumber = 0
    with open(fn, "rb") as f:
        for line in f:
            lineNumber += 1
            rline = next(r)
            if rline != line:
                return "original line {}:\n{}\n differs from:\n{}\n{}". \
                       format(lineNumber,line,rline,diffLine(line,rline) )
    return False

def diffLine(orig,result):
    charNumber = 0
    for c in orig:
        if c != result[charNumber]:
            return "Lines differ at char {}: {} vs {} (of {} vs {})".format(
                    len(result) )
        charNumber += 1

Have a candidate fix but checking a few things. The test code that fails is Jython specific. Python 2.x doesn't run this on Windows, Python 3.x relies on improved string equality (no need for unicode() function).
msg12666 (view) Author: Adam Burke (adamburke) Date: 2019-10-02.02:29:21
msg12966 (view) Author: Jeff Allen (jeff.allen) Date: 2020-02-01.14:59:08
I have pulled the patch from GitHub and it looks fine to me.
msg12968 (view) Author: Jeff Allen (jeff.allen) Date: 2020-02-02.07:33:50
Seems like this would not be specific to Java 11. Anyway, now in at Thanks Adam.
Date User Action Args
2020-02-23 21:52:40jeff.allensetstatus: pending -> closed
2020-02-02 07:33:50jeff.allensetstatus: open -> pending
resolution: accepted -> fixed
messages: + msg12968
components: + Library, - Core
2020-02-01 14:59:08jeff.allensettype: behaviour
nosy: + jeff.allen
messages: + msg12966
priority: normal
milestone: Jython 2.7.2
keywords: + test failure causes
resolution: accepted
2019-10-02 02:29:21adamburkesetmessages: + msg12666
2019-09-30 09:59:16adamburkecreate