Title: Failure in test_sort.TestBase.testStressfully on Java 11
Type: crash Severity: normal
Components: Core Versions:
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: jeff.allen, zyasoft
Priority: normal Keywords: test failure causes

Created on 2019-10-05.09:14:05 by jeff.allen, last changed 2019-10-06.17:05:47 by zyasoft.

msg12678 (view) Author: Jeff Allen (jeff.allen) Date: 2019-10-05.09:14:05
This is showing up intermittently in the CI on Java 11:

    [exec] test test_sort failed -- Traceback (most recent call last):
     [exec]   File "/home/travis/build/jythontools/jython/dist/Lib/test/", line 123, in testStressfully
     [exec]     s.sort()
     [exec]   File "/home/travis/build/jythontools/jython/dist/Lib/test/", line 123, in testStressfully
     [exec]     s.sort()
     [exec] IllegalArgumentException: java.lang.IllegalArgumentException: Comparison method violates its general contract!

I am raising the issue and will reference it in a skip as I don't want to divert from the beta release. We made an attempt to handle this by modifying the test, but it was not fully effective and I don't think we've understood it correctly. (I'll revert that.)

The test fails where we supply a pathological comparison function. I *think* all that is being tested is that in those cases, any error is handled and leaves the list unsorted. However, Java (in later versions) detects that the comparator is badly behaved and raises its own error that is not handled.

See #2399 (and which goes a long way towards solving a related problem in the context of test_list by storing the exception and rethrowing it. An extension to (certain) Java exceptions may be what we need. But also, what about proxied Java lists doing the same?

I don't find the test to be well-designed: it tests too many things at once and I got confused reading it. OTOH it comes to us from CPython and it's nice to use the same source.
msg12696 (view) Author: Jim Baker (zyasoft) Date: 2019-10-06.17:05:47
This is a good example of a leaky abstraction! Previously in Jython we had our own implementation of TimSort, which was subtly buggy. When we switched to Java's version, it works well except for these pathological cases and in this case, throws a bad exception.

What this test is really trying to do is cause a segfault in CPython, which is something that the CPython implementation defensively codes to avoid, much like Java. Therefore  a skip seems reasonable for now (Java does not segfault after all when run). We can revisit with appropriate exception wrapping in the future, however, it seems to me to be of relatively low priority.
Date User Action Args
2019-10-06 17:05:47zyasoftsetnosy: + zyasoft
messages: + msg12696
2019-10-05 09:14:05jeff.allencreate