Title: test_urllib2net fails (on Travis CI)
Type: Severity: normal
Components: Versions:
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: adamburke, darjus, jamesmudd, jeff.allen, stefan.richthofer
Priority: Keywords: test failure causes

Created on 2018-10-13.11:23:54 by stefan.richthofer, last changed 2019-11-02.10:53:06 by jeff.allen.

msg12135 (view) Author: Stefan Richthofer (stefan.richthofer) Date: 2018-10-13.11:23:52
I noticed that rather recently (some months ago) test_urllib2net started to fail on TravisCI. I went through the github commit history to identity the satrting point of failure.
It seems to be this commit:
where it consequently starts failing.
One commit earlier ( the test still went well.

Routinely breaking tests make TravisCI less useful for us, so this should be cleaned up.

Given that the "breaking" PR is titled "Refactor PyJavaType.init for readability (no material change)." I suspect this issue might be induced some change in Travis CI. Traditionally, networking tests were not so reliable anyway. However also in this case we should consider to skip this test or better only the broken bit.

Any ideas what actually goes wrong?
msg12136 (view) Author: Jeff Allen (jeff.allen) Date: 2018-10-13.16:43:33
It is a fairly big refactor ... and it does not quite live up to the description. But the thing I know I broke here (sorry):
I fixed here:

test_urllib2net passes for me at both these revisions on Java 7 & 8. Also at the tip on 11. All on Windows.

I'd been attributing those failures to the network environbment in which the CI is run. I agree it is useless if it just shows red all the time for reasons we think are spurious, but there is a bit to do to fix it. I can't say I really follow the scripts. Our CI needs revisiting on quite a few points.
msg12138 (view) Author: Stefan Richthofer (stefan.richthofer) Date: 2018-10-17.20:15:58
The CI (Travis, not looking at the others) performs some fatally failing runs due to some config issues. However, there are also some okay runs and for these the only failure is test_urllib2net since some time.
So. fixing (or muting) this issue, turns at least some runs green, providing a better look than currently.

     [exec] 379 tests OK.
     [exec] 2 tests skipped:
     [exec]     test_codecmaps_hk test_curses
     [exec] 1 test failed:
     [exec]     test_urllib2net
     [exec] 1 fail unexpected:
     [exec]     test_urllib2net
     [exec] Re-running failed tests in verbose mode
     [exec] Re-running test 'test_urllib2net' in verbose mode
     [exec] test test_urllib2net failed -- multiple errors occurred

Unfortunately the "verbose" rerun of test_urllib2net is not very enlightening. I suspect it will be tedious to find the cause.
msg12139 (view) Author: Stefan Richthofer (stefan.richthofer) Date: 2018-10-17.20:18:23
In another run I found this:

     [exec] Re-running test 'test_urllib2net' in verbose mode
     [exec] test test_urllib2net failed -- Traceback (most recent call last):
     [exec]   File "/home/travis/build/jythontools/jython/dist/Lib/test/", line 294, in test_ftp_basic
     [exec]     u = _urlopen_with_retry(self.FTP_HOST)
     [exec]   File "/home/travis/build/jythontools/jython/dist/Lib/test/", line 27, in wrapped
     [exec]     return _retry_thrice(func, exc, *args, **kwargs)
     [exec]   File "/home/travis/build/jythontools/jython/dist/Lib/test/", line 23, in _retry_thrice
     [exec]     raise last_exc
     [exec] URLError: <urlopen error ftp error: 425 Security: Bad IP connecting.>
msg12488 (view) Author: Adam Burke (adamburke) Date: 2019-05-03.12:37:27
Looks like there was a Travis infrastructure change around this time that causes severe instability to anything using the ftp protocol.

Network calls are often flaky on CI infra. Perhaps the test can be skipped in CI, or tweaked to remove ftp on CI, or similar.
msg12526 (view) Author: Adam Burke (adamburke) Date: 2019-05-22.01:48:56
msg12528 (view) Author: Adam Burke (adamburke) Date: 2019-05-22.01:49:25
Corresponding CPython bug
msg12568 (view) Author: Jeff Allen (jeff.allen) Date: 2019-06-28.16:45:23
Fix by Adam (thanks) now at:
Date User Action Args
2019-11-02 10:53:06jeff.allensetstatus: pending -> closed
2019-06-28 16:45:23jeff.allensetkeywords: + test failure causes
status: open -> pending
resolution: fixed
messages: + msg12568
2019-05-22 01:49:25adamburkesetmessages: + msg12528
2019-05-22 01:48:56adamburkesetmessages: + msg12526
2019-05-03 12:37:28adamburkesetnosy: + adamburke
messages: + msg12488
2018-10-17 20:18:23stefan.richthofersetmessages: + msg12139
2018-10-17 20:15:59stefan.richthofersetmessages: + msg12138
2018-10-13 16:43:34jeff.allensetnosy: + darjus
messages: + msg12136
2018-10-13 11:23:54stefan.richthofercreate