Message10342

Author jeff.allen
Recipients adamburke, darjus, jeff.allen, zyasoft
Date 2015-10-09.17:29:18
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1444411759.51.0.656072594665.issue2393@psf.upfronthosting.co.za>
In-reply-to
Content
A clean-running regrtest is a worthwhile objective. And also the idea that users could help us test more thoroughly on more platforms.

I applied this patch to my current repo tip and it makes regrtest run without reporting errors, as advertised -- a new experience :).

Tests are suppressed in the patch that I don't have any trouble with. When I drop the main expected failure list additions into a file "my.tests" I see:

> dist\bin\jython dist\Lib\test\regrtest.py -f my.tests
...
23 tests OK.
10 tests failed:
    test_file2k test_httpservers test_netrc test_runpy test_shutil
    test_subprocess_jy test_sys test_tarfile test_urllib2 test_zipfile
23 tests passed unexpectedly:
    test___all__ test_bytes test_bz2 test_classpathimporter
    test_import test_import_pep328 test_java_integration
    test_jython_initializer test_list_jy test_logging test_marshal
    test_os_jy test_select test_socket test_sort test_ssl test_str
    test_string test_sys_jy test_unicode test_univnewlines
    test_userstring test_zipimport_jy

My repo is has a few extra fixes, but I doubt I have fixed that many things! So I would not include those unexpected passes. Tests that report skips are not failures (except morally ;). Tests skipped wholesale, can be "expected" separate from the expected failures.

I think the trivial batch script may be overkill: people will surely only run it if we ask (e.g. suggesting it in README.TXT), and we might as well give them a command to type/paste there. If we have jython_regrtest.bat, then I believe SETLOCAL will prevent JY_HOME leaking into the user's environment.

Is this just a typo?
-                            "test_applesingle"]
+                            "test_applesingle.pyingle"]

The thing bothering me, and it may just need getting used to, is that it's so quiet. How do we make regrtest nag *us* continually about things that need fixing, while only complaining (through our users or a build bot) about things that we didn't know? Skips in the test conceal less.

My recommendations (others free to argue):
1. Look into suppressing fewer tests. (Do things actually fail, or just contain skips?)
2. Fix: test_applesingle.pyingle
3. Add an invitation to README.TXT rather adding jython_regrtest.bat.
4. Add your name to ACKNOWLEDGEMENTS!
History
Date User Action Args
2015-10-09 17:29:19jeff.allensetmessageid: <1444411759.51.0.656072594665.issue2393@psf.upfronthosting.co.za>
2015-10-09 17:29:19jeff.allensetrecipients: + jeff.allen, zyasoft, darjus, adamburke
2015-10-09 17:29:19jeff.allenlinkissue2393 messages
2015-10-09 17:29:18jeff.allencreate