Issue2312

classification
Title: Test failures related to encoding in a subprocess on Windows
Type: behaviour Severity: normal
Components: Library Versions: Jython 2.7
Milestone:
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: jeff.allen Nosy List: jeff.allen
Priority: normal Keywords: test failure causes

Created on 2015-04-06.19:48:05 by jeff.allen, last changed 2018-03-22.07:34:07 by jeff.allen.

Messages
msg9770 (view) Author: Jeff Allen (jeff.allen) Date: 2015-04-06.19:48:05
Observed on Windows 7 with Jython 2.7rc2 test.test_file2k reports one failure:

...
test_unicode (__main__.StdoutTests) ... Traceback (most recent call last):
  File "<string>", line 352, in <module>
  File "<string>", line 316, in main
  File "c:\jythondev\jython27\src\shell\build\jython\out00-PYZ.pyz\encodings.utf_16_le", line 16, in decode
UnicodeDecodeError: 'utf16' codec can't decode byte 0x29 in position 58: truncated data
FAIL

======================================================================
FAIL: test_unicode (__main__.StdoutTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\jython\2.7rc2\Lib\test\test_file2k.py", line 737, in test_unicode
    check_message(u'15\u20ac', "utf-16-le", '1\x005\x00\xac\x20')
  File "C:\jython\2.7rc2\Lib\test\test_file2k.py", line 728, in check_message
    stdout = get_message(encoding,
  File "C:\jython\2.7rc2\Lib\test\test_file2k.py", line 724, in get_message
    self.assertEqual(process.returncode, 0)
AssertionError: -1 != 0

----------------------------------------------------------------------

The message about the encoding emerges, I think, from the stderr of the subprocess, which then returns the much less useful error status that fails the test.
msg9774 (view) Author: Jeff Allen (jeff.allen) Date: 2015-04-06.20:28:46
These failures in test_sys and test_sys_jy look like the same issue, being failures to encode/decode correctly in a subprocess.

> jython -m test.test_sys
...
test_ioencoding (__main__.SysModuleTest) ... Traceback (most recent call last):
  File "<string>", line 352, in <module>
  File "<string>", line 316, in main
  File "c:\jythondev\jython27\src\shell\build\jython\out00-PYZ.pyz\encodings.cp424", line 15, in decode
UnicodeDecodeError: 'charmap' codec can't decode byte 0x70 in position 0: character maps to <undefined>
FAIL
test_original_displayhook (__main__.SysModuleTest) ... ok
test_original_excepthook (__main__.SysModuleTest) ... ok
test_recursionlimit (__main__.SysModuleTest) ... ok

======================================================================
FAIL: test_ioencoding (__main__.SysModuleTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\jython\2.7rc2\Lib\test\test_sys.py", line 265, in test_ioencoding
    self.assertEqual(out, unichr(0xa2).encode("cp424"))
AssertionError: '' != 'J'

----------------------------------------------------------------------
...

> jython -m test.test_sys_jy
...
Unicode roundtrips successfully through sys.argv arguments ... FAIL
test_prompts_not_defined_if_not_interactive (__main__.InteractivePromptTest) ... ok

======================================================================
FAIL: test_unicode_argv (__main__.SysArgvTest)
Unicode roundtrips successfully through sys.argv arguments
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\jython\2.7rc2\Lib\test\test_sys_jy.py", line 255, in test_unicode_argv
    self.assertEqual(p.stdout.read().decode("utf-8"), zhongwen)
AssertionError: u'??' != u'\u4e2d\u6587'
- ??
+ \u4e2d\u6587
msg11807 (view) Author: Jeff Allen (jeff.allen) Date: 2018-03-16.07:40:29
Encoding (including the encoding of subprocess streams) has improved since I reased this. Some modules noted here are expected failures in regrtest and others may have needless skips. We should review what's reported here to ensure:

* Tests are not skipped wholesale unnecessarily.
* Fixed things are no longer skipped.
* Anything still skipped or expected to fail in regrtest cites an issue number.

Assigning to me for now.
History
Date User Action Args
2018-03-22 07:34:07jeff.allensetpriority: normal
2018-03-16 07:40:30jeff.allensetassignee: jeff.allen
messages: + msg11807
2015-04-06 20:28:46jeff.allensetmessages: + msg9774
title: test_file2k utf-16 encoding failure on Windows -> Test failures related to encoding in a subprocess on Windows
2015-04-06 19:48:05jeff.allencreate