Issue2397

classification
Title: test.test_os_jy fails on Windows due to non-byte PyString
Type: Severity: normal
Components: Core, Library Versions: Jython 2.7
Milestone:
process
Status: closed Resolution:
Dependencies: Superseder:
Assigned To: jeff.allen Nosy List: jeff.allen
Priority: Keywords:

Created on 2015-09-13.08:39:36 by jeff.allen, last changed 2015-10-26.20:42:02 by zyasoft.

Messages
msg10256 (view) Author: Jeff Allen (jeff.allen) Date: 2015-09-13.08:39:35
Edited for brevity:

> jython -m test.test_os_jy
...
test_env (__main__.UnicodeTestCase) ... Exception in thread "main" java.lang.IllegalArgumentException: Cannot create PyString with non-byte value
        at org.python.core.PyString.<init>(PyString.java:64)
        at org.python.core.PyString.<init>(PyString.java:70)
        at org.python.core.packagecache.PathPackageManager.addDirectory(PathPackageManager.java:201)
        at org.python.core.packagecache.PathPackageManager.addClassPath(PathPackageManager.java:232)
        at org.python.core.packagecache.SysPackageManager.findAllPackages(SysPackageManager.java:96)
        at org.python.core.packagecache.SysPackageManager.<init>(SysPackageManager.java:39)
        at org.python.core.PySystemState.initPackages(PySystemState.java:1122)
        at org.python.core.PySystemState.doInitialize(PySystemState.java:1052)
        at org.python.core.PySystemState.initialize(PySystemState.java:969)
        at org.python.core.PySystemState.initialize(PySystemState.java:925)
        at org.python.core.PySystemState.initialize(PySystemState.java:920)
        at org.python.util.jython.run(jython.java:263)
        at org.python.util.jython.main(jython.java:142)
FAIL
test_getcwd (__main__.UnicodeTestCase) ... Exception in thread "main" java.lang.IllegalArgumentException: Cannot create PyString with non-byte value
          ... identical Java stack dump
FAIL
...
======================================================================
FAIL: test_env (__main__.UnicodeTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\jython\2.7.1b1\Lib\test\test_os_jy.py", line 208, in test_env
    self.assertEqual(p.stdout.read().decode("utf-8"), u"首页")
AssertionError: u'' != u'\u9996\u9875'
+ \u9996\u9875

======================================================================
FAIL: test_getcwd (__main__.UnicodeTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\jython\2.7.1b1\Lib\test\test_os_jy.py", line 216, in test_getcwd
    self.assertEqual(p.stdout.read().decode("utf-8"), temp_cwd)
AssertionError: u'' != u'C:\\Users\\Jeff\\Documents\\Jython\\271b1\\tempcwd-\u4e2d\u6587'
+ C:\Users\Jeff\Documents\Jython\271b1\tempcwd-\u4e2d\u6587

----------------------------------------------------------------------
Ran 24 tests in 8.331s

FAILED (failures=2, skipped=11)
...


I'm fairly sure the Java stack dump is actually from the sub-process invoked in the tests. I think it is connected to non-ascii values in a path #2293 and #2356.

A search through the commits locates the first one where this occurs as last big change to the launcher (585cc365d4bd : views badly in the web interface because of large binary content).
msg10257 (view) Author: Jeff Allen (jeff.allen) Date: 2015-09-13.16:37:53
The problem occurs in processing Java String file or directory names into PyString path elements.

I'm pretty sure CPython 2 would take a byte-smuggling approach to the problem of non-ascii filenames. As long as it does not lead to worse trouble downstream, I think I prefer liberal use of Py.newStringOrUnicode as in this: https://hg.python.org/jython/rev/ea036792f304 .
msg10351 (view) Author: Jeff Allen (jeff.allen) Date: 2015-10-11.16:59:43
I claim this is fixed in https://hg.python.org/jython/rev/b6180b5dc4c6.
History
Date User Action Args
2015-10-26 20:42:02zyasoftsetstatus: pending -> closed
2015-10-11 16:59:43jeff.allensetstatus: open -> pending
messages: + msg10351
2015-09-13 16:37:53jeff.allensetassignee: jeff.allen
messages: + msg10257
2015-09-13 08:39:36jeff.allencreate