Issue2274

classification
Title: os.path.normpath can convert Unicode strings to byte strins
Type: Severity: normal
Components: Versions:
Milestone:
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: zyasoft Nosy List: pekka.klarck, zyasoft
Priority: high Keywords:

Created on 2015-02-25.12:48:34 by pekka.klarck, last changed 2015-03-20.18:28:29 by zyasoft.

Messages
msg9557 (view) Author: Pekka Klärck (pekka.klarck) Date: 2015-02-25.12:48:33
Depending on input, `os.path.normpath` turns Unicode strings to byte strings:

Jython 2.7b4 (default:3672e624962a, Feb 13 2015, 04:59:14) 
[Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)] on java1.7.0_60
Type "help", "copyright", "credits" or "license" for more information.
>>> from os.path import normpath
>>> normpath(u'.')
'.'
>>> normpath(u'/')
'/'
>>> normpath(u'..')
u'..'
>>> normpath(u'./')
'.'
msg9558 (view) Author: Pekka Klärck (pekka.klarck) Date: 2015-02-25.12:55:06
How severe this is depends on do we consider `os.path.abspath` turning byte strings to Unicode a bug or not (I'm in directory `böö` in these examples):

>>> abspath(u'.')
u'/home/peke/b\xf6\xf6'
>>> abspath('.')
u'/home/peke/b\xf6\xf6'
>>>

With CPython the latter returns byte string. Jython nowadays handles environment variables and such as Unicode, so perhaps Unicode is fine here too.

If we agree abspath always returning Unicode is fine, normpath possibly returning bytes is just a minor inconsistency. If abspath should return bytes or Unicode depending on the type of the input, like CPython does, there would be problems with `abspath(normpath(u'.'))` if the cwd would contain non-ASCII characters.
msg9621 (view) Author: Jim Baker (zyasoft) Date: 2015-03-11.20:27:15
This was a bug in CPython (http://bugs.python.org/issue5827), and can be readily be fixed by updating the normpath function (written in Python) to the latest in CPython 2.7 posixpath.py. It has already been updated in Jython's version of ntpath.py

We should also update the corresponding tests
msg9649 (view) Author: Jim Baker (zyasoft) Date: 2015-03-13.21:29:50
Fixed by removing Jython-specific test_posixpath, posixpath, and minor update to pwd - always the best sort of fix, it means underlying Jython runtime is even more compatible

https://hg.python.org/jython/rev/8c1a3b72913e
History
Date User Action Args
2015-03-20 18:28:29zyasoftsetstatus: pending -> closed
2015-03-13 21:29:50zyasoftsetstatus: open -> pending
priority: high
resolution: fixed
messages: + msg9649
assignee: zyasoft
2015-03-11 20:27:15zyasoftsetnosy: + zyasoft
messages: + msg9621
2015-02-25 12:55:07pekka.klarcksetmessages: + msg9558
2015-02-25 12:48:34pekka.klarckcreate