Issue2000

classification
Title: Unicode-related failures in test_codecs
Type: Severity: normal
Components: Library Versions: 2.7a2
Milestone:
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: jeff.allen
Priority: Keywords:

Created on 2012-12-27.14:33:25 by jeff.allen, last changed 2013-01-16.00:30:05 by jeff.allen.

Messages
msg7551 (view) Author: Jeff Allen (jeff.allen) Date: 2012-12-27.14:33:25
dist/Lib/test/test_codecs.py as currently delivered is well out of date, but advancing to the Python 2.7 version raises test failures. Some relate to missing dependencies in the CJK codecs (see issue #1066). This note is to remind us to fix the ones that aren't: absence of UTF-32 codec implementation (needed for test_io) and seeming bugs in other UTF codecs.

In a forthcoming changeset I will update the test and add some skips/commenting out citing #1066 and all that.
msg7563 (view) Author: Jeff Allen (jeff.allen) Date: 2013-01-05.18:18:29
Fixes for UTF-7 and UTF-16 are now in the trunk, and I'll turn to UTF-32.

Absence of an IDNA codec turns out to be issue #1153. I agree with later contributors to that report that it is not resolved until there's a codec. Pointers to java.net.IDN are helpful to that end.
msg7572 (view) Author: Jeff Allen (jeff.allen) Date: 2013-01-16.00:30:04
I've provided UTF-32 codecs and implementations of codecs.encode and codecs.decode, with some modest accommodation in the tests to Jython's
allergy to invalid unicode escapes.

I have chosen not to implement _codecs.readbuffer_encode and _codecs.charbuffer_encode. I don't properly understand the issues, but I gather they were intended as an aid to codec writers in CPython and their retention was controversial. My judgement is they won't be missed.

There's a case for refactoring codecs.java and _codec.java, and for enforcing the bytes vs. unicode distinction in the signatures, but that's another issue.

The other shortcomings (IDNA and CJK) are covered by other issues so I'm closing this.
History
Date User Action Args
2013-01-16 00:30:05jeff.allensetstatus: open -> closed
resolution: fixed
messages: + msg7572
2013-01-05 18:18:30jeff.allensetmessages: + msg7563
2012-12-27 14:33:26jeff.allencreate