Issue1128

...mainly because is the Right Thing. 

PyString expects that its backing String don't have any code point above
255, without actively enforcing it. So whatever character over that
limit is potentially lost when mapping Strings to PyString. 

Also, it is causing other practical problems, such as
unicodedata.normalize returning str instead of unicode.

Here is a patch which doesn't present any regression on my system:

http://codereview.appspot.com/5655

[CC-ing Frank because it is important to know if we are going to push
this for 2.5 or not]

Not sure if it will be relevant for this issue, but here is a pointer to
how CPython is solving the issue of unicode vs bytestrings on file names
and path handling: http://bugs.python.org/issue3187

Does this fix test_doctest?  Looking at this patch is high on my list
but that would put it to the top :)

No, it doesn't. But perhaps it can help once we fix the problems at the
parsing level. See
http://article.gmane.org/gmane.comp.lang.jython.devel/4565

The proposed patch transparently fixes the lossy behavior I see in inline 
web page scripts that consume Unicode from Java code and then send results 
back to Java code.  Without the patch, script authors need to explicitly 
understand and do str/unicode conversions, which they needn't with other 
script engines like Rhino and JRuby.  Very worthwhile proposal!

I had a revised patch a few days back, but wanted to post it to
rietveld. Unfortunately, right now I don't have the time to clean it up
(e.g: removing whitespace-only changes) and adapt it to rietvield
(adding a leading "jython/" to paths inside the diff). 

As I've realized now that I won't have the time until at least the
weekend, I'm attaching the raw patch here...

The last patch was applied on r5390.