Issue2730

classification
Title: datetime.strptime('%b') behaviour inconsistent with CPython on Windows JDK 9+
Type: behaviour Severity: normal
Components: Library Versions: Jython 2.7
Milestone: Jython 2.7.2
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: adamburke, jeff.allen
Priority: Keywords:

Created on 2019-01-08.07:46:47 by adamburke, last changed 2019-11-03.09:13:56 by jeff.allen.

Messages
msg12290 (view) Author: Adam Burke (adamburke) Date: 2019-01-08.07:46:46
It seems there are a few datetime.strptime() / strftime() issues, but this seems like a new variation.

On JDK 9+, Windows 10, the short name for months has changed even for English locales to include a fullstop. This causes incompatibility with CPython and earlier versions of Jython, and breaks regtests, mainly those inherited from CPython itself.

Regrtest sample output
> dist\bin\jython.exe -m test.regrtest -v test_strptime

======================================================================
ERROR: test_feb29_on_leap_year_without_year (test.test_strptime.StrptimeTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\Adam\jython\jython3\dist\Lib\test\test_strptime.py", line 382, in test_feb29_on_leap_year_without_year
    time.strptime("Feb 29", "%b %d")
  File "C:\Users\Adam\jython\jython3\dist\Lib\_strptime.py", line 467, in _strptime_time
    return _strptime(data_string, format)[0]
  File "C:\Users\Adam\jython\jython3\dist\Lib\_strptime.py", line 324, in _strptime
    raise ValueError("time data %r does not match format %r" %
ValueError: time data u'Feb 29' does not match format u'%b %d'

ERROR: test_mar1_comes_after_feb29_even_when_omitting_the_year (test.test_strptime.StrptimeTests)
FAIL: test_pattern (test.test_strptime.TimeRETests)

Demonstration script:
import os
import time
from datetime import date

print (os.uname())
print (os.environ['JAVA_HOME'])

print ( date(2002, 2, 4).strftime('%b %d')  )


print ( time.strptime("Jan. 29", "%b %d") ) # No ValueError
print ( time.strptime("Jan 29", "%b %d") ) # ValueError

C:\Users\Adam\jython\jython3>dist\bin\jython.exe datebug20190108.py
('Windows', '...', '10', '...', 'AMD64')
C:\Program Files\Java\jdk-11.0.1
Feb. 04
time.struct_time(tm_year=1900, tm_mon=1, tm_mday=29, tm_hour=0, tm_min=0, tm_sec=0, tm_wday=0, tm_yday=29, tm_isdst=-1)
Traceback (most recent call last):
  File "datebug20190108.py", line 13, in <module>
    print ( time.strptime("Jan 29", "%b %d") ) # ValueError
  File "C:\Users\Adam\jython\jython3\dist\Lib\_strptime.py", line 467, in _strptime_time
    return _strptime(data_string, format)[0]
  File "C:\Users\Adam\jython\jython3\dist\Lib\_strptime.py", line 324, in _strptime
    raise ValueError("time data %r does not match format %r" %
ValueError: time data u'Jan 29' does not match format u'%b %d'
msg12292 (view) Author: Adam Burke (adamburke) Date: 2019-01-08.07:48:54
Tested on JDK 9 and 11. Causes some related failures across a few reg tests.
msg12294 (view) Author: Adam Burke (adamburke) Date: 2019-01-08.07:51:10
Related 
#2503
#2285
msg12358 (view) Author: Adam Burke (adamburke) Date: 2019-03-15.10:54:53
The local month formatting behaviour is defined at the lowest possible level, for CPython in timemodule.c, calling into either wcsftime(), or the relevant OS-specific library.

In jython there is a reimplementation in org/python/modules/time/Time.java which calls into the Java standard libraries, understandably enough.

The definition of %b, is "Month as locale’s abbreviated name". 

https://docs.python.org/2/library/datetime.html#strftime-and-strptime-behavior
https://docs.python.org/3.8/library/datetime.html#strftime-and-strptime-behavior

One practical if slightly work-avoiding position would be to say jython locales *are* Java locales. This would be a more defensible position if jython locale stuff wasn't weak (as generally admitted), eg if setlocale() wasn't broken (also noted by Wang Yaqiang)

https://sourceforge.net/p/jython/mailman/message/34586162/

Traceback (most recent call last):
  File "showlocale.py", line 10, in <module>
    locale.setlocale(locale.LC_ALL, 'de_DE')
  File "C:\Users\Adam\jython\jython3\dist\Lib\locale.py", line 552, in setlocale
    return _setlocale(category, locale)
  File "C:\Users\Adam\jython\jython3\dist\Lib\locale.py", line 88, in setlocale
    raise Error, '_locale emulation only supports "C" locale'
ValueError: _locale emulation only supports "C" locale

OTOH, you have to fix one thing at a time, and maintaining a Java-C locale mapping sounds like a thankless and moving-target task.

As a side note for anyone fishing in this area, jython has a fork of the datetime module, which doesn't do much except override add a few __tojava__() methods. Reorganizing that in an include / override structure would be good (but not actually address this issue at all).
msg12448 (view) Author: Adam Burke (adamburke) Date: 2019-04-24.01:59:04
PR that fixes this by introducing experimental locale support https://github.com/jythontools/jython/pull/132
msg12738 (view) Author: Jeff Allen (jeff.allen) Date: 2019-11-03.09:13:48
Implementation of locale covers this nicely. See linked GitHub discussion.
History
Date User Action Args
2019-11-03 09:13:56jeff.allensetmilestone: Jython 2.7.2
2019-11-03 09:13:48jeff.allensetstatus: pending -> closed
nosy: + jeff.allen
messages: + msg12738
2019-09-21 15:52:40jeff.allensetstatus: open -> pending
resolution: fixed
2019-04-24 01:59:04adamburkesetmessages: + msg12448
2019-03-15 10:54:54adamburkesetmessages: + msg12358
2019-01-08 07:51:10adamburkesetmessages: + msg12294
2019-01-08 07:48:54adamburkesetmessages: + msg12292
2019-01-08 07:46:47adamburkecreate