Issue1964

classification
Title: time.strptime() does not support %f in format
Type: behaviour Severity: normal
Components: Versions:
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: jeff.allen Nosy List: Arfrever, alex.gronholm, amak, fwierzbicki, jeff.allen, santa4nt
Priority: high Keywords: patch

Created on 2012-08-28.08:38:11 by Arfrever, last changed 2014-01-25.23:02:11 by jeff.allen.

Files
File name Uploaded Description Edit Remove
issue1964.patch santa4nt, 2013-03-24.01:24:09 address both issues
issue1964_2.patch santa4nt, 2014-01-14.01:40:51 Address msg 8224,7.
Messages
msg7429 (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) Date: 2012-08-28.08:38:10
time.strptime() does not support %f in format.
%f was added in CPython 2.6: http://bugs.python.org/issue1158

CPython 2.7:
$ python2.7 -c 'import time; print(time.strptime("0", "%f"))'
time.struct_time(tm_year=1900, tm_mon=1, tm_mday=1, tm_hour=0, tm_min=0, tm_sec=0, tm_wday=0, tm_yday=1, tm_isdst=-1)
$

Jython 2.7:
$ jython2.7 -c 'import time; print(time.strptime("0", "%f"))'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
        at java.text.SimpleDateFormat.compile(SimpleDateFormat.java:845)
        at java.text.SimpleDateFormat.initialize(SimpleDateFormat.java:659)
        at java.text.SimpleDateFormat.<init>(SimpleDateFormat.java:585)
        at java.text.SimpleDateFormat.<init>(SimpleDateFormat.java:560)
        at org.python.modules.time.Time.strptime(Time.java:707)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)

java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: Illegal pattern character 'f'
$

src/org/python/modules/time/Time.java contains the following code, which does not detect format not translatable to Java:
        String jformat = py2java_format(format);
        if (jformat == null) {
            // Format not translatable to java, fallback to _strptime
            return pystrptime(data_string, format);
        }
        SimpleDateFormat d = new SimpleDateFormat(jformat);

Another symptom of probably the same bug is wrong exception (java.lang.IllegalArgumentException instead of ValueError documented in documentation of time.strptime):
$ python2.7 -c 'import time; print(time.strptime("", "%e"))'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib64/python2.7/_strptime.py", line 467, in _strptime_time
    return _strptime(data_string, format)[0]
  File "/usr/lib64/python2.7/_strptime.py", line 317, in _strptime
    (bad_directive, format))
ValueError: 'e' is a bad directive in format '%e'
$ jython2.7 -c 'import time; print(time.strptime("", "%e"))'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
        at java.text.SimpleDateFormat.compile(SimpleDateFormat.java:845)
        at java.text.SimpleDateFormat.initialize(SimpleDateFormat.java:659)
        at java.text.SimpleDateFormat.<init>(SimpleDateFormat.java:585)
        at java.text.SimpleDateFormat.<init>(SimpleDateFormat.java:560)
        at org.python.modules.time.Time.strptime(Time.java:707)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)

java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: Illegal pattern character 'e'
$
msg7440 (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) Date: 2012-08-28.20:52:07
Maybe something like the following could be used:

SimpleDateFormat d;
try {
    d = new SimpleDateFormat(jformat);
} catch (IllegalArgumentException e) {
    return pystrptime(data_string, format);
}
msg7974 (view) Author: Santoso Wijaya (santa4nt) Date: 2013-03-24.01:19:06
Attaching a patch to address both '%f' format support, IllegalArgumentException -> ValueError conversion, and unit tests for both.
msg8188 (view) Author: Jeff Allen (jeff.allen) Date: 2013-12-02.08:14:56
Cheekily stealing this one as I'm considering the related #2071, #2033, all provided with patches.
msg8189 (view) Author: Jeff Allen (jeff.allen) Date: 2013-12-02.20:34:57
Fixed in http://hg.python.org/jython/rev/b31e71644fa8
msg8224 (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) Date: 2014-01-12.22:16:23
Returned value is inconsistent between CPython (time.struct_time) and Jython (tuple with 2 elements: time.struct_time and int):

$ python2.7 -c 'import time; print(time.strptime("10", "%f"))'
time.struct_time(tm_year=1900, tm_mon=1, tm_mday=1, tm_hour=0, tm_min=0, tm_sec=0, tm_wday=0, tm_yday=1, tm_isdst=-1)
$ jython2.7 -c 'import time; print(time.strptime("10", "%f"))'
(time.struct_time(tm_year=1900, tm_mon=1, tm_mday=1, tm_hour=0, tm_min=0, tm_sec=0, tm_wday=0, tm_yday=1, tm_isdst=-1), 100000)
msg8227 (view) Author: Alex Grönholm (alex.gronholm) Date: 2014-01-13.04:25:06
Reopened issue due to reported inconsistency in output between Jython and CPython.
msg8229 (view) Author: Santoso Wijaya (santa4nt) Date: 2014-01-14.01:40:51
Attaching patch #2 to fix it. The fallback to python code from native code used the wrong function.
msg8232 (view) Author: Jeff Allen (jeff.allen) Date: 2014-01-19.22:43:00
That mistake goes back to 2008!
http://hg.python.org/jython/rev/8dd9793817ce
Which demonstrates how rarely we would fall back on _strptime._strptime() until %f and Feb 29 came along.

On the plus side, at least it gets the year right. (Ref #2112)

Thanks for the prompt patch. I'll proceed as before.
msg8234 (view) Author: Jeff Allen (jeff.allen) Date: 2014-01-25.23:02:09
Now in as http://hg.python.org/jython/rev/57232db660c9, thanks Santoso.

This was a latent bug discovered by support for parsing %f. Support will undoubtedly bring out further related inadequacies, such as our behind-the-times datetime module, but IMO we should make them separate issues.
History
Date User Action Args
2014-01-25 23:02:11jeff.allensetstatus: open -> closed
resolution: fixed
messages: + msg8234
2014-01-19 22:43:01jeff.allensetmessages: + msg8232
2014-01-14 01:40:52santa4ntsetfiles: + issue1964_2.patch
messages: + msg8229
2014-01-13 04:25:07alex.gronholmsetstatus: closed -> open
resolution: fixed -> (no value)
messages: + msg8227
nosy: + alex.gronholm
2014-01-12 22:16:23Arfreversetmessages: + msg8224
2013-12-02 20:34:57jeff.allensetstatus: open -> closed
resolution: fixed
messages: + msg8189
2013-12-02 08:14:57jeff.allensetassignee: fwierzbicki -> jeff.allen
messages: + msg8188
nosy: + jeff.allen
2013-03-24 01:24:15santa4ntsetfiles: - issue1964.patch
2013-03-24 01:24:09santa4ntsetfiles: + issue1964.patch
2013-03-24 01:19:11santa4ntsettype: behaviour
2013-03-24 01:19:07santa4ntsetfiles: + issue1964.patch
keywords: + patch
messages: + msg7974
nosy: + santa4nt
2013-02-27 18:31:59fwierzbickisetpriority: high
assignee: fwierzbicki
2012-08-28 22:03:22amaksetnosy: + amak
2012-08-28 20:52:08Arfreversetmessages: + msg7440
2012-08-28 20:46:11Arfreversetnosy: + fwierzbicki
2012-08-28 08:38:11Arfrevercreate