Issue2411

classification
Title: microsecond precision not returning from datetime
Type: behaviour Severity: normal
Components: Library Versions: Jython 2.7
Milestone:
process
Status: open Resolution: accepted
Dependencies: Superseder:
Assigned To: Nosy List: DanglingPointer, zyasoft
Priority: normal Keywords:

Created on 2015-10-11.08:29:42 by DanglingPointer, last changed 2018-03-05.21:56:18 by jeff.allen.

Messages
msg10347 (view) Author: DanglingPointer (DanglingPointer) Date: 2015-10-11.08:29:41
datetime methods are not returning microsecond precision and are only displaying 000s
How do I fix this?
e.g.
>>> datetime.datetime.now().isoformat()
'2015-10-11T19:23:19.377000'
>>> datetime.datetime.now()
datetime.datetime(2015, 10, 11, 19, 23, 23, 862000)
>>> datetime.datetime.utcnow()
datetime.datetime(2015, 10, 11, 8, 24, 20, 467000)
msg10348 (view) Author: DanglingPointer (DanglingPointer) Date: 2015-10-11.08:36:30
From my limited testing it appears to only affect jython 2.7.0
jython 2.5.3 displays microseconds correctly.
e.g.
>>> datetime.datetime.now()
datetime.datetime(2015, 10, 11, 19, 33, 4, 92999)
>>> datetime.datetime.now().isoformat()
'2015-10-11T19:33:21.634999'
msg10370 (view) Author: Jim Baker (zyasoft) Date: 2015-10-22.20:51:08
The best we can do with Java 7 is http://docs.oracle.com/javase/7/docs/api/java/lang/System.html#currentTimeMillis(), which is what is used here, if indirectly from the Calendar class.

Java 8 can give us higher precision, though not necessarily higher accuracy (depends on the underlying clock, such as using an onboard atomic clock source that's synchronized with GPS, yes that does exist): https://docs.oracle.com/javase/8/docs/api/java/time/Instant.html

Something we can look at for 2.7.2
msg10375 (view) Author: DanglingPointer (DanglingPointer) Date: 2015-10-23.11:22:03
The question are, why does it work in jython 2.5.3 and not 2.7.0 ?  
What has regressed ?

Ideally we should not need to import a java class and use python's datetime instead.  Makes it easier for porters to port existing python code.
msg10376 (view) Author: DanglingPointer (DanglingPointer) Date: 2015-10-23.11:25:49
....ACTUALLY....
It doesn't work in 2.5.3 either...
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Jython 2.5.3 (, Nov 21 2013, 23:15:42) 
[OpenJDK 64-Bit Server VM (Oracle Corporation)] on java1.8.0_45-internal
Type "help", "copyright", "credits" or "license" for more information.
>>> import datetime
>>> datetime.datetime.now().isoformat()
'2015-10-23T22:24:36.270999'
>>> datetime.datetime.now().isoformat()
'2015-10-23T22:24:40.836999'
>>> datetime.datetime.now().isoformat()
'2015-10-23T22:24:42.043999'
>>> datetime.datetime.now().isoformat()
'2015-10-23T22:24:43.433000'
>>> datetime.datetime.now().isoformat()
'2015-10-23T22:24:45.757999'
>>> datetime.datetime.now().isoformat()
'2015-10-23T22:24:46.726000'
>>> datetime.datetime.now().isoformat()
'2015-10-23T22:24:47.673000'
>>> datetime.datetime.now().isoformat()
'2015-10-23T22:24:48.796000'
>>> datetime.datetime.now().isoformat()
'2015-10-23T22:24:49.917999'
>>> datetime.datetime.now().isoformat()
'2015-10-23T22:24:50.996000'
>>> datetime.datetime.now().isoformat()
'2015-10-23T22:24:56.303999'
msg10377 (view) Author: Jim Baker (zyasoft) Date: 2015-10-23.15:52:15
So two things here:

1. Jython 2.7 implements the change seen in the Python 2.7 language that the repr of a float uses the shortest decimal fraction that represents the same underlying binary representation. That's why you are seeing in 2.5 formatted values like '2015-10-23T22:24:56.303999' (truncated) vs what would now the more accurate (given truncation) '2015-10-23T22:24:56.304000'

2. I tried java.lang.time.Instant.now() to get instants and corresponding epoch seconds and nanoseconds. On OS X 10.11 and Ubuntu 15.04, using latest Oracle Java 8 for each, the nanoseconds reported are still with millesecond precision:

>>> import java
>>> t = java.time.Instant.now()
>>> t.epochSecond
1445615363L
>>> t.nano
426000000

There may be a configuration of Java and underlying OS and hardware that allows us to get wall clock time with greater precision (and presumably accuracy - I think Java here is preventing false accuracy, which is a good thing), but I don't have that setup.

Please retry this experiment on your own setup. I will also look at Java 9 as part of some other work, but I don't expect any difference.
msg10378 (view) Author: DanglingPointer (DanglingPointer) Date: 2015-10-24.01:43:07
What if we just get the java.lang.System.nanoTime() into a variable whenever jython starts and hybridize the call to datetime?

Something similar to this solution in Java...
/**
 * From http://stackoverflow.com/questions/1712205/current-time-in-microseconds-in-java
 * Class to generate timestamps with microsecond precision
 * For example: MicroTimestamp.INSTANCE.get() = "2012-10-21 19:13:45.267128"
 */ 
public enum MicroTimestamp 
{  INSTANCE ;

   private long              startDate ;
   private long              startNanoseconds ;
   private SimpleDateFormat  dateFormat ;

   private MicroTimestamp()
   {  this.startDate = System.currentTimeMillis() ;
      this.startNanoseconds = System.nanoTime() ;
      this.dateFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss.SSS") ;
   }

   public String get()
   {  long microSeconds = (System.nanoTime() - this.startNanoseconds) / 1000 ;
      long date = this.startDate + (microSeconds/1000) ;
      return this.dateFormat.format(date) + String.format("%03d", microSeconds % 1000) ;
   }
}
msg10379 (view) Author: Jim Baker (zyasoft) Date: 2015-10-24.03:23:07
The hybridization solution is not a good idea:

1. We are essentially lying about the wall clock time.

2. Consider clock drift over a long running process.

So let's not do that.

For Java 8 and later, we should use java.time.Instant.now so that we can access all available precision, so we should address this in a future release, ideally 2.7.2.

Users who want the suggested "hybrid" approach can construct datetime objects with the desired level of precision through their own functions outside of the Jython standard library.
History
Date User Action Args
2018-03-05 21:56:18jeff.allensetmilestone: Jython 2.7.2 ->
2015-10-29 01:28:28zyasoftsetpriority: normal
2015-10-24 03:23:07zyasoftsetresolution: accepted
messages: + msg10379
milestone: Jython 2.7.0 -> Jython 2.7.2
2015-10-24 01:43:08DanglingPointersetmessages: + msg10378
2015-10-23 15:52:16zyasoftsetmessages: + msg10377
2015-10-23 11:25:49DanglingPointersetmessages: + msg10376
2015-10-23 11:22:03DanglingPointersetmessages: + msg10375
2015-10-22 20:51:09zyasoftsetnosy: + zyasoft
messages: + msg10370
2015-10-11 08:36:30DanglingPointersetmessages: + msg10348
2015-10-11 08:29:42DanglingPointercreate