Issue2167

classification
Title: json module is slow
Type: Severity: normal
Components: Versions: Jython 2.7
Milestone:
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: zyasoft Nosy List: amak, pdina, zyasoft
Priority: high Keywords:

Created on 2014-06-16.21:42:20 by zyasoft, last changed 2014-12-17.02:55:29 by zyasoft.

Messages
msg8651 (view) Author: Jim Baker (zyasoft) Date: 2014-06-16.21:42:20
We should reimplement json using Jackson (https://github.com/FasterXML/jackson).

Note there are two similar projects for Jython, both using Jackson:

* Jyson - http://opensource.xhaus.com/projects/jyson
* SimpleJackson - https://bitbucket.org/crankycoder/simplejackson/wiki/Home

The difference is that this will be written in Java for speed and will implement the standard json module.
msg8776 (view) Author: Alan Kennedy (amak) Date: 2014-06-20.17:42:02
Jyson is not implemented with jackson, it's a completely separate, jython-specific implementation that is written from the ground up.

I've been meaning to make the time to make its API compatible with the json module.

After that, I'd be happy to contribute it to jython.
msg8777 (view) Author: Jim Baker (zyasoft) Date: 2014-06-20.18:10:15
Alan, thanks for the clarification! Jackson is definitely overkill for our needs. I don't think we would it use at all for generation, and only its stream support for parsing. So a revised Jyson with a json API could be the best approach.
msg8932 (view) Author: Jim Baker (zyasoft) Date: 2014-08-22.15:20:31
This has recently come up as an issue in #jython. I took at look at lib-python/2.7/json, and it looks like we just need to implement the following functions from _json:

c_make_scanner, c_scanstring, c_encode_basestring_ascii, c_make_encoder 

The only one that's slightly involved is the last, and even then the code in python it's replacing is pretty minimal - couple pages maybe of c-like python.

Note that we can selectively choose which functions to implement - json will fallback to pure Python if a function is not present.
msg8933 (view) Author: Jim Baker (zyasoft) Date: 2014-08-22.15:26:38
This is causing issues for practical usage of Jython, so I'm just going to fix in the next day or so.
msg8976 (view) Author: Jim Baker (zyasoft) Date: 2014-09-09.00:14:28
Just a quick progress report: I spent a couple of days on it, and got 100x speedup on some typical json cases, but I discovered that my original solution, despite just failing one test, would not suffice. (The json module makes some interesting choices/optimizations in support of decoding from bytes.)

So I will have to spend another day or two on it to get it done in a compatible fashion.
msg9029 (view) Author: Jim Baker (zyasoft) Date: 2014-09-24.18:31:55
Target beta 4
msg9225 (view) Author: Jim Baker (zyasoft) Date: 2014-12-07.16:50:08
Fixed as of https://hg.python.org/jython/rev/5f4c860fb58f

Note that there remains a bug still in testing JSON with respect to the command line tool, but that's due to closing sys.stdout (http://bugs.jython.org/issue695383) and is unrelated to this speedup work.
History
Date User Action Args
2014-12-17 02:55:29zyasoftsetstatus: pending -> closed
2014-12-07 16:50:09zyasoftsetstatus: open -> pending
resolution: accepted -> fixed
messages: + msg9225
2014-09-24 18:31:55zyasoftsetpriority: high
messages: + msg9029
2014-09-09 00:14:29zyasoftsetmessages: + msg8976
2014-08-22 15:54:23pdinasetnosy: + pdina
2014-08-22 15:26:39zyasoftsetresolution: accepted
messages: + msg8933
2014-08-22 15:20:32zyasoftsetmessages: + msg8932
2014-06-20 18:10:15zyasoftsetmessages: + msg8777
2014-06-20 17:42:03amaksetmessages: + msg8776
2014-06-20 17:39:44amaksetnosy: + amak
2014-06-16 21:42:20zyasoftcreate