Issue2354

classification
Title: marshal.py does not support serialization of code
Type: behaviour Severity: normal
Components: Library Versions: Jython 2.7
Milestone: Jython 2.7.2
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: fmartins, stefan.richthofer, zyasoft
Priority: Keywords:

Created on 2015-05-11.20:20:21 by fmartins, last changed 2016-11-24.05:54:26 by stefan.richthofer.

Files
File name Uploaded Description Edit Remove
unnamed fmartins, 2015-05-12.10:38:08
Messages
msg10048 (view) Author: fmartins (fmartins) Date: 2015-05-11.20:20:21
as stated in the source code of marshal.py, serialization of code objects not supported. This is supported in Python. At least, dump() should return an error if code is passed in.
msg10050 (view) Author: Jim Baker (zyasoft) Date: 2015-05-11.22:36:33
Throwing an exception is certainly easiest... but I will attempt to sketch out a solution.

Java bytecode cannot be accessed once loaded, and preemptively storing bytecode just-in-case is too much overhead for functionality that is rarely used. Assuming the source jar (or class file) is still available, we could use the inspect module to find the $py.class file. Then we could find the specific code object in the PyFunctionTable by introspecting for the index. Then we could ASM to write out the specific code object. Note that reloading a single code object via a ClassLoader will be quite inefficient, which is a big reason we have the PyFunctionTable approach today.

We probably don't want to do this solution.

Python bytecode is a long term scenario for Jython for code objects that are not hot. Such code objects can be readily dumped and loaded, and there's currently support in marshal for loading them to support pycimport. One we fully implement Python bytecode for this, we will have Python bytecode for every object, and only for hot code we will have - in addition - equivalent, but optimized Java bytecode. But this is not hapenning any time soon.

So throwing an exception is the correct solution for now.
msg10056 (view) Author: fmartins (fmartins) Date: 2015-05-12.10:38:09
I don't see an option to reply in the bug tracker...

The 1st solution seems indeed convoluted and I don't think it would work in my use case since no $py.class file is generated when the python script is compile(). Plus, if it is slow, probably not very useful to my use case either since the scripts are usually small (around 50 LOC avg).

But I would like to propose another alternative. You mention the option of "preemptively storing bytecode". In general it would be bad, but what about having an option in jython allowing the programmer to request the code to be stored. In this case marshal.dump would check for the cached code and save it instead of bailing out. This approach would keep compatibility with Python, I think.

On 12 May 2015, Jim Baker <report@bugs.jython.org> wrote:
> 
> Jim Baker added the comment:
> 
> Throwing an exception is certainly easiest... but I will attempt to sketch out a solution.
> 
> Java bytecode cannot be accessed once loaded, and preemptively storing bytecode just-in-case is too much overhead for functionality that is rarely used. Assuming the source jar (or class file) is still available, we could use the inspect module to find the $py.class file. Then we could find the specific code object in the PyFunctionTable by introspecting for the index. Then we could ASM to write out the specific code object. Note that reloading a single code object via a ClassLoader will be quite inefficient, which is a big reason we have the PyFunctionTable approach today.
> 
> We probably don't want to do this solution.
> 
> Python bytecode is a long term scenario for Jython for code objects that are not hot. Such code objects can be readily dumped and loaded, and there's currently support in marshal for loading them to support pycimport. One we fully implement Python bytecode for this, we will have Python bytecode for every object, and only for hot code we will have - in addition - equivalent, but optimized Java bytecode. But this is not hapenning any time soon.
> 
> So throwing an exception is the correct solution for now.
> 
> ----------
> nosy: +zyasoft
> 
> _______________________________________
> Jython tracker <<report@bugs.jython.org>>
> <<http://bugs.jython.org/issue2354>>
> _______________________________________
>
History
Date User Action Args
2016-11-24 05:54:26stefan.richthofersetnosy: + stefan.richthofer
2015-12-24 17:35:55zyasoftsetmilestone: Jython 2.7.2
2015-05-12 10:38:09fmartinssetfiles: + unnamed
messages: + msg10056
2015-05-11 22:36:33zyasoftsetnosy: + zyasoft
messages: + msg10050
2015-05-11 20:20:21fmartinscreate