Issue1427

classification
Title: Assignment of astnode value fails
Type: behaviour Severity: normal
Components: Core Versions: Jython 2.7
process
Status: open Resolution: remind
Dependencies: Superseder:
Assigned To: fwierzbicki Nosy List: fwierzbicki, kellrott, zyasoft
Priority: normal Keywords:

Created on 2009-08-06.19:01:40 by kellrott, last changed 2013-02-19.18:49:33 by fwierzbicki.

Files
File name Uploaded Description Edit Remove
astBug.tar.gz kellrott, 2009-08-06.19:01:39 Test program to replicate the program
unnamed kellrott, 2009-10-26.19:55:18
Messages
msg4997 (view) Author: Kyle (kellrott) Date: 2009-08-06.19:01:39
Assignment of a newly created ast Node value fails.  The assignment
destination become 'None' rather then accepting the value.

Test case:
Simple code mutation.  Trying manipulate the ast tree to change a
methods call like
a.set( 1 )
to the assignment:
a = 1

By running attached code 'astMutationTest.py' ('unparse.py' code is
obtained from Cpython parse demo code)

the python output is:
Before Mutation

a.set(1)

After Mutation

a = 1


But for Jython:
Before Mutation

a.set(1)

After Mutation

None
msg5250 (view) Author: Kyle (kellrott) Date: 2009-10-21.16:17:51
A simpler example:

import _ast as ast
tmpAssign = ast.Assign()
tmpExpr = ast.Expr()
tmpExpr.value = tmpAssign
print tmpExpr.value

In Jython tmpExpr.value is None

In Python, it's something like:
<_ast.Assign object at 0x8ac50>
msg5257 (view) Author: Kyle (kellrott) Date: 2009-10-22.17:58:34
Is this be caused by the code in org.python.antlr.adapter.ExprAdapter?
The call chain is:
org.python.antlr.ast.Expr.setValue
org.python.antlr.adapter.AstAdapters.py2expr
org.python.antlr.adapter.ExprAdapter.iter2ast
org.python.antlr.adapter.ExprAdapter.py2ast


The py2ast method only includes type checks for PyInteger, PyLong,
PyFloat, PyComplex, PyString, and PyUnicode.  All other elements (like
an 'Assign' class) would default to null, if they were set via 'setValue'.
msg5266 (view) Author: Frank Wierzbicki (fwierzbicki) Date: 2009-10-26.18:55:28
Kyle: Thanks for the extra analysis, it looks like it could be the
problem. I'll take a look at that part of the code soon to see if this
will fix your issues.  FYI down the road I'm planning a rewrite of
ast.py based on a better understanding of CPython's ast.py.
msg5267 (view) Author: Kyle (kellrott) Date: 2009-10-26.19:55:19
I've been looking at it, and I'm trying to figure out if it's a byproduct of
me using the ast wrong.  I'm putting the Assign object in the value field of
the Expr object (changing the expression a.set(1) into the assignment a=1).
An Assign is a stmt, but the value field for the Expr in the Java code
expects a expr.  Looking back at the ast description at
http://docs.python.org/library/ast.html  it looks like expr is a descendant
of stmt.  So my code may be constructing a illegal tree.  The unparse.py
code is still able to deconstruct it back into code, but adding in the java
type checking reveals the bad tree...
I'm going back to my original code to see if replacing the Expr node in it's
parent's list fixes the problem.

Don't kill the issue just yet, I need to make sure I'm right on this one.

Kyle

On Mon, Oct 26, 2009 at 11:55 AM, Frank Wierzbicki
<report@bugs.jython.org>wrote:

>
> Frank Wierzbicki <fwierzbicki@users.sourceforge.net> added the comment:
>
> Kyle: Thanks for the extra analysis, it looks like it could be the
> problem. I'll take a look at that part of the code soon to see if this
> will fix your issues.  FYI down the road I'm planning a rewrite of
> ast.py based on a better understanding of CPython's ast.py.
>
> _______________________________________
> Jython tracker <report@bugs.jython.org>
> <http://bugs.jython.org/issue1427>
> _______________________________________
>
msg5268 (view) Author: Kyle (kellrott) Date: 2009-10-26.21:47:26
I've confirmed that in the original code a list was passed to the
'value' field.  In python this didn't cause any problems, despite the
fact that the official AST description for the value field of an Assign
is for a single expression, and not for a list of expressions.  The type
checking in Java caused the bad assignment to default to None.

Fixing this issue would only be useful if Jython needs to handle badly
formed AST trees the same way as Python.  Otherwise, raising an
exception for a bad assignment might be worth while.
msg5269 (view) Author: Frank Wierzbicki (fwierzbicki) Date: 2009-10-27.02:01:55
Kyle: thanks for the further analysis. CPython accepts arbitrary Python
objects pretty much anywhere in an ast.py tree, even if it fails at
parse time. Matching up with this behavior will require a complete
re-design (which I plan to do, but it's going to take a while). CPython
separates ast.py from the internal ast -- they are deliberately
different (ast.py is only a mirror of the internal ast so that it can
evolve). Not understanding this, I actually made Jython internal ast
implementation the same as ast.py. I plan to separate these in the
future (CPython design makes loads of sense *slaps forehead*)
msg6077 (view) Author: Jim Baker (zyasoft) Date: 2010-09-20.18:58:05
Related to #1497

Deferring to 2.6
History
Date User Action Args
2013-02-19 18:49:33fwierzbickisetkeywords: - patch
2013-02-19 18:49:27fwierzbickisetkeywords: + patch
resolution: accepted -> remind
versions: + Jython 2.7, - Deferred
2010-09-20 18:58:06zyasoftsetpriority: high -> normal
nosy: + zyasoft
messages: + msg6077
versions: + Deferred, - 2.5.1
2009-10-27 02:01:56fwierzbickisetmessages: + msg5269
2009-10-26 21:47:27kellrottsetmessages: + msg5268
2009-10-26 19:55:19kellrottsetfiles: + unnamed
messages: + msg5267
2009-10-26 18:55:28fwierzbickisetmessages: + msg5266
2009-10-22 17:58:35kellrottsetmessages: + msg5257
2009-10-21 16:17:52kellrottsetmessages: + msg5250
2009-08-06 19:05:28fwierzbickisetpriority: high
assignee: fwierzbicki
resolution: accepted
nosy: + fwierzbicki
2009-08-06 19:01:40kellrottcreate