Issue1900

classification
Title: Python imports from Java cause some Python imports to fail
Type: behaviour Severity: major
Components: Core Versions: 2.7a2
Milestone:
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: fwierzbicki Nosy List: Arfrever, alex.gronholm, fwierzbicki, pjenvey
Priority: Keywords: patch

Created on 2012-05-26.10:35:14 by alex.gronholm, last changed 2012-08-03.16:37:29 by fwierzbicki.

Files
File name Uploaded Description Edit Remove
ImportTest.zip alex.gronholm, 2012-05-26.10:35:14
imp.diff fwierzbicki, 2012-06-12.23:18:24
Messages
msg7157 (view) Author: Alex Grönholm (alex.gronholm) Date: 2012-05-26.10:35:14
If I import Python modules from Java (using org.python.core.imp) it will cause later Python imports of the same module/package to fail.
Attached is a trivial codebase that demonstrates the problem.
msg7177 (view) Author: Frank Wierzbicki (fwierzbicki) Date: 2012-05-31.20:44:18
Hey Alex, I get "Exception in thread "main" ImportError: No module named testpkg.submodule" from imp.load("testpkg.submodule") in the Java code before it can even run the "test" function from the python code. FYI it runs imp.load("testpkg") just fine if I comment the other call to load out.
msg7184 (view) Author: Frank Wierzbicki (fwierzbicki) Date: 2012-06-07.17:42:55
My original theory was wrong, this is almost certainly due to http://hg.python.org/jython/rev/4c672dbbcdb2?revcount=960 I'm still investigating.
msg7187 (view) Author: Frank Wierzbicki (fwierzbicki) Date: 2012-06-07.21:24:10
Philip, do you remember why you removed __findattr_ex__ from PyModule? Not calling impAttr is part of this bug.
msg7188 (view) Author: Philip Jenvey (pjenvey) Date: 2012-06-07.21:40:02
Alex or anyone else, it'd be great if you could commit this as a formal test, then I can just deal with solely the import crazyness =]
msg7201 (view) Author: Alex Grönholm (alex.gronholm) Date: 2012-06-11.03:14:39
While attempting to create a python-only version of this, I ran into a possibly related issue. The code:

from org.python.core import imp
imp.load('xml.dom')

does not work. If I import xml.dom with an import statement, it works fine. Loading just the "xml" package works fine too.
msg7208 (view) Author: Frank Wierzbicki (fwierzbicki) Date: 2012-06-12.23:15:08
It may be related, but calling imp.load("xml.dom") didn't work in 2.5.2 either so it is not (at least by itself) the cause of your troubles.
msg7209 (view) Author: Frank Wierzbicki (fwierzbicki) Date: 2012-06-12.23:18:24
This patch seems to fix the original issue (though not the new imp.load("xml.dom") one) -- I still wonder if src/org/python/compiler/PyModule should get its __findattr_ex__ method back - it calls impAttr if __findattr__ returns null.
msg7210 (view) Author: Frank Wierzbicki (fwierzbicki) Date: 2012-06-12.23:19:58
BTW that patch is against the 2.5 branch
msg7220 (view) Author: Frank Wierzbicki (fwierzbicki) Date: 2012-06-15.04:49:46
As Philip requested, I submitted a test for this bug. It is run via ant like this:

ant importest

I needed a new task because the code needed tests/python in the classpath, and I didn't want to muck up other tests with new classpath items. I also applied the small fix that I put here before to see if it looks to others like it might be enough.
msg7221 (view) Author: Frank Wierzbicki (fwierzbicki) Date: 2012-06-15.04:50:19
Oh yeah I only pushed the test and fix to 2.5 for now.
msg7288 (view) Author: Frank Wierzbicki (fwierzbicki) Date: 2012-06-29.22:12:29
Alex: I pushed a fix to the 2.5 branch - have you checked to see if this resolves your issue? I know you have found an additional problem (summarized by imp.load("xml.dom") fails) but I'm more worried about the specific problem that prompted this bug report. I don't currently see the new imp.load problem as a blocker (at least for beta) since it does not seem to be a regression (2.5.2 has the same behavior)
msg7289 (view) Author: Frank Wierzbicki (fwierzbicki) Date: 2012-06-29.22:18:45
Alex: in fact I just checked and imp.load('xml.dom') isn't even a regression from 2.2.
msg7290 (view) Author: Alex Grönholm (alex.gronholm) Date: 2012-06-30.14:13:33
I tested this on my JDK7 machine (by setting jdk.target.version=1.6 in ant.properties) and ran into a different error:

Item in ``from list'' not a string

This happened when executing:

module = __import__(moduleName, fromlist=[className])

where className was a unicode string. If I convert it to a bytestring, it works fine. Of course the import machinery _should_ accept unicode strings too.
msg7298 (view) Author: Frank Wierzbicki (fwierzbicki) Date: 2012-07-01.17:34:23
Alex: is the new bug a regression? I'd like this bug report to focus on just those and maybe add new bug reports for non-regressions. If it is a regression than of course this is a fine place to track it.
msg7299 (view) Author: Alex Grönholm (alex.gronholm) Date: 2012-07-01.17:35:12
Yes, it's a regression from 2.5.2 -- the same code worked just fine before.
msg7300 (view) Author: Frank Wierzbicki (fwierzbicki) Date: 2012-07-01.17:42:19
So should I expect this to fail? It doesn't fail for me:

module = __import__("java.lang", fromlist=[u"String",])

or am I not understanding the problem?
msg7301 (view) Author: Alex Grönholm (alex.gronholm) Date: 2012-07-01.17:48:22
I'm at a loss -- I can't reproduce the issue any more. Same Jython build, same application code.
msg7302 (view) Author: Frank Wierzbicki (fwierzbicki) Date: 2012-07-01.17:57:48
:)

So is this fixed (other than making a new bug to track imp.load("xml.dom") ?
msg7303 (view) Author: Alex Grönholm (alex.gronholm) Date: 2012-07-01.20:16:01
I'm kinda concerned about that latest error, but since I can't reproduce it, I can't argue with you.
msg7321 (view) Author: Alex Grönholm (alex.gronholm) Date: 2012-07-16.08:24:51
I am able to reproduce it again. No matter how many times I restart, my app won't start. This was triggered by adding new jars to the class path. I get the same "Item in ``from list'' not a string" error every time.
I tracked this error to line 872 in imp.java:

if (!Py.isInstance(item, PyString.TYPE)) {
                    throw Py.TypeError("Item in ``from list'' not a string");
                }

It only checks for the string type, not unicode. If the element is a unicode string, it'll fail. That's my theory anyway.
msg7322 (view) Author: Alex Grönholm (alex.gronholm) Date: 2012-07-16.08:35:34
Except that in Jython, PyUnicode inherits from PyString...
msg7323 (view) Author: Alex Grönholm (alex.gronholm) Date: 2012-07-16.08:48:00
The problem occurs even with the -Dpython.cachedir.skip=true option.
msg7326 (view) Author: Frank Wierzbicki (fwierzbicki) Date: 2012-07-17.16:49:44
>> Except that in Jython, PyUnicode inherits from PyString...
Aha - but that's not enough -- what really needs to be checked is PyBaseString due to the annotation in PyUnicode (base = PyBaseString.class) -- so this should be a simple fix.
msg7327 (view) Author: Frank Wierzbicki (fwierzbicki) Date: 2012-07-17.16:50:46
Can't wait to move to Jython3 so I can merge PyString and PyUnicode.
msg7331 (view) Author: Frank Wierzbicki (fwierzbicki) Date: 2012-07-18.16:31:32
Oops -forgot to push this change - it's in now. I'm going to revert a little of it - I was overzealous in changing the checks in PyClass and PyFunction.
History
Date User Action Args
2012-08-03 16:37:29fwierzbickisetstatus: open -> closed
resolution: fixed
2012-07-18 16:31:33fwierzbickisetmessages: + msg7331
2012-07-17 16:50:46fwierzbickisetmessages: + msg7327
2012-07-17 16:49:45fwierzbickisetmessages: + msg7326
2012-07-16 08:48:00alex.gronholmsetmessages: + msg7323
2012-07-16 08:35:34alex.gronholmsetmessages: + msg7322
2012-07-16 08:24:52alex.gronholmsetmessages: + msg7321
2012-07-01 20:16:01alex.gronholmsetmessages: + msg7303
2012-07-01 17:57:48fwierzbickisetmessages: + msg7302
2012-07-01 17:48:22alex.gronholmsetmessages: + msg7301
2012-07-01 17:42:19fwierzbickisetmessages: + msg7300
2012-07-01 17:35:12alex.gronholmsetmessages: + msg7299
2012-07-01 17:34:23fwierzbickisetmessages: + msg7298
2012-06-30 14:13:34alex.gronholmsetmessages: + msg7290
2012-06-29 22:18:45fwierzbickisetmessages: + msg7289
2012-06-29 22:12:29fwierzbickisetmessages: + msg7288
2012-06-15 04:50:19fwierzbickisetmessages: + msg7221
2012-06-15 04:49:47fwierzbickisetmessages: + msg7220
2012-06-12 23:19:59fwierzbickisetmessages: + msg7210
2012-06-12 23:18:25fwierzbickisetfiles: + imp.diff
keywords: + patch
messages: + msg7209
2012-06-12 23:15:08fwierzbickisetmessages: + msg7208
2012-06-12 16:30:06Arfreversetnosy: + Arfrever
2012-06-11 03:14:39alex.gronholmsetmessages: + msg7201
versions: + 2.7a2, - 2.7a1
2012-06-07 21:40:02pjenveysetmessages: + msg7188
2012-06-07 21:24:10fwierzbickisetnosy: + pjenvey
messages: + msg7187
2012-06-07 17:42:56fwierzbickisetmessages: + msg7184
2012-05-31 20:44:18fwierzbickisetmessages: + msg7177
2012-05-26 10:35:14alex.gronholmcreate