Issue1879
Created on 2012-04-30.08:37:52 by conflatedauto, last changed 2015-05-27.14:53:30 by zyasoft.
Messages | |||
---|---|---|---|
msg7074 (view) | Author: Adam (conflatedauto) | Date: 2012-04-30.08:37:52 | |
jython -m broken for scripts inside jar files The workaround jython -c "import module; module.blah()" will work, but not if you don't have a blah function. This is jy.bat Version 2.5.2 C:\Working\working>java -classpath C:\Working\tools\maven\repository2\org\robotf ramework\robotframework\2.7.1\robotframework-2.7.1.jar;C:\Working\tools\maven\re pository2\tools\tools-core\1.3.1-SNAPSHOT\tools-core-1.3.1-SNAPSHOT.jar org.python .util.jython Jython 2.5.2 (Release_2_5_2:7206, Mar 2 2011, 23:12:06) [Java HotSpot(TM) Client VM (Sun Microsystems Inc.)] on java1.6.0_26 Type "help", "copyright", "credits" or "license" for more information. C:\Working\working>jy -m editor.vimgen C:\Working\working>java -classpath C:\Working\tools\maven\repository2\org\robotf ramework\robotframework\2.7.1\robotframework-2.7.1.jar;C:\Working\tools\maven\re pository2\tools\tools-core\1.3.1-SNAPSHOT\tools-core-1.3.1-SNAPSHOT.jar org.python .util.jython -m editor.vimgen Traceback (most recent call last): File "<string>", line 1, in <module> File "C:\Working\tools\maven\repository2\org\robotframework\robotframework\2.7 .1\robotframework-2.7.1.jar\Lib\runpy.py", line 88, in run_module AttributeError: 'ClasspathPyImporter' object has no attribute 'get_code' C:\Working\working>jy -m editor/vimgen C:\Working\working>java -classpath C:\Working\tools\maven\repository2\org\robotf ramework\robotframework\2.7.1\robotframework-2.7.1.jar;C:\Working\tools\maven\re pository2\tools\tools-core\1.3.1-SNAPSHOT\tools-core-1.3.1-SNAPSHOT.jar org.python .util.jython -m editor/vimgen Traceback (most recent call last): File "<string>", line 1, in <module> File "C:\Working\tools\maven\repository2\org\robotframework\robotframework\2.7 .1\robotframework-2.7.1.jar\Lib\runpy.py", line 88, in run_module AttributeError: 'ClasspathPyImporter' object has no attribute 'get_code' C:\Working\working>jy -m editor/blah Looking at the source it seems that ClasspathPyImporter is indeed lacking this method. No such module is a different, correct error: C:\Working\working>java -classpath C:\Working\tools\maven\repository2\org\robotf ramework\robotframework\2.7.1\robotframework-2.7.1.jar;C:\Working\tools\maven\re pository2\tools\tools-core\1.3.1-SNAPSHOT\tools-core-1.3.1-SNAPSHOT.jar org.python .util.jython -m editor/blah Traceback (most recent call last): File "<string>", line 1, in <module> File "C:\Working\tools\maven\repository2\org\robotframework\robotframework\2.7 .1\robotframework-2.7.1.jar\Lib\runpy.py", line 87, in run_module ImportError: No module named editor/blah Still present under the latest beta release C:\Working\working>jy -m editor/vimgen C:\Working\working>rem set JYTHON=C:\Working\tools\maven\repository2\org\robotfr amework\robotframework\2.7.1\robotframework-2.7.1.jar C:\Working\working>set JYTHON=C:\Working\tools\jython\jython2.5.3b1\jython.jar C:\Working\working>java -classpath C:\Working\tools\jython\jython2.5.3b1\jython. jar;C:\Working\tools\maven\repository2\tools\tools-core\1.3.1-SNAPSHOT\tools-core- 1.3.1-SNAPSHOT.jar org.python.util.jython -m editor/vimgen Traceback (most recent call last): File "<string>", line 1, in <module> File "C:\Working\tools\jython\jython2.5.3b1\Lib\runpy.py", line 88, in run_mod ule code = loader.get_code(mod_name) AttributeError: 'ClasspathPyImporter' object has no attribute 'get_code' Works for local scripts C:\Working\working>dir *.py Volume in drive C has no label. Volume Serial Number is 7402-3955 Directory of C:\Working\working 04/30/2012 11:11 AM 15 test.py 1 File(s) 15 bytes 0 Dir(s) 208,984,723,456 bytes free C:\Working\working>jy -m test C:\Working\working>rem set JYTHON=C:\Working\tools\maven\repository2\org\robotfr amework\robotframework\2.7.1\robotframework-2.7.1.jar C:\Working\working>set JYTHON=C:\Working\tools\jython\jython2.5.3b1\jython.jar C:\Working\working>java -classpath C:\Working\tools\jython\jython2.5.3b1\jython. jar;C:\Working\tools\maven\repository2\tools\tools-core\1.3.1-SNAPSHOT\tools-core- 1.3.1-SNAPSHOT.jar org.python.util.jython -m test test.py This is jy.bat just to show there is nothing weird there. Jython 2.5.2 was the version disted with the robot framework. rem set JYTHON=%M2_REPO%\org\robotframework\robotframework\2.7.1\robotframework- 2.7.1.jar set JYTHON=C:\Working\tools\jython\jython2.5.3b1\jython.jar java -classpath %JYTHON%;%M2_REPO%\ets\tools-core\1.3.1-SNAPSHOT\tools-core-1.3. 1-SNAPSHOT.jar org.python.util.jython %1 %2 %3 |
|||
msg7124 (view) | Author: Darjus Loktevic (darjus) | Date: 2012-05-21.19:57:41 | |
Hey Adam, Where is this editor.vimgen coming from? I am trying to reproduce your issue, but can't. I agree that get_code is missing, but first of all i want to make sure we can reproduce your problem. Could you build a test that reproduces it (or at least clarify a bit more on reproducing it)? java -cp ./dist/robotframework-trunk20120521.jar org.python.util.jython -m robot.run [ ERROR ] Expected at least 1 argument, got 0. Thanks! Darjus |
|||
msg7144 (view) | Author: Adam (conflatedauto) | Date: 2012-05-25.00:02:47 | |
Hi Darjus ā thanks for looking into this. The vimgen.py script is inside the tools-core jar ā I can see how it would be ambiguous without a jar listing now. Tools-core is part of the internal project Iām working on. $ jar tfv tools-core-1.3.2.jar 0 Sun May 06 23:18:56 CST 2012 META-INF/ 127 Sun May 06 23:18:54 CST 2012 META-INF/MANIFEST.MF 0 Sun May 06 23:18:34 CST 2012 editor/ 630 Sun May 06 23:18:34 CST 2012 editor/nameutil.py 0 Sun May 06 23:18:34 CST 2012 editor/__init__.py 7381 Sun May 06 23:18:34 CST 2012 editor/vimgen.py Does that help? Adam |
|||
msg9369 (view) | Author: Jim Baker (zyasoft) | Date: 2015-01-09.03:50:49 | |
Related issue is #2058 - no get_data support We should implement all the optional features in PEP 302: https://www.python.org/dev/peps/pep-0302/#optional-extensions-to-the-importer-protocol |
|||
msg9371 (view) | Author: Jim Baker (zyasoft) | Date: 2015-01-09.03:53:27 | |
Let's see if we can get into RC1 |
|||
msg9531 (view) | Author: Jim Baker (zyasoft) | Date: 2015-02-19.15:02:32 | |
Necessary for PEP 338, which is used by wrapper exe scripts as needed for #2251 |
|||
msg9553 (view) | Author: Jim Baker (zyasoft) | Date: 2015-02-23.04:50:56 | |
Demoting to high, but we will still try to get in for RC1 It's not actually the problem we saw for ensurepip, which was about making zip files executable by Jython. |
|||
msg10037 (view) | Author: Jeff Allen (jeff.allen) | Date: 2015-05-08.15:37:42 | |
I don't understand much of the above, and some of it turned out to be a red herring. Does the following scenario represent the issue? I have a jarred-up the files I used in #2158 (and hidden the original): > jar tf ulib.jar META-INF/ META-INF/MANIFEST.MF u/ u/v/ u/v/w/ u/v/w/x.py u/v/w/y.py u/v/w/__init__.py u/v/y.py u/v/__init__.py u/y.py u/__init__.py When you invoke x.py, each of these will tell you its path. Now, from 2.7.0+ I get: > java -cp ulib.jar;%JT%\dist\jython-dev.jar;%JT%\dist\javalib\* org.python.util.jython -c"import u.v.w.x" Init: u\__init__.py Init: u\v\__init__.py Init: u\v\w\__init__.py File: u\v\w\x.py File: u\y.py > java -cp ulib.jar;%JT%\dist\jython-dev.jar;%JT%\dist\javalib\* org.python.util.jython -m u.v.w.x Init: u\__init__.py Init: u\v\__init__.py Init: u\v\w\__init__.py ... stack dump ... AttributeError: 'ClasspathPyImporter' object has no attribute 'is_package' The failure of the second one constitutes the issue, and is because our loaders do not implement the "optional" methods that are indispensible to runpy. Right? |
|||
msg10038 (view) | Author: Jim Baker (zyasoft) | Date: 2015-05-08.15:46:23 | |
Jeff, I believe we just have to implement the "optional" features of PEP 302, per msg9369. Should be straightforward, since #2058 does have an attached PR that just needs to be reviewed, and Raphael has contributed other good commits in the past. |
|||
msg10039 (view) | Author: Jeff Allen (jeff.allen) | Date: 2015-05-08.16:31:52 | |
I think that says I'm reproducing the right thing. Thanks, I'd missed Raphael's PR for get_data. The message here is about is_package, earlier it was get_code, and I suppose a failing test would be nice. So it's not the whole story, but #2058 is a good start. |
|||
msg10063 (view) | Author: Jeff Allen (jeff.allen) | Date: 2015-05-16.10:01:51 | |
I've made some decent progress with this, committed #2058 locally, and implemented the optional methods. I'm not entirely sure what to test for, as I have no reference semantics when using a ClasspathPyImporter directly. (There is no direct CPython counterpart.) The requirement as I see it is to work correctly with import, -m (runpy) and pkgutil (which runpy uses) so that's what my tests look like. At the moment I can do this: >jar tf rlib.jar META-INF/ META-INF/MANIFEST.MF data/ data/r.dat data/__init__.py >java -cp rlib.jar;dist\jython-dev.jar;dist\javalib\* org.python.util.jython Jython 2.7.0+ (default:67f36ecd9103+, May 16 2015, 00:30:35) [Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)] on java1.7.0_60 Type "help", "copyright", "credits" or "license" for more information. >>> import sys, pkgutil >>> from java.lang import System >>> 'rlib.jar' in System.getProperty('java.class.path').split(';') True >>> i = pkgutil.get_importer('__pyclasspath__/') >>> i.is_package('data') True >>> i.get_code('data') <code object <module> at 0x2, file "__pyclasspath__/data/__init__.py", line 0> >>> import data Init data package >>> i.get_data('data\\r.dat') 'some data' >>> i.get_data('data\\r.dat') '' That last bit is clearly wrong, and stems from re-using the cached InputStream from "entries". This puzzles me: why do we have that cache if it contains things we can only use once? zipimporter avoids this by re-opening the entire archive and finding the resource again. I'll emulate that, and same for get_source. They could usefully share code, but it all seems a bit fragile to do in one step. The entries cache is still there in zipimporter, and up it looks up modules in it, but then it doesn't use the result. I wonder if these open streams are behind the 100% unlink() failure I get from test_zipimporter? It seems like a part of the code that could benefit from a cache, but not quite the one we have. My impression generally of Jython's import mechanism is that it contains some good stuff, but we lost our way. I can see that CPython wandered a long time before settling, so it's hardly surprising. It has to be revisited for Jy3 anyway: chance to re-think? |
|||
msg10064 (view) | Author: Jeff Allen (jeff.allen) | Date: 2015-05-16.14:52:02 | |
Correction, zipimporter is doing its own thing with a table of contents. It's still a helpful study. In ClasspathPyImporter, a sneaky entries.remove happens inside makeBundle (if you use it), where one-time use is assured, and fixes my bug. This is a very odd proceeding. |
|||
msg10065 (view) | Author: Jim Baker (zyasoft) | Date: 2015-05-16.15:21:30 | |
Jeff, sounds like the existing implementation was half baked, and it introduces a resource leak in the entries map. I think the best thing to do now is remove the entries map completely, and avoid caching in this layer. Instead let the underlying OS filesystem determine what caching needs to be done, if any. Alternatively users of the importer can do their own caching. |
|||
msg10066 (view) | Author: Jeff Allen (jeff.allen) | Date: 2015-05-16.15:41:14 | |
No I was confused. It's not caching. It's holding something (in this case the InputStream around the resource) between finding it and using it. The trick (missed by Raphael in #2058) is to remove it when you use it. If you don't do that it *looks* like a cache because there's an entry for each thing you've accessed. This bit's good now. I think refactoring this is a separate little project. End in sight. |
|||
msg10068 (view) | Author: Jeff Allen (jeff.allen) | Date: 2015-05-17.08:43:19 | |
I claim this is fixed, including the addition of desired PEP 302 optional methods, by change sets culminating in: https://hg.python.org/jython/rev/dccf5d04d58e I'm not *absolutely* sure that all the possible ways we might want to use PEP 302 methods are supported with least surprise semantics. (See also additions to test_classpathimporter.) Where there was no established expectation from CPython, I've used my judgement. Bottom line: > java -cp ulib.jar;%JT%\dist\jython-dev.jar;%JT%\dist\javalib\* org.python.util.jython -m u.v.w.x Init: u\__init__.py Init: u\v\__init__.py Init: u\v\w\__init__.py File: u\v\w\x.py File: u\y.py |
History | |||
---|---|---|---|
Date | User | Action | Args |
2015-05-27 14:53:30 | zyasoft | set | status: pending -> closed |
2015-05-17 08:43:20 | jeff.allen | set | status: open -> pending resolution: accepted -> fixed messages: + msg10068 |
2015-05-16 15:41:14 | jeff.allen | set | messages: + msg10066 |
2015-05-16 15:21:30 | zyasoft | set | messages: + msg10065 |
2015-05-16 14:52:03 | jeff.allen | set | messages: + msg10064 |
2015-05-16 10:01:53 | jeff.allen | set | messages:
+ msg10063 versions: + Jython 2.7, - Jython 2.5 |
2015-05-08 16:31:52 | jeff.allen | set | messages: + msg10039 |
2015-05-08 15:46:23 | zyasoft | set | messages: + msg10038 |
2015-05-08 15:37:43 | jeff.allen | set | messages: + msg10037 |
2015-04-19 22:40:47 | jeff.allen | set | assignee: jeff.allen nosy: + jeff.allen |
2015-02-23 04:50:56 | zyasoft | set | priority: urgent -> high messages: + msg9553 |
2015-02-19 15:02:32 | zyasoft | set | priority: high -> urgent messages: + msg9531 |
2015-01-09 03:53:27 | zyasoft | set | priority: high resolution: accepted messages: + msg9371 |
2015-01-09 03:50:49 | zyasoft | set | nosy:
+ zyasoft messages: + msg9369 |
2013-02-19 21:50:47 | fwierzbicki | set | nosy:
+ fwierzbicki versions: + Jython 2.5, - 2.5.2, 2.5.3b2 |
2012-05-25 00:02:47 | conflatedauto | set | messages: + msg7144 |
2012-05-21 19:57:41 | darjus | set | nosy:
+ darjus messages: + msg7124 |
2012-04-30 08:37:52 | conflatedauto | create |
Supported by Python Software Foundation,
Powered by Roundup