Issue1161

classification
Title: imp.find_module can't find many modules (was: pylint & logilab ASTNG library)
Type: Severity: normal
Components: Library Versions: 2.5alpha3
Milestone:
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: pjenvey Nosy List: pjenvey, sreimers
Priority: Keywords:

Created on 2008-10-28.19:47:00 by sreimers, last changed 2008-12-11.05:38:44 by pjenvey.

Messages
msg3717 (view) Author: Sven Reimers (sreimers) Date: 2008-10-28.19:46:59
After happily installing pylint with jython (thanks for the quick fix) I
now get the following exception:
Traceback (most recent call last):
  File "c:\source\jython\scripts\run-pylint.py", line 3, in <module>
    from pylint import lint
  File "C:\source\jython\Lib\site-packages\pylint\lint.py", line 31, in
<module>
    from pylint.checkers import utils
  File "C:\source\jython\Lib\site-packages\pylint\checkers\__init__.py",
line 40, in <module>
    from logilab.astng.utils import ASTWalker
  File "C:\source\jython\Lib\site-packages\logilab\astng\__init__.py",
line 256, in <module>
    List._proxied = MANAGER.astng_from_class(list)
  File "C:\source\jython\Lib\site-packages\logilab\astng\manager.py",
line 191, in astng_from_class
    modastng = self.astng_from_module_name(modname)
  File "C:\source\jython\Lib\site-packages\logilab\astng\manager.py",
line 135, in astng_from_module_name
    filepath = self.file_from_module_name(modname, context_file)
  File "C:\source\jython\Lib\site-packages\logilab\astng\manager.py",
line 159, in file_from_module_name
    raise value
logilab.astng._exceptions.ASTNGBuildingException: Unable to load module
__builtin__ (No module named __builtin__)
msg3758 (view) Author: Philip Jenvey (pjenvey) Date: 2008-11-07.23:34:46
How do I reproduce this? Do I have to install the logilab-common and 
logilab-astng packages manually for pylint to work?

I tried easy_installing pylint, logilab-common and then logilab-astng. 
That ended up with the error 'No module named astng.utils' when running 
pylint.

That's due to the fact that the logilab package isn't setup as a 
namespace package

You can use the pkg_resources (or setuptools) module to do that. If 
pylint's setup.py is using setuptools, a proper install_requires= 
argument would also be nice =]
msg3857 (view) Author: Philip Jenvey (pjenvey) Date: 2008-11-27.06:55:36
To answer my own question (from http://tarekziade.wordpress.com/2008/02/20/pylint-installation-made-
easier/ ):

easy_install logilab.pylintinstaller
msg3858 (view) Author: Philip Jenvey (pjenvey) Date: 2008-11-27.07:26:19
this is caused by a bug in our imp.find_module; it raises ImportErrors on 
builtin modules, and also anything found on the meta_path
msg3884 (view) Author: Philip Jenvey (pjenvey) Date: 2008-12-08.07:35:04
Ideally our imp stuff would get a big cleanup (pretty much a rewrite). 
It's a bit crufty, and what's particularly lame is the imp module not 
totally relying on the main import code's facilities. Which is why we've 
had numerous imp bugs like this

e.g. the imp module's findFromSource is really duplicating 
core.imp.loadFromSource (but core.imp.loadFromSource doesn't return the 
data the imp module needs). CPython's import.c is much cleaner

Anyway I'm just venting my long desire to clean this stuff up the right 
way, I doubt it will happen for 2.5. We'll try to fix this with a 
simpler bandaid for before 2.5
msg3914 (view) Author: Philip Jenvey (pjenvey) Date: 2008-12-11.05:38:08
We only needed to support builtin modules in find_module, not other 
things (like meta_path), so this was actually a fairly easy fix -- r5735

With this change pylint gets farther, but I ran into 3 more issues:

  File "/Users/pjenvey/src/java/jython/jython-pylons2/Lib/site-
packages/logilab.pylintinstaller-0.15.2-
py2.5.egg/logilab/astng/builder.py", line 34, in <module>
    from parser import ParserError
ImportError: No module named parser

Which is an unused import that can be safely removed

  File "/Users/pjenvey/src/java/jython/jython-pylons2/Lib/site-
packages/pylint-0.15.2-py2.5.egg/pylint/checkers/utils.py", line 77, in 
<module>
    builtins = __builtins__.copy()

__builtins__ is a CPython implementation detail that pylint/logilab 
references a few times. You'll want to import the __builtin__ module and 
use that instead

The final issue is probably more dire, in that logilab.astng seems to be 
a heavy user of the compiler module. Jython doesn't support it and never 
will, because a) it generates .pyc bytecode that we don't use b) it 
heavily relies on the parser module which we will never support either 
c) is pending deprecation and was removed in Python 3

The parser module provides the low level innards of CPython's parser 
(which is really nothing like ours), and the way to deal with AST going 
forward is via the _ast module, which we support and was originally 
added in CPython 2.5. We also support the ast module, as well as the 
ability to pass modified/generated AST to the compile function, which 
were both added in CPython 2.6.

So migrating away from the compiler module will benefit pylint on Jython 
2.5 as well as Python 3

$ pylint b.py 
No config file found, using default configuration
Traceback (most recent call last):
  File "/Users/pjenvey/src/java/jython/jython-pylons2/Lib/site-
packages/logilab.pylintinstaller-0.15.2-
py2.5.egg/logilab/astng/manager.py", line 110, in astng_from_file
    astng = ASTNGBuilder(self).file_build(filepath, modname)
  File "/Users/pjenvey/src/java/jython/jython-pylons2/Lib/site-
packages/logilab.pylintinstaller-0.15.2-
py2.5.egg/logilab/astng/builder.py", line 219, in file_build
    node = self.string_build(data, modname, path)
  File "/Users/pjenvey/src/java/jython/jython-pylons2/Lib/site-
packages/logilab.pylintinstaller-0.15.2-
py2.5.egg/logilab/astng/builder.py", line 229, in string_build
    return self.ast_build(parse(data + '\n'), modname, path)
  File 
"/Users/pjenvey/src/java/jython/dist/Lib/compiler/transformer.py", line 
51, in parse
    return Transformer().parsesuite(buf)
  File 
"/Users/pjenvey/src/java/jython/dist/Lib/compiler/transformer.py", line 
121, in parsesuite
    return self.transform(parser.suite(text))
AttributeError: 'NoneType' object has no attribute 'suite'
************* Module b
F:  1: <class 'logilab.astng._exceptions.ASTNGBuildingException'>: 
Unable to load module b ('NoneType' object has no attribute 'suite')
History
Date User Action Args
2008-12-11 05:38:44pjenveysetstatus: open -> closed
resolution: fixed
messages: + msg3914
2008-12-08 07:35:04pjenveysetassignee: pjenvey
messages: + msg3884
title: pylint & logilab ASTNG library -> imp.find_module can't find many modules (was: pylint & logilab ASTNG library)
2008-11-27 07:26:19pjenveysetmessages: + msg3858
2008-11-27 06:55:37pjenveysetmessages: + msg3857
2008-11-07 23:34:46pjenveysetnosy: + pjenvey
messages: + msg3758
2008-10-28 19:47:00sreimerscreate