Issue2545

(Testing with Java 8 on Linux)

Jython 2.7.1b3 (default:77f1bcd04321+, Feb 1 2017, 19:21:14) 
[OpenJDK 64-Bit Server VM (Oracle Corporation)] on java1.8.0_121
Type "help", "copyright", "credits" or "license" for more information.
>>> import _imp
>>> print _imp.__doc__
This module provides the components needed to build your own
__import__ function.  Undocumented functions are obsolete.

>>> help(_imp)

#Help on class _imp:
#
#_imp = <type 'org.python.modules._imp'>

>>> import _weakref
>>> print _weakref.__doc__
Weak-reference support module.
>>> help(_weakref)
#Help on class _weakref in module __builtin__:
#
#_weakref = <type '_weakref'>
#(END)

>>> 


(By hand I inserted the text displayed by help in each case.)
I imported _weakref, because it sets up __doc__ using classDictInit while _imp only stores a plain __doc__-named static string.
In both cases __doc__ is present in dict correctly, but help fails to use it.

This looks like an easy enough bug for a first-time contributor.

https://hg.python.org/jython/file/tip/src/org/python/modules/_imp.java#l27
    public static PyString __doc__ = new PyString(
        "This module provides the components needed to build your own\n"+
        "__import__ function.  Undocumented functions are obsolete.\n"
    );

https://hg.python.org/jython/file/tip/src/org/python/modules/_weakref/WeakrefModule.java#l15

    public static final PyString __doc__ = new PyString("Weak-reference support module.");

A strategy to write a unit test for this:
* Monkey-patch the pager target, pydoc.pager, so that we can capture the text that is sent to the pager (pipe to less/more or stdout). Pass text through pydoc.plain to strip bold markup bytes to make unit tests more readable.

# Back up global variables so they can be restored at the exit of the unit test
# Assume this unit test is executed atomically or wrap it in a lock.
@BeforeClass {
    import pydoc
    defaultpager = pydoc.pager
}

# Monkey-patch the pager target
# Basically the same as pydoc.plainpager, but capturing the output to StringIO rather than stdout)
@Before {
    out = StringIO.StringIO()
    stringiopager = lambda text: out.write(pydoc.plain(text))
    pydoc.pager = stringiopager
}

# Run the unit test
    help('_imp')

# Compare the output
    assertEquals(expectedText, out.getvalue())
    # We expect out.getvalue() to contain the text from _imp.__doc__ if we ignore whitespace. This currently fails.

# Cleanup and undo the monkey-patching
@After {
    out.close()
}
@AfterClass {
    pydoc.pager = defaultpager
}

based lib-python/2.7/test/test_help.py off of lib-python/2.7/test/test_pydoc.py

help(module)
calls pydoc.help(module)
calls pydoc.doc(module, 'Help on %s:')
calls pydoc.render_doc(module, title='Help on %s:', forceload=0)
calls pydoc.text.document(module, modulename)

Which will only add the docstring if the object is a module. Since inspect identifies modules implemented in Java as classes instead of modules, the module section is not created for Java classes.

inspect.ismodule(urllib) == True #a pure python module
inspect.ismodule(_imp) == False #a java implemented module/class
inspect.isclass(_imp) == True

https://hg.python.org/jython/file/tip/Lib/pydoc.py#l321

A quick hack of writing a module section when the object is either a module or a class fixed the missing docstring for Java docs, but likely has negative consequences for Python classes (both old and new style).

diff -r 0d2b840f6df8 Lib/pydoc.py
--- a/Lib/pydoc.py      Mon Feb 27 01:22:01 2017 +0100
+++ b/Lib/pydoc.py      Mon Feb 27 03:13:18 2017 -0800
@@ -329,7 +331,7 @@
         if inspect.ismemberdescriptor(object): return self.docdata(*args)
         try:
             if inspect.ismodule(object): return self.docmodule(*args)
-            if inspect.isclass(object): return self.docclass(*args)
+            if inspect.isclass(object): return self.docmodule(*args)
             if inspect.isroutine(object): return self.docroutine(*args)
         except AttributeError:
             pass

I'm leaning towards fixing inspect.ismodule and inspect.isclass to identify Java static (all static fields, private constructor, shouldn't have instances) classes as modules. Thoughts?

Javen, thanks for looking into this!
I'd prefer to not change inspect, because long-term goal is to use an unmodified std-lib.
Also, it would be nice if the call inspect uses to detect a module (isinstance(object, types.ModuleType)) would be workable outside of inspect for Java-modules.

I see two places where this might be fixable inside of Jython:

1) Look at
PyJavaType.init
It sets up baseClass and bases (List<PyObject> visibleBases).
Maybe adding ModuleType somehow to bases could fix according isinstance-calls. (Not sure if plainly adding that type goes well through isinstance; additional tweaks might be required.)

2) Look at
Py.isInstance
Thant method backs Python's isinstance on Java-side. You could add special treatment for PyJavaType to fix inspect's module-detection call.

I feel like 1) would be the most "down to the root"-fix possible. If 1) isn't workable, 2) would be a good still Jython-internal alternative.

Maybe a combination of 1) and 2) would also be an option. E.g. in 1) add ModuleType to bases and fix 2) if it doesn't properly take them into account.