Issue1133

classification
Title: Support ipython and other completers with readline emulation
Type: Severity: normal
Components: Core Versions: Jython 2.5
Milestone:
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: zyasoft Nosy List: alex001, fwierzbicki, mboersma, pjenvey, russell, talisker, vanvorst, walton, zyasoft
Priority: high Keywords: patch

Created on 2008-09-17.16:20:37 by zyasoft, last changed 2014-05-22.00:46:33 by zyasoft.

Files
File name Uploaded Description Edit Remove
completion.patch zyasoft, 2010-08-12.05:43:04
ipython-0_10_1-os_name.patch zyasoft, 2010-10-17.05:13:25
ipython-0.10.2-os_name-sys.platform.patch talisker, 2011-09-23.01:00:10 patch to make ipython 0.10.2 happier under jython2.5.2 (addresses os.name and sys.platform calls)
Messages
msg3606 (view) Author: Jim Baker (zyasoft) Date: 2008-09-17.16:20:36
ipython has some explicit dependencies on the underlying OS, which
doesn't work so well on Jython because it's dispatching on os.name
(=='java' on Jython). 

We also need signal support (see #1074)

There are a number of other issues, which Matt Boersma can no doubt
elaborate on!
msg3607 (view) Author: Philip Jenvey (pjenvey) Date: 2008-09-17.17:25:28
Chris McClimans also had a patch on the pastebin against IPython trunk 
(from earlier this year) to solve some issues he found. It's here:

http://pylonshq.com/pasties/645
msg4917 (view) Author: Nate Van Vorst (vanvorst) Date: 2009-07-17.21:08:57
The patch from Chris McClimans did not work for me. However, I spent
some time playing and got some functionality working for my ubuntu box.
There are patches to ipython, and a java class. Here they are:

java  --> http://pylonshq.com/pasties/e7fdb01924b15c9bcdc9966883f8eb01
patch --> http://pylonshq.com/pasties/5eff7080b77f01c7e8aaf128aebebb6e

Disclaimer: I am very much a newbie; this was pretty much the first
thing I have done with jython.
msg5919 (view) Author: Jim Baker (zyasoft) Date: 2010-07-24.15:45:10
This needs to be in the context of JLine, which we bundle with Jython.

We should have the readline module provide the same API by wrapping JLine appropriately. This also means exposing the JLineConsole, and its ConsoleReader.

Lastly, should we change the logic for org.python.util.jython such that it attempts to use JLineConsole (unless python.console is registered), then fallbacks to InteractiveConsole? The current interactive test logic doesn't seem to work for scripts that have access to the tty, eg something that might want to do getpass.
msg5952 (view) Author: Jim Baker (zyasoft) Date: 2010-08-12.05:43:04
Attached my first working version of a JLine readline that supports completion. Note that you need to define a ~/.jlinebindings.properties file with this entry:

9: COMPLETE

for completions to be activated in JLine.
msg5953 (view) Author: Jim Baker (zyasoft) Date: 2010-08-12.05:45:18
I should point out - this requires some more work for completeness. But still useful as it is. To get completions working, do the standard:

import readline
import rlcompleter
readline.parse_and_bind("tab: complete")

although this last line is just ignored for now (and part of finishing things up is to work around that, if possible).
msg5997 (view) Author: Jim Baker (zyasoft) Date: 2010-08-21.04:12:38
Applied my completion patch as of r7098. This nuked the readline that bound against GNU readline (in favor of JLine), we can resurrect that if someone cares and can test.

Readline module emulation is still not complete, or well tested, but it is definitely *very* useful.
msg6085 (view) Author: Jim Baker (zyasoft) Date: 2010-09-21.03:53:43
Plan to have this completed for 2.5.2rc1
msg6102 (view) Author: Alex G (alex001) Date: 2010-09-28.03:44:48
tab completion on an invalid object crashes jython console:

Steps to reproduce:  (linux 32 bit)

alex@meow ~ $ ./jython/bin/jython 
Jython 2.5.2b2 (Release_2_5_2beta2:7124, Sep 14 2010, 00:11:45) 
[Java HotSpot(TM) Client VM (Sun Microsystems Inc.)] on java1.6.0_21
Type "help", "copyright", "credits" or "license" for more information.
>>> a.
  File "<stdin>", line 1
    a.
     ^
SyntaxError: mismatched input '\n' expecting set null
>>> import readline                        
>>> import rlcompleter                     
>>> readline.parse_and_bind("tab:complete")
>>> a.<tab pressed>Traceback (most recent call last):
  File "/home/alex/jython/Lib/readline.py", line 97, in complete_handler
    completion = function(buffer[:cursor], state)
  File "/home/alex/jython/Lib/rlcompleter.py", line 87, in complete
    self.matches = self.attr_matches(text)
  File "/home/alex/jython/Lib/rlcompleter.py", line 130, in attr_matches
    object = eval(expr, self.namespace)
  File "<string>", line 1, in <module>
NameError: name 'a' is not defined
alex@meow ~ $
msg6175 (view) Author: Jim Baker (zyasoft) Date: 2010-10-17.05:13:24
Just committed r7153 which fixes up readline so that it can be used by ipython. Still lacks support for readline.parse_and_bind, but that's pretty minor. This means that for now you have to create a copy of the properties file, see msg5952

The other requirement is to specifically make it interactive:
jython -i ipython.py

I haven't been able to figure out a workaround on determining this (see org.python.util.jython).

ipython also needs to be patched, specifically its use of os.name. If os.name == 'java', it should be using os._name instead. This seems to be the extent of it, so hopefully it can be pushed into ipython reasonably soon. See the patch.
msg6182 (view) Author: Jim Baker (zyasoft) Date: 2010-10-17.16:59:42
readline.parse_and_bind('tab: complete') support added in r7154
msg6189 (view) Author: Jim Baker (zyasoft) Date: 2010-10-19.03:10:52
It turns out the need to use -i to force interactive was because of a mistake in the logical test in org.python.util.jython#run. Turning the test to a boolean implication fixed the problem.

Also this got ipython to support colorization too! (Not certain how that change got propagated, maybe ipython accesses these options? Some magic at work, which is what we might expect for something like ipython.)
msg6190 (view) Author: Jim Baker (zyasoft) Date: 2010-10-19.03:53:02
As I mentioned in #jython:
figured out the ipython colorization. too many ipython installs lying around. it turns out that the os.name fixes i made had this impact of turning *off* colorization. (some complex logic no doubt.) the os.name fixes were necessary prior to the readline fixes i made to support ipython. but now readline does the right thing. but we still need os.name fixes for 1) install, otherwise get "Unsupported operating system: java" ; 2) OS integration

The other change we need here is to tamp down the NotImplementedWarning in readline for unsupported key bindings. Probably just makes this warning visible when debugging. Re-opening the issue for this.
msg6443 (view) Author: Philip Jenvey (pjenvey) Date: 2011-03-20.18:49:00
We should really submit the latest patch of jython support to IPython for inclusion if we haven't already. They're getting ready to start the upcoming 0.11 release cycle: http://old.nabble.com/0.10.2.rc1-ready-for-testing.-td31136040.html
msg6651 (view) Author: Andrew Myers (talisker) Date: 2011-09-23.01:00:10
Nothing really new to add to the Jython side of this issue; attaching an updated patch for ipython 0.10.2 in case it's helpful to anyone.  It's working thus far w/jython2.5.2 under Ubuntu, including syntax coloration, ls, etc etc etc. Should work under win32 etc, but I haven't tested it.  The patch addresses references to both os.name and sys.platform.
msg8524 (view) Author: Jim Baker (zyasoft) Date: 2014-05-22.00:46:33
Closing out this bug. readline works just fine now, it's just a larger issue with supporting ipython - a rapidly moving target if there's ever one ;).

Let's track instead in the corresponding ipython issue:
https://github.com/ipython/ipython/issues/5032

There's a pull request that has a lot of useful commentary as well: 
https://github.com/ipython/ipython/pull/4226 and
History
Date User Action Args
2014-05-22 00:46:33zyasoftsetstatus: open -> closed
messages: + msg8524
2013-03-05 22:36:04amaksetkeywords: + patch
2013-03-05 22:35:49amaksetkeywords: - patch, console
2013-02-25 22:00:07amaksetkeywords: + console
2013-02-25 18:40:57fwierzbickisetversions: + Jython 2.5, - 2.5.1
2011-09-23 01:00:11taliskersetfiles: + ipython-0.10.2-os_name-sys.platform.patch
nosy: + talisker
messages: + msg6651
2011-03-20 18:49:00pjenveysetmessages: + msg6443
2010-10-19 03:53:03zyasoftsetstatus: closed -> open
messages: + msg6190
2010-10-19 03:10:53zyasoftsetmessages: + msg6189
2010-10-17 16:59:42zyasoftsetstatus: pending -> closed
messages: + msg6182
2010-10-17 05:13:28zyasoftsetstatus: open -> pending
files: + ipython-0_10_1-os_name.patch
resolution: fixed
messages: + msg6175
2010-09-28 03:44:51alex001setnosy: + alex001
messages: + msg6102
2010-09-21 03:53:44zyasoftsetpriority: normal -> high
messages: + msg6085
2010-08-21 04:12:38zyasoftsetmessages: + msg5997
2010-08-12 05:45:19zyasoftsetmessages: + msg5953
2010-08-12 05:43:08zyasoftsetfiles: + completion.patch
keywords: + patch
messages: + msg5952
2010-07-24 15:45:11zyasoftsetassignee: zyasoft
messages: + msg5919
title: Support ipython -> Support ipython and other completers with readline emulation
2009-09-17 03:45:14russellsetnosy: + russell
2009-07-17 21:08:57vanvorstsetnosy: + vanvorst
messages: + msg4917
2009-03-14 14:15:01fwierzbickisetpriority: normal
versions: + 2.5.1
2009-01-21 17:27:14fwierzbickisetnosy: + fwierzbicki
2009-01-20 15:35:22waltonsetnosy: + walton
2008-11-01 13:57:43fwierzbickisetcomponents: + Core
2008-09-17 17:25:29pjenveysetnosy: + pjenvey
messages: + msg3607
2008-09-17 16:23:12mboersmasetnosy: + mboersma
2008-09-17 16:20:37zyasoftcreate