Title: Popen.terminate closes pipes
Type: Severity: normal
Components: Versions: Jython 2.7
Status: open Resolution: accepted
Dependencies: Superseder:
Assigned To: Nosy List: pekka.klarck, zyasoft
Priority: low Keywords:

Created on 2014-10-13.21:13:43 by pekka.klarck, last changed 2015-01-19.11:12:14 by pekka.klarck.

File name Uploaded Description Edit Remove pekka.klarck, 2014-10-13.21:13:42
msg9135 (view) Author: Pekka Klärck (pekka.klarck) Date: 2014-10-13.21:13:42
The bug can be reproduced by running the attached script. With Python 2.7 you get "start", as expected, bu with Jython 2.7b3 you get:

Traceback (most recent call last):
  File "", line 15, in <module>
IOError: Stream closed
msg9136 (view) Author: Pekka Klärck (pekka.klarck) Date: 2014-10-13.21:40:21
Notice also that in the example `p.stdout.closed is False` before calling ``.
msg9138 (view) Author: Jim Baker (zyasoft) Date: 2014-10-14.01:47:59
This is not currently possible. For details, please see

                    # Coupling stdin is not supported: there's no way to
                    # cleanly interrupt it if it blocks the
                    # _CouplerThread forever (we can Thread.interrupt()
                    # its _CouplerThread but that closes stdin's
                    # Channel)

which means we don't have the ability to put a coupling thread to ensure simulating selectable, plus the underlying channel is buffered. Now there are other possibilities as seen in JRuby, with details given here:

But as we see in the final analysis by Charlie Nutter, one hits a wall here.

For now, likely the best approach is to let CPython control the subprocess, then control the CPython with some other tooling like rpyc.

Perhaps Java 9 will offer some improvements, although it looks like will be more limited. Let's revisit at that time.
msg9140 (view) Author: Pekka Klärck (pekka.klarck) Date: 2014-10-14.08:27:31
I was afraid this would be due to JVM limitations. Luckily in our situation it's easy catch the resulting IOError and return an empty string instead. Obviously that will lose everything written to pipes before termination, but that's not too severe. I already tested that the problem can be avoided altogether if using files as streams.
msg9141 (view) Author: Pekka Klärck (pekka.klarck) Date: 2014-10-14.08:47:52
Interestingly this problem does not occur on Windows.
msg9425 (view) Author: Pekka Klärck (pekka.klarck) Date: 2015-01-19.11:12:14
I'm testing new POSIX Popen.kill implementation (#2220) that uses java.lang.Process.destroyForcibly in Java 8+ and sends SIGKILL otherwise. Interestingly pipes are closed in the former case but not in the latter. The process is killed just fine in both cases.
Date User Action Args
2015-01-19 11:12:14pekka.klarcksetmessages: + msg9425
2014-10-14 08:47:52pekka.klarcksetmessages: + msg9141
2014-10-14 08:27:32pekka.klarcksetmessages: + msg9140
2014-10-14 01:48:00zyasoftsetpriority: low
resolution: accepted
messages: + msg9138
nosy: + zyasoft
2014-10-13 21:40:21pekka.klarcksetmessages: + msg9136
2014-10-13 21:13:43pekka.klarckcreate