Issue1367

I am using the popen2.popen2 function to open a child and get a pipe to
him. Unfortunatelly when i print to the pipe the child does not get any
output.
Unfortunatelly i dont have internet connection in the computer i work
with so i am not able to submit exactly the same program that caused the
error, however i will post here some similar code, as similar as my
memory allows.

<Code>
1:  child_stdout, chld_stdin = popen("/usr/bin/tee")
2:  child_stdin.write("This is a line\n")
3:  child_stdin.flush()
4:  out = child_stdout.readline()
5:  print "from child:", out
</code>

The code above should print “from child: This is a line” because tee
prints back its line it receives. The code works as it should in cpython
but in jython it gets stucked in line 4 when trying to read. 
The problem in jython is that the pipe is not flushed to ‘tee’, ‘tee’
gets not input and consequently it does not print back any value. Thus
trying to read from child_stdout blocks the program to the end of the days.
I have been doing some debugging and the problem seems to happen also
with subprocess.Popen
Since I have not enough knowledge of jython and the Popen implementation
I cannot tell exactly what is the problem. In spite of that, I still
could understand some of the code and I see that the pipe is implemented
as a python file wrapping an object of type org.python.core.io.StreamIO
wrapping the output stream returned by
‘java.lang.Process.getOutputStream(..)’ . I copied-pasted that code and
 to a function of mine and tried to play with it. I arrived at the
conclution that there are two places to flush.
The jython file handler has a flush function and the java output stream
has another flush function that never gets flushed, that means that even
when the jython file is not buffered, output is never printed. 
This pseudo-code will do the work that the previous pseudo-code should
have done:
<code>
procBuild = java.lang.ProcessBuilder(["/usr/bin/tee"])
proc = procBuild.start()
proc_out = proc.getOutputStream()
child_stdout = org.python.core.io.StreamIO (proc_out, True) #copied True
from the Popen code
child_stdin = org.python.core.io.StreamIO (proc.getInputStream()
, True)
 child_stdin.write("This is a line\n")
 child_stdin.flush()
proc_out.flush() #NOTICE THIS LINE!!!
out = child_stdout.readline()
print "from child:", out
</code>

You're right, we definitely need to be flushing the OutputStream wrapped 
by StreamIO. I've fixed that in r6449

Did your extracted out example with the extra flush work? If so, jython 
trunk should be the equivalent

I do fear that this example still might block due to our file 
implementation. Our readline reads in large chunks as an optimization 
(this is copied from py3k), and I think that could still deadlock the 
stream to the process. I would like to see the actual code you're trying

Since the flush issue is solved I'm closing this issue. Feel free to log 
another if we need to continue pressing forward on this example

Thanks. That is a fast answer, this will let me erase the workaround 
from my project when you release the next version of jython2.5.

The exact example i was using is running Rpyc in jython so i can have 
the best of both worlds, the java api and the cpython api which is more 
complete that jython right now. This is mandatory for my project since i 
have some code which uses Tkinter to draw its GUI and rewritting it is 
not an option.

So i installed the last jython and Rpyc. After some modifications to 
Rpyc i managed to make it load under jython but when I tried to make the 
jython and cpython communicate both process would freeze forever. It 
took me a long time till i tracked down the problem to the pipe 
implementation, at first i thought that the problem was inside Rpyc.

Rpyc sends information between the two processes using the 
write()+flush() to send data, and read() to read it.  It does not use 
directly print or readline, so i think it wont be a problem.

In spite of that I think it is quite dangerous to make a process stuck 
because of an optimization, imagine you are implementing a unix shell in 
jython, the user writes 'ls' + <enter> but nothing happens.

ציטוט Philip Jenvey:

> Philip Jenvey <pjenvey@users.sourceforge.net> added the comment:
>
> You're right, we definitely need to be flushing the OutputStream wrapped 
> by StreamIO. I've fixed that in r6449
>
> Did your extracted out example with the extra flush work? If so, jython 
> trunk should be the equivalent
>
> I do fear that this example still might block due to our file 
> implementation. Our readline reads in large chunks as an optimization 
> (this is copied from py3k), and I think that could still deadlock the 
> stream to the process. I would like to see the actual code you're trying
>
> Since the flush issue is solved I'm closing this issue. Feel free to log 
> another if we need to continue pressing forward on this example
>
> ----------
> resolution:  -> fixed
> status: open -> closed
>
> _______________________________________
> Jython tracker <report@bugs.jython.org>
> <http://bugs.jython.org/issue1367>
> _______________________________________
>