Issue2525

classification
Title: Jython incorrectly buffers file pipe output with Subprocess(command,stdout='/some/file/path')
Type: behaviour Severity: normal
Components: Core Versions: Jython 2.7
Milestone:
process
Status: open Resolution: remind
Dependencies: Superseder:
Assigned To: Nosy List: amak, jeff.allen, shadowhawk100, zyasoft
Priority: low Keywords:

Created on 2016-10-12.22:02:37 by shadowhawk100, last changed 2018-03-03.17:07:04 by jeff.allen.

Messages
msg10965 (view) Author: Ibraheem (shadowhawk100) Date: 2016-10-12.22:02:35
I created this bug in RobotFramework:
https://github.com/robotframework/robotframework/issues/2457

After further investigation, it seems that this is actually a bug with Jython's implementation of Subprocess and not an issue with RobotFramework itself (see comments in github issue).

Basically, when using Subprocess to run a db-query script which generates 174 MB of stdout data piped directly to a file, Subprocess with stdout redirected to a file takes about 150 seconds to complete while the script normally completes in about 17 seconds in the shell or using Python System calls.

Jython's implementation of Subprocess when using file pipes needs to be fixed to pipe directly to a file instead of doing whatever it is doing now.

I'm kind of new to Jython and how exactly it does everything it does so I hope you'll excuse me if I misdiagnosed the cause a bit :) ... I am pretty sure this is a Jython problem though!
msg11739 (view) Author: Jeff Allen (jeff.allen) Date: 2018-03-03.17:06:46
It does seem as though something silly is happening (10 bytes at a time) in the circumstances detailed at the github link.

Text i/o will never have a high performance. The subprocess pipes necessarily feed back to Jython's i/o system in order to reach your output redirection. There will be a codec or two on the way. It's not obvious how one would short-circuit that.

Not a blocker for 2.7.2 as it is functionally correct.
History
Date User Action Args
2018-03-03 17:07:04jeff.allensetseverity: major -> normal
2018-03-03 17:06:46jeff.allensetpriority: low
nosy: + jeff.allen
messages: + msg11739
milestone: Jython 2.7.2 ->
2017-06-23 16:55:09zyasoftsetresolution: remind
milestone: Jython 2.7.2
2016-10-14 16:20:36amaksetnosy: + amak
2016-10-12 23:31:52zyasoftsetnosy: + zyasoft
2016-10-12 22:02:38shadowhawk100create