Issue1389

classification
Title: subprocess execution is way too slow
Type: Severity: normal
Components: Library Versions: 2.5b1
Milestone:
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: ita, nriley, pjenvey
Priority: Keywords:

Created on 2009-06-30.20:56:18 by ita, last changed 2013-02-19.21:26:50 by fwierzbicki.

Messages
msg4867 (view) Author: ita (ita) Date: 2009-06-30.20:56:17
/home/waf/jython/bin/jython -c "import subprocess,
time;t=time.time();[subprocess.Popen(['true'], shell=False).wait() for x
in xrange(1000)];print time.time()-t"

cpython: 3 seconds
jython: 25 seconds
msg5241 (view) Author: Philip Jenvey (pjenvey) Date: 2009-10-20.04:42:09
Is this really our fault? How does Java's plain ProcessBuilder fair 
against CPython?
msg5255 (view) Author: Philip Jenvey (pjenvey) Date: 2009-10-22.03:08:09
Thomas Nagy emailed me and says it is our fault as Java's plain 
ProcessBuilder is faster. If someone could post a code snippet of the 
Java ProcessBuilder running the same task, that would help for someone 
looking deeper into this, wanting to compare the two

I think we're obviously going to be slower than CPython and 
ProcessBuilder, the question is by how much. Seems like there could be 
room for improvement
msg5261 (view) Author: Nicholas Riley (nriley) Date: 2009-10-24.16:01:04
Some of this is Java's fault; some of this is ours.  Profiling it, all I 
could see was a lot of time attributed to calling through 
PyReflectedFunction.

In a quick test, subprocess.Popen in Jython took 28 seconds to spawn 1000 
processes, using ProcessBuilder from Jython took 22 and using 
ProcessBuilder from Java took 12.  subprocess from CPython took 3.
msg5262 (view) Author: Nicholas Riley (nriley) Date: 2009-10-24.16:02:02
What I used to test with Java:

class sample {
    public static void main(String[] args) {
        try {
        long start = System.currentTimeMillis();
        for (int i = 0 ; i < 1000 ; i++) {
            new ProcessBuilder("true").start().waitFor();
        }
        System.out.println(System.currentTimeMillis() - start);
        } catch (Exception e) {
            e.printStackTrace();
        }

    }
}
History
Date User Action Args
2013-02-19 21:26:50fwierzbickisetstatus: open -> closed
resolution: wont fix
2009-10-24 16:02:02nrileysetmessages: + msg5262
2009-10-24 16:01:05nrileysetnosy: + nriley
messages: + msg5261
2009-10-22 03:08:11pjenveysetmessages: + msg5255
2009-10-20 04:42:10pjenveysetmessages: + msg5241
2009-06-30 22:19:16pjenveysetnosy: + pjenvey
2009-06-30 20:56:18itacreate