Issue1765
Created on 2011-07-03.06:00:35 by mkz, last changed 2014-06-19.05:56:40 by zyasoft.
File name |
Uploaded |
Description |
Edit |
Remove |
proctest.py
|
mkz,
2011-07-03.06:00:34
|
python code |
|
|
msg6561 (view) |
Author: Marko Kreen (mkz) |
Date: 2011-07-03.06:00:34 |
|
Code that repeatedly calls subprocess.Popen() will crash with
'Too many files open' error.
Attached files reproduce it. Crash is reproducable also with default
ulimit (1024).
Python code contains several patterns of Popen usage, all leaky.
Excpected: .communicate() and code that closes fds excplicitly
should be safe.
test script:
--------------------------
#! /bin/sh
# crash faster
ulimit -n 32
jython ./proctest.py
# jython leaves tty messed up
stty sane
--------------------------
Output:
$ ./test.sh
0
1
2
3
4
5
Traceback (most recent call last):
File "./proctest.py", line 9, in <module>
p = Popen(['true'])
File "/opt/apps/jython/Lib/subprocess.py", line 751, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "/opt/apps/jython/Lib/subprocess.py", line 1265, in _execute_child
raise OSError(e.getMessage() or e)
OSError: Cannot run program "true" (in directory "/home/marko/src/niisama/rarfile/jxx"): java.io.IOException: error=24, Too many open files
|
msg6562 (view) |
Author: Marko Kreen (mkz) |
Date: 2011-07-03.06:11:08 |
|
My setup: Jython 2.5.2, Ubuntu 10.04
$ java -version
java version "1.6.0_20"
OpenJDK Runtime Environment (IcedTea6 1.9.8) (6b20-1.9.8-0ubuntu1~10.04.1)
OpenJDK 64-Bit Server VM (build 19.0-b09, mixed mode)
|
msg6814 (view) |
Author: halfie (halfie) |
Date: 2012-03-19.08:45:43 |
|
I can re-produce the bug (using proctest.py) on Oracle JDK and Open JDK 6.x series. However, the bug magically disappears when using Oracle JDK 7. Seems to be a JVM bug (and not a Jython bug).
|
msg6815 (view) |
Author: Marko Kreen (mkz) |
Date: 2012-03-19.09:24:37 |
|
The question is, does subprocess module leaves fd.close() to GC?
If yes it's subprocess problem, either in Jython-specific code, or maybe even generic core subprocess problem, except it only is seen on non-refcounting python implementations.
Not sure why newer JVM makes issue disapper. More aggressive GC? But does it guarantee the issue is fixed now?
OTOH, if subprocess closes it's objects properly, but still JVM leaves thing hanging, then it's indeed JVM problem.
|
msg8717 (view) |
Author: Jim Baker (zyasoft) |
Date: 2014-06-19.05:56:39 |
|
Looks like an earlier JDK problem, since it also works for me on 2.7 trunk against JDK 7. Re subprocess and closing FD resources, I would think the underlying Process.waitFor, as used by our implementation of communicate for example, will take care of closing.
|
|
Date |
User |
Action |
Args |
2014-06-19 05:56:40 | zyasoft | set | status: open -> closed resolution: invalid messages:
+ msg8717 nosy:
+ zyasoft |
2013-02-25 20:16:57 | fwierzbicki | set | priority: normal nosy:
+ fwierzbicki versions:
+ Jython 2.5, - 2.5.2 |
2012-03-19 09:24:37 | mkz | set | messages:
+ msg6815 |
2012-03-19 08:45:43 | halfie | set | nosy:
+ halfie messages:
+ msg6814 |
2011-07-03 06:11:08 | mkz | set | messages:
+ msg6562 |
2011-07-03 06:00:35 | mkz | create | |
|