Issue1466
Created on 2009-09-13.14:38:29 by gaddo, last changed 2010-03-21.17:32:01 by pjenvey.
msg5144 (view) |
Author: (gaddo) |
Date: 2009-09-13.14:38:28 |
|
jython does not respect the append flag to
open, causing data loss.
this is on linux.
stracing jython, i can se that the file
is opened O_APPEND in cpython but is not
with jython
i discovered this bug while trying to run
mercuria with jython.
gaddo@enterprise:~/jython$ ./jython
Jython 2.5.0 (Release_2_5_0:6476, Jun 16 2009, 13:33:26)
[OpenJDK Client VM (Sun Microsystems Inc.)] on java1.6.0_0
Type "help", "copyright", "credits" or "license" for more information.
>>> fd = open('test', 'ab')
>>> fd.write('test1\n')
>>> fd2 = open('test','ab')
>>> fd2.write('test2\n')
>>> quit()
gaddo@enterprise:~/jython$ cat test
test2
|
msg5145 (view) |
Author: Jim Baker (zyasoft) |
Date: 2009-09-13.22:43:13 |
|
May require RC3
|
msg5146 (view) |
Author: Frank Wierzbicki (fwierzbicki) |
Date: 2009-09-13.22:50:15 |
|
Agreed, nothing is worse than potential data loss -- and my first
response -- just use close() -- looks insufficient in this case --
pjenvey: what do you think?
|
msg5147 (view) |
Author: Jim Baker (zyasoft) |
Date: 2009-09-13.23:02:59 |
|
Repeated bug on OS X 10.5.8 on trunk, including using close.
|
msg5148 (view) |
Author: Philip Jenvey (pjenvey) |
Date: 2009-09-13.23:16:10 |
|
For this smaller example you just need to fd.flush() or close() in line
3 to get the right output.
The bigger problem is that 'a' mode doesn't use O_APPEND, which makes
writes always append to EOF regardless of the fd's position. You could
use it to open a log file twice, for example, and both fds would always
append to the end (though in reality I'm not sure how safe that actually
is when files are buffered by the underlying OS)
So the JVM provides one single way of getting a file in O_APPEND, via
FileOutputStream(file, appending=True). Unfortunately that fd is opened
write only, so we couldn't support the 'a+' mode with it
|
msg5149 (view) |
Author: Philip Jenvey (pjenvey) |
Date: 2009-09-13.23:26:40 |
|
Unfortunately I guess all we can do is fake 'a+' by seek()ing to the end
before every write.. which is pretty lame
|
msg5150 (view) |
Author: Philip Jenvey (pjenvey) |
Date: 2009-09-14.01:08:22 |
|
applied said workarounds in r6796. It's a bit invasive so we need an rc3
|
msg5151 (view) |
Author: (gaddo) |
Date: 2009-09-14.06:11:26 |
|
i am worried that this can fail
in a multithreaded program, where 2 threads
append to the same log.
if both threads seek and then write, they can
see the same file length, and overwrite
each other data.
maybe this is not a problem with jython
right now ...
|
msg5152 (view) |
Author: Philip Jenvey (pjenvey) |
Date: 2009-09-14.06:30:08 |
|
with the fix I applied it's still a potential issue in 'a+' mode.
Unfortunately there's nothing else we can really do about that except
document the problem
However regular 'a' mode is now safe as it's using O_APPEND
Also it's not a problem at all if threads are writing to the same file
object, which I would recommend. Python file operations are atomic
|
msg5581 (view) |
Author: Philip Jenvey (pjenvey) |
Date: 2010-03-21.17:32:01 |
|
This fix introduced a regression in 2.5.1, see #1576
|
|
Date |
User |
Action |
Args |
2010-03-21 17:32:01 | pjenvey | set | messages:
+ msg5581 |
2009-09-14 06:30:08 | pjenvey | set | messages:
+ msg5152 |
2009-09-14 06:11:27 | gaddo | set | messages:
+ msg5151 |
2009-09-14 01:08:22 | pjenvey | set | status: open -> closed resolution: fixed messages:
+ msg5150 |
2009-09-13 23:26:40 | pjenvey | set | messages:
+ msg5149 |
2009-09-13 23:16:11 | pjenvey | set | messages:
+ msg5148 |
2009-09-13 23:02:59 | zyasoft | set | messages:
+ msg5147 |
2009-09-13 22:50:15 | fwierzbicki | set | messages:
+ msg5146 |
2009-09-13 22:48:49 | pjenvey | set | assignee: pjenvey |
2009-09-13 22:43:14 | zyasoft | set | priority: urgent nosy:
+ fwierzbicki, pjenvey, zyasoft messages:
+ msg5145 versions:
+ 2.5.1, - 2.5b1 |
2009-09-13 14:38:29 | gaddo | create | |
|