Issue2068

classification
Title: os.write does not properly support os.O_APPEND
Type: behaviour Severity: normal
Components: Core Versions: Jython 2.7
Milestone:
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: jamesls, santa4nt, zyasoft
Priority: high Keywords: patch

Created on 2013-07-14.19:32:15 by jamesls, last changed 2015-03-27.23:39:07 by zyasoft.

Files
File name Uploaded Description Edit Remove
test_issue2068.patch santa4nt, 2015-02-20.19:42:57 unittest patch to reproduce/regress this issue
issue2068.patch santa4nt, 2015-02-20.22:25:35
issue2068.patch santa4nt, 2015-02-20.22:57:16 A better patch.
Messages
msg8063 (view) Author: James (jamesls) Date: 2013-07-14.19:32:14
Mac OS X 10.8.3, using jython 2.7b1 I get:

import os

fd = os.open('testfile', os.O_RDWR|os.O_CREAT|os.O_APPEND)
os.write(fd, bytes('one'))
os.write(fd, bytes('two'))
os.write(fd, bytes('three'))
os.close(fd)

with open('testfile', 'rb') as f:
    print(f.read())

# Output, first time run:
$ jython /tmp/repro.py
three
# Second time
$ jython /tmp/repro.py
threethree

On CPython 2.7.5 I get:
$ python /tmp/repro.py
onetwothree
$ python /tmp/repro.py
onetwothreeonetwothree
msg8064 (view) Author: James (jamesls) Date: 2013-07-14.19:39:26
The actual issue appears to be with the O_APPEND flag, if I remove this option, I get the correct behavior the first time the script is run, but then of course the second time around the file is not appended to (as expected since O_APPEND is not specified).

import os

fd = os.open('testfile', os.O_RDWR|os.O_CREAT)
os.write(fd, bytes('one'))
os.write(fd, bytes('two'))
os.write(fd, bytes('three'))
os.close(fd)

with open('testfile', 'rb') as f:
    print(f.read())


$ jython /tmp/repro.py
onetwothree
msg8484 (view) Author: Jim Baker (zyasoft) Date: 2014-05-21.21:45:36
Removing bytes(...) as an issue, it's really just OS_APPEND

Target beta 4
msg9494 (view) Author: Jim Baker (zyasoft) Date: 2015-02-07.01:36:17
Still an issue. Maybe we can get this fixed for RC1
msg9532 (view) Author: Santoso Wijaya (santa4nt) Date: 2015-02-20.22:11:04
It seems like the call:

    fileChannel.write(buf, fileChannel.position());

always has the second parameter returning 3 (the number of bytes written after the first .write() call) on subsequent write() calls. This is puzzling to me since the JavaDoc for FileChannel#write [1] says:

    [...] the file position is updated with the number of bytes actually written. 

[1] http://docs.oracle.com/javase/7/docs/api/java/nio/channels/FileChannel.html#write(java.nio.ByteBuffer)
msg9533 (view) Author: Santoso Wijaya (santa4nt) Date: 2015-02-20.22:25:35
I don't know what the deal with #position() is--why it does not return updated value after each #write() call--but the file channel's #size() does.

What about this tentative patch?
msg9534 (view) Author: Santoso Wijaya (santa4nt) Date: 2015-02-20.22:33:58
>> why it does not return updated value after each #write() call

D'oh. I was looking at the wrong overloaded #write() JavaDoc. The correct one [1] does say:

    """ This method does not modify this channel's position. """

[1] http://docs.oracle.com/javase/7/docs/api/java/nio/channels/FileChannel.html#write(java.nio.ByteBuffer,%20long)
msg9535 (view) Author: Santoso Wijaya (santa4nt) Date: 2015-02-20.22:57:16
Attaching a better patch: manually adjust a file channel's position in emulated append writing mode.
msg9681 (view) Author: Santoso Wijaya (santa4nt) Date: 2015-03-19.17:49:30
Is this reviewed? I can commit it if so.
msg9682 (view) Author: Jim Baker (zyasoft) Date: 2015-03-19.18:59:29
+1, LGTM
msg9683 (view) Author: Jim Baker (zyasoft) Date: 2015-03-19.23:34:00
Santoso, if you could get this in today, that'd be great. Trying to close out the release candidate.
msg9684 (view) Author: Santoso Wijaya (santa4nt) Date: 2015-03-20.00:01:22
Done. Changeset e0f84f668a19.
History
Date User Action Args
2015-03-27 23:39:07zyasoftsetstatus: pending -> closed
2015-03-20 17:21:24zyasoftsetstatus: open -> pending
resolution: accepted -> fixed
2015-03-20 00:01:22santa4ntsetmessages: + msg9684
2015-03-19 23:34:00zyasoftsetmessages: + msg9683
2015-03-19 18:59:29zyasoftsetmessages: + msg9682
2015-03-19 17:49:30santa4ntsetmessages: + msg9681
2015-02-20 22:57:17santa4ntsetfiles: + issue2068.patch
messages: + msg9535
2015-02-20 22:33:58santa4ntsetmessages: + msg9534
2015-02-20 22:25:36santa4ntsetfiles: + issue2068.patch
messages: + msg9533
2015-02-20 22:11:05santa4ntsetmessages: + msg9532
2015-02-20 19:42:57santa4ntsetfiles: + test_issue2068.patch
keywords: + patch
2015-02-07 01:36:17zyasoftsetpriority: high
messages: + msg9494
2014-05-21 21:45:36zyasoftsetresolution: accepted
messages: + msg8484
nosy: + zyasoft
title: Incorrect behavior with bytes() and with os.write with os.O_APPEND -> os.write does not properly support os.O_APPEND
2013-07-15 18:20:07santa4ntsetnosy: + santa4nt
type: behaviour
2013-07-14 19:39:26jameslssetmessages: + msg8064
title: Incorrect behavior with bytes() and with os.write -> Incorrect behavior with bytes() and with os.write with os.O_APPEND
2013-07-14 19:32:15jameslscreate