Issue1736

classification
Title: Missing os.fstat function
Type: rfe Severity: normal
Components: Library Versions: Jython 2.7
Milestone:
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: zyasoft Nosy List: amak, dstromberg, fwierzbicki, zyasoft
Priority: high Keywords:

Created on 2011-04-19.02:11:29 by dstromberg, last changed 2015-02-04.21:15:56 by zyasoft.

Files
File name Uploaded Description Edit Remove
unnamed dstromberg, 2013-03-01.00:16:28
unnamed dstromberg, 2013-03-01.00:53:33
Messages
msg6500 (view) Author: Dan Stromberg (dstromberg) Date: 2011-04-19.02:11:29
os.fstat appears to be in the Jython 2.5.2 documentation, but not in the 2.5.2 os module.

os.fstat is important for avoiding symlink races - a security issue.

AFAIK, Java runtimes offer no direct way of accessing the C libray's fstat(), but fstat remains important for about anything that runs as root and accesses files not owned by root.
msg7709 (view) Author: Frank Wierzbicki (fwierzbicki) Date: 2013-02-19.23:50:52
We should probably update the documentation.
msg7871 (view) Author: Alan Kennedy (amak) Date: 2013-02-28.22:39:00
This bug is also the cause of a failure in our test_httplib.py

http://hg.python.org/jython/file/e80a189574d0/Lib/test/test_httplib.py#l204

The failure of os.fstat(body.fileno()).st_size causes a failure to set a content-length.

The jruby folks have more details about this issue (but they haven't fixed it either)

IO#stat blows up 
http://jira.codehaus.org/browse/JRUBY-3769
msg7874 (view) Author: Alan Kennedy (amak) Date: 2013-03-01.00:08:42
Thinking further, we could actually implement a limited version of os.fstat in pure java.

On java 6, we could support size and modification time, through the use of java.io.File. Which is better than nothing, but would not address the requesters issue of dealing with symbolic links.

On java 7, we could implement almost everything, using the new java java.nio.file.Files class, which has extensive facilities, including symbolic link handling, file permissions, etc.

http://docs.oracle.com/javase/7/docs/api/index.html?java/nio/file/Files.html

Thoughts?
msg7875 (view) Author: Dan Stromberg (dstromberg) Date: 2013-03-01.00:16:28
I like anything that'll get me st_ino, st_dev, st_uid, st_gid and st_mode.

I'm currently using each of these to detect race condition tampering in a
filesystem backup application.
msg7876 (view) Author: Alan Kennedy (amak) Date: 2013-03-01.00:26:09
> "st_ino, st_dev, st_uid, st_gid and st_mode."

On java 7, we can do uid, gid and mode.

But we can't do inode or device, unless there's something I'm missing in the docs.

This is all really low level filesystem stuff that java tries to avoid, because it's so platform specific.
msg7877 (view) Author: Alan Kennedy (amak) Date: 2013-03-01.00:31:25
Here is the only mention of inode anywhere in the java docs.

http://docs.oracle.com/javase/7/docs/api/java/nio/file/attribute/BasicFileAttributes.html#fileKey%28%29

"""
Object fileKey()

Returns an object that uniquely identifies the given file, or null if a file key is not available. On some platforms or file systems it is possible to use an identifier, or a combination of identifiers to uniquely identify a file. Such identifiers are important for operations such as file tree traversal in file systems that support symbolic links or file systems that allow a file to be an entry in more than one directory. On UNIX file systems, for example, the device ID and inode are commonly used for such purposes.

The file key returned by this method can only be guaranteed to be unique if the file system and files remain static. Whether a file system re-uses identifiers after a file is deleted is implementation dependent and therefore unspecified.

File keys returned by this method can be compared for equality and are suitable for use in collections. If the file system and files remain static, and two files are the same with non-null file keys, then their file keys are equal.
"""
msg7878 (view) Author: Dan Stromberg (dstromberg) Date: 2013-03-01.00:53:33
The existence of fileKey() suggests to me that the os module should have a
new callable that'll detect filesystem race conditions, so that it can be
abstracted as needed in different interpreters.

Naturally, it may never find its way into Python 2.x, but maybe for 3.x.
msg8310 (view) Author: Jim Baker (zyasoft) Date: 2014-04-25.16:56:36
Fix for 2.7.

Among other things impacts httplib.HTTPConnection send file support, which I'm reexposing as a failure in test_httplib
msg9454 (view) Author: Jim Baker (zyasoft) Date: 2015-01-26.04:07:46
fstat on file descriptors is supported by JNR Posix (https://github.com/jnr/jnr-posix/blob/master/src/main/java/jnr/posix/LazyPOSIX.java#L124), so we should look at supporting this feature now in 2.7

This lack of support also impacts logging.handlers.WatchedFileHandler
msg9459 (view) Author: Jim Baker (zyasoft) Date: 2015-01-28.02:58:31
Fixed as of https://hg.python.org/jython/rev/e1a246f6a178
History
Date User Action Args
2015-02-04 21:15:56zyasoftsetstatus: pending -> closed
2015-01-28 02:58:31zyasoftsetstatus: open -> pending
resolution: accepted -> fixed
messages: + msg9459
2015-01-26 04:07:59zyasoftsetpriority: normal -> high
assignee: zyasoft
2015-01-26 04:07:47zyasoftsetmessages: + msg9454
title: Jython 2.5.2 lacks os.fstat -> Missing os.fstat function
2014-04-25 16:56:44zyasoftsetversions: + Jython 2.7, - Jython 2.5
2014-04-25 16:56:36zyasoftsetpriority: low -> normal
resolution: accepted
messages: + msg8310
nosy: + zyasoft
2013-03-01 00:53:33dstrombergsetfiles: + unnamed
messages: + msg7878
2013-03-01 00:31:25amaksetmessages: + msg7877
2013-03-01 00:26:09amaksetmessages: + msg7876
2013-03-01 00:16:28dstrombergsetfiles: + unnamed
messages: + msg7875
2013-03-01 00:08:43amaksetmessages: + msg7874
2013-02-28 22:39:00amaksetnosy: + amak
messages: + msg7871
2013-02-19 23:50:52fwierzbickisetpriority: low
nosy: + fwierzbicki
messages: + msg7709
versions: + Jython 2.5, - 2.5.2rc
2011-04-19 02:11:29dstrombergcreate