Issue2014

Jython 2.7b1 (default:ac42d59644e9, Feb 9 2013, 15:24:52)
[Java HotSpot(TM) Client VM (Sun Microsystems Inc.)] on java1.6.0_06
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.mkdir("\\temp\\ddir")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 20047] Unknown error: 20047: '\\temp\\ddir'

The mkdir call does not create intermediate directories.

http://docs.python.org/2/library/os.html

Does the intermediate directory "\\temp" exist on your machine?

If it does not, and you want to create 2 directories, i.e. "\\temp" and then create "ddir" inside it, then use the os.makedirs function, which "recursively ... makes all intermediate-level directories needed to contain the leaf directory. Raises an error exception if the leaf directory already exists or cannot be created."

http://docs.python.org/2/library/os.html

Sorry for not being clear enough (late hours I guess).

I was using Sphinx (Python Doc Generator) with Jython 2.5.3 and everything worked for me. But when I switched to Jython 2.7b1 my doc builds started to fail.

The problem is that the same exception is raised with different error numbers.

Jython 2.5 raises the following exception:
OSError: [Errno 17] File exists: ...
This exception is caught by Sphinx and err.errno is checked. Errno 0 & 17 are ignored.

But Jython 2.7 raises:
OSError: [Errno 20047] Unknown error: 20047: ...
which causes Sphinx to reraise the exception and ultimately to fail.

Hi Jeff, I added you to the nosy list as you might have some insight here?

In order to fix the users issue, we need to be able to reproduce it.

To reproduce it, we need to know what directories did and did not exist before the call was attempted.

Did the "\\temp"" directory exist or not?

Did the "\\ddir" directory exist or not?

It appears that the error number 20047 is an error that jnr raises on Windows, when stat() is called.

http://jira.codehaus.org/browse/JRUBY-6136

I did a quick test on my Windows 7 x64 box and may have the explanation.

Jython 2.7b1+ (default:ebb67f175c9f+, Feb 17 2013, 23:49:57)
[Java HotSpot(TM) 64-Bit Server VM (Sun Microsystems Inc.)] on java1.6.0_35
Type "help", "copyright", "credits" or "license" for more information.
>>> import os

Done progressively, locally, it works:
>>> os.mkdir("junk")
>>> os.mkdir("junk\\1")
>>> os.mkdir("junk\\2")

I deleted the junk folder here in Windows Explorer. Done as a path all in one go, it fails, but with error 3 and a not very good message:

>>> os.mkdir("junk\\1")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 3] No such process: 'C:\\Users\\Jeff\\Documents\\Eclipse\\jython-trunk\\junk\\1'

Then I tried again in the root (of C:):

>>> os.mkdir("\\junk\\1")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 3] No such process: '\\junk\\1'

But done progressively it works:
>>> os.mkdir("\\junk")
>>> os.mkdir("\\junk\\1")

If I try to create it again (no intervening delete) I get the error we're looking for:
>>> os.mkdir("\\junk\\1")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 20047] Unknown error: 20047: '\\junk\\1'

I get similar results with CPython 2.7:

Python 2.7.2 (default, Jun 12 2011, 14:24:46) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.mkdir("junk2\\1")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
WindowsError: [Error 3] The system cannot find the path specified: 'junk2\\1'
>>> os.mkdir("junk2\\1")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
WindowsError: [Error 3] The system cannot find the path specified: 'junk2\\1'
>>> os.mkdir("junk2")
>>> os.mkdir("junk2\\1")
>>> os.mkdir("\\junk2\\2")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
WindowsError: [Error 3] The system cannot find the path specified: '\\junk2\\2'
>>> os.mkdir("\\junk2")
>>> os.mkdir("\\junk2\\2")
>>> os.mkdir("\\junk2\\2")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
WindowsError: [Error 183] Cannot create a file when that file already exists: '\\junk2\\2'
>>>

Different number and a more helpful message, but it works and fails in the same places.

As it appears os.mkdir cannot and never could make paths, we should assume that in Paul's case, \temp exists beforehand and only ddir is being created. We know that file deletion on Windows often raises error 17 (no permission) when Python tries to to unlink a file à la Unix. (The theory is that something else is still using the file, maybe a virus checker: Windows is another country.)

Since Paul states that errors 0 and 17 are ignored, I speculate that in this case, a previous pass has failed to delete ddir, and the error (maybe 13) ignored. Perhaps the present unexpected error results from trying to create a directory that already exists?

phinx should not be checking raw error numbers, i.e. error 0 and 17, since these values are platform specific. This is a common problem with cpython code. Instead the code should be checking for the symbolic constants in errno, not the raw numbers. (On jython, these numbers will often be different between platforms, just as they are between cpython on *nix and cpython in windows)

I can see that this has been a problem with sphinx before.

http://stackoverflow.com/questions/1085791/does-sphinx-run-on-jython

Which resulted in this bug report

https://bitbucket.org/birkenfeld/sphinx/issue/265/sphinx-doesnt-run-with-jython251

Which resulted in this code patch, which did not properly fix the problem.

https://bitbucket.org/birkenfeld/sphinx/commits/6948941768d2

I think this should be reported to the sphinx people, with a request that they use symbolic constants for errors, not platform-specific integers.

Alen, the answer to your question is:

The directory "\\temp\\ddir" already exists on the machine.


As for Sphinx, it actually doesn't check raw error numbers. Previously I just shortened the code a bit. The actual code looks like this (only the relevant parts):

# -------------------------------------------------
import os
import errno

EEXIST = getattr(errno, 'EEXIST', 0)

def ensuredir(path):
    """Ensure that a path exists."""
    try:
        os.makedirs(path)
    except OSError, err:
        # 0 for Jython/Win32
        if err.errno not in [0, EEXIST]:
            raise
# -------------------------------------------------

Regardless of the Sphinx issue (which doesn't occur when run in Jython 2.5), in Jython 2.7 the raised exception doesn't look good:

Jython 2.7b1 (default:ac42d59644e9, Feb 9 2013, 15:24:52)
[Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)] on java1.7.0_11
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.mkdir("\\temp\\ddir")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 20047] Unknown error: 20047: '\\temp\\ddir'


While on the same machine, with the same JVM the Jython 2.5.3 exception looks like this:

Jython 2.5.3 (2.5:c56500f08d34+, Aug 13 2012, 14:48:36)
[Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)] on java1.7.0_11
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.mkdir("\\temp\\ddir")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 17] File exists: '\\temp\\ddir'

Is this still an issue?

I reviewed http://jira.codehaus.org/browse/JRUBY-6136, and it's clear we need to do something similar to https://github.com/jruby/jruby/commit/947c661e46683ea82f8016dde9d3fa597cd10e56

which I copied here since it's so short

+    public RaiseException newErrnoFromInt(int errno, String methodName, String message) {
+        if (Platform.IS_WINDOWS && ("stat".equals(methodName) || "lstat".equals(methodName))) {
+            if (errno == 20047) return newErrnoENOENTError(message); // boo:bar UNC stat failure
+            if (errno == Errno.ESRCH.intValue()) return newErrnoENOENTError(message); // ESRCH on stating ""
+        }
+        
+        return newErrnoFromInt(errno, message);
+    }

basically we just to hardcode the workaround to treat 20047 and errno.ESRCH as special on Windows for stat, lstat so they work like on CPython (and presumably on C version of Ruby) and map to what we expect.

Interestingly, JRuby chooses to map to errno.ENOENT instead of errno.EEXIST...

Need to fix for 2.7.0

This problem causes that weird error also when installing like `jython setup.py install`. For details see #2257 that was closed as a dupe of this issue.

Inadvertently fixed to produce correct errno in https://hg.python.org/jython/rev/ca92ffdff93f, should have been in a separate commit, but no worries