Issue2474

To reproduce:

E:\robotframework>jython
Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11)
[Java HotSpot(TM) Client VM (Oracle Corporation)] on java1.8.0_73
Type "help", "copyright", "credits" or "license" for more information.
>>> import os, shutil
>>> os.mkdir('foo')
>>> os.mkdir('bar')
>>> shutil.copytree('foo', 'bar/foo')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\jython2.7.0\Lib\shutil.py", line 208, in copytree
    raise Error, errors
shutil.Error: [('foo', 'bar/foo', "[Errno 1] Operation not permitted: 'bar/foo'")]
>>>

The failure seems to be caused by the fact that with Jython WindowsError doesn't exist and shutil has first this code

    try:
        WindowsError
    except NameError:
        WindowsError = None

and later in copytree

    try:
        copystat(src, dst)
    except OSError, why:
        if WindowsError is not None and isinstance(why, WindowsError):
            # Copying file access times may fail on Windows
            pass
        else:
            errors.append((src, dst, str(why)))

I already tested that always ignoring error here makes copytree pass. Even without the change the actual directory is copied fine.

Notice that there is already issue #2314 about related test_shutil failures on Windows. This issue is likely the root cause for those problems.

This problem was initially reported to Robot Framework as https://github.com/robotframework/robotframework/issues/2321. I'm surprised we haven't noticed this problem earlier, because we ought to have tests for the affected functionality that are also run on Windows. It could be that the offending exception doesn't occur always, but I couldn't easily figure out when it occurs and when not.

It turned out we don't have adequate tests for the affected functionality in Robot Framework. That explains why we haven't noticed this earlier and likely means there are no cases where shutil.copytree would work.

Peke, could you please also run your test suite against Jython 2.7.1 beta 3? I have no doubts about this issue, just want to see if there are other problems we need to resolve.

One possibility is to move the problem to Java, which as of Java 7, does a good job in managing Windows differences. We can look at using this sort of approach: http://docs.oracle.com/javase/tutorial/essential/io/examples/Copy.java

Jim, testing Jython 2.7.1 beta 3 has been on my todo list since it was released. Running tests now. Already tested that `jython -m pip install robotframework` works fine, so pip issues on Ubuntu based Linuxes seem to be fixed.

---------

What benefits would switching from Python to Java bring in this case? I guess performance benefits aren't that big. I'd probably go with a fix on the Python side if it is easy. Switching to Java would require more changes and more changes would mean more possible bugs.

Do you see problems in adding something like below to __builtin__?

    if os._name == 'nt':
        WindowsError = OSError

It wouldn't be exactly like with CPython, but it would at least satisfy `issubclass(WindowsError, OSError)` contract. Should probably still investigate where copystat actually can fail to see could catching all OSErrors on Windows hide some real problems.

Peke, we should just do a Jython-specific version of shutil, with a fix to detect the specific OSError raised when on Windows. I'm pretty sure OSError = WindowsError will cause other problems, based on my short investigation of that possible solution.

It's also possible we should look at rmtree as well - currently pip is noisy on Windows because of removing a directory.

I think I may be solving this over in #2314.