Issue1238

By running pyftpdlib [1] test suite on Jython 2.5b1 I get this exception
from select().
I can't understand the reason since it happens randomly.


back (most recent call last):
  File "C:\jython2.5b0\Lib\threading.py", line 247, in _Thread__bootstrap
    self.run()
  File "test_ftpd.py", line 2003, in run
    self.server.serve_forever(timeout=self.__timeout, count=1,
  File "C:\jython2.5b0\bin\ftpserver.py", line 3126, in serve_forever
    poll_fun(timeout, map)
  File "C:\jython2.5b0\Lib\asyncore.py", line 108, in poll
    r, w, e = select.select(r, w, e, timeout)
  File "C:\jython2.5b0\Lib\select.py", line 171, in native_select
    pobj = poll()
  File "C:\jython2.5b0\Lib\select.py", line 63, in __init__
    self.selector = java.nio.channels.Selector.open()
IOException: java.io.IOException: Unable to establish loopback connection


[1] http://code.google.com/p/pyftpdlib

I downloaded pyftpdlib and tried to run the test suite.

It fails completely because pyftpdlib uses the cpython specific "buffer"
builtin, which is not available on jython.

The test suite then proceeds to hang, not sure why.

More on this later; I'm working on asyncore at the moment, and that
should be resolved before addressing this.

Have you contacted the pyftpdlib authors? Are they interested in having
their code run on jython?

See here for a good description of the buffer interface and its
problems, written by Guido himself.

http://mail.python.org/pipermail/python-dev/2000-October/009974.html

I'm the pyftpdlib author and even if it's not high priority I'd like to
see pyftpdlib work on Jython too.
pyftpdlib on Jython fails for many reasons.
By monkey patching such issues I managed to run the test suite despite a
lots of tests still keep failing.
Here's a list of some reasons why pyftpdlib test suite does not work on
Jython:


- Missing IPv6 support ( http://bugs.jython.org/issue1210 ) solved by
disabling the test

- asyncore.dispatcher._map attribute on Jython is deleted for no reason
( http://bugs.jython.org/issue1226 ) solved by monkey patching asyncore.py 

- socket.getservbyport not implemented (
http://bugs.jython.org/issue1225 ), solved by disabling the test

- missing buffer() function, solved by putting this piece of code
straight into ftpserver.py

try:
    buffer
except NameError:
    # hack for Jython which does not implement buffer() 
    def buffer(obj, start=None, stop=None):
        if not hasattr(obj, "__getitem__"):
            raise TypeError
        if start == None:
            start = 0
        if stop == None:
            stop = len(obj)
        return obj[start:stop]


Despite all these modifications I still get the following exception
(amongst others):
IOException: java.io.IOException: Unable to establish loopback connection

In case you want to try I attach the modified ftpserver.py and
test_ftpd.py scripts I modified to make them partially work with Jython,
and also the traceback message I get from the failing tests.

Note that I also get this error which is quite strange:


======================================================================
ERROR: test_retr_ascii (__main__.TestFtpRetrieveData)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_ftpd.py", line 1213, in test_retr_ascii
    expected = data.replace(os.linesep, '\r\n')
OutOfMemoryError: java.lang.OutOfMemoryError: Java heap space

I found this:
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5073504

I have checked in a new version of asyncore at r6013.

Please, can you try your code again with the new version?

https://jython.svn.sourceforge.net/svnroot/jython/trunk/jython/Lib/asyncore.py

About this error:

======================================================================
ERROR: test_retr_ascii (__main__.TestFtpRetrieveData)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_ftpd.py", line 1213, in test_retr_ascii
    expected = data.replace(os.linesep, '\r\n')
OutOfMemoryError: java.lang.OutOfMemoryError: Java heap space

I notice that you're using the cpython "buffer" construct. Because
buffer is not available on jython, you're using a simple replacement,
shown above.

And now you're getting an "Out of memory" error. When you get this
error, how big is the content of data variable, i.e. what is len(data)?

> Please, can you try your code again with the new version?

Using 2.5 asyncore.py looks fine but I still get that select() related
error, which seems not related with asyncore itself.

> And now you're getting an "Out of memory" error. When you get this
> error, how big is the content of data variable, i.e. what is 
> len(data)?

>>> data = ('abcde12345' + os.linesep) * 100000
>>> len(data)
1200000

If I execute:

>>> expected = data.replace(os.linesep, '\r\n')

...from the Jython prompt instead of from the test suite I don't get any
exception.

Let's stay focussed on the select() issues in this bug; please consider
the out of memory error a separate issue.

In relation to the select bug, please can you provide the following?

1. What operating systems ad versions does it happen on?
2. What JVMs and versions does it happen on?
3. Are you running a firewall?

Windows XP Prof SP3, Java JRE 1.6.0_11, no firewall.

Please can you upload new versions of ftpserver.py and test_ftpd.py
which work with the new asyncore module?

I'll check it on windows and see if I can make any sense out of the bug,
which seems to happen in many different circumstances.

http://www.google.com/search?q=selector.open+%22java.io.IOException%3A+Unable+to+establish+loopback+connection%22

In attachment.

Any developments on this bug?

I will close the bug if there is no more activity on it.

I don't think that closing the report is a good idea when the problem is
still there.
Aside from asyncore/pyftpdlib it seems a problem related to select()
itself so any other application relying on it will most likely not work.

It looks like this is the same bug as 

http://bugs.jython.org/issue1291

Reported by Irmen de Jong.

All activity on this bug will continue on 1291, because Irmen has
isolated the bug with a small piece of code, thus making it far easier
reproduce and address.

I am fairly sure that this bug is a duplicate of bug 1291.
I intend to close this bug as a duplicate, but will give the reporter a
final chance to disagree.

It seems to be the same bug.
Please go on and close this one.
Thanks.

Closing this bug as a duplicate of 1291, by agreement with the reporter.

http://bugs.jython.org/issue1291