Title: randomly select() exception on 2.5b1
Type: Severity: normal
Components: Versions: 2.5.1, 2.5b1, 2.5b0
Status: closed Resolution: duplicate
Dependencies: Superseder:
Assigned To: amak Nosy List: amak, billiejoex
Priority: normal Keywords:

Created on 2009-01-14.15:30:48 by billiejoex, last changed 2009-04-03.16:49:50 by amak.

File name Uploaded Description Edit Remove billiejoex, 2009-02-03.18:18:54 Modified which partially works on Jython billiejoex, 2009-02-03.18:19:48 Modified test suite which partially works on Jython
traceback.txt billiejoex, 2009-02-03.18:20:45 Test suite results billiejoex, 2009-02-04.17:20:00 billiejoex, 2009-02-04.17:20:18
msg4045 (view) Author: Giampaolo Rodola' (billiejoex) Date: 2009-01-14.15:30:48
By running pyftpdlib [1] test suite on Jython 2.5b1 I get this exception
from select().
I can't understand the reason since it happens randomly.

back (most recent call last):
  File "C:\jython2.5b0\Lib\", line 247, in _Thread__bootstrap
  File "", line 2003, in run
    self.server.serve_forever(timeout=self.__timeout, count=1,
  File "C:\jython2.5b0\bin\", line 3126, in serve_forever
    poll_fun(timeout, map)
  File "C:\jython2.5b0\Lib\", line 108, in poll
    r, w, e =, w, e, timeout)
  File "C:\jython2.5b0\Lib\", line 171, in native_select
    pobj = poll()
  File "C:\jython2.5b0\Lib\", line 63, in __init__
    self.selector =
IOException: Unable to establish loopback connection

msg4120 (view) Author: Alan Kennedy (amak) Date: 2009-02-03.17:27:09
I downloaded pyftpdlib and tried to run the test suite.

It fails completely because pyftpdlib uses the cpython specific "buffer"
builtin, which is not available on jython.

The test suite then proceeds to hang, not sure why.

More on this later; I'm working on asyncore at the moment, and that
should be resolved before addressing this.

Have you contacted the pyftpdlib authors? Are they interested in having
their code run on jython?
msg4121 (view) Author: Alan Kennedy (amak) Date: 2009-02-03.17:28:27
See here for a good description of the buffer interface and its
problems, written by Guido himself.
msg4122 (view) Author: Giampaolo Rodola' (billiejoex) Date: 2009-02-03.18:18:53
I'm the pyftpdlib author and even if it's not high priority I'd like to
see pyftpdlib work on Jython too.
pyftpdlib on Jython fails for many reasons.
By monkey patching such issues I managed to run the test suite despite a
lots of tests still keep failing.
Here's a list of some reasons why pyftpdlib test suite does not work on

- Missing IPv6 support ( ) solved by
disabling the test

- asyncore.dispatcher._map attribute on Jython is deleted for no reason
( ) solved by monkey patching 

- socket.getservbyport not implemented ( ), solved by disabling the test

- missing buffer() function, solved by putting this piece of code
straight into

except NameError:
    # hack for Jython which does not implement buffer() 
    def buffer(obj, start=None, stop=None):
        if not hasattr(obj, "__getitem__"):
            raise TypeError
        if start == None:
            start = 0
        if stop == None:
            stop = len(obj)
        return obj[start:stop]

Despite all these modifications I still get the following exception
(amongst others):
IOException: Unable to establish loopback connection

In case you want to try I attach the modified and scripts I modified to make them partially work with Jython,
and also the traceback message I get from the failing tests.

Note that I also get this error which is quite strange:

ERROR: test_retr_ascii (__main__.TestFtpRetrieveData)
Traceback (most recent call last):
  File "", line 1213, in test_retr_ascii
    expected = data.replace(os.linesep, '\r\n')
OutOfMemoryError: java.lang.OutOfMemoryError: Java heap space
msg4126 (view) Author: Giampaolo Rodola' (billiejoex) Date: 2009-02-04.13:16:08
I found this:
msg4131 (view) Author: Alan Kennedy (amak) Date: 2009-02-04.15:55:25
I have checked in a new version of asyncore at r6013.

Please, can you try your code again with the new version?
msg4132 (view) Author: Alan Kennedy (amak) Date: 2009-02-04.15:58:19
About this error:

ERROR: test_retr_ascii (__main__.TestFtpRetrieveData)
Traceback (most recent call last):
  File "", line 1213, in test_retr_ascii
    expected = data.replace(os.linesep, '\r\n')
OutOfMemoryError: java.lang.OutOfMemoryError: Java heap space

I notice that you're using the cpython "buffer" construct. Because
buffer is not available on jython, you're using a simple replacement,
shown above.

And now you're getting an "Out of memory" error. When you get this
error, how big is the content of data variable, i.e. what is len(data)?
msg4135 (view) Author: Giampaolo Rodola' (billiejoex) Date: 2009-02-04.16:09:39
> Please, can you try your code again with the new version?

Using 2.5 looks fine but I still get that select() related
error, which seems not related with asyncore itself.

> And now you're getting an "Out of memory" error. When you get this
> error, how big is the content of data variable, i.e. what is 
> len(data)?

>>> data = ('abcde12345' + os.linesep) * 100000
>>> len(data)

If I execute:

>>> expected = data.replace(os.linesep, '\r\n')

...from the Jython prompt instead of from the test suite I don't get any
msg4136 (view) Author: Alan Kennedy (amak) Date: 2009-02-04.16:23:02
Let's stay focussed on the select() issues in this bug; please consider
the out of memory error a separate issue.
msg4137 (view) Author: Alan Kennedy (amak) Date: 2009-02-04.16:29:02
In relation to the select bug, please can you provide the following?

1. What operating systems ad versions does it happen on?
2. What JVMs and versions does it happen on?
3. Are you running a firewall?
msg4138 (view) Author: Giampaolo Rodola' (billiejoex) Date: 2009-02-04.17:00:03
Windows XP Prof SP3, Java JRE 1.6.0_11, no firewall.
msg4139 (view) Author: Alan Kennedy (amak) Date: 2009-02-04.17:11:18
Please can you upload new versions of and
which work with the new asyncore module?

I'll check it on windows and see if I can make any sense out of the bug,
which seems to happen in many different circumstances.
msg4140 (view) Author: Giampaolo Rodola' (billiejoex) Date: 2009-02-04.17:19:58
In attachment.
msg4183 (view) Author: Alan Kennedy (amak) Date: 2009-03-07.16:30:11
Any developments on this bug?

I will close the bug if there is no more activity on it.
msg4186 (view) Author: Giampaolo Rodola' (billiejoex) Date: 2009-03-07.17:00:52
I don't think that closing the report is a good idea when the problem is
still there.
Aside from asyncore/pyftpdlib it seems a problem related to select()
itself so any other application relying on it will most likely not work.
msg4368 (view) Author: Alan Kennedy (amak) Date: 2009-03-28.18:18:52
It looks like this is the same bug as

Reported by Irmen de Jong.

All activity on this bug will continue on 1291, because Irmen has
isolated the bug with a small piece of code, thus making it far easier
reproduce and address.
msg4416 (view) Author: Alan Kennedy (amak) Date: 2009-04-03.16:34:07
I am fairly sure that this bug is a duplicate of bug 1291.
I intend to close this bug as a duplicate, but will give the reporter a
final chance to disagree.
msg4417 (view) Author: Giampaolo Rodola' (billiejoex) Date: 2009-04-03.16:42:34
It seems to be the same bug.
Please go on and close this one.
msg4418 (view) Author: Alan Kennedy (amak) Date: 2009-04-03.16:49:49
Closing this bug as a duplicate of 1291, by agreement with the reporter.
Date User Action Args
2009-04-03 16:49:50amaksetstatus: open -> closed
resolution: duplicate
messages: + msg4418
2009-04-03 16:42:34billiejoexsetmessages: + msg4417
2009-04-03 16:34:08amaksetmessages: + msg4416
2009-03-28 18:18:52amaksetmessages: + msg4368
2009-03-14 14:30:40fwierzbickisetpriority: normal
2009-03-07 17:00:52billiejoexsetmessages: + msg4186
2009-03-07 16:30:11amaksetmessages: + msg4183
2009-02-04 17:20:19billiejoexsetfiles: +
2009-02-04 17:20:04billiejoexsetfiles: +
messages: + msg4140
2009-02-04 17:11:18amaksetmessages: + msg4139
2009-02-04 17:00:03billiejoexsetmessages: + msg4138
2009-02-04 16:29:04amaksetmessages: + msg4137
2009-02-04 16:23:02amaksetmessages: + msg4136
2009-02-04 16:09:40billiejoexsetmessages: + msg4135
2009-02-04 15:58:19amaksetmessages: + msg4132
2009-02-04 15:55:25amaksetmessages: + msg4131
2009-02-04 13:16:08billiejoexsetmessages: + msg4126
2009-02-03 18:20:45billiejoexsetfiles: + traceback.txt
2009-02-03 18:19:48billiejoexsetfiles: +
2009-02-03 18:18:59billiejoexsetfiles: +
messages: + msg4122
2009-02-03 17:28:27amaksetmessages: + msg4121
2009-02-03 17:27:10amaksetmessages: + msg4120
2009-01-29 21:07:34amaksetassignee: amak
nosy: + amak
2009-01-14 15:30:49billiejoexcreate