Message4485

Author kfitch42
Recipients amak, billiejoex, irmen, kfitch42, zyasoft
Date 2009-04-08.12:35:41
SpamBayes Score 1.10204e-10
Marked as misclassified No
Message-id <1239194142.02.0.278751741755.issue1291@psf.upfronthosting.co.za>
In-reply-to
Content
From what I can tell, the underlying problem here is that there is an
implicit leak in the java Selector object related to lookback
connections, that it uses for whatever reason. So perhaps the best
workaround would be to introduce our own 'leak' that will cover up the
java one :)

What if we had a poll pool (say that 10 times fast :)? The naive
implementation would make sure we never created more poll objects (and
hence selectors) than the maximum number of simultaneous calls to
select, instead of creating as many as there are calls to select.

The only outstanding question would  be: when do we clean the pool?
Perhaps it would be good enough to just let things accumulate. In
'normal' applications (at least in my definition of normal) we would
only end up with a small handful of poll objects that live forever in
the pool.

A quick and dirty pool implementation (I haven't run/tested this yet):
class _selectPollPool():
    _pollPool=[]
    _mtx = threading.mutex()
    
    @staticmethod
    def getPollObject():
        with _mtx:
            if _pollPool:
                return _pollPool.pop()
            else:
                return poll()

    def putPollObject(pobj)
        pobj.close()
        with _mtx:
            _pollPool.append(pobj)

We could then just replace the 'pobj = poll()' line in native_select
with 'pobj = _selectPollPool.getPollObject()' and the pobj.close() in
the finally block with _selectPollPool.putPollObject(pobj)
History
Date User Action Args
2009-04-08 12:35:42kfitch42setmessageid: <1239194142.02.0.278751741755.issue1291@psf.upfronthosting.co.za>
2009-04-08 12:35:42kfitch42setrecipients: + kfitch42, amak, irmen, billiejoex, zyasoft
2009-04-08 12:35:41kfitch42linkissue1291 messages
2009-04-08 12:35:41kfitch42create