Message10873

Author ryan.springer
Recipients ryan.springer
Date 2016-07-05.15:50:43
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1467733845.54.0.835093021935.issue2508@psf.upfronthosting.co.za>
In-reply-to
Content
We have noticed a problem that occurs sometimes when using non-blocking sockets with Jython.  Our application will call send() on a socket and then check the amount of bytes that are returned to see if the data has been sent.  If the number of bytes returned matched the number of bytes sent, then close() is called on the socket.  This logic works without problems when using cPython.  However, in Jython, if Netty has not finished delivering the data, then the close() will cause Netty to throw away whatever bytes have not been delivered yet.  Our application is unaware that this situation has occurred, and the client of the application receives a truncated response.

We can only reproduce this problem when the application and our client are communication over the slower network conditions of a VPN.  Normally Netty delivers the data before our application calls close() on the socket.

There is a FIXME in the send() method in Lib/_socket.py that reads:

  # FIXME are we sure we are going to be able to send this much data, especially async?
  return len(data)


I wanted to make sure that I understood the behavior of send() when used with non-blocking sockets, so I reviewed the Python documentation and the Posix specification for send() and write().  Two details in the Posix specification seem related to this situation:

1) If no flags are passed to send(), it is equivalent to write().  Jython is currently ignoring the flags that are passed to send().

    If the socket argument refers to a socket and the flags argument is 0, the send() function is equivalent to write().

See: http://pubs.opengroup.org/onlinepubs/9699919799/functions/send.html

2) write() will return the number of bytes written, even in non-blocking mode:

    If the O_NONBLOCK flag is set, write() shall not block the thread. If some data can be written without blocking the thread, write() shall write what it can and return the number of bytes written. Otherwise, it shall return -1 and set errno to EAGAIN.

See: http://pubs.opengroup.org/onlinepubs/9699919799/functions/write.html

This is the Posix non-blocking socket behavior.  For Python non-blocking sockets, the behavior is the same.  send() will return the number of bytes actually sent.

See: https://docs.python.org/2/library/socket.html - The description of send() says "Returns the number of bytes sent. " 
also see: http://stackoverflow.com/a/6718824

I did not see a way to easily implement the Posix behavior using the Netty API.  It might be possible to maintain some state concerning sockets that have not delivered all of their data so that a socket close() could internally delay calling close on the Netty channel until the data was fully delivered.
History
Date User Action Args
2016-07-05 15:50:45ryan.springersetmessageid: <1467733845.54.0.835093021935.issue2508@psf.upfronthosting.co.za>
2016-07-05 15:50:45ryan.springersetrecipients: + ryan.springer
2016-07-05 15:50:45ryan.springerlinkissue2508 messages
2016-07-05 15:50:43ryan.springercreate