Title: urllib2 POST > 64k fails (RESOLVED)
Type: crash Severity: normal
Components: Library Versions: Jython 2.7.2
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: gbach
Priority: Keywords:

Created on 2020-05-26.09:20:17 by gbach, last changed 2020-08-19.08:49:35 by gbach.

File name Uploaded Description Edit Remove gbach, 2020-05-26.09:20:17 gbach, 2020-08-17.18:08:14
msg13064 (view) Author: Gunter (gbach) Date: 2020-05-26.09:20:17
When urllib2-POSTing data > 64k the request times out and returns:

httplib.BadStatusLine: ''

<= 64k works fine.

Tested in newest (2.7.2 and 2.7.2b3, 2.7.2b2).
msg13066 (view) Author: Gunter (gbach) Date: 2020-05-26.09:21:25
To test modify line 5 to make the request bigger than 64k
msg13118 (view) Author: Gunter (gbach) Date: 2020-08-17.16:11:40
Sending mails with seems to suffer the same issue.
When sending an message > 64k (e.g. adding an attachment) the system stops for 5 minutes and then reports:
smtplib.SMTPServerDisconnected: Server not connected
msg13120 (view) Author: Gunter (gbach) Date: 2020-08-17.18:08:14
I managed to create a workaround:
removing sock.sendall in and replacing it by sending out 0xffff-sized chunks.
Implementation is inefficient but works...
msg13122 (view) Author: Gunter (gbach) Date: 2020-08-17.18:09:08
...workaround is only for sending mails of course!
msg13124 (view) Author: Gunter (gbach) Date: 2020-08-19.08:49:27
Inspired by my own workaround ;-)
Found the issue in

-    sendall = send   # FIXME see note above!

replace with:

    def sendall(self, data, flags=0):
        chunk_size = 8192
        length = len(data)
        data_view = memoryview(data)
        idx = 0
        while idx < length:
            bytes_sent = self.send(data_view[idx:idx + chunk_size], flags=flags)
            idx += bytes_sent

then all is working.

(Sorry for not making a proper Whatever-Git-Pull-Request/Hg-Thingy...)
Date User Action Args
2020-08-19 08:49:35gbachsettitle: urllib2 POST > 64k fails -> urllib2 POST > 64k fails (RESOLVED)
2020-08-19 08:49:27gbachsetmessages: + msg13124
severity: major -> normal
2020-08-17 18:09:08gbachsetmessages: + msg13122
2020-08-17 18:08:15gbachsetfiles: +
messages: + msg13120
severity: urgent -> major
2020-08-17 16:11:40gbachsetmessages: + msg13118
severity: major -> urgent
2020-06-24 14:24:43gbachsetcomponents: + Library
2020-05-26 09:21:25gbachsetmessages: + msg13066
2020-05-26 09:20:17gbachcreate