Issue1290

classification
Title: 600x slower network download speed with urllib.urlretrieve() or urllib2.urlopen() compared to CPython
Type: behaviour Severity: normal
Components: Library Versions: 2.5.1, 2.5b1, 2.5b0
Milestone:
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: amak Nosy List: amak, dregan
Priority: Keywords:

Created on 2009-03-27.23:50:06 by dregan, last changed 2009-03-31.12:47:43 by amak.

Files
File name Uploaded Description Edit Remove
downloadTest dregan, 2009-03-27.23:50:05 script showing problem
Messages
msg4363 (view) Author: dan regan (dregan) Date: 2009-03-27.23:50:04
600x slower network download speed with urllib.urlretrieve() or
urllib2.urlopen() compared to CPython

When I run the attached script using Jython 2.5b0 or Jython 2.5b3, it
performs file downloads 600 times slower than it does when running under
CPython.  I had to write a Java class in order to accomplish the same
thing, because it was so slow in Jython.

I have not tried it with any other versions of Jython.

There may not be as much of a difference on a slow network connection,
but on a fast network connection, it took me about 600 seconds to
download a 16MB file vs 1 second with CPython.  I tried downloading
using HTTP and FTP protocols.

Just thought somebody might be interested.
msg4364 (view) Author: dan regan (dregan) Date: 2009-03-28.00:11:58
This seems to be something new with 2.5.  I tried it with Jython 2.2.1
and a 900K file and it wasn't slow:

mb:~ cephalopod$ jython2.5b3/jython /tmp/downloadTest 
time elapsed for urllib.urlretrieve(): 46.4300000668
Downloading 'http://.../testfile' to '/tmp/testfile'
time elapsed for urllib2.urlopen(): 47.4960000515
http://.../testfile
/tmp/testfile
mb:~ cephalopod$ jython2.2.1/jython /tmp/downloadTest 
time elapsed for urllib.urlretrieve(): 0.18600010871887207
Downloading 'http://.../testfile' to '/tmp/testfile'
time elapsed for urllib2.urlopen(): 0.1659998893737793
http://.../testfile
/tmp/testfile
mb:~ cephalopod$ python /tmp/downloadTest 
time elapsed for urllib.urlretrieve(): 0.161434173584
Downloading 'http://.../testfile' to '/tmp/testfile'
time elapsed for urllib2.urlopen(): 0.102355957031
http://.../testfile
/tmp/testfile
msg4401 (view) Author: Alan Kennedy (amak) Date: 2009-03-31.12:33:54
Just a quick note before I check in the fix: the code that you posted
results from is obviously different to the code that you attached to the
bug. 

The reported results show times with float values, but the uploaded code
prints times as integers.
msg4402 (view) Author: Alan Kennedy (amak) Date: 2009-03-31.12:47:42
This was caused by a wrong calculation of read buffer sizes; this change
originated in cpython here

http://svn.python.org/view?view=rev&revision=61008

and made it into jython here

http://fisheye3.atlassian.com/changelog/jython/?cs=4791

Fix checked into trunk to r6132 and release_22 at r6133.

Dan, if you don't want to wait for a new release, you can fix your local
socket.py module by changing the min() function at line 1269 to max().
History
Date User Action Args
2009-03-31 12:47:43amaksetstatus: open -> closed
resolution: fixed
messages: + msg4402
2009-03-31 12:33:55amaksetassignee: amak
messages: + msg4401
nosy: + amak
2009-03-28 00:11:58dregansetmessages: + msg4364
2009-03-27 23:50:06dregancreate