Issue2021

Hi

After changing to jython2.7beta1 from Jython2.5.3 I am no longer able to read content of webpages using SSL, http and using java "trusting all certificates". The response from the https-page is always an empty string, resulting in httplib.BadStatusLine exception from httplib.py in Jython.

I have described the problem on stack overflow: http://stackoverflow.com/questions/15133691/https-with-jython2-7-trusting-all-certificates-does-not-work-result-httplib

I think this is a duplicate of #2016

ssl sockets have broken recv() and makefile()
http://bugs.jython.org/issue2016

The recv() part of that issue is already fixed.

httplib switched to using makefile() for HTTPSConnections in python 2.7, which our SSL sockets do not yet support: I am working on a fix.

It is not related to the "trust all certificates" thing, because if certificates were an issue, you would have gotten a certificate verification exception.

I think I'm going to close this issue as a duplicate of #2016.

Please update your stackoverflow post with this information: I'm busy working on the fix :-)

I've checked i a temporary fix for the lack of makefile here

http://hg.python.org/jython/rev/baf84d8e91d0

Which should make httplib.HTTPSConnection start working.

This is just a temporary fix, so I have not added tests yet.

I have a more permanent fix in the works, which will be extensively tested.

Please retest with the latest code.

I will leave this bug open for a few days for fix verification.

Hi Alan, thank you for your reply and fix. I have tested my code with your latest commit and the HTTPSConnection does indeed work!

But if I change to use urllib2 with HTTPBasicAuthHandler to request my https-url, I still receive socket problems. 

My urrlib2 test works fine with jython2.5.3, does not work at all with 2.7b1 (i.e. same result as prior of your commit), and with your latest commit (baf84d8e91d0) I get the following problem:

[...]
File "jytest.py", line 59, in urllib2test
    data = urllib2.urlopen(URL).read()
[...]
File "/Users/erikiveroth/Workspace/sandbox/jython/dist/Lib/socket.py", line 1669, in read
    data = self._sock.recv(recv_size)
File "/Users/erikiveroth/Workspace/sandbox/jython/dist/Lib/socket.py", line 174, in handle_exception
    raise _map_exception(jlx)
socket.error: [Errno 32] Socket closed

Full stack-trace and testcode: http://247bc66123446939.paste.se/

urllib2.urlopen("https://blah/").read() is working for me

Jython 2.7b1+ (default:e80a189574d0+, Feb 28 2013, 22:54:17)
[Java HotSpot(TM) Client VM (Sun Microsystems Inc.)] on java1.6.0_41
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib2
>>> f = urllib2.urlopen('https://httpbin.org/ip')
>>> f.read()
'{\n  "origin": "10.120.118.223"\n}'

I note that you re using basic authorization. On a private server, so I can get a look at that the server is returning.

I suspect that the server is returning a www-authenticate header without quoting the realm name.

Please can you paste the output of this command: I'm specifically interested in the response headers.

curl -v -u admin:test https://192.168.1.101:8888/rest/system

Hi Alan, sorry for the delayed response.

It looks like my problem is with the Java SSLContext, which I use to accept all certificates. 
I have removed the basic authorization and simplified the code, still receiving same problem: http://49c386d0057a547c.paste.se/

And it is probably due to the response from the server, since my tests works with url 'https://httpbin.org/ip'.
But why does it work with earlier version of jython?


Response header looks ok to me (some parts masked): http://d795a35132c048bb.paste.se/

OK, I see you're Brandon Pedersens recipe for trusting all certificates.

http://tech.pedersen-live.com/2010/10/trusting-all-certificates-in-jython/

But you've commented out the key line from his context manager declaration

SSLContext.setDefault(TRUST_ALL_CONTEXT)

So the SSLContexts you're creating do not inherit your trust manager.

Ah, no. I accidentally pasted the code I used on my first run. I executed my code without SSLContext.setDefault(TRUST_ALL_CONTEXT) (Lines 47-70) and with SSLContext.setDefault(TRUST_ALL_CONTEXT) (Lines 73-104) to see the difference.

Sorry for the confusion.

> "Response header looks ok to me (some parts masked): http://d795a35132c048bb.paste.se/"

OK, line 29 of that output looks interesting.

> 29: < HTTP/1.1 302 Found

Is your urlopener setup correctly to handle redirects?

(It's quite frustrating having to debug something that's behind a firewall that I can't access and cannot run code against. Is there anyway you can get this running somewhere I can access it and really see what's going on?)

I have experience that this sort of trust all certs works well on Jython 2.7. Here's another example of how to set this up:
https://github.com/rackerlabs/romper/blob/master/romper/trust.py