Issue2016

classification
Title: ssl sockets have broken recv() and makefile()
Type: behaviour Severity: normal
Components: Library Versions: Jython 2.7
Milestone:
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: amak Nosy List: amak, fwierzbicki, t-8ch, zyasoft
Priority: normal Keywords:

Created on 2013-02-20.18:46:39 by t-8ch, last changed 2014-05-10.08:48:20 by t-8ch.

Messages
msg7726 (view) Author: Thomas Weißschuh (t-8ch) Date: 2013-02-20.18:46:39
When creating a ssl socket with ssl.socket(some_sock) the returned object is missing proper recv() and makefile() methods.

If one uses those methods they are taken form the underlying socket.
This leads to unencrypted data being sent over the underlying socket which will be rejected by the remote host with a ssl error.
The data of this error is then returned on rcv() to the caller.

Testcases:

recv():

import socket
import ssl
import time

sock = socket.create_connection(('httpbin.org', 443))
ssl_sock = ssl.wrap_socket(sock)
ssl_sock.write(b"GET /ip HTTP/1.1\r\nHost:httpbin.org\r\n\r\n")
while True:
    time.sleep(1)
    print(ssl_sock.recv(10000))

makefile():

import httplib
conn = httplib.HTTPSConnection("httpbin.org")
conn.request("GET", "/ip")
r = conn.getresponse()

print(r.read())


The issue with recv() can be fixed by aliasing it to read()

recv = read
msg7731 (view) Author: Alan Kennedy (amak) Date: 2013-02-23.15:21:39
Hmm, there seems to have been an implicit API change between cpython 2.5 socket.ssl and cpython 2.7 ssl module.

Cpython 2.5 ssl sockets, as returned from socket.ssl, do not have recv() (et al) methods.

C:\>C:\Python25\python.exe
Python 2.5.4 (r254:67916, Dec 23 2008, 15:10:54) [MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> import time
>>> sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>>> sock.connect( ('httpbin.org', 443) )
>>> ssl_sock = socket.ssl(sock)
>>> ssl_sock.write("GET /ip HTTP/1.1\r\nHost:httpbin.org\r\n\r\n")
38
>>> print(ssl_sock.recv(10000))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: recv
>>>

But SSLSockets, as returned from ssl.wrap_socket do have the method

X:\xhaus\jython\jython_25\dist>C:\Python27\python.exe
Python 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>>> import socket
>>> import time
>>> sock = socket.create_connection(('httpbin.org', 443))
>>> ssl_sock = ssl.wrap_socket(sock)
>>> ssl_sock.write("GET /ip HTTP/1.1\r\nHost:httpbin.org\r\n\r\n")
38
>>> print(ssl_sock.recv(10000))
HTTP/1.1 200 OK
>>>

Even though the methods are undocumented, as can be seen on the module documentation page, where only the read and write methods are documented, not recv() et al.

http://docs.python.org/2.7/library/ssl.html#sslsocket-objects

I think I'm going to have to seek clarity from the cpython folks on this one. Either

A: The methods should be supported, in which case they should be documented or
B: The methods should not be supported

It is indeed the case that the current implementation delegates the recv() et al methods to the underlying socket, and should not, because they operate on the unwrapped/unencrypted socket. I will fix this straightaway. The immediate fix will be to not support the methods for now, until the cpython question of undocumented methods is resolved.
msg7732 (view) Author: Alan Kennedy (amak) Date: 2013-02-23.15:29:36
"""
It is indeed the case that the current implementation delegates the recv() et al methods to the underlying socket, and should not, because they operate on the unwrapped/unencrypted socket. I will fix this straightaway. The immediate fix will be to not support the methods for now, until the cpython question of undocumented methods is resolved.
"""

It is indeed the case that the current *jython 2.7* implementation ....
msg7733 (view) Author: Thomas Weißschuh (t-8ch) Date: 2013-02-23.17:35:57
According to the documentation of ssl.wrap_socket() SSLSockets are a subtype of socket.socket which defines rcv() and send().
msg7734 (view) Author: Alan Kennedy (amak) Date: 2013-02-23.18:27:01
> "According to the documentation of ssl.wrap_socket() SSLSockets are a subtype of socket.socket which defines rcv() and send()."

I see that now: The documentation says

"""
ssl.wrap_socket(sock, keyfile=None, certfile=None, server_side=False, cert_reqs=CERT_NONE, ssl_version={see docs}, ca_certs=None, do_handshake_on_connect=True, suppress_ragged_eofs=True, ciphers=None)

    Takes an instance sock of socket.socket, and returns an instance of ssl.SSLSocket, a subtype of socket.socket, which wraps the underlying socket in an SSL context
"""

The cpython 2.7 httplib also makes use of the send() and sendall() methods for transmitting data over SSLSockets.

But recv() is not used: instead the read() method of the makefile wrapper is used. An asymmetry in its design.

It's also the case that jython 2.5 never supported the makefile() method on ssl sockets. Which never affected httplib in 2.5, because it used a FakeSocket wrapper for ssl sockets.

In summary, the whole ssl support situation, including makefle, needs thorough revision jython 2.7, in the light of cpython 2.7 API and library changes.

Thanks for taking the time to report this issue: a fix will be forthcoming soon.
msg7735 (view) Author: Alan Kennedy (amak) Date: 2013-02-25.09:09:58
I have checked in a temporary fix for recv, send and sendall here.

http://hg.python.org/jython/rev/ae103c4e1aef

This will at least address the problem of unencrypted information being transmitted over the socket.
msg7872 (view) Author: Alan Kennedy (amak) Date: 2013-02-28.23:08:20
I've checked i a temporary fix for the lack of makefile here

http://hg.python.org/jython/rev/baf84d8e91d0

Which should make httplib.HTTPSConnection start working.

This is just a temporary fix, so I have not added tests yet.

I have a more permanent fix in the works, which will be extensively tested.

Leaving this bug open as a reminder of that.
msg8355 (view) Author: Jim Baker (zyasoft) Date: 2014-05-10.04:41:41
Resolved in Jython trunk
msg8369 (view) Author: Jim Baker (zyasoft) Date: 2014-05-10.06:02:59
I'm going to close this out. This functionality is extremely well tested by such packages as urllib3, which is used by requests, which is used by pip.

You will need to use trunk as of 7222:107fe4a4c96b
msg8370 (view) Author: Thomas Weißschuh (t-8ch) Date: 2014-05-10.08:48:20
Thanks!
Now we have come full circle, as I stumbled upon this while working on urllib3 :-)
History
Date User Action Args
2014-05-10 08:48:20t-8chsetmessages: + msg8370
2014-05-10 06:02:59zyasoftsetstatus: pending -> closed
messages: + msg8369
2014-05-10 04:41:42zyasoftsetstatus: open -> pending
resolution: fixed
messages: + msg8355
nosy: + zyasoft
2013-02-28 23:08:20amaksetmessages: + msg7872
2013-02-26 18:40:42fwierzbickisetpriority: normal
nosy: + fwierzbicki
2013-02-25 09:09:59amaksetmessages: + msg7735
2013-02-23 18:27:02amaksetmessages: + msg7734
2013-02-23 17:35:57t-8chsetmessages: + msg7733
2013-02-23 15:29:36amaksetmessages: + msg7732
2013-02-23 15:21:40amaksetassignee: amak
messages: + msg7731
nosy: + amak
2013-02-20 18:46:39t-8chcreate