SSL (HTTPS) with 2.4

[pyguy2]

The code I have wriitten that does auth ssl through a proxy with python
_works_ with a proxy I tested with earlier. I finally got a squid proxy
server running in which it does not work. As I get time, during the
next few days, I am going to try to debug the difference and let you
know what's up.


And I agree, getting a fix included in the dist would be a good idea.

john

 

[Bloke]

Andrew and John,

Any chance of you sending me your code to have a look at?

Rob

 

[pyguy2]

After failed attempts at trying to get my code to work with squid.

I did some research into this and came up with some info.

http://www.python.org/peps/pep-0320.txt

"- It would be nice if the built-in SSL socket type
could be used for non-blocking SSL I/O. Currently
packages such as Twisted which implement async
servers using SSL have to require third-party packages
such as pyopenssl. "

My guess is that the squid proxy server uses
non-blocking sockets which python ssl does not
support.

And, of course after looking at the squid site, I found this:

"Unlike traditional caching software, Squid handles
all requests in a single, non-blocking, I/O-driven
process."

Now, I haven't had time to verify this. But, it can explain why the
non-ssl proxy authentication works and the ssl partially works. And,
also why I get success with a different type of proxy server.

For a clue as to why there is this problem I would also recommend
looking at http://www.openssl.org/support/faq.html, specifically the
section on non-blocking i/o.

It looks like pyopenssl would be an option:
http://pyopenssl.sourceforge.net/

It's docs comment that it was written because m2crypto error handeling
was not finished for non-blocking i/o:

http://pyopenssl.sourceforge.net/pyOpenSSL.txt

The reason this module exists at all is that the SSL support in the
socket module in the Python 2.1 distribution (which is what we used,
of course I cannot speak for later versions) is severely limited.

When asking about SSL on the comp.lang.python newsgroup (or on
(e-mail address removed)) people usually pointed you to the M2Crypto
package. The M2Crypto.SSL module does implement a lot of OpenSSL's
functionality but unfortunately its error handling system does not
seem to be finished, especially for non-blocking I/O. I think that
much of the reason for this is that M2Crypto^1 is developed using
SWIG^2. This makes it awkward to create functions that e.g. can
return
both an integer and NULL since (as far as I know) you basically
write
C functions and SWIG makes wrapper functions that parses the Python
argument list and calls your C function, and finally transforms your
return value to a Python object.

john

 

[Bloke]

OK.

I try pyopenssl and can get a secure socket to the server, but am
unsure how to use this socket with urllib2 or even httplib.

Here's the code I'm using:

import sys, socket, string, base64, httplib
from OpenSSL import SSL


# Connects to the server, through the proxy
def run(server, proxy):
user='me';passwd='pass'
#setup basic authentication
if user and passwd:
user_pass=base64.encodestring(user+':'+passwd)
proxy_authorization='Proxy-authorization: Basic
'+user_pass+'\r\n'
else:
proxy_authorization=''

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
s.connect(proxy)
print 'Socket established'
except socket.error, e:
print "Unable to connect to %s:%s %s" % (proxy[0], proxy[1],
str(e))
sys.exit(-1)

# Use the CONNECT method to get a connection to the actual server
connectMessage = "CONNECT %s:%s HTTP/1.0\r\n" % (server[0],
server[1]) + \
proxy_authorization #+ 'Proxy-Connection:
Keep-Alive\r\n'
print connectMessage
s.send(connectMessage)
print '\nConnect sent...'
print "Proxy response: %s" % string.strip(s.recv(1024))

ctx = SSL.Context(SSL.SSLv2_METHOD)
conn = SSL.Connection(ctx, s)

# Go to client mode
conn.set_connect_state()

# start using HTTP

conn.send("HEAD / HTTP/1.0\n\n")
print "Server response:"
print "-" * 40
while 1:
try:
buff = conn.recv(4096)
except SSL.ZeroReturnError:
# we're done
break

print buff,

#initalize httplib and replace with your socket
sock = httplib.FakeSocket(s, conn)
print 'Fake socket installed'
h=httplib.HTTPSConnection(server[0],server[1])
h.sock=sock
print 'Sock installed'
h.request('GET','/')
print 'Request sent.'
r=h.getresponse()
print r.read()

if __name__ == '__main__':
server = ('www.anz.com', 443)
proxy = ('proxy.company.com, 8008)
run(server, proxy)

I get the following response at line
59 r=h.getresponse()

Socket established
CONNECT www.anz.com:443 HTTP/1.0

Proxy-authorization: Basic cmhhbGw6YWxlbW0y




Connect sent...
Proxy response: HTTP/1.0 200 Connection established
conn established
conn connect state set
Server response:
----------------------------------------
HTTP/1.1 200 OK

Server: Microsoft-IIS/4.0

Date: Thu, 26 May 2005 09:33:26 GMT

Content-Type: text /html

Set-Cookie: ASPSESSIONIDCRADCCBB=JPGLOCLDMMFNKJKCMIBADHOH; path=/

Cache-control: private



Fake socket installed
Sock installed
Request sent.
Traceback (most recent call last):
File "C:\Documents and
Settings\rhall\Desktop\software\python\tunnel\proxy-openssl.py", line
65, in ?
run(server, proxy)
File "C:\Documents and
Settings\rhall\Desktop\software\python\tunnel\proxy-openssl.py", line
59, in run
r=h.getresponse()
File "C:\Python24\Lib\httplib.py", line 862, in getresponse
response.begin()
File "C:\Python24\Lib\httplib.py", line 333, in begin
version, status, reason = self._read_status()
File "C:\Python24\Lib\httplib.py", line 291, in _read_status
line = self.fp.readline()
File "C:\Python24\Lib\httplib.py", line 981, in readline
s = self._read()
File "C:\Python24\Lib\httplib.py", line 937, in _read
buf = self._ssl.read(self._bufsize)
ZeroReturnError

I tried enabling 'Proxy-Connection: Keep-Alive' but then it hangs for
ages at:
conn.send("HEAD / HTTP/1.0\n\n")
and eventually returns a 'handshaking' error.

Any pointers anyone?

Rob