Cannot connect to IMAP server in Python 3.2

[Steven D'Aprano]

I can connect to an IMAP server using Python 2.6:

steve@runes:~$ python2.6
Python 2.6.6 (r266:84292, Dec 27 2010, 00:02:40)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.<imaplib.IMAP4_SSL instance at 0xb7183c4c>


But when I try with Python 3.2, it just sits there until it times out:

steve@runes:~$ python3.2
Python 3.2.2 (default, Feb 29 2012, 18:11:33)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.Traceback (most recent call last):
[...]
socket.timeout: timed out


What am I doing wrong?

 

[Steve Howell]

I can connect to an IMAP server using Python 2.6:

steve@runes:~$ python2.6
Python 2.6.6 (r266:84292, Dec 27 2010, 00:02:40)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.>>>import imaplib
<imaplib.IMAP4_SSL instance at 0xb7183c4c>

But when I try with Python 3.2, it just sits there until it times out:

steve@runes:~$ python3.2
Python 3.2.2 (default, Feb 29 2012, 18:11:33)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.>>>import imaplib
Traceback (most recent call last):
[...]
socket.timeout: timed out

What am I doing wrong?

Is it simply the wrong port? (IMAP4_SSL_PORT vs. IMAP4_PORT)

How long do you wait before seeing the timeout?

Have you tried print-debugging within your local copy of imaplib.py?
The code related to making the connection just wraps
socket.create_connection:


234
235 def _create_socket(self):
236 return socket.create_connection((self.host, self.port))
237
238 def open(self, host = '', port = IMAP4_PORT):
239 """Setup connection to remote server on "hostort"
240 (default: localhost:standard IMAP4 port).
241 This connection will be used by the routines:
242 read, readline, send, shutdown.
243 """
244 self.host = host
245 self.port = port
246 self.sock = self._create_socket()
247 self.file = self.sock.makefile('rb')


http://hg.python.org/cpython/file/3.2/Lib/imaplib.py

 

[Steven D'Aprano]

I can connect to an IMAP server using Python 2.6:

steve@runes:~$ python2.6
Python 2.6.6 (r266:84292, Dec 27 2010, 00:02:40) [GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more
information.>>> import imaplib

server = imaplib.IMAP4_SSL('xxxxx')
print server

<imaplib.IMAP4_SSL instance at 0xb7183c4c>

But when I try with Python 3.2, it just sits there until it times out:

steve@runes:~$ python3.2
Python 3.2.2 (default, Feb 29 2012, 18:11:33) [GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more
information.>>> import imaplib

server = imaplib.IMAP4('xxxxx', imaplib.IMAP4_SSL_PORT)

Traceback (most recent call last):
[...]
socket.timeout: timed out

What am I doing wrong?

Is it simply the wrong port? (IMAP4_SSL_PORT vs. IMAP4_PORT)

No, it is the correct port, 993.

How long do you wait before seeing the timeout?
Indefinitely.


Have you tried print-debugging within your local copy of imaplib.py? The
code related to making the connection just wraps
socket.create_connection:

I'm not going to start debugging the standard library until after I'm
satisfied that I'm not doing something wrong.

 

[Steve Howell]

I can connect to an IMAP server using Python 2.6:
steve@runes:~$ python2.6
Python 2.6.6 (r266:84292, Dec 27 2010, 00:02:40) [GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more
information.>>> import imaplib
server = imaplib.IMAP4_SSL('xxxxx')
print server
<imaplib.IMAP4_SSL instance at 0xb7183c4c>
But when I try with Python 3.2, it just sits there until it times out:
steve@runes:~$ python3.2
Python 3.2.2 (default, Feb 29 2012, 18:11:33) [GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more
information.>>> import imaplib
server = imaplib.IMAP4('xxxxx', imaplib.IMAP4_SSL_PORT)
Traceback (most recent call last):
[...]
socket.timeout: timed out
What am I doing wrong?

Is it simply the wrong port?  (IMAP4_SSL_PORT vs. IMAP4_PORT)

No, it is the correct port, 993.

How long do you wait before seeing the timeout?
Indefinitely.

Have you tried print-debugging within your local copy of imaplib.py? The
code related to making the connection just wraps
socket.create_connection:

I'm not going to start debugging the standard library until after I'm
satisfied that I'm not doing something wrong.

I'm not suggesting that you debug the standard library because I think
the library itself is broken. I'm suggesting that using the source
code that's freely available to you can help you have some insight on
what's going wrong.

Do you have a working theory on what you're doing wrong? You've
already ruled out the port. Why are you changing the invocation
between versions of Python?

imaplib.IMAP4_SSL('xxxxx') # 2.6
imaplib.IMAP4('xxxxx', imaplib.IMAP4_SSL_PORT) # 3.2

I'm sure the standard library works fine, and you're just doing
something silly, like mistyping the host name or forgetting to start
the server. Maybe you're running 3.2 in a slightly different OS
environment?

The ONE thing that you can find out almost immediately is what
self.host and self.post are set to when _create_socket gets called.
You could almost immediately narrow down the problem to
socket.create_connection() instead of IMAP.

 

[Steve Howell]

I can connect to an IMAP server using Python 2.6:
steve@runes:~$ python2.6
Python 2.6.6 (r266:84292, Dec 27 2010, 00:02:40) [GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more
information.>>> import imaplib
server = imaplib.IMAP4_SSL('xxxxx')
print server
<imaplib.IMAP4_SSL instance at 0xb7183c4c>
But when I try with Python 3.2, it just sits there until it times out:
steve@runes:~$ python3.2
Python 3.2.2 (default, Feb 29 2012, 18:11:33) [GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more
information.>>> import imaplib
server = imaplib.IMAP4('xxxxx', imaplib.IMAP4_SSL_PORT)
Traceback (most recent call last):
[...]
socket.timeout: timed out
What am I doing wrong?

Is it simply the wrong port?  (IMAP4_SSL_PORT vs. IMAP4_PORT)

No, it is the correct port, 993.

How long do you wait before seeing the timeout?
Indefinitely.

Have you tried print-debugging within your local copy of imaplib.py? The
code related to making the connection just wraps
socket.create_connection:

I'm not going to start debugging the standard library until after I'm
satisfied that I'm not doing something wrong.

What happens when you do this?

 

[Steven D'Aprano]

I'm not suggesting that you debug the standard library because I think
the library itself is broken. I'm suggesting that using the source code
that's freely available to you can help you have some insight on what's
going wrong.

Do you have a working theory on what you're doing wrong?

Actually, between you and me and the thousands of others reading this
thread, I actually do suspect it is a bug in the std library
implementation. But the suspicion is only that, and my first *assumption*
is that it is more likely I am doing something wrong. Hence my question.

You've already
ruled out the port. Why are you changing the invocation between
versions of Python?

Because imaplib.IMAP4_SSL apparently no longer exists in Python 3.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'IMAP4_SSL'

imaplib.IMAP4_SSL('xxxxx') # 2.6
imaplib.IMAP4('xxxxx', imaplib.IMAP4_SSL_PORT) # 3.2

I'm sure the standard library works fine, and you're just doing
something silly, like mistyping the host name or forgetting to start the
server. Maybe you're running 3.2 in a slightly different OS
environment?

If you look carefully at my copy-and-pasted sessions, you will see that
both of them are on the same host, "runes". While I suppose that
technically I could have two hosts both called runes, in fact I only have
one

And no, I have not mistyped the host name.

 

[Steven D'Aprano]

Because imaplib.IMAP4_SSL apparently no longer exists in Python 3.

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'IMAP4_SSL'

Wait a minute...


IMAP4_SSL is documented as existing in Python 3. And when I run Python
3.2 on a Centos machine, instead of Debian, it includes IMAP4_SSL which
works fine.


[steve@ando ~]$ python3.2
Python 3.2.2 (default, Mar 4 2012, 10:50:33)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-51)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
py> import imaplib
py> server = imaplib.IMAP4_SSL('xxxxx')
py> server
<imaplib.IMAP4_SSL object at 0xb7b8632c>


So there's something screwy going on here. Why does my Python 3.2 on
Debian not include IMAP4_SSL, but Python 2.6 does?


<raises eyebrow>

 

[Damien Wyart]

IMAP4_SSL is documented as existing in Python 3. And when I run Python
3.2 on a Centos machine, instead of Debian, it includes IMAP4_SSL which
works fine.
So there's something screwy going on here. Why does my Python 3.2 on
Debian not include IMAP4_SSL, but Python 2.6 does?

Could you try "import ssl" on your Debian machine and see if some errors
are reported? Some ssl libs used by ssl.py might not be installed.

 

[Steve Howell]

Because imaplib.IMAP4_SSL apparently no longer exists in Python 3.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'IMAP4_SSL'

Wait a minute...

IMAP4_SSL is documented as existing in Python 3. And when I run Python
3.2 on a Centos machine, instead of Debian, it includes IMAP4_SSL which
works fine.

[steve@ando ~]$ python3.2
Python 3.2.2 (default, Mar  4 2012, 10:50:33)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-51)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
py> import imaplib
py> server = imaplib.IMAP4_SSL('xxxxx')
py> server
<imaplib.IMAP4_SSL object at 0xb7b8632c>

So there's something screwy going on here. Why does my Python 3.2 on
Debian not include IMAP4_SSL, but Python 2.6 does?

Looking at the source can give you some insight:

http://hg.python.org/cpython/file/3.2/Lib/imaplib.py

Look for where IMAP4_SSL is defined. On your 3.2/Debian setup, the
HAVE_SSL flag (see two lines above the "class" statement) is
apparently false:

1174 if HAVE_SSL:
1175
1176 class IMAP4_SSL(IMAP4):

If you look for the first mention of HAVE_SSL, it becomes apparent
that you have some issue with importing the ssl module, which
unfortunately gets buried fairly silently by imaplib:

27 try:
28 import ssl
29 HAVE_SSL = True
30 except ImportError:
31 HAVE_SSL = False

Damien, in his response, asks you to try "import ssl" on your Debian
machine. I think he's on the right track in identifying the problem,
based on the simple code above.

Once you are able to import ssl, you should be able to use IMAP4_SSL,
but that still doesn't entirely explain to me why you got a timeout
error with plain IMAP4 and the proper port. (I would have expected a
failure, but of a different kind.)

I'd still be curious to see what happens when you try this:

import socket, imaplib
your_host_name = # ...
socket.create_connection((your_host_name, imaplib.IMAP4_SSL_PORT))

 

[Keith Medcalf]

Once you are able to import ssl, you should be able to use IMAP4_SSL,
but that still doesn't entirely explain to me why you got a timeout
error with plain IMAP4 and the proper port. (I would have expected a
failure, but of a different kind.)

Connecting to the SSL socket requires that one initiate the TLS handshake
forthwith. Establishing a connection to an "I expect SSL from the get-
go" using a protocol that speaks "I am a normal unencrypted socket but
you can initiate TLS using the starttls command" is not the same thing.

In other words, you are violating the requirements of the protocol, and
you are timing out. This is because the first protocol step in a
standard connection is to wait for the plain-text greeting, where the
first step in connecting to the SSL socket is to do a TLS dance, then
initiate the IMAP protocol by sending the greeting.

If you connect with a non-SSL initiator to an SSL endpoint, you will get
a timeout. If you connect with an SSL initiator to a non-SSL endpoint,
you will timeout. It is not the connection that is timing out, it is the
protocol.

I'd still be curious to see what happens when you try this:

import socket, imaplib
your_host_name = # ...
socket.create_connection((your_host_name, imaplib.IMAP4_SSL_PORT))

This will, of course, work just fine. You will not see a +Hello however
until you have completed the TLS negotiation.