urllib2 test fails (2.7, linux)

[guandalino]

Hi, running Python 2.7 test suite for urllib2 there is a test that
doesn't pass.
Do you have an idea about where the problem could be and how to solve
it?

Thanks,
best regards.

$ # ubuntu 8.04
$ pwd
~/sandbox/2.7/lib/python2.7/test
$ python test_urllib2.py
======================================================================
ERROR: test_file (__main__.HandlerTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "test_urllib2.py", line 711, in test_file
h.file_open, Request(url))
File "/home/redt/sandbox/2.7/lib/python2.7/unittest/case.py", line
456, in assertRaises
callableObj(*args, **kwargs)
File "/home/redt/sandbox/2.7/lib/python2.7/urllib2.py", line 1269,
in file_open
return self.open_local_file(req)
File "/home/redt/sandbox/2.7/lib/python2.7/urllib2.py", line 1301,
in open_local_file
(not port and socket.gethostbyname(host) in self.get_names()):
gaierror: [Errno -5] No address associated with hostname


Notes:
$ hostname
speedy

$ cat /etc/hosts
127.0.0.1 localhost localhost.localdomain speedy

::1 localhost speedy ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts

Finally, I don't know if this matters but the tests have been executed
offline
(without an internet connection).

 

[Terry Reedy]

Hi, running Python 2.7 test suite for urllib2 there is a test that
doesn't pass.
Do you have an idea about where the problem could be and how to solve
it?

Thanks,
best regards.

$ # ubuntu 8.04
$ pwd
~/sandbox/2.7/lib/python2.7/test
$ python test_urllib2.py
======================================================================
ERROR: test_file (__main__.HandlerTests)

Look there to find the complete statement. For 3.1, it would be

self.assertRaises(urllib.error.URLError,
h.file_open, Request(url))
(urllib2 is now urllib.request)

You could insert a print to find what url caused a problem.

File "/home/redt/sandbox/2.7/lib/python2.7/unittest/case.py", line

This puzzles me. In 3.1, unittest is a module, unittest.py, not a
package containing modules like 'case.py'. I though it was the same for 2.7

456, in assertRaises
callableObj(*args, **kwargs)

This is in unittest.py. It says that this test case *should* fail, but
with a different error (urllib.error.URLError) than the one you got
(gaierror).

File "/home/redt/sandbox/2.7/lib/python2.7/urllib2.py", line 1269,
in file_open
return self.open_local_file(req)
File "/home/redt/sandbox/2.7/lib/python2.7/urllib2.py", line 1301,
in open_local_file
(not port and socket.gethostbyname(host) in self.get_names()):
gaierror: [Errno -5] No address associated with hostname

gaierror comes from socket.gethostbyname

Finally, I don't know if this matters but the tests have been executed
offline (without an internet connection).

Since error is in open_local_file, I would think not.

 

[guandalino]

Hi Terry, thanks for your reply.

======================================================================
ERROR: test_file (__main__.HandlerTests)

[cut]
You could insert a print to find what url caused a problem.

Output:
file://localhost:80/home/redt/sandbox/2.7/lib/python2.7/test/%40test_29416_tmp
file:///file_does_not_exist.txt
file://127.0.0.1:80/home/home/redt/sandbox/2.7/lib/python2.7/test/@test_29416_tmp
file://somerandomhost.ontheinternet.com/home/home/redt/sandbox/2.7/lib/python2.7/test/@test_29416_tmp

Offending line is the the 4th.

This is in unittest.py. It says that this test case *should* fail, but
with a different error (urllib.error.URLError) than the one you got
(gaierror).

   File "/home/redt/sandbox/2.7/lib/python2.7/urllib2.py", line 1269,
in file_open
     return self.open_local_file(req)
   File "/home/redt/sandbox/2.7/lib/python2.7/urllib2.py", line 1301,
in open_local_file
     (not port and socket.gethostbyname(host) in self.get_names()):
gaierror: [Errno -5] No address associated with hostname

gaierror comes from socket.gethostbyname

When I print the value of the variable 'host' in urllib2.py line 1301
I get this: somerandomhost.ontheinternet.com. This is why
socket.gethostbyname(host) raises gaierror -5, there is no address
associated to somerandomhost.ontheinternet.com. Instead the values
that 'host' takes for the other urls are localhost or 127.0.0.1, both
valid for gethostbyname().

Any hint?

Thanks.

 

[Terry Reedy]

Hi Terry, thanks for your reply.

======================================================================
ERROR: test_file (__main__.HandlerTests)

[cut]
You could insert a print to find what url caused a problem.

Output:
file://localhost:80/home/redt/sandbox/2.7/lib/python2.7/test/%40test_29416_tmp
file:///file_does_not_exist.txt
file://127.0.0.1:80/home/home/redt/sandbox/2.7/lib/python2.7/test/@test_29416_tmp
file://somerandomhost.ontheinternet.com/home/home/redt/sandbox/2.7/lib/python2.7/test/@test_29416_tmp
Offending line is the the 4th.

Ah. I did not see anything like that in 3.1 test_urllib2. Either I just
simply missed it, or it is obscured, or it was removed because it causes
failures;-).

This is in unittest.py. It says that this test case *should* fail, but
with a different error (urllib.error.URLError) than the one you got
(gaierror).

File "/home/redt/sandbox/2.7/lib/python2.7/urllib2.py", line 1269,
in file_open
return self.open_local_file(req)
File "/home/redt/sandbox/2.7/lib/python2.7/urllib2.py", line 1301,
in open_local_file
(not port and socket.gethostbyname(host) in self.get_names()):
gaierror: [Errno -5] No address associated with hostname

gaierror comes from socket.gethostbyname

When I print the value of the variable 'host' in urllib2.py line 1301
I get this: somerandomhost.ontheinternet.com. This is why
socket.gethostbyname(host) raises gaierror -5, there is no address
associated to somerandomhost.ontheinternet.com. Instead the values
that 'host' takes for the other urls are localhost or 127.0.0.1, both
valid for gethostbyname().

Any hint?

Remove the offending fake url.

If you want to investigate the history of the file, go to
http://svn.python.org/view/python/branches/release27-maint/Lib/test/test_urllib2.py?view=log

If you do not get an answer here, file a bug report. If you can, put
orsenthil,benjamin.peterson,ezio.melotti
on the nosy list, as they are recent committers to this file. Say I said
to do so if you want. Add tjreedy also so I can see responses and learn too.

 

[guandalino]

Remove the offending fake url.

This makes tests to pass, but I have concrete reasons to think that
urllib2 is really broken on my system. Btw, a(nother) weird thing is
that I wrote a standalone test file based on test_urllib2.py and the
test that there failed now passes.

import unittest, urllib2

class Test(unittest.TestCase):
def runTest(self):
h = urllib2.FileHandler()
urls = (
"file://localhost:80/home/redt/sandbox/2.7/lib/python2.7/
test/%40test_29416_tmp",
"file:///file_does_not_exist.txt",
"file://127.0.0.1:80/home/home/redt/sandbox/2.7/lib/
python2.7/test/@test_29416_tmp",
"file://somerandomhost.ontheinternet.com/home/home/redt/
sandbox/2.7/lib/python2.7/test/@test_29416_tmp",)

for url in urls:
self.assertRaises(urllib2.URLError, h.file_open,
urllib2.Request(url))
print "PASSED"

PASSED
PASSED
PASSED
PASSED <-- this doesn't pass in test_urllib2 because gaierror is
raised instead of URLError.

I'm going to file an issue on python's bug tracker. Just the last
thing to be sure it's not my fault... My system already had Python
installed system wide (2.5.2) and I did the new installation in my
home directory. To "activate" the new interpreter I added /home/redt/
sandbox/2.7/bin as first element in PATH. PYTHONPATH is empty. python -
V gives 2.7.

Is it OK or new and old pythons step on each other's foot?

Thanks again,
best regards.

 

[guandalino]

If you do not get an answer here, file a bug report. If you can, put
orsenthil,benjamin.peterson,ezio.melotti
on the nosy list, as they are recent committers to this file. Say I said
to do so if you want. Add tjreedy also so I can see responses and learn too.

Done here: http://bugs.python.org/issue9446.

Best regards, HAND.