urllib2 content-type headers

[TYR]

I have a little application that wants to send data to a Google API.
This API requires an HTTP header to be set as follows:

Authorization: GoogleLogin auth=[value of auth token goes here]

Unfortunately, I'm getting nothing but 400 Bad Requests. I suspect
this is due to an unfeature of urllib2. Notably, although you can use
urllib2.Request's add_header method to append a header, the
documentation (http://docs.python.org/library/urllib2.html) says that:

remember that a few standard headers (Content-Length, Content-Type and
Host) are added when the Request is passed to urlopen() (or
OpenerDirector.open()).

And:

Note that there cannot be more than one header with the same name, and
later calls will overwrite previous calls in case the key collides.

To put it another way, you cannot rely on Content-Type being correct
because whatever you set it to explicitly, urllib2 will silently
change it to something else which may be wrong, and there is no way to
stop it. What happened to "explicit is better than implicit"?

 

[Philip Semanchuk]

I have a little application that wants to send data to a Google API.
This API requires an HTTP header to be set as follows:

Authorization: GoogleLogin auth=[value of auth token goes here]

Unfortunately, I'm getting nothing but 400 Bad Requests. I suspect
this is due to an unfeature of urllib2. Notably, although you can use
urllib2.Request's add_header method to append a header, the
documentation (http://docs.python.org/library/urllib2.html) says that:

remember that a few standard headers (Content-Length, Content-Type and
Host) are added when the Request is passed to urlopen() (or
OpenerDirector.open()).

And:

Note that there cannot be more than one header with the same name, and
later calls will overwrite previous calls in case the key collides.

To put it another way, you cannot rely on Content-Type being correct
because whatever you set it to explicitly, urllib2 will silently
change it to something else which may be wrong, and there is no way to
stop it. What happened to "explicit is better than implicit"?

Hi TYR,

I'm confused, are you having a problem with the Content-Type or
Authorization headers?

Some suggestions --
- Try sending the request to a server you control so you can see what
it is actually receiving. Maybe urllib2 isn't overwriting your header
at all, but you're sending (e.g.) a misconfigured auth token.

- Use a protocol sniffer like Wireshark to see what's going out over
the wire

- Urrlib2 automates some things for you (like building headers), but
as with all automated magic sometimes it's not what you want. The
httplib module offers finer control over HTTP conversations. You might
want to use httplib to debug this problem even if you find you don't
need it in your final solution.


good luck
Philip

 

[Miles Kaufmann]

Unfortunately, I'm getting nothing but 400 Bad Requests. I suspect
this is due to an unfeature of urllib2. Notably, although you can use
urllib2.Request's add_header method to append a header, the
documentation (http://docs.python.org/library/urllib2.html) says that:

remember that a few standard headers (Content-Length, Content-Type and
Host) are added when the Request is passed to urlopen() (or
OpenerDirector.open()).

And:

Note that there cannot be more than one header with the same name, and
later calls will overwrite previous calls in case the key collides.

To put it another way, you cannot rely on Content-Type being correct
because whatever you set it to explicitly, urllib2 will silently
change it to something else which may be wrong, and there is no way to
stop it. What happened to "explicit is better than implicit"?

Those headers are added (by AbstractHTTPHandler.do_request_) only if
they are missing.

-Miles