ClientCookie problem - Difference between 'post' on the local networkand the internet

M

Max M

I am using ClientCookie for login on to servers and browsing them as
authenticated users.

I kept getting "HTTP Error 400: Bad Request" errors when submitting my
forms. So I boiled it down to a simple example.

When I try to use ClientCookie.urlopen() on my private network with ip
numbers like "http://localhost:8081/test_site/logged_in", it works fine.

If I try to call the same site through the Internet, with a url like:
http://www.mxm.dk/test_site/logged_in I get the 400 message. But only
when using "post". 'get' works fine :-s

On the Internet I go out through a gateway, so my external IP is
different than my internal.

I assume that it causes an url mismatch somewhere. With my local ip
being set in some header by urlopener, and the server expecting the
remote ip.

But I have no idea as to how I can avoid it? Any takers?


#########################################

import ClientCookie, urllib

req = ClientCookie.Request
urlopen = ClientCookie.urlopen

def login(came_from, login_form):
data = {
'came_from' : came_from,
'__ac_name' : name,
'__ac_password' : password,
'__ac_persistent' : '1',
'submit' : 'Log ind',
}
encoded_data = urllib.urlencode(data)

request = req(login_form, encoded_data)
response = urlopen(request)
content = response.read()
return content

# This works
came_from = 'http://localhost:8081/test_site/request_tester'
login_form = "http://localhost:8081/test_site/logged_in"
print login(came_from, login_form)

# This results in "HTTP Error 400: Bad Request"
came_from = 'http://www.mxm.dk/test_site/request_tester'
login_form = "http://www.mxm.dk/test_site/logged_in"
print login(came_from, login_form)



############################################
# traceback from the last call to login()

Traceback (most recent call last):
File "E:\zopes\test_site\PRODUCTS\MXMPRO~1\z.py", line 35, in ?
print login(came_from, login_form)
File "E:\zopes\test_site\PRODUCTS\MXMPRO~1\z.py", line 22, in login
response = urlopen(request)
File
"G:\uninstalled\Python\ClientCookie-1.0.0a\ClientCookie\_urllib2_support.py",
line 715, in urlopen
return _opener.open(url, data)
File
"G:\uninstalled\Python\ClientCookie-1.0.0a\ClientCookie\_Opener.py",
line 125, in open
response = meth(req, response)
File
"G:\uninstalled\Python\ClientCookie-1.0.0a\ClientCookie\_urllib2_support.py",
line 387, in http_response
response = self.parent.error(
File
"G:\uninstalled\Python\ClientCookie-1.0.0a\ClientCookie\_Opener.py",
line 189, in error
return apply(self._call_chain, args)
File "C:\pytyons\Python23\lib\urllib2.py", line 306, in _call_chain
result = func(*args)
File "C:\pytyons\Python23\lib\urllib2.py", line 412, in
http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 400: Bad Request
--

hilsen/regards Max M, Denmark

http://www.mxm.dk/
IT's Mad Science
 
F

Fuzzyman

I am using ClientCookie for login on to servers and browsing them as
authenticated users.

I kept getting "HTTP Error 400: Bad Request" errors when submitting my
forms. So I boiled it down to a simple example.

When I try to use ClientCookie.urlopen() on my private network with ip
numbers like "http://localhost:8081/test_site/logged_in", it works fine.

If I try to call the same site through the Internet, with a url like:
http://www.mxm.dk/test_site/logged_in I get the 400 message. But only
when using "post". 'get' works fine :-s

Are you using Version 1.0.0a ?
There is a bug (well actually a bug in urllib2 - but it amoutns to the
same thing) that is fixed in CVS. Either use CVS or go back to 0.4
whatever which is the latest stable version.

Regards,

Fuzzy
 
M

Max M

Are you using Version 1.0.0a ?
Yes

There is a bug (well actually a bug in urllib2 - but it amoutns to the
same thing) that is fixed in CVS. Either use CVS or go back to 0.4
whatever which is the latest stable version.

argh! Well that was a day of my life wasted :-( Well at least *this* day
was saved. Thank you.

It's one of the damnest problems with programming.

Either you write a module like ClientCookie yourself. Then you can find
bugs rather easily. But it takes so long to work out the special cases.

Or you can use an existing library. But if that has a bug, or is poorly
documented, you need to understand the library pretty well to bug fix
it. Leaving you esssentially in the hands of the author.

I find that moments like these are among the most frustrating in
programming.

Unfurtunately there is no easy way out, but some of the best stuff I
have seen to solve these kind of documentation problems, is the doctest
based code in Zope 3.

This approach to testing makes their code *really* easy to understand.



--

hilsen/regards Max M, Denmark

http://www.mxm.dk/
IT's Mad Science
 
M

Max M

Fuzzyman said:
Are you using Version 1.0.0a ?
There is a bug (well actually a bug in urllib2 - but it amoutns to the
same thing) that is fixed in CVS. Either use CVS or go back to 0.4
whatever which is the latest stable version.


I cannot seem to find a cvs repository for ClientCookie or wwwsearch
anywhere. Or do you mean that I should get urllib2 from cvs?


--

hilsen/regards Max M, Denmark

http://www.mxm.dk/
IT's Mad Science
 
M

Michael Foord

Max M said:
I cannot seem to find a cvs repository for ClientCookie or wwwsearch
anywhere. Or do you mean that I should get urllib2 from cvs?

You're right.. I can't find it either. At least it's not on
sourceforge anyway..
John J Lee is the author of the module and he usually hangs around on
the wwwsearch mailing list (and sometimes here). When I reported the
bug to him a while ago he said he has fixed it in CVS. I just went
back to using 0.9 which I'm not sure is now easily available....

If you want I can mail that one to you.
Mail me on fuzzyman AT voidspace DOT org DOT uk and I'll see if I can
dig it out.

Regards,

Fuzzy

http://www.voidspace.org.uk/atlantibots/pythonutils.html
 
J

John J Lee

I kept getting "HTTP Error 400: Bad Request" errors when submitting my
forms. So I boiled it down to a simple example. [...]
#########################################

import ClientCookie, urllib [...]
urllib2.HTTPError: HTTP Error 400: Bad Request

I'm not certain this is the problem fuzzyman refers to.

Thanks for sample, but I get NameError instead, so your script needs
fixing before I can easily investigate:

Traceback (most recent call last):
File "/home/john/bin/tst.py", line 59, in ?
print login(came_from, login_form)
File "/home/john/bin/tst.py", line 42, in login
data = {
NameError: global name 'name' is not defined


John
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,968
Messages
2,570,149
Members
46,695
Latest member
StanleyDri

Latest Threads

Top