how to browse using urllib2 and cookeilib the correct way

[jnair]

Hi ,

I am using python2.4 "urllib2" and "cookelib".
In line "5" below i provide my credentials to
login into a web site.During the first attempt i "fail",
judging from the output of line "6".
I try again and the second time i succeed,judging
from the output of line "8".

Now using the "twill" module (http://www.idyll.org/~t/www-tools/twill/)
it suceeds in the first attempt(lines "10" to "19").

can anybody explain, in the first case why i need to do two attempts.

1) >>>import urllib2,urllib,cookielib
2) >>>cj = cookielib.CookieJar()
3) >>>opener = urllib2.build_opener( urllib2.HTTPCookieProcessor(cj))
4) >>>data = urllib.urlencode ( { "username" : "user" ,
"password" :"****" } )
5) >>>fp = opener.open(
"https://rhn.redhat.com/rhn/LoginSubmit.do",data )
6) >>>fp.url

'https://rhn.redhat.com/rhn/ReLogin.do?url_bounce=/network/index.pxt'
7) >>>fp = opener.open(
"https://rhn.redhat.com/rhn/LoginSubmit.do",data )
8) >>>fp.url
'https://rhn.redhat.com/network/index.pxt'

#twill module

10) >>>from twill.commands import *
11) >>>go("https://rhn.redhat.com")
12) ==> at https://rhn.redhat.com
13) 'https://rhn.redhat.com'
14) >>>fv("2","username","user")
15) >>>fv("2","password","****")
16) >>>submit()
17) Note: submit is using submit button: name="None", value=" Â Sign
In "
18) >>>url('https://rhn.redhat.com/network/index.pxt')
19) 'https://rhn.redhat.com/network/index.pxt'

Thanks
jn
jn

 

[John J. Lee]

1) >>>import urllib2,urllib,cookielib
2) >>>cj = cookielib.CookieJar()
3) >>>opener = urllib2.build_opener( urllib2.HTTPCookieProcessor(cj))
4) >>>data = urllib.urlencode ( { "username" : "user" ,
"password" :"****" } )
5) >>>fp = opener.open(
"https://rhn.redhat.com/rhn/LoginSubmit.do",data )
6) >>>fp.url

..url is not a public interface. Use .geturl() .

'https://rhn.redhat.com/rhn/ReLogin.do?url_bounce=/network/index.pxt'
7) >>>fp = opener.open(
"https://rhn.redhat.com/rhn/LoginSubmit.do",data )
8) >>>fp.url
'https://rhn.redhat.com/network/index.pxt'

Dunno. Perhaps because a Refresh is used to handle the login? twill
is based on mechanize, which handles Refresh headers.


John