ClientCookie bug

[Anand Pillai]

In fact, I am not aware of how Nirmal has done it.
He is wrapping up the coding soon, when I will get to
see how he has done it. He is doing the persistency part
of the cookies, and I guess he is using anydbm for it. Details
later.

I will send a mail to you, once I get the code from him
and review it. Meanwhile you can visit his homepage here

http://nirmalc.freezope.org

There might be some details on the work there.

Regards

-Anand

'We', nothing royal about it. It is just me and my friend
& co-developer Nirmal Chidambaram. Apparently he has found
a way around some of the bugs in Clientcookie. He has written

It'd be great if you made me aware what those bugs are!

(BTW, no intent to offend with my comment about your plurality, or
lack thereof -- it's just that the convention of using 'we' in source
code comments is common enough that I've sometimes found myself using
it even when writing code alone, which is funny.)

a new module using the existing Cookie module of python &
urllib2. One of the problems 'we' had with Clientcookie is that
it uses its own 'urlopen' methods which does not fit our
applications needs, so 'we' had to find a way around it.

As I said before, if you know how to do that, please comment on the
RFE I referenced in my last post. Jeremy Hylton is planning to look
at the patch associated with that RFE in detail sometime, and you
could save him some time if you know a way to do this without patching
urllib2. And I'd like to know how to do it, too

Once the code is ready, I will post it on my webpage, and
of course it is not a module in itself, so I think an
announcement to c.l.py is out of place.

[...]

Would you mind sending me an email?

Thanks


John

 

[John J. Lee]

I must admit that I don't really know what I am doing. How would you
simplify the following code:

[...]

It doesn't tend to encourage people to help you if you don't even
*try* to apply their advice, and just expect them to do all your work
for you. Read that paragraph you quoted (and the rest I posted in
that message and previously), try doing what it suggests, then come
back here if you need to.


John

 

[Robin Munn]

It'd be great if you made me aware what those bugs are!

(BTW, no intent to offend with my comment about your plurality, or
lack thereof -- it's just that the convention of using 'we' in source
code comments is common enough that I've sometimes found myself using
it even when writing code alone, which is funny.)

I do that all the time; "we" is the default in my code comments and I
have rarely, I think never, used "I". I usually think of it as myself
reading through the code with another coder, who is tasked with taking
over code maintenance from me. I'm explaining to him what the code is
intended to do: "So here, we want to search through the list for..."

 

[John J. Lee]

I do that all the time; "we" is the default in my code comments and I
have rarely, I think never, used "I". I usually think of it as myself
reading through the code with another coder, who is tasked with taking
over code maintenance from me. I'm explaining to him what the code is
intended to do: "So here, we want to search through the list for..."

That's a good excuse.


John

 

[Mark Carter]

I must admit that I don't really know what I am doing. How would you
simplify the following code:

[...]

It doesn't tend to encourage people to help you if you don't even
*try* to apply their advice, and just expect them to do all your work
for you. Read that paragraph you quoted (and the rest I posted in
that message and previously), try doing what it suggests, then come
back here if you need to.

I've looked again, and add_cookie_header() definitely(!) is required
for the following code to work:

def go10():
#works with win xp
import ClientCookie
c = ClientCookie.MSIECookieJar(delayload=1)
c.load_from_registry(username='mcarter') #only need username for
win9x

import urllib2
url = 'http://businessplus.hemscott.net/corp/crp03733.htm'
request = urllib2.Request(url)
response = urllib2.urlopen(request)
request2 = urllib2.Request(url)
c.add_cookie_header(request2)
response2 = urllib2.urlopen(request2)

print response2.geturl()
print response2.info() # headers
for line in response2.readlines(): # body
print line

If I take out add_cookie_header(), then the wrong page (i.e. the
registration page) is returned.

I will investigate further and let you know what I find.

 

[John J. Lee]

(e-mail address removed) (Mark Carter) writes:

[...]
I've looked again, and add_cookie_header() definitely(!) is required
for the following code to work:

Oh, sorry, I missed something: you're using urllib.urlopen. You need
to call ClientCookie.urlopen instead (this is still using urllib2 --
ClientCookie uses urllib2 to do most of the work).

You seem to be opening the same URL twice for no obvious reason, so I
removed that below:


def go10():
#works with win xp
import ClientCookie
c = ClientCookie.MSIECookieJar(delayload=1)
c.load_from_registry(username='mcarter') #only need username for win9x

url = 'http://businessplus.hemscott.net/corp/crp03733.htm'
response = ClientCookie.urlopen(url)

print response.info() # headers
print response.read() # body


John

 

[Mark Carter]

Oh, sorry, I missed something: you're using urllib.urlopen. You need

to call ClientCookie.urlopen instead (this is still using urllib2 --
ClientCookie uses urllib2 to do most of the work).

You seem to be opening the same URL twice for no obvious reason, so I
removed that below:


def go10():
#works with win xp
import ClientCookie
c = ClientCookie.MSIECookieJar(delayload=1)
c.load_from_registry(username='mcarter') #only need username for win9x

url = 'http://businessplus.hemscott.net/corp/crp03733.htm'
response = ClientCookie.urlopen(url)

print response.info() # headers
print response.read() # body

At the risk of sounding thick ... this doesn't work either! I couldn't
get load_cookie_data() to work using the above template, either. What
seems odd for starters is the line
response = ClientCookie.urlopen(url)

What makes it odd is that to an offhand observer, there is nothing
visibly connecting the urlopen() command to c. Sure, some fancy stuff
may be going on under the hood; but like I say, ClientCookie.urlopen()
doesn't seem connected with the MSIECookieJar instance just by looking
at it. It looks like I really want to be saying something like
response = c.urlopen(url)
but that just gives me
AttributeError: MSIECookieJar instance has no attribute 'urlopen'

 

[John J. Lee]

At the risk of sounding thick ... this doesn't work either! I couldn't
get load_cookie_data() to work using the above template, either. What
seems odd for starters is the line
response = ClientCookie.urlopen(url)

What makes it odd is that to an offhand observer, there is nothing
visibly connecting the urlopen() command to c. Sure, some fancy stuff

[...]

D'Oh!

def go10():
#works with win xp
import ClientCookie
c = ClientCookie.MSIECookieJar(delayload=1)
c.load_from_registry(username='mcarter') #only need username for win9x
opener = ClientCookie.build_opener(ClientCookie.HTTPCookieProcessor(c))

url = 'http://businessplus.hemscott.net/corp/crp03733.htm'
response = opener.open(url)

print response.info() # headers
print response.read() # body


All of this is explained in the web page, though. [attempts to looks
at web page] Well, it would be if the link weren't broken or I could
log in to my SF account to fix it... Sigh.

BTW, if you want to use ClientCookie.urlopen rather than opener.open,
you can call ClientCookie.install_opener(opener) to install a global
opener, just as you would with urllib2.


John

 

[John J. Lee]

(e-mail address removed) (Mark Carter) writes:

[...]

All of this is explained in the web page, though.

You are right! All the technical information is there. But can I
suggest a change in the docs that might spell it out a bit more
obviously to someone coming to the module from fresh? Something
like:
Certainly!


[...]
h2: Using Internet Explorer or Netscape/Mozilla cookies

c = ClientCookie.MSIECookieJar(delayload=1) # see note 1
c.load_from_registry(username='joe bloggs') # see note 2
opener = ClientCookie.build_opener(ClientCookie.HTTPCookieProcessor(c))

url = 'http://foo.bar.com/'
response = opener.open(url)

print response.info() # headers
print response.read() # body

Possibly, but there's a balance to be struck between convenience and
documentation bloat. Since "Cooperating with Netscape/Mozilla and
Internet Explorer" explains the CookieJar part, and the very next
section is "Using your own CookieJar instance", it seems to flow quite
well to me. The first sentence of that section is kind of misleading
though, so I'll change that.

[...]

2. Usually only win98 and 95 users need supply the username. As an
alternative
to loading from the registry, consider using the line
c.load_cookie_data('cookie-file.txt')
instead.

Thanks -- worthwhile addition.

[...]

should be at the top, where people will see it first, and before
those more complicated examples.

It is at the top -- after an example of the simplest usages and an
explanation of what CookieJar is actually for, which I think have to
come first.


John