problems with module Cookie

[Manlio Perillo]

Hi.
I'm using the Cookie module (on the client side).
I have found a problem trying to parse the cookie:

"Set-Cookie: value=thevalue; path=/;
expires=Fri, 21-May-2004 10:40:51 GMT"

The date is not parsed correctly, only "Fri," is matched.


Thanks and regards Manlio Perillo

 

[Jarek Zgoda]

I'm using the Cookie module (on the client side).
I have found a problem trying to parse the cookie:

"Set-Cookie: value=thevalue; path=/;
expires=Fri, 21-May-2004 10:40:51 GMT"

You must use date in UTC format.

 

[Manlio Perillo]

^^^

You must use date in UTC format.

This is the date format used by the server cookie!




Thanks and Regards Manlio Perillo

 

[Jarek Zgoda]

This is the date format used by the server cookie!

See http://www.w3.org/TR/NOTE-datetime on how should date look in
cookies. If date is not in valid format, the module functions may have
trouble decoding it.

 

[Manlio Perillo]

See http://www.w3.org/TR/NOTE-datetime on how should date look in
cookies. If date is not in valid format, the module functions may have
trouble decoding it.

It is not a my problem!
The web server follow the Netscape specification for Cookies and in
this spec the date is in the format I have posted.

I'm trying to fix the regular expression patternin Cookie.py but it
does not work:


_LegalCharsPatt = r"[\w\d!#%&'~_`><@,:/\$\*\+\-\.\^\|\)\(\?\}\{\=]"

_DatePatt = r"(?:\D{3}, \d{2}-\D{3}-\d{4} \d{2}:\d{2}:\d{2} GMT)" <---

_CookiePattern = re.compile(
r"(?x)"
r"(?P<key>"
""+ _LegalCharsPatt +"+?"
r")"
r"\s*=\s*"
r"(?P<val>"
r'"(?:[^\\"]|\\.)*"'
r"|"
""+ _DatePatt + "" <-------------
r"|"
""+ _LegalCharsPatt +"*"
r")"
r"\s*;?"
)



Thanks and regards Manlio Perillo

 

[Jarek Zgoda]

It is not a my problem!

As you see, it's your problem, since it's you who cann't decode this
cookie.

The web server follow the Netscape specification for Cookies and in
this spec the date is in the format I have posted.

Netscape is not internet God, W3C is.

I never tried to read cookies using Cookie module (I used Python only
for writing), so I cann't help more here. Good luck.

 

[JanC]

I'm using the Cookie module (on the client side).

Do you know the ClientCookie module?
<http://wwwsearch.sourceforge.net/ClientCookie/>

 

[Manlio Perillo]

Do you know the ClientCookie module?
<http://wwwsearch.sourceforge.net/ClientCookie/>

Yes, I know; but the standard Cookie module is sufficient.



Thanks and regards

 

[Manlio Perillo]

As you see, it's your problem, since it's you who cann't decode this
cookie.

The problem is also of Cookie module.

Netscape is not internet God, W3C is.

Unfortunately we don't live in a perfect world...

I never tried to read cookies using Cookie module (I used Python only
for writing), so I cann't help more here. Good luck.

Ok, thanks.



Regards Manlio Perillo

 

[John J. Lee]

I'm trying to fix the regular expression patternin Cookie.py but it
does not work:

[...]

Yeah, IIRC there's some odd stuff in there, that doesn't even seem to
come from the standards, let alone reality ;-)

The Cookie module really doesn't know how to handle cookies on the
client side. Use this, which does:

http://wwwsearch.sf.net/ClientCookie/


You can just say:

ClientCookie.urlopen("http://www.example.com/")

and be done with it.


John

 

[Manlio Perillo]

I'm trying to fix the regular expression patternin Cookie.py but it
does not work:

[...]

Yeah, IIRC there's some odd stuff in there, that doesn't even seem to
come from the standards, let alone reality ;-)

I have fixed the pattern.
For matching spaces it is needed '\ ' and not ' '.

Here is the code.
Now the Cookie parse the Netscape format.


_LegalCharsPatt = r"[\w\d!#%&'~_`><@,:/\$\*\+\-\.\^\|\)\(\?\}\{\=]"

_WeekPatt = r"(?:Mon|Tue|Wed|Thu|Fri|Sat|Sun)"
_MonthPatt = r"(?:Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)"

_DatePatt = r"(?:" + _WeekPatt + r",\ \d{2}-" + _MonthPatt +\
r"-\d{4}\ \d{2}:\d{2}:\d{2}\ GMT)"

_CookiePattern = re.compile(
r"(?x)" # This is a Verbose pattern
r"(?P<key>" # Start of group 'key'
""+ _LegalCharsPatt +"+?" # Any word of at least one letter,\
nongreedy
r")" # End of group 'key'
r"\s*=\s*" # Equal Sign
r"(?P<val>" # Start of group 'val'
r'"(?:[^\\"]|\\.)*"' # Any doublequoted string
r"|" # or
""+ _DatePatt + "" # A date as specified by Netscape\
spec
r"|" # or
""+ _LegalCharsPatt +"*" # Any word or empty string
r")" # End of group 'val'
r"\s*;?" # Probably ending in a semi-colon
)



I also have added a method to BaseCookie that behaves like
Morsel.OutputString:

def OutputString(self, attrs=None, sep='\n'):
"""Return a string suitable for HTTP.
"""
result = []
items = self.items()
items.sort()
for K,V in items:
result.append( V.OutputString(attrs) )
return sep.join(result)



Now the Cookie is usable on the client side too.

The Cookie module really doesn't know how to handle cookies on the
client side.

It does not matter, all the cookie logic for my program is very very
simple and standard Cookie is all I need.

Use this, which does:

http://wwwsearch.sf.net/ClientCookie/


You can just say:

ClientCookie.urlopen("http://www.example.com/")

and be done with it.

I have seen the module, but it is too complicated.
Standard Cookie module (with my corrections) plus httplib module is
really all I need.



Thanks and regards Manlio Perillo

 

[John J. Lee]

On 27 May 2004 22:52:10 +0100, (e-mail address removed) (John J. Lee) wrote: [...]

You can just say:

ClientCookie.urlopen("http://www.example.com/")

and be done with it.

I have seen the module, but it is too complicated.

Standard Cookie module (with my corrections) plus httplib module is
really all I need.

Cool.


John

 

[Manlio Perillo]

On 27 May 2004 22:52:10 +0100, (e-mail address removed) (John J. Lee) wrote: [...]

You can just say:

ClientCookie.urlopen("http://www.example.com/")

and be done with it.

I have seen the module, but it is too complicated.

Having written it, I agree, but I don't think it's my fault <wink>.

Well, it is too complicated for me.

Cool.

Yes, there exists programs that are simple (as there exist programming
languages that are simple)!

Actually what I do is to download several files from a server.
Some files/pages are generated by a script (so I have to post an
x-www-form-urlencoded string).

I don't want to use urllib2 because it (as I think) for every request
connects to the server, do the request and disconnect.

I need cookies because the server (as many other) authenticate user
with cookies.
So the simple algorithm is:

-connect to the server
-read a cookie from a file
-send the cookie to the server
-if the server send a cookie, the old one must be updated: with
standard Cookie this is simple: cookie.update(newcookie)
- ...
- save the cookie to a file


This is very simple to do with httplib and Cookie modules, so why to
use more involved modules?



Regards Manlio Perillo

 

[John J. Lee]

Yes, there exists programs that are simple (as there exist programming
languages that are simple)!

Actually what I do is to download several files from a server.
Some files/pages are generated by a script (so I have to post an
x-www-form-urlencoded string).

I don't want to use urllib2 because it (as I think) for every request
connects to the server, do the request and disconnect.

Yes, that's true. I should fix it...

I need cookies because the server (as many other) authenticate user
with cookies.
So the simple algorithm is:

-connect to the server
-read a cookie from a file
-send the cookie to the server
-if the server send a cookie, the old one must be updated: with
standard Cookie this is simple: cookie.update(newcookie)
- ...
- save the cookie to a file


This is very simple to do with httplib and Cookie modules, so why to
use more involved modules?

No reason at all if you're happy with it, of course. That was what my
"Cool" was meant to communicate.

*Using* urllib2 is less involved even than your little algorithm
above, of course (neglecting bugs, of course, including the persistent
connection bug, which -- though it could certainly be a problem -- I
confess has never actually troubled me, despite having used it to
repeatedly fetch tens of millions of records in the past).

Why use modules whose *implementation* is more involved? Because
they're even easier to use, and because, even for simple scripts like
your's, not every case is as simple as the one you happen to have met.
I know from experience that it can quickly get *very* tiresome to do
some of this stuff by hand (even if only a few things like
redirections and cookies are involved). And personally, I don't even
like to *think* about it, or to have to maintain it in future if I can
avoid it, regardless of how simple (ignoring for the moment that I
have to maintain the library itself ;-).

As for why the complications... well, if you want to know, read the
code (no, don't .


John

 

[Manlio Perillo]

No reason at all if you're happy with it, of course. That was what my
"Cool" was meant to communicate.

*Using* urllib2 is less involved even than your little algorithm
above, of course (neglecting bugs, of course, including the persistent
connection bug, which -- though it could certainly be a problem -- I
confess has never actually troubled me, despite having used it to
repeatedly fetch tens of millions of records in the past).

Why use modules whose *implementation* is more involved? Because
they're even easier to use, and because, even for simple scripts like
your's, not every case is as simple as the one you happen to have met.

You are right, but: what means 'easier to use'?
There is less code to write?
There is less theory to learn?

For my script using urllib2 does not make the program easier.

I know from experience that it can quickly get *very* tiresome to do
some of this stuff by hand (even if only a few things like
redirections and cookies are involved).

In my case cookie are very easy to use and I simply ignore
redirection...

Of course I agree with you for all other cases, but there exist
programs that really needs only low level library.

Actually, ad example, standard Cookie module is low level.
It only parses key=value pairs, and, more important, it is 'other
library' neutral.
That is, BaseCookie class has a parse method for parsing a string
(as SetCookie: key=value; ....), and an OutputString (added by me)
that returns the cookie data.



Thanks and regards Manlio Perillo

 

[John J. Lee]

On 31 May 2004 01:56:10 +0100, (e-mail address removed) (John J. Lee) wrote: [...]

No reason at all if you're happy with it, of course. That was what my
"Cool" was meant to communicate.

[...]
Of course I agree with you for all other cases, but there exist
programs that really needs only low level library.

Was that not what I said? Sorry if I'm not making myself clear!


(What follows is unrelated to your (quite unnecesary!) extended
defence of your use of Cookie in your script, but just by the way of
commentary on the points you make)

Actually, ad example, standard Cookie module is low level.

Yes. The low-level stuff it does is not not always the right thing
for client-side code, though.

It only parses key=value pairs, and, more important, it is 'other
library' neutral.

[...]

Same goes for ClientCookie. The interface required of request and
response objects is defined in the docs. For doing what ClientCookie
does (automatic cookie handling), I don't think it can get much
simpler.


John

 

[Manlio Perillo]

On 31 May 2004 01:56:10 +0100, (e-mail address removed) (John J. Lee) wrote: [...]

This is very simple to do with httplib and Cookie modules, so why to
use more involved modules?

No reason at all if you're happy with it, of course. That was what my
"Cool" was meant to communicate.

[...]
Of course I agree with you for all other cases, but there exist
programs that really needs only low level library.

Was that not what I said? Sorry if I'm not making myself clear!

I'm sorry but I not a very expert in english language...

(What follows is unrelated to your (quite unnecesary!) extended
defence of your use of Cookie in your script, but just by the way of
commentary on the points you make)

Actually, ad example, standard Cookie module is low level.

Yes. The low-level stuff it does is not not always the right thing
for client-side code, though.

Why?

It only parses key=value pairs, and, more important, it is 'other
library' neutral.

[...]

Same goes for ClientCookie. The interface required of request and
response objects is defined in the docs.

Ok, but I don't want to write additional code for implementing request
and response interface...

For doing what ClientCookie
does (automatic cookie handling), I don't think it can get much
simpler.

Ok. But my program is already very simple using Cookie.



Thanks and regards Manlio Perillo