using httplib for authentication

[James Stroud]

Hello All,

I want to use python to download files from sites where authentication is
required. The page appears to send a form with the login and pass by post. I
would like to log in and keep this session open within python and
download a number of files automatically. Could anyone kindly point me in the
right direction? I have discovered httplib and read the documentation and
looked at the examples, but they don't seem helpful for this.

James

--
James Stroud
UCLA-DOE Institute for Genomics and Proteomics
Box 951570
Los Angeles, CA 90095

http://www.jamesstroud.com/

 

[Fuzzyman]

Hello All,

I want to use python to download files from sites where authentication is
required. The page appears to send a form with the login and pass by post. I
would like to log in and keep this session open within python and
download a number of files automatically. Could anyone kindly point me in the
right direction? I have discovered httplib and read the documentation and
looked at the examples, but they don't seem helpful for this.

Hello James,

httplib is built on top of the socket module - but is still a library
meant for doing fairly low level http operations. For fetching URLs,
urllib2 is probably the way to go. It handles GET and POST with ease.

If you think this will do the job then you could try reading the
urllib2 tutorial at :

http://www.voidspace.org.uk/python/articles.shtml#http

You might wnat to look at other extension modules too - ClientCookie
for handling cookies, ClientForm for automatically filling in the
forms, BeautifulSoup for parsing HTML pages....

All the Best,

Fuzzy
http://www.voidspace.org.uk/python

 

[James Stroud]

Thank you Fuzzy, I will look into these things. Maybe the site is setting a
cookie, as you have suggested. I have never delved into the ways of http
except to configure apache and write some very bare-bones web pages, so I
have to say that some very obvious things do not occur to me.

James