how to spider part of the content of a webpage ?

[jiehuang001]

This page http://www.dpgn.com/resultDetail.cfm?resultID=1 has data on
physican networks. ("jieACS" is the username and password). When I
change the resumtID on the URL to a different number such as 2, a
different record will show up.

Now I need to write a program, say, to collect all the records into a
single file, from resultID=1 to resultID=10000.

Can someone please tell me how to write such a perl program. Of course,
I don't need the content on the left and bottom of the webpage.

Thank you very very much!

JIe

 

[Gunnar Hjalmarsson]

This page http://www.dpgn.com/resultDetail.cfm?resultID=1 has data on
physican networks. ("jieACS" is the username and password). When I
change the resumtID on the URL to a different number such as 2, a
different record will show up.

Now I need to write a program, say, to collect all the records into a
single file, from resultID=1 to resultID=10000.

Can someone please tell me how to write such a perl program.

1. Learn Perl.

2. Write the program.

Of course,
I don't need the content on the left and bottom of the webpage.

No, of course not.

Thank you very very much!

You are welcome.

 

[Tad McClellan]

This page [snip URL] has data on
physican networks. ( [snip pword] is the username and password).

Did you just really do that!?

You are willing to have the whole World use that pay-for service
under your account information?

Are you crazy or just incompetent?

Go change your password right now.

Don't read the rest of this message yet, go change your password right now.

We'll wait...

....

....

Good. Thank you, and don't ever do that again.

Now I need to write a program, say, to collect all the records into a
single file, from resultID=1 to resultID=10000.

What do you plan to do with the information that you are collecting?

Does the website's Terms Of Service allow you to do that?

A compentent webmaster is bound to notice a usage spike when you
do ten thousand lookups in 20 minutes. It might look like you
are trying to copy the entire database.

You should ask the site's webmaster if you are allowed to do this
before spending a lot of time coding it up.

Can someone please tell me how to write such a perl program.

Sounds illegal to me, so I figure you are probably on your own...

Thank you very very much!

You *did* already change your password, didn't you?

 

[Bart Van der Donck]

This page http://www.dpgn.com/resultDetail.cfm?resultID=1 has data on
physican networks. ("jieACS" is the username and password). When I
change the resumtID on the URL to a different number such as 2, a
different record will show up.

Now I need to write a program, say, to collect all the records into a
single file, from resultID=1 to resultID=10000.

That is very poor system design, it's totally insecure.

Can someone please tell me how to write such a perl program. Of course,
I don't need the content on the left and bottom of the webpage.

Why don't you just ask the datafile from sysadmin ?