reading website

[fabsy]

Hey!
I have been going through some basic IO stuff and want to get into
something more "advanced".
I would like to make a script who asks for a website and takes the
information (the text) on the website and put it into a textfile..

So now im wondering.. how do I tell the script to download/read
text/content from a website.. How do I do it? ( of course i do not want
anyone to actually write this script for me, i want to write it myself
but im a newbie and don't know which libs and etc to read in)

Thanks!

 

[James Edward Gray II]

Hey!
I have been going through some basic IO stuff and want to get into
something more "advanced".
I would like to make a script who asks for a website and takes the
information (the text) on the website and put it into a textfile..

So now im wondering.. how do I tell the script to download/read
text/content from a website.. How do I do it? ( of course i do not
want
anyone to actually write this script for me, i want to write it myself
but im a newbie and don't know which libs and etc to read in)

You are looking for the open-uri library. See if these docs are
enough to get you going:

http://www.ruby-doc.org/stdlib/libdoc/open-uri/rdoc/index.html

James Edward Gray II

 

[Patrick Hurley]

Hey!
I have been going through some basic IO stuff and want to get into
something more "advanced".
I would like to make a script who asks for a website and takes the
information (the text) on the website and put it into a textfile..

So now im wondering.. how do I tell the script to download/read
text/content from a website.. How do I do it? ( of course i do not want
anyone to actually write this script for me, i want to write it myself
but im a newbie and don't know which libs and etc to read in)

Thanks!

Just

require "open-uri"

open(url) do |io|
# the ensues
end

 

[ChrisH]

Hey!
I have been going through some basic IO stuff and want to get into
something more "advanced".
I would like to make a script who asks for a website and takes the
information (the text) on the website and put it into a textfile..

Have a look at http://ruby-doc.org/docs/ProgrammingRuby/, the section
titled 'Network and Web', for the low level details.

Open-uri is a nice wrapper to make web IO simple as File IO

Look into Mechanize/Hpricot/etc if you want to get fancy 9^)

Cheers

 

[fabsy]

Thanks!
I've got it working..

---
require "open-uri"


print 'Skriv in adress: '
addr = gets.chomp


open(addr) do |addr|
fil = File.open('file', 'a')
fil << addr.read
fil.close
end

 

[fabsy]

Thanks!
I've got it working..Is this a good way to write it?

---
require "open-uri"

print 'Skriv in adress: '
addr = gets.chomp

open(addr) do |addr|
fil = File.open('file', 'a')
fil << addr.read
fil.close
end
---

 

[David Vallner]

Thanks!
I've got it working..

---
require "open-uri"


print 'Skriv in adress: '
addr =3D gets.chomp


open(addr) do |addr|
fil =3D File.open('file', 'a')
fil << addr.read
fil.close
end
---

Is that a good way to write it?
I also saw that the first "word" in the file 'file' was
#<StringIO:0x25350>.. What does that mean?

It means you wrote an input stream object into the file. Which seems =

Weird. Wasn't there anything in 'file' before this script started? You s=
et =

it to append, so that might be a leftover from some other script you wro=
te =

that used that file. Delete it, or open the file with the 'w' flag inste=
ad =

of 'a', and run it again?

<rant>
Also, you might want to avoid open(addr) {|addr| ... }, reusing the same=
=

variable name. It's probably not the issue here, but shadowing variables=
=

is inherently confusing. YMMV.
</rant>

David Vallner

 

[fabsy]

Aha! Yes it was something old.
It opened the file with append..

And, sorry for all the questions.. (im really trying to learn)
But is there any thing like this in ruby?

--VB code--

If InStr(1, Text1.Text, "String to find", 1) > 0 Then Msgbox "I found
the string!"

 

[David Vallner]

Aha! Yes it was something old.
It opened the file with append..

And, sorry for all the questions.. (im really trying to learn)
But is there any thing like this in ruby?

--VB code--

If InStr(1, Text1.Text, "String to find", 1) > 0 Then Msgbox "I found
the string!"

Does VB index strings starting with 1?

For very simple seaching, use String#include?
=3D> true

If you want to check if there's a substring to be found -after- (and not=
=

including) the second character, for example, then I'd use a substring:

"foobar"[2..-1].include? "ooba"

=3D> false

"foobar"[2..-1].include? "obar"

=3D> true

What VB's CompareMode is, I have no idea. Case sensitivity? If so, I'd =

personally manually use String#downcase in the conditional.

And, of course, there's always regular expressions, the swiss army knife=
=

of string searching, but those are for a longer discussion and you can =

probably find material for those out there that explains them rather =

nicely.

David Vallner

 

[fabsy]

David Vallner skrev:

Aha! Yes it was something old.
It opened the file with append..

And, sorry for all the questions.. (im really trying to learn)
But is there any thing like this in ruby?

--VB code--

If InStr(1, Text1.Text, "String to find", 1) > 0 Then Msgbox "I found
the string!"

Does VB index strings starting with 1?

For very simple seaching, use String#include?

>> "foobar".include? "ooba"
=> true

If you want to check if there's a substring to be found -after- (and not
including) the second character, for example, then I'd use a substring:

>> "foobar"[2..-1].include? "ooba"
=> false

>> "foobar"[2..-1].include? "obar"
=> true

What VB's CompareMode is, I have no idea. Case sensitivity? If so, I'd
personally manually use String#downcase in the conditional.

And, of course, there's always regular expressions, the swiss army knife
of string searching, but those are for a longer discussion and you can
probably find material for those out there that explains them rather
nicely.

David Vallner

Ok!
But that doesn't tell me the position of the string..
Say that you have a textfile with the sentence "Could you pass me the
milk please?" and I make a search for the string "milk". Then I want to
know the position of where the search word starts so I can extract the
word into a variable.... How do I do that?

And yes VB index strings start with 1..

 

[David Vallner]

Ok!
But that doesn't tell me the position of the string..
Say that you have a textfile with the sentence "Could you pass me the
milk please?" and I make a search for the string "milk". Then I want t= o
know the position of where the search word starts so I can extract th= e
word into a variable.... How do I do that?

Use String#index, or String#=3D~. The former works with strings and regu=
lar =

expressions, the latter requires regular expressions, but I think it als=
o =

saves the match data into the special global variable I don't use or =

remember the name of... Anyhoo:
=3D> 1
=3D> 3
=3D> 3
=3D> 1

For word extraction, I'm usually very, very bad doing those with substri=
ng =

indexes and abuse regexps wildly each and every time I need to get some =
=

piece of text out of another one.

David Vallner

 

[Mat Schaffer]

Hey!
I have been going through some basic IO stuff and want to get into
something more "advanced".
I would like to make a script who asks for a website and takes the
information (the text) on the website and put it into a textfile..

So now im wondering.. how do I tell the script to download/read
text/content from a website.. How do I do it? ( of course i do not
want
anyone to actually write this script for me, i want to write it myself
but im a newbie and don't know which libs and etc to read in)

Thanks!

I'm a little late to this party, and it might be a little advanced
for you yet, but if you're dealing with scripting against websites
you can do some pretty sick stuff using 'mechanize' (http://
mechanize.rubyforge.org/) and 'hpricot' (http://
code.whytheluckystiff.net/hpricot/).

Mechanize can click links and fill out forms. Hpricot can look for
text/tags in the resulting HTML. And Mechanize supports pluggable
parsers which lets you drop Hpricot right into Mechanize. It's so
buttery...
-Mat