Scraping <table> from a website

[cskilbeck]

Hi,

I need to extract everything between <table> and </table> on a website
(there's only one table on the page. So far I have:

require 'open-uri'
page = open('http://xxx.html').read
page.gsub!(/\n/,"")
page.gsub!(/\r/,"")
inner = page.scan(%r{.*<table.*>(.*)</table>.*}m)
print inner

but inner is empty - any ideas?

If I substitute line 2 with

page = '123<table>456</table>789

I get inner = 456, which is correct.

 

[Alex LeDonne]

Hi,

I need to extract everything between <table> and </table> on a website
(there's only one table on the page. So far I have:

require 'open-uri'
page = open('http://xxx.html').read
page.gsub!(/\n/,"")
page.gsub!(/\r/,"")
inner = page.scan(%r{.*<table.*>(.*)</table>.*}m)

Untested, but try:

print inner

but inner is empty - any ideas?

If I substitute line 2 with

page = '123<table>456</table>789

I get inner = 456, which is correct.

If you try page = '123<table><tr><td>456</td></tr></table>789', it
will fail again.

You only want to capture up to the next closing angle bracket. What's
happening is that the second .* is matching the contents of the entire
table, up to the closing angle bracket of the last tag (probably
</tr>) right before the </table>, and inner gets only the leftover
whitespace inbetween. So only capture characters that are NOT a
closing angle bracket.

-Alex

 

[Rolando Abarca]

Hi,

I need to extract everything between <table> and </table> on a website
(there's only one table on the page. So far I have:

require 'open-uri'
page = open('http://xxx.html').read
page.gsub!(/\n/,"")
page.gsub!(/\r/,"")
inner = page.scan(%r{.*<table.*>(.*)</table>.*}m)
print inner

but inner is empty - any ideas?

If I substitute line 2 with

page = '123<table>456</table>789

I get inner = 456, which is correct.

use the right tools for the right job

require 'hpricot'
require 'open-uri'

doc = Hpricot(open('http://xxx.html'))
table = doc.at('table')
puts table.inner_html

(not tested)
regards,

 

[William James]

Hi,

I need to extract everything between <table> and </table> on a website
(there's only one table on the page. So far I have:

require 'open-uri'
page = open('http://xxx.html').read
page.gsub!(/\n/,"")
page.gsub!(/\r/,"")
inner = page.scan(%r{.*<table.*>(.*)</table>.*}m)
print inner

but inner is empty - any ideas?

If I substitute line 2 with

page = '123<table>456</table>789

I get inner = 456, which is correct.

inner = page[ %r{<table.*?>(.*?)</table>}mi, 1]

 

[cskilbeck]

Hi,

I need to extract everything between <table> and </table> on a website
(there's only one table on the page. So far I have:

require 'open-uri'
page = open('http://xxx.html').read
page.gsub!(/\n/,"")
page.gsub!(/\r/,"")
inner = page.scan(%r{.*<table.*>(.*)</table>.*}m)
print inner

but inner is empty - any ideas?

If I substitute line 2 with

page = '123<table>456</table>789

I get inner = 456, which is correct.

inner = page[ %r{<table.*?>(.*?)</table>}mi, 1]

Thanks all for your help. non greedy matching is the key.

 

[Thufir]

require 'hpricot'
require 'open-uri'

doc = Hpricot(open('http://xxx.html')) table = doc.at('table')
puts table.inner_html

Amazing -- I thought that the above would be a massive project, not what
appears to be pseudo-code! Not everything in Ruby is magically easy, but
the above is pretty good



-Thufir