Timeout exception open-uri

[Luis Some]

Hello people... I'm kinda new to Ruby, so probably this is just a stupid
question...

I'm crawling some websites to get some information and I'm getting this
error:

/usr/lib/ruby/1.8/net/http.rb:560:in `initialize':
Connection timed out - connect(2) (Errno::ETIMEDOUT)
from /usr/lib/ruby/1.8/net/http.rb:560:in `open'
from /usr/lib/ruby/1.8/net/http.rb:560:in `connect'
from /usr/lib/ruby/1.8/timeout.rb:53:in `timeout'
from /usr/lib/ruby/1.8/timeout.rb:93:in `timeout'
from /usr/lib/ruby/1.8/net/http.rb:560:in `connect'
from /usr/lib/ruby/1.8/net/http.rb:553:in `do_start'
from /usr/lib/ruby/1.8/net/http.rb:542:in `start'
from /usr/lib/ruby/1.8/open-uri.rb:242:in `open_http'
from /usr/lib/ruby/1.8/open-uri.rb:616:in `buffer_open'
from /usr/lib/ruby/1.8/open-uri.rb:164:in `open_loop'
from /usr/lib/ruby/1.8/open-uri.rb:162:in `catch'
from /usr/lib/ruby/1.8/open-uri.rb:162:in `open_loop'
from /usr/lib/ruby/1.8/open-uri.rb:132:in `open_uri'
from /usr/lib/ruby/1.8/open-uri.rb:518:in `open'
from /usr/lib/ruby/1.8/open-uri.rb:30:in `open'
from ./folder/file.rb:111:in `get_stuff'
from /usr/lib/ruby/1.8/timeout.rb:62:in `timeout'
from ./folder/file.rb:109:in `get_stuff'
from ./folder/file.rb:88:in `each'
from ./folder/file.rb:88:in `get_stuff'
from ./folder/file.rb:77:in `get_stuff_subcategories'
from ./folder/file.rb:74:in `each'
from ./folder/file.rb:74:in `get_stuff_subcategories'
from ./folder/file.rb:72:in `each'
from ./folder/file.rb:72:in `get_stuff_subcategories'
from ./folder/site.rb:51:in `run'
from /home/user/NetBeansProjects/project/lib/main.rb:30

This only happen sometimes, and happen specially when my internet
connection is slow. So I assume is because of that (a timeout, or
something like that).

Complete code:

begin
Timeout::timeout(40){
doc = Nokogiri::HTML(open(begin_url))
#do some stuff with doc variable
}
rescue Net::HTTPBadResponse => e
log.error 'Fail!'
retries_torr -= 1
if retries_torr > 0
retry
end
rescue Mechanize::ResponseCodeError => ex
log.error 'Fail'
if retries_torr > 0
retry
end
rescue Timeout::Error => e
log.error 'Fail'
retries -= 1
if retries > 0
retry
else
log.error 'Fail opening the page. Check the internet connection.'
end
end

The exception is generated in this line:

doc = Nokogiri::HTML(open(begin_url))

The thing is, I'm trying to catch that exception and I'm not getting
success with that task. I tried:
rescue Timeout::Error

But didn't work.
Can any of you give me some help here?

Thanks,

Luis

 

[Brian Candler]

The thing is, I'm trying to catch that exception and I'm not getting
success with that task. I tried:
rescue Timeout::Error

That's not the exception you're getting (read the error message
carefully). It's:

rescue Errno::ETIMEDOUT

Or better,

rescue SystemCallError

which is the superclass of all the Errno::* errors. These are errors
generated by the operating system itself, and this one is occuring
before your 40 second timeout has expired.

 

[Luis Some]

That's not the exception you're getting (read the error message
carefully). It's:

rescue Errno::ETIMEDOUT

Or better,

rescue SystemCallError

which is the superclass of all the Errno::* errors. These are errors
generated by the operating system itself, and this one is occuring
before your 40 second timeout has expired.

Hey Brian

Thanks for your reply.

I'm gonna do some tests with that options you gave me.
And yes, was a stupid question...

If I get other errors, I'll let you know.

Thanks,

Luis