Trying to add threading to parse a .txt file.

[jdevito01]

Hi All,

I have code that will parse a text file line by line and then send the
variable I want to a sub routine to send pages to people. My question
is can I use threads to parse that text file with out duplicating the
output. Below is the code it may make more sense to look at it.

#!/usr/bin/ruby

trap('SIGHUP','IGNORE')

i = 0

def autopage(pagenum,count)

begin

`snpp -s SERVER ADDRESS -f 'EOC Notify' -m 'test' #{pagenum}`
rescue
puts "#{pagenum} has NOT been paged."
else
puts "#{pagenum} has been paged. #{count} "
end
end


File.open('//var//lib//asterisk//agi-bin//pager_list.txt', 'r') do |
f1|
while line = f1.gets
result = {}
pagenum,fname,lname=line.split(" ")
if line.split.empty?
else
if pagenum == "####"
else
count = (i += 1)
autopage(pagenum,count)
end
end
end
end


Thanks!!

 

[Avdi Grimm]

Hi All,

I have code that will parse a text file line by line and then send the
variable I want to a sub routine to send pages to people. My question
is can I use threads to parse that text file with out duplicating the
output. Below is the code it may make more sense to look at it.

Why do you want to use threads?

--
Avdi

Home: http://avdi.org
Developer Blog: http://avdi.org/devblog/
Twitter: http://twitter.com/avdi
Journal: http://avdi.livejournal.com

 

[jdevito01]

Why do you want to use threads?

--
Avdi

Home:http://avdi.org
Developer Blog:http://avdi.org/devblog/
Twitter:http://twitter.com/avdi
Journal:http://avdi.livejournal.com

As this .txt file is 200 lines long. I can send 200 individual pages
now in 3:min33sec i would like to cut that time by a forth by using 4
threads. I just don't think I know enough enough about threads yet to
do so. The closest thing I have come up with so far is to have 2
threads parsing two .txt file with 100 lines each in them.

 

[Avdi Grimm]

As this .txt file is 200 lines long. I can send 200 individual pages
now in 3:min33sec i would like to cut that time by a forth by using 4
threads. I just don't think I know enough enough about threads yet to
do so. The closest thing I have come up with so far is to have 2
threads parsing two .txt file with 100 lines each in them.

Chances are, with the MRI threading model, you're not going to get any
speed increase from threading this. Ruby IO tends to block the entire
interpreter.

--
Avdi

Home: http://avdi.org
Developer Blog: http://avdi.org/devblog/
Twitter: http://twitter.com/avdi
Journal: http://avdi.livejournal.com

 

[Eric I.]

Hi All,

I have code that will parse a text file line by line and then send the
variable I want to a sub routine to send pages to people. My question
is can I use threads to parse that text file with out duplicating the
output. Below is the code it may make more sense to look at it.

#!/usr/bin/ruby

trap('SIGHUP','IGNORE')

i = 0

def autopage(pagenum,count)

 begin

  `snpp -s SERVER ADDRESS -f 'EOC Notify' -m 'test' #{pagenum}`
  rescue
    puts "#{pagenum} has NOT been paged."
  else
    puts "#{pagenum} has been paged. #{count} "
  end
end

File.open('//var//lib//asterisk//agi-bin//pager_list.txt', 'r') do |
f1|
   while line = f1.gets
      result = {}
      pagenum,fname,lname=line.split(" ")
      if line.split.empty?
      else
        if pagenum == "####"
        else
           count = (i += 1)
           autopage(pagenum,count)
           end
         end
       end
     end

Thanks!!

The parsing of the file itself, should not be threaded. What you
should do is every time you read a line and determine that you need to
send a page, put the paging information into a Queue.

Then you can creat four, ten, or some other number of threads to
consume the paging information from the queue. After pulling
something from the queue, they'll use the information to call your
autopage method.

For a good example, see the the short producer/consumer program
created by Robert Kellner that appears in the 'thread' library section
of the Pick Axe book (2nd ed). In my version of the book it's on page
722, but page numbers seem to vary b/w printings and format.

If you don't mind my offering some additional unsolicited notes on
your code:

You have a few "if"s with empty "then"s. Ruby has an "unless" which
is essentially the opposite of "if" that would make this code cleaner
(IMHO).

You can put a "rescue" at the top level inside a method without having
a begin/end block. But you're right in that in other cases you do
need the begin/end block.

Hope that's helpful,

Eric

====

LearnRuby.com offers Rails & Ruby HANDS-ON public & ON-SITE
workshops.
Ruby Fundamentals Wkshp June 16-18 Ann Arbor, Mich.
Ready for Rails Ruby Wkshp June 23-24 Ann Arbor, Mich.
Ruby on Rails Wkshp June 25-27 Ann Arbor, Mich.
Ruby Plus Rails Combo Wkshp June 23-27 Ann Arbor, Mich
Please visit http://LearnRuby.com for all the details.