threads, mutex, regexp and trouble

[Mark Probert]

Hello, rubyists.

I have a bit of a problem suing threads. The program reads in a list of
commands, then fires them off, via a telnet session, to another device,
collecting data as it goes.

On a one-by-one basis, everything is fine. the commands works,
everything is peachy. The problem is I have to do 250+ sessions in a
fairly limited time. Ok, use threads.

The problem is that I appear to be occassional tromping a variable, and
mutexing doesn't seem to work. Sometimes its ok, often it not.

The code in question looks like:

@threads << Thread.new(node) { |idx|
nn, ip, usr, pwd = idx.split(/:/)

begin
bsn = BSN.new(ip, @timeout)

# login and run each command
if bsn.login then

@cmds.each do |cmd|
mutex.synchronize do
puts "~~>> cmd=#{cmd}"
bsn.cmd(cmd)
end
end
bsn.logout
else
puts " !! Unable to reach node= #{nn}"
end
}
@threads.each { |thr| thr.join }

The problem is that the "cmd" variable seems to get mashed by a regexp
match within bsn.cmd(). A good run looks like:

.. Running 2 commands on 2 nodes.
.. threading now ...
~~>> cmd=show subs isp=*_eeua
~~>> cmd=show subs isp=*_eeua

A bad run looks like:

.. Running 2 commands on 2 nodes.
.. threading now ...
~~>> cmd=show subs isp=*_eeua
~~>> cmd=show subs isp=ebg1_eeua

These commands come from a text file and the line is "show subs
isp=*_eeua".

The regexp does a match on data returned from the node. The first match
for the first node is "ebg1_eeua". So, it seems that the regexp is
confusing the thread.

Any thoughts?

Thanks,

-mark.

 

[Robert Klemme]

Hello, rubyists.

I have a bit of a problem suing threads. The program reads in a list of
commands, then fires them off, via a telnet session, to another device,
collecting data as it goes.

On a one-by-one basis, everything is fine. the commands works,
everything is peachy. The problem is I have to do 250+ sessions in a
fairly limited time. Ok, use threads.

The problem is that I appear to be occassional tromping a variable, and
mutexing doesn't seem to work. Sometimes its ok, often it not.

The code in question looks like:

@threads << Thread.new(node) { |idx|
nn, ip, usr, pwd = idx.split(/:/)

begin
bsn = BSN.new(ip, @timeout)

# login and run each command
if bsn.login then

@cmds.each do |cmd|
mutex.synchronize do
puts "~~>> cmd=#{cmd}"
bsn.cmd(cmd)
end
end
bsn.logout
else
puts " !! Unable to reach node= #{nn}"
end
}
@threads.each { |thr| thr.join }

The problem is that the "cmd" variable seems to get mashed by a regexp
match within bsn.cmd(). A good run looks like:

.. Running 2 commands on 2 nodes.
.. threading now ...
~~>> cmd=show subs isp=*_eeua
~~>> cmd=show subs isp=*_eeua

A bad run looks like:

.. Running 2 commands on 2 nodes.
.. threading now ...
~~>> cmd=show subs isp=*_eeua
~~>> cmd=show subs isp=ebg1_eeua

These commands come from a text file and the line is "show subs
isp=*_eeua".

The regexp does a match on data returned from the node. The first match
for the first node is "ebg1_eeua". So, it seems that the regexp is
confusing the thread.

Any thoughts?

Is it guaranteed that all threads use the same instance via 'mutex', i.e.
does synchronization work properly? Or maybe you use gsub! instead of
gsub (which you should since apparently all threads share @cmds).

Additionally I'd add a rescue clause at the end of the thread's main block
to catch and display any exceptions. Plus I'd put the bsn.logout into an
ensure clause:

@threads << Thread.new(node) { |idx|
nn, ip, usr, pwd = idx.split(/:/)

begin
bsn = BSN.new(ip, @timeout)

# login and run each command
if bsn.login then
begin
@cmds.each do |cmd|
mutex.synchronize do
puts "~~>> cmd=#{cmd}"
bsn.cmd(cmd)
end
end
ensure
bsn.logout
end
else
$stderr.puts " !! Unable to reach node= #{nn}"
end
rescue Exception => e
$stderr.puts e
end
}

@threads.each { |thr| thr.join }

Btw: Your indentation was garbled. Better use spaces.

Kind regards

robert

 

[Robert Klemme]

Thank you very much for your input. It is was the @cmds that was in fact
getting overwritten. I changed how I processed it, to rereading the file
for each thread, and the problem went away.

Rereading the file is a bad solution (performance wise). Why don't you
just copy it? Depending on whether only the reference @cmd was
overwritte, the collection changed or even elements were changed one of
these three options is most appropriate (in order):

@threads << Thread.new(node, @cmds) { |idx, commands|
nn, ip, usr, pwd = idx.split(/:/)

begin
bsn = BSN.new(ip, @timeout)

# login and run each command
if bsn.login then
begin
commands.each do |cmd|
mutex.synchronize do
puts "~~>> cmd=#{cmd}"
bsn.cmd(cmd)
end
end
ensure
bsn.logout
end
else
$stderr.puts " !! Unable to reach node= #{nn}"
end
rescue Exception => e
$stderr.puts e
end
}

@threads << Thread.new(node, @cmds.dup) { |idx, commands|
nn, ip, usr, pwd = idx.split(/:/)
....
end


@threads << Thread.new(node, Marshal.load(Marshal.dump(@cmds))) { |idx,
commands|
nn, ip, usr, pwd = idx.split(/:/)
....
end

Thanks also for the exception handling suggestions. Implemented!

Yeah, that's an often seen problem: exceptions that terminate threads are
normally not shown so that threads seem to die silently.

Regards

robert