Database or large array faster?

[Jim]

I'm starting to re-write an SNMP Trap catcher that I originally did in
Visual Basic a long time ago. In VB, I had an Access database that
had all the SNMP traps that I wanted to take action on: the rest were
simply ignored. I opened the database and did queries everytime a trap
came in. If the database was open for awhile with no queries I then
closed the database until needed again.

I have my perl SNMP catcher working great. Now I'm at the point where
I need to determine if the traps I'm receiving require some kind of
action. Would it be better for me to do it the same way I did it in
VB, and use database queries everytime a trap comes in (what if the
database is a remote connection) or should I maintain the actionable
traps in a large multidimensional array and search the array for each
trap (suggestions on best/fastest way to do this)? The application
currently receives an average of 40 traps per second.

Thanks
Jim

 

[gnari]

I have my perl SNMP catcher working great. Now I'm at the point where
I need to determine if the traps I'm receiving require some kind of
action. Would it be better for me to do it the same way I did it in
VB, and use database queries everytime a trap comes in (what if the
database is a remote connection) or should I maintain the actionable
traps in a large multidimensional array and search the array for each
trap (suggestions on best/fastest way to do this)? The application
currently receives an average of 40 traps per second.

unless the number of actionable traps make memory issues, I would
keep them in a simple hash, keyed to the full trap number
%traps=(
'x.y.foo.z.1' => 'action',
'x.y.bar.z.10' => 'someotheraction',
);

gnari

 

[ctcgag]

I'm starting to re-write an SNMP Trap catcher that I originally did in
Visual Basic a long time ago. In VB, I had an Access database that
had all the SNMP traps that I wanted to take action on: the rest were
simply ignored. I opened the database and did queries everytime a trap
came in. If the database was open for awhile with no queries I then
closed the database until needed again.

I have my perl SNMP catcher working great. Now I'm at the point where
I need to determine if the traps I'm receiving require some kind of
action. Would it be better for me to do it the same way I did it in
VB, and use database queries everytime a trap comes in (what if the
database is a remote connection) or should I maintain the actionable
traps in a large multidimensional array and search the array for each
trap (suggestions on best/fastest way to do this)? The application
currently receives an average of 40 traps per second.

I would think a hash would be faster than either an array or a database.

Which is better depends on a whole host of things like how many traps
there are, the exact format of their representation/searching, how much
memory they take, how much memory you have, and how (and how often) the
database is updated. The easiest way to tell is probably just to try it
and see, but I my money is on the hash.


Xho

 

[Jim]

I would think a hash would be faster than either an array or a database.

Which is better depends on a whole host of things like how many traps
there are, the exact format of their representation/searching, how much
memory they take, how much memory you have, and how (and how often) the
database is updated. The easiest way to tell is probably just to try it
and see, but I my money is on the hash.


Xho

That's what I figured the answer would be. A hash isn't too hard to
deal with but what I need to do is create either a hash of arrays or a
hash of hashes. For example, I need the following data:

[OID], [Criticality], [If it has this text], [but not this text]

'1.3.4.435.32.12.1.10', '2', 'User', 'Group'
'1.3.4.435.32.99.2', '1', '', ''
'1.3.4.2606.1.2.4.4.4.2.2.2.1.1.52', '1', '', ''


What I want to do is get the fastest way to see if the SNMP OID is in
my list of first fields and if so, react to it according to the data
in the other fields. At first, I was thinking a multidimensional
array. How would I do this with a hash and what would be the fastest
way to search it? Thanks!

Jim

 

[Tore Aursand]

[...] but what I need to do is create either a hash of arrays or a
hash of hashes. For example, I need the following data:

[OID], [Criticality], [If it has this text], [but not this text]

'1.3.4.435.32.12.1.10', '2', 'User', 'Group'
'1.3.4.435.32.99.2', '1', '', ''
'1.3.4.2606.1.2.4.4.4.2.2.2.1.1.52', '1', '', ''

What I want to do is get the fastest way to see if the SNMP OID is in
my list of first fields and if so, react to it according to the data
in the other fields. At first, I was thinking a multidimensional
array. How would I do this with a hash and what would be the fastest
way to search it?

I'm not quite sure what you really mean with "react to it according to the
data in the other fields". You don't even have a sample application for
us to look at.

Anyway. Consider this:

my %hash = (
'1.3.4.435.32.12.1.10' => ['2', 'User', 'Group'],
'1.3.4.435.32.99.2' => ['1', '', ''],
'1.3.4.2606.1.2.4.4.4.2.2.2.1.1.52' => ['1', '', ''],
);

if ( exists $hash{$snmp_oid} ) {
# ...
}

That's the best I can offer you until you come up with something yourself.


--
Tore Aursand <[email protected]>
"Omit needless words. Vigorous writing is concise. A sentence should
contain no unnecessary words, a paragraph no unnecessary sentences,
for the same reason that a drawing should have no unnecessary lines
and a machine no unnecessary parts." (William Strunk Jr.)

 

[ctcgag]

(e-mail address removed) wrote in message

I would think a hash would be faster than either an array or a
database.

Which is better depends on a whole host of things like how many traps
there are, the exact format of their representation/searching, how much
memory they take, how much memory you have, and how (and how often) the
database is updated. The easiest way to tell is probably just to try
it and see, but I my money is on the hash.


Xho

That's what I figured the answer would be. A hash isn't too hard to
deal with but what I need to do is create either a hash of arrays or a
hash of hashes. For example, I need the following data:

[OID], [Criticality], [If it has this text], [but not this text]

'1.3.4.435.32.12.1.10', '2', 'User', 'Group'
'1.3.4.435.32.99.2', '1', '', ''
'1.3.4.2606.1.2.4.4.4.2.2.2.1.1.52', '1', '', ''

For example, to build a hash of arrays:

my %hash;
while (<FH>) { chomp;
my ($oid, $crit,$if,$not) = parse_line($_)
$hash{$oid} = [$oid,$crit,$if,$not] ;
};

I will probably retrieve the other data using $oid as the hash key,
meaning that I already know $oid, so I really don't need to store
it in the referenced array, too. But it hurts nothing but memory to do
so; and might make things easier, especially if code already exists
which expects to work on arrays formatted thus.

What I want to do is get the fastest way to see if the SNMP OID is in
my list of first fields and if so, react to it according to the data
in the other fields. At first, I was thinking a multidimensional
array. How would I do this with a hash and what would be the fastest
way to search it? Thanks!

my ($it, $incoming_oid) = parse_incoming_request();

if (exists $hash{$incoming_oid}) {
react_to_it($it, @{$hash{$incoming_oid}} )
} else {
die "didn't find incoming oid $incoming_oid in oid list";
}


Obviously this is abstract to the point of being psuedocode, but that's
because I don't know anything about SNMP oids or reacting to them.
If you show us some (short) code for how you do this with a
database, we could provide more specific advice for adapting it to
hashes.

Xho