All,
I run a mysql db 5.0.11. With a table that contains a few columns
defined as type "text" with indexes..
I running this on a Arch Linux 0.7.1.. 2.6.15 custom.. perl 5.8.8
I run some scripts to parse data from cisco/network devices using snmp
and telnet/ssh..
I retrieve the name, mac, serial, location, description, interfaces,
etc..
The problem I have is the names I recieve from the
devices I run threw a "clean subroutine"..
##########
sub clean
{
my $debug = 0;
my $item = $_[0];
$item =~ s/^\s+//; #remove Leading whitespace
$item =~ s/\s+$//; #remove trailing whitespace
$item =~ s/\r/ /g; #remove those Damn ^M
$item =~ s/\f/ /g; #remove those Damn ^M
$item =~ s/\t/ /g; #remove those Damn ^M
$item =~ s/\n/ /g; #remove those Damn ^M
$item =~ s/\s+/ /g; #replace multiple spaces with one
chomp($item); #remove newline character
return $item;
}
##########
And for some reason when I compare to different names
From DB: $name = "00a45fde0032(sw1)"
And the name I pulled from the device $name: "00a45fde0032(sw1)"
and compare them for exact match
##########
if($name eq $mname)
{ print "MATCH ON NAME: \"$name\" MNAME: \"$mname\"\n"; }
else
{ print "NO MATCH\n"; }
##########
Nothing happens.. No Match.
However when I did a
##########
if($name == $mname)
{ print "MATCH ON NAME: \"$name\" MNAME: \"$mname\"\n"; }
else
{ print "NO MATCH\n"; }
##########
I recieved an error stating "cannot equal on non numeric" which is
what I expected.. However it showed a hidden character of
"MNAME: "00a45fde0032(sw1)\o"
What does the "\o" mean? I have never seen this before..
In the variable their has to be hidden characters that I cannot see.
I thought about converting the string to hex then back to see if
any characters show up..
Or "$item =~ s/[^ A-Za-z0-9\-\:\.\(\)\@]//g;"
A note.. I use the "sub clean" for everthing I receive from my
scripts.. And then insert/update/replace into the database..
I don't know if the column type is causing this error or the
devices that are returning the value.. Due to I cannot see the
hidden values...
If their is a better way of "cleaning" the variables please let me
know..
Joe
All,
Thanks for all your help.. I found out what the problem was...
Their was a hidden "null" in the variable.. hex code of "00"..
This is the sub I wrote that fixes hidden characters...
########################################################
# Sub caschex
#
# USAGE: Removes hidden strings in variables.. Addition to Clean
#
# v1.0.0 -> 2006-04-20
# Born
#
######
# my ($item) = cipdec(1, $ip); #1 = A->H, 2 = H->A ######
#
sub caschex
{
# 1 = ASCII TO HEX
# 2 = HEX TO ASCII
##########
$debug = 0;
##########
if($debug == 1) { print "---------------------------------- ENTERED
SUB: \"caschex\"\n"; }
my $opt = undef; my $item = undef; my $ret = undef; my $val = undef;
$opt = shift(@_);
$item = shift(@_);
if($debug == 1)
{
print "\n\n";
print "OPT: \"$opt\"\n";
print "ITEM: \"$item\"\n";
}
############################# OPT 1
if($opt == 1)
{
if($debug == 1) { print "CONVERT ASCII TO HEX\n"; }
$key = undef; $val = undef;
foreach $key (split//,$item)
{
if($debug == 1) { print "KEY: \"$key\"\n"; }
($key) = sprintf("%02lx", ord $key);
if($debug == 1) { print "HEX KEY: \"$key\"\n"; }
if($key eq "00")
{
if($debug == 1) { print "FOUND NULL IN ASCII VARIABLE...
REPLACE WITH SPACE\n"; }
$key = " ";
#NOTE: A SPACE IN HEX IS "20"
($key) = sprintf("%02lx", ord $key);
if($debug == 1) { print "HEX KEY: \"$key\"\n"; }
}
$val .= $key;
if($debug == 1) { print "VAL: \"$val\"\n"; }
}
if($debug == 1) { print "COMPLETE VAL: \"$val\"\n"; }
}
############################# EO OPT 1
############################# OPT 2
if($opt == 2)
{
if($debug == 1) { print "CONVERT HEX TO ASCII\n"; }
$key = undef; $val = undef;
foreach $key ($item =~ /[a-fA-F0-9]{2}/g)
{
if($debug == 1) { print "HEX KEY: \"$key\"\n"; }
if($key eq "00")
{
if($debug == 1) { print "FOUND NULL IN HEX.. REPLACE WITH
SPACE\n"; }
$key = 20;
if($debug == 1) { print "HEX KEY: \"$key\"\n"; }
}
($key) = chr(hex $key);
if($debug == 1) { print "ASC KEY: \"$key\"\n"; }
$val .= $key;
if($debug == 1) { print "VAL: \"$val\"\n"; }
}
if($debug == 1) { print "COMPLETE VAL: \"$val\"\n"; }
}
############################# EO OPT 2
$ret = $val;
if($debug == 1) { print "RET: \"$ret\"\n"; }
if($debug == 1) { print "---------------------------------- LEAVING
SUB: \"caschex\"\n"; }
$debug = 0;
return($ret);
}
#
#
############################################## EO SUB CASCHEX
I also editted my "clean" sub...
########################################################
# Sub clean
# USAGE: Removes leading and trailing whitespace
#
# HIS: v1.0.5 -> 2005-01-24
# Added to "joes_pm";
#
# HIS: v1.0.6 -> 2005-07-28
# Added to "chomp";
# Added "s/\r/\n/g" for those damn ^Ms..
#
# HIS: v1.0.7 -> 2006-01-18
# Added "s/\s+/ /g" for replace multiple spaces with one
#
# HIS: v1.0.8 -> 2006-04-20
# Added "caschex" sub for removing hidden characters
#
# Usage: line below for results
######
# $line = &clean($line);
######
#
sub clean
{
my $debug = 0;
if($debug == 1) { print "ENTERED SUB \"CLEAN\"\n"; }
my $item = shift(@_);
if($debug == 1) { print "ITEM: \"$item\"\n"; }
$item =~ s/\r/ /g; #remove those Damn ^M
$item =~ s/\f/ /g; #remove those Damn ^M
$item =~ s/\t/ /g; #remove those Damn ^M
$item =~ s/\n/ /g; #remove those Damn ^M
$item = caschex(1, $item); #A->H
if($debug == 1) { print "HEX: \"$item\"\n"; }
$item = caschex(2, $item); #H->A
if($debug == 1) { print "ASCII: \"$item\"\n"; }
$item =~ s/^\s+//; #remove Leading whitespace
$item =~ s/\s+$//; #remove trailing whitespace
$item =~ s/\s+/ /g; #replace multiple spaces with one
chomp($item); #remove newline character
if($debug == 1) { print "RET: \"$item\"\n"; }
if($debug == 1) { print "LEAVING SUB \"CLEAN\"\n"; }
$debug = 0;
return $item;
}
################### EO SUB CLEAN ######################
It turns out the string Pulled from a cisco device from CDP (cdp name
from a snmp oid) has a hidden null character @ the end of the
string...
###
NOTE: The cisco devices I'm referring too are Menu/IOS/Cat based
from every model (around 200 different sysobject oids).
We had a problem with CiscoWorks getting us a complete inventory so I
wrote my own using Net::SNMP, Net::Telnet, DBI, Perl, Mysql 5.x,
Linux/Sun 2.9..
###
I did not see it until I converted it to hex and printed the output...
Their is where the devil was hiding..
When i reran all my scripts with the new subroutine it found hundreds
of instances where "null" characters where hidden in the variables..
Shame I didn't find out this problem until I went to remove duplicates
out of a few tables of mine and that is where I 1st noticed the
problem..
BTW.. In case someone wants to double check my work and has a stash of
cisco products on-hand they can use the following oids..
##########
my $cdpnintoid = ".1.3.6.1.4.1.9.9.23.1.2.1.1.3"; # CDP NEAR
INTERFACE OID
my $cdpfipoid = ".1.3.6.1.4.1.9.9.23.1.2.1.1.4"; # CDP FAR IP
ADD OID
my $cdpfsysdoid = ".1.3.6.1.4.1.9.9.23.1.2.1.1.5"; # CDP FAR
DESCRIPTION
my $cdpfsysnoid = ".1.3.6.1.4.1.9.9.23.1.2.1.1.6"; # CDP FAR
SYSN
my $cdpfintoid = ".1.3.6.1.4.1.9.9.23.1.2.1.1.7"; # CDP FAR
INT
my $cdpfsysmoid = ".1.3.6.1.4.1.9.9.23.1.2.1.1.8"; # CDP FAR
MODEL
##########
########## THe code for dumping the trees
if($debug == 1) { print "\n----- START CDPFSYSNOID\n"; }
$res = undef; $roid = undef;
if(defined($res = $sess->get_table(-baseoid => $cdpfsysnoid)))
{
foreach $roid (keys(%{$res}))
{
$key = undef; $val = undef;
if($debug == 1) { print "ROID: \"$roid\"\n"; }
$val = $res->{$roid};
$val = clean($val);
$val = lc($val);
#ROID: ".1.3.6.1.4.1.9.9.23.1.2.1.1.3.30.1"
@array = split(/\./, $roid);
$key .= "$array[$#array - 1].$array[$#array]";
if($debug == 1)
{
print "KEY: \"$key\"\n";
print "VAL: \"$val\"\n";
print "\n";
}
$key = clean($key);
$key = lc($key);
$key =~ s/\,//g;
$val = clean($val);
$val = lc($val);
$val =~ s/\,//g;
if($val =~ /\./)
{
$val = (split(/\./, $val))[0]; #GET ME THE EVERYTHING B4
THE 1ST PERIOD
}
if(exists $cdpfsysn{$key}) { print "KEY: \"$key\" EXISTS
IN HASH \"CDPFSYSN\".. NEXT\n"; next; }
else { $cdpfsysn{$key} = $val; }
if($debug == 1) { print "\n"; }
} #EO FE ROID
if($debug == 1) { print "--- START DUMP KEY/VAL\n";
$key = undef; $val = undef;
foreach $key (keys %cdpfsysn)
{ print "KEY: \"$key\"\nVAL: \"$cdpfsysn{$key}\"\n\n"; }
print "--- END DUMP KEY/VAL\n"; }
} #EO IF DEF CDPFSYSNOID
if($debug == 1) { print "----- DONE CDPFSYSNOID\n"; }
########## EO The code for dumping the trees
I don't know if the hidden null is vendor specific.. but you never
know....
Thanks again,
Joe