Processing Qmail Smtp Session Log

[BadApple]

Hi Group ,

I am trying to process a Qmail Smtp session Log file ( multilog ).

The section of the Log file is give below :

@400000004123d44320c51f3c tcpserver: ok 9198
mailgateway.foo.com:10.10.0.1:25 :20.132.29.1::60433
@400000004123d44320c52edc qmail-smtpd 9198: connection from
200.12.239.1 (unknown) to mailgateway.foo.com
@400000004123d4451529c72c qmail-smtpd 9198: authentication success,
user Authenticated user:[email protected]
@400000004123d4452fcdffbc qmail-smtpd 9198: mail from:
(e-mail address removed)
@400000004123d4460f3cbbe4 qmail-smtpd 9198: rcpt to: (e-mail address removed)
@400000004123d452240346c4 tcpserver: status: 4/150
@400000004123d455142c5844 qmail-smtpd 9198: message queued: 1092867147
qp 9200 size 84902 bytes
@400000004123d455341b60dc tcpserver: end 9198 status 0

For every user that authenticates a "Authenticated user:" line is
generated also a "pid" is assigned to this session .The pid is the
number visible after the "qmail-smtpd" field in this case it's
"9198".After the message is transferred the line " message queued " is
generated and the no of bytes transferred is printed just before the
"bytes" field .
The pid assigned is constant till the smtp session is live . The start
of any smtp session is the line "tcpserver: ok <pid no>
<hostname>...ipaddress ..."
The end of the smtp session is marked by the line ."tcpserver: end
<pid> status 0".

I am trying to match this pid for the "Authenticated user:" to the
bytes transferred.

I have written following code ,which works a bit ,but it fails if
another smtp session start before the end of the smtp session which I
am processing .

My Code .
-------------------
#!/usr/bin/perl -w
# The Log File
$logfile = shift || die "Usage:$0 <logfile>";
open FLE, "< $logfile" if defined ($logfile);
# While Start
while (<FLE>) {
chomp;
# We get the Authenticated Line
if (/\s(\d{1,}): authentication success, user Authenticated
user.{1,})$/){ # Start IF AUTHENTICATED
# We assign the Pid and Auth User
$authpid = $1; # Auth pid
$authuser = $2; # Auth User
$HoH{$authpid}{user} = $authuser; # Just create a Hash
for each Pid
print "Pid: $authpid User: $authuser \n";
} # END IF AUTHENTICATED
# We search the bytes transferred line
if (/qmail-smtpd\s(\d{1,}):\smessage
queued:\s\d{1,}\sqp\s\d{1,}\ssize\s(\d{1,})\sbytes.*$/){ # IF MESSAGE
QUEUED
# We define Pid and Bytes
$pid = $1;
$bytes = $2;
#$HoH{$pid}{Bytes} = $bytes if defined ( $HoH{$pid} );
if (defined ( $HoH{$pid})) { # Check if it's pid
of an authenticated smtp session ,if not it's mostly a non
authenticated session
print "Pid :$pid Bytes :$bytes\n";
}
} # END IF QUEUED

}

-------------------

And the output is :

-----------------
Pid: 10554 User: (e-mail address removed)
Pid :10554 Bytes :6385
Pid: 11315 User: (e-mail address removed)
Pid :11315 Bytes :1605
Pid: 11547 User: (e-mail address removed)
Pid: 11842 User: (e-mail address removed)
Pid: 11844 User: (e-mail address removed)
Pid :11844 Bytes :1112
------------------

As you can see till (e-mail address removed) I am getting the Username & bytes
properly but later after (e-mail address removed) the bytes are lost .

My questions are :
1. When I get a pid of Authenticated User how do I store it till the
bytes are found.
2. When I get the bytes how do I destroy the Hash so that the bytes
won't get overwritten
3. When handling more than one "Authenticated user:" pids how to
store them (pids) till the corresponding "bytes" line is not received
..

Thanx in advance for any help
BadApple

 

[Gunnar Hjalmarsson]

My questions are :
1. When I get a pid of Authenticated User how do I store it till
the bytes are found.
2. When I get the bytes how do I destroy the Hash so that the bytes
won't get overwritten
3. When handling more than one "Authenticated user:" pids how to
store them (pids) till the corresponding "bytes" line is not
received

I believe this suggestion contains possible answers to those questions:

my %hash;
while (<FLE>) {
chomp;
if ( /qmail-smtpd\s+(\d+).+Authenticated user.+)/ ) {
$hash{$1} = $2;
}
if ( /qmail-smtpd\s+(\d+).+queued.+size\s+(\d+)/ ) {
if ( $hash{$1} ) {
print "Pid: $1 User: $hash{$1}\n",
"Pid: $1 Bytes: $2\n";
delete $hash{$1};
}
}
}

 

[Gunnar Hjalmarsson]

my %hash;
while (<FLE>) {
chomp;
if ( /qmail-smtpd\s+(\d+).+Authenticated user.+)/ ) {
$hash{$1} = $2;
}
if ( /qmail-smtpd\s+(\d+).+queued.+size\s+(\d+)/ ) {
if ( $hash{$1} ) {
print "Pid: $1 User: $hash{$1}\n",
"Pid: $1 Bytes: $2\n";
delete $hash{$1};
}
}
}

If the first regex matches, there is no reason to test the second
regex, so this makes more sense:

my %hash;
while (<FLE>) {
chomp;
if ( /qmail-smtpd\s+(\d+).+Authenticated user.+)/ ) {
$hash{$1} = $2;
} elsif ( /qmail-smtpd\s+(\d+).+queued.+size\s+(\d+)/ ) {
if ( $hash{$1} ) {
print "Pid: $1 User: $hash{$1}\n",
"Pid: $1 Bytes: $2\n";
delete $hash{$1};
}
}
}

 

[BadApple]

If the first regex matches, there is no reason to test the second
regex, so this makes more sense:

my %hash;
while (<FLE>) {
chomp;
if ( /qmail-smtpd\s+(\d+).+Authenticated user.+)/ ) {
$hash{$1} = $2;
} elsif ( /qmail-smtpd\s+(\d+).+queued.+size\s+(\d+)/ ) {
if ( $hash{$1} ) {
print "Pid: $1 User: $hash{$1}\n",
"Pid: $1 Bytes: $2\n";
delete $hash{$1};
}
}
}

Many Many Many Many ...... Thanx
Regards
BadApple