Question about file management [read]

[Panagiotis Atmatzidis]

A salut to the list,

I need read info from a file, extract the info that I want which is the =
date and IP and dump them into a database (mysql).

However, this file gets constantly updated, sits on /var/log/.

Until now I have used just the file.open and file.read methods. However, =
I feel that both ways are not suitable for what I want to do, because I =
will need to make a selection between old and new entries every time my =
script reads the file to extract info.

What is the *standard* (or best) way to manage this situation?

Here is the class which reads the file:

class ReadIPs
def self.ip(log)=20
file =3D File.open(log)
file.each_line do |line|=20
if line.include?("Ban")
a =3D line.split(" ")
date =3D a[0]
ip =3D a[6]
puts "date: #{date} IP: #{ip}"
end
end
file.close
end
end

I use File.open now but I should use something like File.append or =
similar? Note that I will never write to this file, it's just the file =
that contains the info. Every time a new line appears that contains the =
word "Ban" I want to trigger this...

Best regards

Panagiotis (atmosx) Atmatzidis

email: (e-mail address removed)
URL: http://www.convalesco.org
GnuPG ID: 0xFC4E8BB4=20
gpg --keyserver x-hkp://pgp.mit.edu --recv-keys 0xFC4E8BB4

 

[Robert Klemme]

A salut to the list,

I need read info from a file, extract the info that I want which is the date and IP and dump them into a database (mysql).

However, this file gets constantly updated, sits on /var/log/.

Until now I have used just the file.open and file.read methods. However, I feel that both ways are not suitable for what I want to do, because I will need to make a selection between old and new entries every time my script reads the file to extract info.

What is the *standard* (or best) way to manage this situation?

Here is the class which reads the file:

class ReadIPs
def self.ip(log)
file = File.open(log)

Ha! There you are not using block form of File.open and you do not have
the #close in ensure!

file.each_line do |line|
if line.include?("Ban")
a = line.split(" ")
date = a[0]
ip = a[6]
puts "date: #{date} IP: #{ip}"
end
end
file.close
end
end

I use File.open now but I should use something like File.append or similar? Note that I will never write to this file, it's just the file that contains the info. Every time a new line appears that contains the word "Ban" I want to trigger this...

Best regards

I can't test it right now but this might be an option.

File.open log do |io|
loop do
io.each do |line|
# whatever...
end

# v2: pos = io.tell
sleep 10 # whatever interval is appropriate
# v1: io.seek(0, IO::SEEK_SET)
# v2: io.seek(pos, IO::SEEK_CUR)
end
end

Maybe you have to do one of the variants to clear eof flag:

v1: seek to offset 0 of current position
v2: fetch the position before the sleep and then seek to that absolute
position

Kind regards

robert

 

[Panagiotis Atmatzidis]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello,

However, I feel that both ways are not suitable for what I want to do, =
because I will need to make a selection between old and new entries =
every time my script reads the file to extract info.

=20
Ha! There you are not using block form of File.open and you do not =

have the #close in ensure!

I don't understand what do you mean by 'ensure'.

=20

file.each_line do |line|
if line.include?("Ban")
a =3D line.split(" ")
date =3D a[0]
ip =3D a[6]
puts "date: #{date} IP: #{ip}"
end
end
file.close
end
end
=20
I use File.open now but I should use something like File.append or =

similar? Note that I will never write to this file, it's just the file =
that contains the info. Every time a new line appears that contains the =
word "Ban" I want to trigger this...

=20
I can't test it right now but this might be an option.
=20
File.open log do |io|
loop do
io.each do |line|
# whatever...
end
=20
# v2: pos =3D io.tell
sleep 10 # whatever interval is appropriate
# v1: io.seek(0, IO::SEEK_SET)
# v2: io.seek(pos, IO::SEEK_CUR)
end
end
=20
Maybe you have to do one of the variants to clear eof flag:
=20
v1: seek to offset 0 of current position
v2: fetch the position before the sleep and then seek to that absolute = position
=20
Kind regards
=20
robert

Oh, this solutions, might work but I think that it's a bit =
overcomplicated for me. I did not used IO and "seek" yet...

I found online [1] a few examples but I did not had the time to test =
them.

Maybe something like:=20

file =3D File.open(path, File::WRONLY|File::APPEND)

Will do.

--=20
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/

[1] http://pleac.sourceforge.net/pleac_ruby/fileaccess.html

Panagiotis (atmosx) Atmatzidis

email: (e-mail address removed)
URL: http://www.convalesco.org
GnuPG ID: 0xFC4E8BB4=20
gpg --keyserver x-hkp://pgp.mit.edu --recv-keys 0xFC4E8BB4
- --
The wise man said: "Never argue with an idiot. They bring you down to =
their level and beat you with experience."

-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.12 (Darwin)

iEYEARECAAYFAksk7g0ACgkQrghUb/xOi7Q+LwCdGmvHDHouNAz5lLSd4H84M/1F
+csAn398ysHU8qJc4vVTGtpcjHzRHX2k
=3DpOA2
-----END PGP SIGNATURE-----

 

[Robert Klemme]

I don't understand what do you mean by 'ensure'.

There full syntax you can have with begin end blocks is

begin
rescue
else
ensure
end

See also
http://blog.rubybestpractices.com/posts/rklemme/002_Writing_Block_Methods.html
http://blog.rubybestpractices.com/posts/rklemme/001-Using_blocks_for_Robustness.html

file.each_line do |line|
if line.include?("Ban")
a = line.split(" ")
date = a[0]
ip = a[6]
puts "date: #{date} IP: #{ip}"
end
end
file.close
end
end

I use File.open now but I should use something like File.append or similar? Note that I will never write to this file, it's just the file that contains the info. Every time a new line appears that contains the word "Ban" I want to trigger this...

Best regards

I can't test it right now but this might be an option.

File.open log do |io|
loop do
io.each do |line|
# whatever...
end

# v2: pos = io.tell
sleep 10 # whatever interval is appropriate
# v1: io.seek(0, IO::SEEK_SET)
# v2: io.seek(pos, IO::SEEK_CUR)
end
end

Maybe you have to do one of the variants to clear eof flag:

v1: seek to offset 0 of current position
v2: fetch the position before the sleep and then seek to that absolute position

Oh, this solutions, might work but I think that it's a bit overcomplicated for me. I did not used IO and "seek" yet...

#seek is not used in the simplest case. What about reading up on #seek
and #tell or trying out the code? I don't see where this is
complicated. If that's too complicated for you then I either my
explanation made it look more complicated that it actually is or you
should abandon software development. You'll encounter far more
complicated things in the course of it.

I found online [1] a few examples but I did not had the time to test them.

Maybe something like:

file = File.open(path, File::WRONLY|File::APPEND)

Will do.

Will do what?

[1] http://pleac.sourceforge.net/pleac_ruby/fileaccess.html

I am not sure whether pleac is a proper source to enter the world of
ruby. AFAIK their aim is to compare languages - not to present an
introductory read of particular languages. There are quite a few
tutorials and books out there for Ruby.

Kind regards

robert

 

[Panagiotis Atmatzidis]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

hi,

On 13.12.2009 14:37, Panagiotis Atmatzidis wrote:
=20

However, I feel that both ways are not suitable for what I want to do, =
because I will need to make a selection between old and new entries =
every time my script reads the file to extract info.

have the #close in ensure!
I don't understand what do you mean by 'ensure'.

=20
There full syntax you can have with begin end blocks is
=20
begin
rescue
else
ensure
end
=20
See also
= http://blog.rubybestpractices.com/posts/rklemme/002_Writing_Block_Methods.=
html

=

http://blog.rubybestpractices.com/posts/rklemme/001-Using_blocks_for_Robus=
tness.html

Okay

=20

file.each_line do |line|
if line.include?("Ban")
a =3D line.split(" ")
date =3D a[0]
ip =3D a[6]
puts "date: #{date} IP: #{ip}"
end
end
file.close
end
end
=20
I use File.open now but I should use something like File.append or =

similar? Note that I will never write to this file, it's just the file =
that contains the info. Every time a new line appears that contains the =
word "Ban" I want to trigger this...

absolute position
=20

overcomplicated for me. I did not used IO and "seek" yet...

=20
#seek is not used in the simplest case. What about reading up on =

#seek and #tell or trying out the code? I don't see where this is =
complicated. If that's too complicated for you then I either my =
explanation made it look more complicated that it actually is or you =
should abandon software development. You'll encounter far more =
complicated things in the course of it.

Given the fact that I'm messing with Ruby less than 8 days, I'll pass =
that note.

By complicated I meant that, ruby might have a build-in function or gem =
that already deals with this situation of monitoring etc.

=20

I found online [1] a few examples but I did not had the time to test = them.
Maybe something like: file =3D File.open(path, = File::WRONLY|File::APPEND)
Will do.

=20
Will do what?
=20

[1] http://pleac.sourceforge.net/pleac_ruby/fileaccess.html

=20
I am not sure whether pleac is a proper source to enter the world of =

ruby. AFAIK their aim is to compare languages - not to present an =
introductory read of particular languages. There are quite a few =
tutorials and books out there for Ruby.

Okay, thanks

=20
Kind regards
=20
robert
=20
--=20
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/
=20

Panagiotis (atmosx) Atmatzidis

email: (e-mail address removed)
URL: http://www.convalesco.org
GnuPG ID: 0xFC4E8BB4=20
gpg --keyserver x-hkp://pgp.mit.edu --recv-keys 0xFC4E8BB4
- --
The wise man said: "Never argue with an idiot. They bring you down to =
their level and beat you with experience."

-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.12 (Darwin)

iEYEARECAAYFAkslIAwACgkQrghUb/xOi7RSswCfeBZr7n9l4Yl3l6XRjqx2/w6v
vd8AnA+7ArZL4CCVuaoQRqt6QTZJxxHk
=3D0pF6
-----END PGP SIGNATURE-----