Dir, recursive filescan

[Rebhan, Gilbert]

Hi,

i want to count the occurence of a searchstring in all files
below a directory and all its subdirectories

i have =3D

matches=3D0
Dir.glob("E:/test/foobar/scripts/**/**.xml").each do |f|
matches +=3D File.new(f).read.scan('<if>').size
puts f
end

puts "Matchcount =3D=3D " + matches.to_s
puts "Scanned Files =3D=3D " +
Dir.glob("E:/test/foobar/scripts/**/**.xml").size.to_s

Is there a better way ?

Regards,
Gilbert

 

[Robert Klemme]

i want to count the occurence of a searchstring in all files
below a directory and all its subdirectories

i have =

matches=0
Dir.glob("E:/test/foobar/scripts/**/**.xml").each do |f|
matches += File.new(f).read.scan('<if>').size
puts f
end

puts "Matchcount == " + matches.to_s
puts "Scanned Files == " +
Dir.glob("E:/test/foobar/scripts/**/**.xml").size.to_s

Is there a better way ?

Yes, you should at least avoid globbing twice. This is really slow.
Also, you do not need two stars for the file name.

Then, you do not close files properly when reading. A simplified
version looks like this

files = Dir["E:/test/foobar/scripts/**/*.xml"]
matches = files.inject(0) {|sum,f| sum + File.read(f).scan('<if>').size}
printf "%10d matches\n%10d files\n", matches, files.size

If you have a lot of files it may pay off to use Find.find instead of Dir[].

Kind regards

robert

 

[Rebhan, Gilbert]

=20
Hi, Robert

-----Original Message-----
From: Robert Klemme [mailto:[email protected]]=20
Sent: Friday, August 24, 2007 10:30 PM
To: ruby-talk ML
Subject: Re: Dir, recursive filescan

/*
files =3D Dir["E:/test/foobar/scripts/**/*.xml"]
matches =3D files.inject(0) {|sum,f| sum + =
File.read(f).scan('<if>').size}
printf "%10d matches\n%10d files\n", matches, files.size

If you have a lot of files it may pay off to use Find.find instead of
Dir[].
*/

thanks, works like a charm
Never used the inject method until now, i'm still on newbie level.

Regards, Gilbert

 

[Robert Klemme]

Hi, Robert

-----Original Message-----
From: Robert Klemme [mailto:[email protected]]
Sent: Friday, August 24, 2007 10:30 PM
To: ruby-talk ML
Subject: Re: Dir, recursive filescan

/*
files = Dir["E:/test/foobar/scripts/**/*.xml"]
matches = files.inject(0) {|sum,f| sum + File.read(f).scan('<if>').size}
printf "%10d matches\n%10d files\n", matches, files.size

If you have a lot of files it may pay off to use Find.find instead of
Dir[].
*/

thanks, works like a charm
Never used the inject method until now, i'm still on newbie level.

Took me a while, too. But if you get the hang of it you'll see it's a
really cool utility. At one point I rewrote all the Enumerable
methods via #inject just for the learning experience.

Kind regards

robert

 

[Rebhan, Gilbert]

=20


-----Original Message-----
From: Robert Klemme [mailto:[email protected]]=20
Sent: Monday, August 27, 2007 9:01 AM
To: ruby-talk ML
Subject: Re: Dir, recursive filescan

files =3D Dir["E:/test/foobar/scripts/**/*.xml"]
matches =3D files.inject(0) {|sum,f| sum +

printf "%10d matches\n%10d files\n", matches, files.size

If you have a lot of files it may pay off to use Find.find instead of
Dir[].
*/

thanks, works like a charm
Never used the inject method until now, i'm still on newbie level.

/*
Took me a while, too. But if you get the hang of it you'll see it's a
really cool utility. At one point I rewrote all the Enumerable
methods via #inject just for the learning experience.
*/

one thing i noticed when using Dir[...] on large dirtrees i.e. C:/ruby
i get a Permission denied =3D

filescan.rb:21:in `read': Permission denied - C:/ruby/doc/Expat-1.95.8
(Errno::EACCES)

I think that's the reason you mentioned to use Find.find yor large
dirtrees ?!

Regards, Gilbert

 

[Robert Klemme]

-----Original Message-----
From: Robert Klemme [mailto:[email protected]]
Sent: Monday, August 27, 2007 9:01 AM
To: ruby-talk ML
Subject: Re: Dir, recursive filescan

files = Dir["E:/test/foobar/scripts/**/*.xml"]
matches = files.inject(0) {|sum,f| sum +

printf "%10d matches\n%10d files\n", matches, files.size

If you have a lot of files it may pay off to use Find.find instead of
Dir[].
*/

thanks, works like a charm
Never used the inject method until now, i'm still on newbie level.

/*
Took me a while, too. But if you get the hang of it you'll see it's a
really cool utility. At one point I rewrote all the Enumerable
methods via #inject just for the learning experience.
*/

one thing i noticed when using Dir[...] on large dirtrees i.e. C:/ruby
i get a Permission denied =

filescan.rb:21:in `read': Permission denied - C:/ruby/doc/Expat-1.95.8
(Errno::EACCES)

I think that's the reason you mentioned to use Find.find yor large
dirtrees ?!

Not exactly. My reasoning was that the Dir[] returns a large array
while Find invokes the block once per file found.

But your problem is related to error handling. Dunno what Find does
here but the root cause of your issue is that you seem to not have
permissions for one of the directories.

Kind regards

robert

 

[Rebhan, Gilbert]

=20
-----Original Message-----
From: Robert Klemme [mailto:[email protected]]=20
Sent: Monday, August 27, 2007 10:14 AM
To: ruby-talk ML
Subject: Re: Dir, recursive filescan

one thing i noticed when using Dir[...] on large dirtrees i.e. C:/ruby
i get a Permission denied =3D

filescan.rb:21:in `read': Permission denied - C:/ruby/doc/Expat-1.95.8
(Errno::EACCES)

I think that's the reason you mentioned to use Find.find yor large
dirtrees ?!

/*
Not exactly. My reasoning was that the Dir[] returns a large array
while Find invokes the block once per file found.

But your problem is related to error handling. Dunno what Find does
here but the root cause of your issue is that you seem to not have
permissions for one of the directories.
*/

Hm, i regularly get those errors here, when doing dir related stuff.
I have all permissions as this is my local C: partition and i'm admin.
Maybe a windows only problem ?!

C:\WINNT\system32>ruby -v
ruby 1.8.4 (2006-04-14) [i386-mswin32]

Regards, Gilbert