trouble reading a set of files recursively

[katari]

Hello,
I am doing a batch run, where I have several .csv files with the same
name in different sub-directories. At the end of the batch run, I want
to cat all the individual files together, and load them into the
database.

Here is the code:
Dir["**/store_need*"].each do |path|
single_file_nm = path.to_s
single_file = File.open("#{single_file_nm}", "r")
single_file.each do |line|
store_need_file << line
puts line
end
single_file.close
end

I am recursively looking for the store_need.csv files, and then
reading them into a single store_need_file. The problem is that this
code is not reading the file at all. I can see that the path is
defined correctly.

Any help is appreciated. I am running this on windows XP.

Kaushik

 

[Don Wood]

Here is the code:
Dir["**/store_need*"].each do |path|
single_file_nm = path.to_s
single_file = File.open("#{single_file_nm}", "r")
single_file.each do |line|
store_need_file << line
puts line
end
single_file.close
end

I just tried your code on a Linux box, and it ran as it should. I don't
do Windows, but at least this tells you to focus on OS specific issues.
Could it have something to do with drive letters or slashes instead of
backslashes?

 

[Robert Klemme]

I am doing a batch run, where I have several .csv files with the same
name in different sub-directories. At the end of the batch run, I want
to cat all the individual files together, and load them into the
database.

Here is the code:
Dir["**/store_need*"].each do |path|
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0single_file_nm =3D path.to=

_s

Why do you convert a string to a string and then do string
interpolation in the next line?

=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0single_file =3D File.open(=

"#{single_file_nm}", "r")

The string interpolation is superfluous plus you should rather use the
block form of File.open.

=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0single_fil= e.each do |line|
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =

=A0 =A0store_need_file << line

Where do you open store_need_file?

=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0puts line
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0end
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0single_file.close
end

I am recursively looking for the store_need.csv files, and then
reading them into a single store_need_file. The problem is that this
code is not reading the file at all. I can see that the path is
defined correctly.

This is a shorter variant if your individual CSV files are small:

File.open "load.csv", "w" do |io|
Dir["**/store_need*"].each do |path|
# note: puts instead of write to get the terminating newline
io.puts(File.read(path))
end
end

If your individual files are large I would probably not bother to
combine them. If you still want to do it then I would do something
like this

# untested
NL =3D "\r\n".freeze
BL =3D 1024

File.open "load.csv", "wb" do |fout|
buffer =3D ""

Dir["**/store_need*"].each do |path|
File.open path, "rb" do |fin|
while fin.read(BL, buffer)
fout.write(buffer)
end
end

# only needed if files are not terminated properly:
# fout.write(NL)
end
end

Cheers

robert

--=20
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/