S
Szymon Rozga
I have encountered a problem with zlib at work. I have to parse through
huge gzipped text files (500000+ lines). However, it seems that zlib
only reads the first 66000 or so lines.
Initially I used the code:
Zlib::GZipReader.open(file) do |f|
# process text
puts f.readlines.size
end
However:
I
pen("zcat #{file}",'r') do |io|
puts io.readlines.size
end
I went through about 2 hours of confusion and struggling not believing
that I couldn't find what I expected using the code, but grepping
returned results. Then I somehow stumbled upon this unexpected cause of
my problems.
Is this a known issue?
-Szymon
huge gzipped text files (500000+ lines). However, it seems that zlib
only reads the first 66000 or so lines.
Initially I used the code:
Zlib::GZipReader.open(file) do |f|
# process text
puts f.readlines.size
end
However:
I
pen("zcat #{file}",'r') do |io|puts io.readlines.size
end
I went through about 2 hours of confusion and struggling not believing
that I couldn't find what I expected using the code, but grepping
returned results. Then I somehow stumbled upon this unexpected cause of
my problems.
Is this a known issue?
-Szymon