Zlib::BufError when extracting file from a large .zip

[rpardee]

Hey All,

I'm having trouble unzipping a large .zip file w/rubyzip on winxp. The
file is actually in the public domain--it's this 59MB chunk of census
data:

ftp://ftp2.census.gov/census_2000/datasets/Summary_File_1/California/ca00002_uf1.zip

If I point the following script at it:

require "rubygems"
require_gem "rubyzip"

folder = "//mpe04n/vdw$/census/"
fname = folder + "ca00008_uf3.zip"

z = Zip::ZipFile.new(fname)

Zip::ZipFile.foreach(fname) do |zipentry|
# puts(zipentry.methods)
z.extract(zipentry, folder + zipentry.to_s)
end

I get:

c:/ruby/lib/ruby/gems/1.8/gems/rubyzip-0.5.12/lib/zip/zip.rb:201:in
`inflate': buffer error (Zlib::BufError)
from
c:/ruby/lib/ruby/gems/1.8/gems/rubyzip-0.5.12/lib/zip/zip.rb:201:in
`internal_produce_input'
from
c:/ruby/lib/ruby/gems/1.8/gems/rubyzip-0.5.12/lib/zip/zip.rb:178:in
`sysread'
from
c:/ruby/lib/ruby/gems/1.8/gems/rubyzip-0.5.12/lib/zip/zip.rb:124:in
`sysread'
from
c:/ruby/lib/ruby/gems/1.8/gems/rubyzip-0.5.12/lib/zip/ioextras.rb:54:in
`read'
from
c:/ruby/lib/ruby/gems/1.8/gems/rubyzip-0.5.12/lib/zip/zip.rb:1242:in
`write_file'
from
c:/ruby/lib/ruby/gems/1.8/gems/rubyzip-0.5.12/lib/zip/zip.rb:1242:in
`get_input_stream'
from
c:/ruby/lib/ruby/gems/1.8/gems/rubyzip-0.5.12/lib/zip/zip.rb:1242:in
`write_file'
from
c:/ruby/lib/ruby/gems/1.8/gems/rubyzip-0.5.12/lib/zip/zip.rb:1240:in
`open'
... 6 levels...
from
c:/ruby/lib/ruby/gems/1.8/gems/rubyzip-0.5.12/lib/zip/zip.rb:1062:in
`foreach'
from
c:/ruby/lib/ruby/gems/1.8/gems/rubyzip-0.5.12/lib/zip/zip.rb:1060:in
`open'
from
c:/ruby/lib/ruby/gems/1.8/gems/rubyzip-0.5.12/lib/zip/zip.rb:1060:in
`foreach'
from C:/Documents and Settings/pardre1/Desktop/DeleteMe.rb:11

PKZip can unzip this file w/out errors or warnings.

Can anybody advise me on how to successfully unzip this file w/ruby?

Thanks!

-Roy

 

[Thomas Sondergaard]

I think this is the same problem rubygems have been having with zlib. If
you look at package.rb from rubygems you can see that they apply some
workaround if the ruby-version is older than 1.2.1. I've also seen
postings that 1.2.1 itself may have problems. Could you check your version:

require 'zlib'
Zlib::ZLIB_VERSION

I don't have the time and energy to put into researching it, but I'm not
sure ruby's zlib wrapper does the right thing for Z_BUF_ERROR (see
http://www.zlib.net/zlib_faq.html#faq05)

Thomas

 

[Thomas Sondergaard]

workaround if the ruby-version is older than 1.2.1. I've also seen

postings that 1.2.1 itself may have problems. Could you check your version:

not ruby version - zlib version!

 

[rpardee]

Thanks Thomas!

I have 1.2.1 installed:

irb(main):002:0> Zlib::ZLIB_VERSION
=> "1.2.1"

Do you know if there's another version that should work better?

Thanks!

-Roy

 

[rpardee]

Ach, now I'm finding that even files that unzipped w/out complaint, and
which have the expected file sizes, are filled with ascii nulls.

I may wind up just trying to drive a command-line unzip utility...
What a drag...