Reading online zip files - zipfile and zlib, wbits

[Brendan]

I am fooling around with accessing contents of zip files online. I
download the tail end of the zip and use zipfile to get the zip
central directory structure. I download the section of the zip file I
need, directly read the zip file headers and use that information with
zlib to uncompress the data. The files I am examining will always be
compressed using deflate, with a wbits value of -15(minus for
headerless data because I am unsure whether the zip file header is
what zlib expects).

I can not find anywhere in the PK Zip Application notes (
http://www.pkware.com/documents/casestudies/APPNOTE.TXT ) how to
determine the value I should uze for wbits with zlib.decompress. I
have determined it is -15 from experimentation. Does anyone know the
answer to this?

 

[Brendan]

I am fooling around with accessing contents of zip files online. I
download the tail end of the zip and use zipfile to get the zip
central directory structure. I download the section of the zip file I
need, directly read the zip file headers and use that information with
zlib to uncompress the data. The files I am examining will always be
compressed using deflate, with a wbits value of -15(minus for
headerless data because I am unsure whether the zip file header is
what zlib expects).

I can not find anywhere in the PK Zip Application notes (http://www.pkware.com/documents/casestudies/APPNOTE.TXT) how to
determine the value I should uze for wbits with zlib.decompress. I
have determined it is -15 from experimentation. Does anyone know the
answer to this?

Okay, I found part of the answer here in the zip app notes

general purpose bit flag: (2 bytes)

Bit 0: If set, indicates that the file is encrypted.

(For Method 6 - Imploding)
Bit 1: If the compression method used was type 6,
Imploding, then this bit, if set, indicates
an 8K sliding dictionary was used. If clear,
then a 4K sliding dictionary was used.
Bit 2: If the compression method used was type 6,
Imploding, then this bit, if set, indicates
3 Shannon-Fano trees were used to encode the
sliding dictionary output. If clear, then 2
Shannon-Fano trees were used.

(For Methods 8 and 9 - Deflating)
Bit 2 Bit 1
0 0 Normal (-en) compression option was used.
0 1 Maximum (-exx/-ex) compression option was
used.
1 0 Fast (-ef) compression option was used.
1 1 Super Fast (-es) compression option was used.

Now I just don't understand Why Normal deflate corresponds to 15
wbits, and why I have to use headerless for the data, i.e. wbits = -15.

 

[Brendan]

I am fooling around with accessing contents of zip files online. I
download the tail end of the zip and use zipfile to get the zip
central directory structure. I download the section of the zip file I
need, directly read the zip file headers and use that information with
zlib to uncompress the data. The files I am examining will always be
compressed using deflate, with a wbits value of -15(minus for
headerless data because I am unsure whether the zip file header is
what zlib expects).

I can not find anywhere in the PK Zip Application notes (http://www.pkware.com/documents/casestudies/APPNOTE.TXT) how to
determine the value I should uze for wbits with zlib.decompress. I
have determined it is -15 from experimentation. Does anyone know the
answer to this?

Okay, I found part of the answer here in the zip app notes

general purpose bit flag: (2 bytes)

          Bit 0: If set, indicates that the file is encrypted.

          (For Method 6 - Imploding)
          Bit 1: If the compression method used was type 6,
                 Imploding, then this bit, if set, indicates
                 an 8K sliding dictionary was used.  If clear,
                 then a 4K sliding dictionary was used.
          Bit 2: If the compression method used was type 6,
                 Imploding, then this bit, if set, indicates
                 3 Shannon-Fano trees were used to encode the
                 sliding dictionary output.  If clear, then 2
                 Shannon-Fano trees were used.

          (For Methods 8 and 9 - Deflating)
          Bit 2  Bit 1
            0      0    Normal (-en) compression option was used.
            0      1    Maximum (-exx/-ex) compression option was
used.
            1      0    Fast (-ef) compression option was used.
            1      1    Super Fast (-es) compression option was used.

Now I just don't understand Why Normal deflate corresponds to 15
wbits, and why I have to use headerless for the data, i.e. wbits = -15.

Seems the bit flags are not properly set, bit 2 should be 0 and bit 1
should be 1, to correspond to maximum compression i.e. wbit = 15.
Still don't know why wbits has to be negative.

 

[Brendan]

Okay, I found part of the answer here in the zip app notes

general purpose bit flag: (2 bytes)

          Bit 0: If set, indicates that the file is encrypted..

          (For Method 6 - Imploding)
          Bit 1: If the compression method used was type 6,
                 Imploding, then this bit, if set, indicates
                 an 8K sliding dictionary was used.  If clear,
                 then a 4K sliding dictionary was used.
          Bit 2: If the compression method used was type 6,
                 Imploding, then this bit, if set, indicates
                 3 Shannon-Fano trees were used to encode the
                 sliding dictionary output.  If clear, then 2
                 Shannon-Fano trees were used.

          (For Methods 8 and 9 - Deflating)
          Bit 2  Bit 1
            0      0    Normal (-en) compression option was used.
            0      1    Maximum (-exx/-ex) compression option was
used.
            1      0    Fast (-ef) compression option was used.
            1      1    Super Fast (-es) compression option was used.

Now I just don't understand Why Normal deflate corresponds to 15
wbits, and why I have to use headerless for the data, i.e. wbits = -15.

Seems the bit flags are not properly set, bit 2 should be 0 and bit 1
should be 1, to correspond to maximum compression i.e. wbit = 15.
Still don't know why wbits has to be negative.- Hide quoted text -

- Show quoted text -

Arg! Tried a different tack. Took the file header plus compressed
file and concatenated with central directory. Now need only zipfile
module.