zip without compression extracts to empty files #236

ericcj · 2015-06-15T21:15:57Z

i created a zip using https://github.com/pixelglow/zipzap with compress:NO on the entries and it unzips fine using the normal unzip command line utility (Info-ZIP) and Archive Utility on my os x laptop. however, when i try to extract it with rubyzip it creates zero-byte files for all the entries. here's what one of the entries and its input stream looks like. sysread on that stream always returns empty string.

#<Zip::Entry:0x007f9abcab60e0 @local_header_offset=99599183, @local_header_size=nil, @internal_file_attributes=0, @external_file_attributes=2175008768, @header_signature=33639248, @version_needed_to_extract=10, @version=30, @ftype=:file, @filepath=nil, @gp_flags=2056, @follow_symlinks=false, @restore_times=true, @restore_permissions=false, @restore_ownership=false, @unix_uid=nil, @unix_gid=nil, @unix_perms=420, @dirty=false, @fstype=3, @zipfile="/Users/ej/17ba3bfa-139f-11e5-9c00-2a828cf49bc3.zip", @name="C45664C1-ACC5-413E-9534-C7BF05DC5338.jpg", @comment="", @extra={}, @compressed_size=2239128, @crc=375996240, @compression_method=0, @size=2239128, @time=2015-06-15 16:42:50 -0400, @last_mod_time=34137, @last_mod_date=18127, @name_length=40, @extra_length=0, @comment_length=0>

#<Zip::InputStream:0x007f9ac2549a98 @lineno=0, @pos=0, @output_buffer="", @archive_io=#<File:/Users/ej/17ba3bfa-139f-11e5-9c00-2a828cf49bc3.zip>, @decompressor=#<Zip::PassThruDecompressor:0x007f9ac25495e8 @input_stream=#<File:/Users/ej/17ba3bfa-139f-11e5-9c00-2a828cf49bc3.zip>, @chars_to_read=0, @read_so_far=0, @has_returned_empty_string=false>, @decrypter=#<Zip::NullDecrypter:0x007f9ac25499d0>, @current_entry=#<Zip::Entry:0x007f9ac25499a8 @local_header_offset=99599183, @local_header_size=70, @internal_file_attributes=1, @external_file_attributes=0, @header_signature=67324752, @version_needed_to_extract=20, @version=10, @ftype=:file, @filepath=nil, @gp_flags=2056, @follow_symlinks=false, @restore_times=true, @restore_permissions=false, @restore_ownership=false, @unix_uid=nil, @unix_gid=nil, @unix_perms=nil, @dirty=false, @fstype=0, @zipfile=#<File:/Users/ej/17ba3bfa-139f-11e5-9c00-2a828cf49bc3.zip>, @name="C45664C1-ACC5-413E-9534-C7BF05DC5338.jpg", @comment="", @extra={}, @compressed_size=0, @crc=0, @compression_method=0, @size=0, @time=2015-06-15 16:42:50 -0400, @last_mod_time=34137, @last_mod_date=18127, @name_length=40, @extra_length=0>>

The text was updated successfully, but these errors were encountered:

simonoff · 2015-07-28T18:15:35Z

Need to look on zip file structure what this library making.

jlhonora · 2015-07-31T06:33:21Z

I'm facing the same issue. Basically the problem is that in lib/zip/input_stream.rb:147

when @current_entry.compression_method == ::Zip::Entry::STORED
   ::Zip::PassThruDecompressor.new(@archive_io, @current_entry.size)

@current_entry.size == 0. When unpacking the values from the buffer, the size is indeed zero, lib/zip/entry.rb:

def unpack_local_entry(buf)
  puts "unpack local entry buf: #{buf.bytes.to_a.map {|c| '%02x' % c}.join(' ')}"

prints 50 4b 03 04 0a 00 08 08 00 00 96 bb fe 46 00 00 00 00 00 00 00 00 00 00 00 00 0f 00 00 00, so effectively size = 0 (and compressed_size = 0).

I'm using this zip to replicate it. The test case is:

  def test_extract_zipzap
    filename = 'test/data/zipzap.zip'
    ::Zip::File.open(filename) {
      |zf|
      assert_operator zf.read('evaluation.json').size, :>, 0
    }
  end

Which of course fails.

I've been trying to follow rubyzip's code in order to fix it, but this is my first time dealing with zip formats and the like. Any help is appreciated.

pharmazone · 2016-05-13T15:52:15Z

Is there any updates or solutions?
thnx

herdst · 2017-12-07T06:00:07Z

I'm having the same issue. Does anyone have a solution? Thanks

rlgomes · 2018-03-23T20:48:38Z

Any update here ? I have zip files with NO_COMPRESSION and basically rubzip says there are no entries when uncompressing

ameyer-pivotal · 2018-04-03T20:16:48Z

@ericcj Take a look at PR #358. It looks like your general purpose bit 3 is set (2056 & 8 != 0), and this may be a fix for you too.

- Add "stack" to buildpack model - At migration time, buildpacks will be assigned default stack from stacks.yml file if its path is set in the STACKS_YML env var - hwc_buildpack is assigned the newer of windows2016,windows2012R2 stacks if present in stacks.yml - if STACKS_YML is not set, buildpacks' stacks will be set to nil - Buildpacks are now unique over name AND stack - Sets buildpack stack from manifest.yml in buildpack zip on creation - Validate buildpack model stack against stack in buildpack zip manifest.yml - Validate stack exists upon buildpack bits upload - Include stack name in serialized buildpack filename - Only provide buildpacks for the relevant stack to the staging container - Handle buildpack stacks appropriately in the buildpack installer - Use fork of rubyzip/rubyzip to work around rubyzip/rubyzip#236 NOTE: The API checkshum has changed due to adding stack as an input Signed-off-by: Dave Goddard <dave@goddard.id.au> Signed-off-by: Victoria Henry <vhenry@pivotal.io> Signed-off-by: Jackson Feeny <jacksonfeeny@gmail.com> Signed-off-by: Tyler Phelan <tphelan@pivotal.io> Signed-off-by: Andrew Meyer <ameyer@pivotal.io> Signed-off-by: Leah Hanson <lhanson@pivotal.io> Signed-off-by: Sam Coward <scoward@pivotal.io>

- Add "stack" to buildpack model - At migration time, buildpacks will be assigned default stack from stacks.yml file if its path is set in the STACKS_YML env var - hwc_buildpack is assigned the newer of windows2016,windows2012R2 stacks if present in stacks.yml - if STACKS_YML is not set, buildpacks' stacks will be set to nil - Buildpacks are now unique over name AND stack - Sets buildpack stack from manifest.yml in buildpack zip on creation - Validate buildpack model stack against stack in buildpack zip manifest.yml - Validate stack exists upon buildpack bits upload - Include stack name in serialized buildpack filename - Only provide buildpacks for the relevant stack to the staging container - Handle buildpack stacks appropriately in the buildpack installer - Use fork of rubyzip/rubyzip to work around rubyzip/rubyzip#236 NOTE: The API checkshum has changed due to adding stack as an input Signed-off-by: Dave Goddard <dave@goddard.id.au> Signed-off-by: Victoria Henry <vhenry@pivotal.io> Signed-off-by: Jackson Feeny <jacksonfeeny@gmail.com> Signed-off-by: Tyler Phelan <tphelan@pivotal.io> Signed-off-by: Andrew Meyer <ameyer@pivotal.io> Signed-off-by: Leah Hanson <lhanson@pivotal.io> Signed-off-by: Sam Coward <scoward@pivotal.io> Signed-off-by: Leah Hanson <lhanson@pivotal.io>

- Add "stack" to buildpack model - At migration time, buildpacks will be assigned default stack from stacks.yml file if its path is set in the STACKS_YML env var - hwc_buildpack is assigned the newer of windows2016,windows2012R2 stacks if present in stacks.yml - if STACKS_YML is not set, buildpacks' stacks will be set to nil - Buildpacks are now unique over name AND stack - Sets buildpack stack from manifest.yml in buildpack zip on creation - Validate buildpack model stack against stack in buildpack zip manifest.yml - Validate stack exists upon buildpack bits upload - Include stack name in serialized buildpack filename - Only provide buildpacks for the relevant stack to the staging container - Handle buildpack stacks appropriately in the buildpack installer - Use fork of rubyzip/rubyzip to work around rubyzip/rubyzip#236 NOTE: The API checkshum has changed due to adding stack as an input Signed-off-by: Dave Goddard <dave@goddard.id.au> Signed-off-by: Victoria Henry <vhenry@pivotal.io> Signed-off-by: Jackson Feeny <jacksonfeeny@gmail.com> Signed-off-by: Tyler Phelan <tphelan@pivotal.io> Signed-off-by: Andrew Meyer <ameyer@pivotal.io> Signed-off-by: Leah Hanson <lhanson@pivotal.io> Signed-off-by: Sam Coward <scoward@pivotal.io>

- Add "stack" to buildpack model - At migration time, buildpacks will be assigned default stack from stacks.yml file if its path is set in the STACKS_YML env var - hwc_buildpack is assigned the newer of windows2016,windows2012R2 stacks if present in stacks.yml - if STACKS_YML is not set, buildpacks' stacks will be set to nil - Buildpacks are now unique over name AND stack - Sets buildpack stack from manifest.yml in buildpack zip on creation - Validate buildpack model stack against stack in buildpack zip manifest.yml - Validate stack exists upon buildpack bits upload - Include stack name in serialized buildpack filename - Only provide buildpacks for the relevant stack to the staging container - Handle buildpack stacks appropriately in the buildpack installer - Use fork of rubyzip/rubyzip to work around rubyzip/rubyzip#236 NOTE: The API checkshum has changed due to adding stack as an input Signed-off-by: Dave Goddard <dave@goddard.id.au> Signed-off-by: Victoria Henry <vhenry@pivotal.io> Signed-off-by: Jackson Feeny <jacksonfeeny@gmail.com> Signed-off-by: Tyler Phelan <tphelan@pivotal.io> Signed-off-by: Andrew Meyer <ameyer@pivotal.io> Signed-off-by: Leah Hanson <lhanson@pivotal.io>

[#153256959] [#154527662] - Add "stack" to buildpack model, which will be nil by default - Buildpacks are now unique over name AND stack - Sets buildpack stack from manifest.yml in buildpack zip on creation - Validate buildpack model stack against stack in buildpack zip manifest.yml - Validate stack exists upon buildpack bits upload - Include stack name in serialized buildpack filename - Only provide buildpacks for the relevant stack (or those with a nil stack) to the staging container - Handle buildpack stacks appropriately in the buildpack installer - Use fork of rubyzip/rubyzip to work around rubyzip/rubyzip#236 NOTE: The API checkshum has changed due to adding stack as an input Signed-off-by: Dave Goddard <dave@goddard.id.au> Signed-off-by: Victoria Henry <vhenry@pivotal.io> Signed-off-by: Jackson Feeny <jacksonfeeny@gmail.com> Signed-off-by: Tyler Phelan <tphelan@pivotal.io> Signed-off-by: Andrew Meyer <ameyer@pivotal.io> Signed-off-by: Leah Hanson <lhanson@pivotal.io> Signed-off-by: Eric Promislow <eric.promislow@suse.com> Signed-off-by: Lisa Cho <lcho@pivotal.io>

jdleesmiller · 2019-03-22T15:06:48Z

(I'm helping Alex with maintenance.) I see a 👍 and no further reports since #358 landed in 1.2.2, so I will close the ticket. Please comment if it is still a problem.

Thanks for the fix @ameyer-pivotal and @idoru.

mattn mentioned this issue Aug 3, 2016

Block parameters are possible to be in next line. vim-ruby/vim-ruby#308

Merged

idoru mentioned this issue Apr 3, 2018

Handle stored files with general purpose bit 3 set #358

Merged

jdleesmiller closed this as completed Mar 22, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

zip without compression extracts to empty files #236

zip without compression extracts to empty files #236

ericcj commented Jun 15, 2015

simonoff commented Jul 28, 2015

jlhonora commented Jul 31, 2015

pharmazone commented May 13, 2016

herdst commented Dec 7, 2017

rlgomes commented Mar 23, 2018

ameyer-pivotal commented Apr 3, 2018

jdleesmiller commented Mar 22, 2019

zip without compression extracts to empty files #236

zip without compression extracts to empty files #236

Comments

ericcj commented Jun 15, 2015

simonoff commented Jul 28, 2015

jlhonora commented Jul 31, 2015

pharmazone commented May 13, 2016

herdst commented Dec 7, 2017

rlgomes commented Mar 23, 2018

ameyer-pivotal commented Apr 3, 2018

jdleesmiller commented Mar 22, 2019