extra bytes in zip archive when used with logrotate

[Martha Greenberg]

Run zip with the -q option.  Otherwise, the output of zip ends up in the created zip file.  You can see this by running strings on the zip file, which (without -q) shows:

 (deflated 3%)

Adding -q to zip fixes this.


Martha Greenberg

On Aug 6, 2012, at 17:35 , Oleg Goldshmidt wrote:

> Hi everyone,
> 
> I am looking at another weird problem on CentOS 6.2. Here is the
> outline - hope someone will be able to help me figure it out.
> 
> Many (most?) of you are probably familiar with logrotate(8), and know
> that it is capable of a) compressing the rotated logs, and b) mailing
> them to a specified address. Logrotate is smart enough to know what it
> has compressed and takes care to uncompress the logs before mailing,
> so that the plain text logs become the body of the email.
> 
> I need to send zipped logs as attachments as they are rotated. Since
> logrotate allows specifying the mailer program I wrote a trivial
> script that accepts the same arguments as the default "mail", takes
> the stdin, zips it, and calls "mutt -a" to send the archive as an
> attachment. During testing I even commented out the mutt part, so that
> the (simplified, debug version of the) script looks like this:
> 
> ---------------------- file: zipit --------------------------------
> #!/bin/bash
> 
> archive=/tmp/file.log.zip
> input=/tmp/zipit.$$
> 
> # read the input - we want a filename in the archive
> /bin/cat - > $input
> 
> # create a new archive
> /bin/rm -f $archive
> /usr/bin/zip -r $archive -j $input
> 
> # clean up, keeping the archive for checking
> /bin/rm -f $input
> --------------------------------------------------------------------
> 
> As long as I do
> 
> $ cat file.log | ./zipit
> 
> or
> 
> $ gunzip -c file.log.gz | ./zipit
> 
> everything is fine. The zip archive is created, i can open it, no problem.
> 
> As soon as I try
> 
> $ logrotate -vm ./zipit logrotate.conf
> 
> everything is also fine, but subsequent unzip results in a warning
> 
> $ unzip -l file.log.zip
> Archive:  file.log.zip
> warning [file.log.zip]:  16 extra bytes at beginning or within zipfile
>  (attempting to process anyway)
>  Length      Date    Time    Name
> ---------  ---------- -----   ----
>    71992  08-06-2012 16:05   zipit.10182
> ---------                     -------
>    71992                     1 file
> 
> and the archive cannot be opened:
> 
> $ unzip -c file.log.zip
> Archive:  file.log.zip
> warning [file.log.zip]:  16 extra bytes at beginning or within zipfile
>  (attempting to process anyway)
> file #1:  bad zipfile offset (local header sig):  16
>  (attempting to re-compensate)
>  inflating: zipit.10182
> 
>  error:  invalid compressed data to inflate
> $
> 
> The file cannot be opened either on Linux (the same machine) or on
> Windows (after receiving in the mail) - basically, the archive is
> corrupted. The number of "extra bytes" is always 16, by the way.
> 
> I googled and only saw similar complaints about *very* large files or
> when non-binary FTP or similar was involved - neither is the case
> here. I went over the code of logrotate (very readable, creates a pipe
> and calls the configured uncompress command, gunzip by default, before
> passing to mail command), zip, and unzip (very UNreadable) in an
> attempt to find out what could cause the problem, all to no avail. I
> saw that someone succeeded fixing the archive using zip -FFv after
> such a warning. Didn't work for me, but it would be a non-starter,
> anyway.
> 
> Packages and versions:
> 
> zip-3.0-1.el6.x86_64
> unzip-6.0-1.el6.x86_64
> logrotate-3.7.8-12.el6_0.1.x86_64
> 
> Any ideas? For a solution or an explanation?
> 
> -- 
> Oleg Goldshmidt | pub at goldshmidt.org
> 
> _______________________________________________
> Linux-il mailing list
> Linux-il at cs.huji.ac.il
> http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il