GNU bug report logs - #23107
new snapshot available: gzip-1.6.56-9190

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 23107 in the body.
You can then email your comments to 23107 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: bug-gzip <at> gnu.org
Cc: platform-testers <at> gnu.org
Subject: new snapshot available: gzip-1.6.56-9190
Date: Thu, 24 Mar 2016 09:37:17 -0700

With this, I think we're finally ready to release.
Paul fixed an ancient zdiff bug and a Solaris portability problem.
I fixed gzip so that we honor the new --rsyncable option
in the "GZIP" environment variable.

gzip snapshot:
  http://meyering.net/gzip/gzip-ss.tar.xz      748 KB
  http://meyering.net/gzip/gzip-ss.tar.xz.sig
  http://meyering.net/gzip/gzip-1.6.56-9190.tar.xz

Changes in gzip since 1.6.52-30cc:

Jim Meyering (1):
      gzip: also honor GZIP=--rsyncable

Paul Eggert (3):
      zgrep: with -f SPECIAL, read SPECIAL just once
      Port to Solaris 11 /bin/sh
      Fix typo in previous patch

Message #10 received at 23107-done <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: 23107-done <at> debbugs.gnu.org
Subject: Re: bug#23107: new snapshot available: gzip-1.6.56-9190
Date: Thu, 24 Mar 2016 09:47:46 -0700

tags 23107 notabug
thanks

Message #13 received at 23107 <at> debbugs.gnu.org (full text, mbox):

From: Assaf Gordon <assafgordon <at> gmail.com>
To: Jim Meyering <jim <at> meyering.net>
Cc: 23107 <at> debbugs.gnu.org
Subject: Re: bug#23107: new snapshot available: gzip-1.6.56-9190
Date: Thu, 24 Mar 2016 22:56:41 -0400

[Message part 1 (text/plain, inline)]

Hello,

> On Mar 24, 2016, at 12:37, Jim Meyering <jim <at> meyering.net> wrote:
>  http://meyering.net/gzip/gzip-1.6.56-9190.tar.xz


No test failures on
  CentOS 7 + 6.5,   amd64
  Ubuntu 14.04,   amd64
  Ubuntu 15.04   amd64 + i686
  openSUSE 42.1, 13.1,   amd64
  Fedora 23, 22, 21,   amd64
  Debian 8.1, 7.6,   amd64
  Trisquel 7.0, 6.0.1,   amd64

  Fedora-21, ppc64le
  Fedora-20, ppc64
  Ubuntu-14, aarch64

  OpenBSD 5.8, 5.7, 5.6, amd64
  FreeBSD 10.1, 9.3,   amd64
  GNU Hurd 0.7,   i686
  Debian-7/kFreeBSD-9,  amd64

  AIX-7.1,power7
  OpenSolaris 5.10/5.11 on i86pc and suv4u/sub4v
  Mac OS X 10.11.3, 10.10.5, 10.9.5

On NetBSD 7.0, 6.1.4, One test fails (zgrep-f) perhaps due to filename issues?
log attached.

[NetBSD7-gzip 1.6.56-9190-test-suite-2.log (application/octet-stream, attachment)]

[Message part 3 (text/plain, inline)]

Minix 3.3.0 fails as before (with decision not to fix).



One more failure on a not-so-common system:
I'm adding Alpine Linux to my test suite ( https://alpinelinux.org/ ).
Alpine is a lightweight Linux-based system, with musl libc and busybox utilities - so not very GNU-like, and possibly not POSIX compliant.
On this system, zdiff check fails because busybox's diff does not support the "-c" option
( https://busybox.net/downloads/BusyBox.html#diff ).
I suspect such systems will become more common in the future (due to usage in small VMs / containers).
 
Example:

    $ uname -a
    Linux alp333 4.1.20-0-grsec #1-Alpine SMP Mon Mar 21 15:49:51 GMT 2016 x86_64 Linux

    $ make check
    ...
    make[3]: Entering directory '/tmp/gzip-1.6.56-9190.CaiIeG/gzip-1.6.56-9190'
     GEN      check-local
    diff: unrecognized option: c
    [...]
    Makefile:2196: recipe for target 'check-local' failed
    make[3]: *** [check-local] Error 1
    make[3]: Leaving directory '/tmp/gzip-1.6.56-9190.CaiIeG/gzip-1.6.56-9190'
    Makefile:1985: recipe for target 'check-am' failed
    make[2]: *** [check-am] Error 2


    $ grep -A 5 check-local Makefile.am
    check-local: $(FILES_TO_CHECK) $(bin_PROGRAMS) gzip.doc.gz
        $(AM_V_GEN)$(new_path); { test '$(srcdir)' != . \
                                    || zdiff -c gzip.doc.gz; }
        $(AM_V_at)$(new_path); zdiff -c $(srcdir)/gzip.doc $(srcdir)/gzip.doc
        $(AM_V_at)$(new_path); zdiff $(srcdir)/gzip.doc gzip.doc.gz
        $(AM_V_at)$(new_path); zdiff -c - $(srcdir)/gzip.doc <gzip.doc.gz



    $ diff -c
    diff: unrecognized option: c
    BusyBox v1.24.1 (2015-12-16 08:00:02 GMT) multi-call binary.

    Usage: diff [-abBdiNqrTstw] [-L LABEL] [-S FILE] [-U LINES] FILE1 FILE2

    Compare files line by line and output the differences between them.
    This implementation supports unified diffs only.

        -a      Treat all files as text
        -b      Ignore changes in the amount of whitespace
        -B      Ignore changes whose lines are all blank
        -d      Try hard to find a smaller set of changes
        -i      Ignore case differences
        -L      Use LABEL instead of the filename in the unified header
        -N      Treat absent files as empty
        -q      Output only whether files differ
        -r      Recurse
        --no-dereference Don't follow symlinks
        -S      Start with FILE when comparing directories
        -T      Make tabs line up by prefixing a tab when necessary
        -s      Report when two files are the same
        -t      Expand tabs to spaces in output
        -U      Output LINES lines of context
        -w      Ignore all whitespace



regards,
 - assaf

Message #16 received at 23107 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Assaf Gordon <assafgordon <at> gmail.com>, Jim Meyering <jim <at> meyering.net>
Cc: 23107 <at> debbugs.gnu.org
Subject: Re: bug#23107: new snapshot available: gzip-1.6.56-9190
Date: Fri, 25 Mar 2016 00:27:12 -0700

[Message part 1 (text/plain, inline)]

Thanks again for checking all that.

Assaf Gordon wrote:

> On NetBSD 7.0, 6.1.4, One test fails (zgrep-f) perhaps due to filename issues?


No, NetBSD's mktemp command is incompatible with GNU's, in terms of the 
interpretation of -t. The simplest fix is to stop using mktemp's -t option 
(which has been "deprecated" in the GNU version for many years anyway).

> I'm adding Alpine Linux to my test suite ( https://alpinelinux.org/ ).
> ...
> On this system, zdiff check fails because busybox's diff does not support the "-c" option

Wow, *that* is stripped down. Well, it shouldn't be that hard to port to. Does 
the attached patch fix the problem (as well as the NetBSD problem)?

[gzip.patch (text/x-diff, attachment)]

Message #19 received at 23107 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Jim Meyering <jim <at> meyering.net>
Cc: Assaf Gordon <assafgordon <at> gmail.com>,
 "bug-gnulib <at> gnu.org List" <bug-gnulib <at> gnu.org>, 23107 <at> debbugs.gnu.org
Subject: Re: [PATCH 1/2] test-framework-sh: port to NetBSD 7.0
Date: Fri, 25 Mar 2016 13:01:06 -0700

[Message part 1 (text/plain, inline)]

On 03/25/2016 08:16 AM, Jim Meyering wrote:
> I know that mktemp's -t is not portable, but the code in
> init.sh was supposed to work around that. Did you see
> some place where that work-around failed?

Ah, sorry, I misinterpreted the debugging output. I saw a 'fail=1' and 
thought, "Oh, the script is failing". But it was working correctly 
(albeit confusingly). I reverted that part of the change to 
tests/init.sh in gnulib and installed further changes that I hope make 
the test output clearer.  Revised proposed patches to gzip attached.

[0001-Port-to-Alpine-Linux-which-uses-Busybox.patch (application/x-patch, attachment)]

[0002-tests-init.sh-Sync-from-Gnulib.patch (application/x-patch, attachment)]

[0003-Port-to-NetBSD-7.0.patch (application/x-patch, attachment)]

Message #22 received at 23107 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: Assaf Gordon <assafgordon <at> gmail.com>,
 "bug-gnulib <at> gnu.org List" <bug-gnulib <at> gnu.org>, 23107 <at> debbugs.gnu.org
Subject: Re: [PATCH 1/2] test-framework-sh: port to NetBSD 7.0
Date: Fri, 25 Mar 2016 19:07:13 -0700

On Fri, Mar 25, 2016 at 1:01 PM, Paul Eggert <eggert <at> cs.ucla.edu> wrote:
> On 03/25/2016 08:16 AM, Jim Meyering wrote:
>>
>> I know that mktemp's -t is not portable, but the code in
>> init.sh was supposed to work around that. Did you see
>> some place where that work-around failed?
>
> Ah, sorry, I misinterpreted the debugging output. I saw a 'fail=1' and
> thought, "Oh, the script is failing". But it was working correctly (albeit
> confusingly). I reverted that part of the change to tests/init.sh in gnulib
> and installed further changes that I hope make the test output clearer.
> Revised proposed patches to gzip attached.

Thank you for all of those patches. All look fine, so please push.
That fail=1 in init.sh has misled me, too, so I'm glad it's gone, now.

Message #25 received at 23107 <at> debbugs.gnu.org (full text, mbox):

From: Assaf Gordon <assafgordon <at> gmail.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: Jim Meyering <jim <at> meyering.net>, 23107 <at> debbugs.gnu.org
Subject: Re: bug#23107: new snapshot available: gzip-1.6.56-9190
Date: Fri, 25 Mar 2016 23:17:29 -0400

[Message part 1 (text/plain, inline)]

Hello,

On that minimalist Alpine 3.3.3 system, the attached patch fixed the "check-local" step, but then two other tests fail (help-version and zless).
The 'help-version' might be related to 'mktemp' (not sure),
while the 'zless' fails because it assumes 'less' supports '-V', which busybox's 'less' does not.
log attached.

[gzip-alpine-3.3.3-test-suite.log (application/octet-stream, attachment)]

[Message part 3 (text/plain, inline)]

It might be a bit tricker than expected to adapt to such minimalist system - whether it's a show-stopper or not is a different matter.
I'm happy to test further (and I'll soon send similar reports for other packages). 
	

Regarding netbsd/mktemp - sorry for creating some confusion. I noticed gnulib was updated then reverted with some related changes, please let me know if there's a specific version to check.

Examining 'mktemp' support on various systems, I see the following:
===
FreeBSD 10.1:           mktemp [-d] [-q] [-t prefix] [-u] template ...
       (and)            mktemp [-d] [-q] [-u] -t prefix 

Mac OS X:               (same as FreeBSD)

OpenBSD 5.8:            mktemp [-dqtu] [-p directory] [template]

NetBSD 7:		mktemp [-dqu] [-p <tmpdir>] {-t prefix | template ...}

Alpine (BusyBox):	mktemp [-dt] [-p DIR] [TEMPLATE]

OpenSolaris 5.10        mktemp [-dqtu] [-p prefix_dir] [template]

OpenSolaris 5.11        mktemp [-dqtu] [-p prefix_dir] [--directory][--dry-run] [--help] [--quiet] [--suffix=SUFF] [--tmpdir[=DIR]] [tmeplate]

AIX 7:                  (missing?)
===

Specifically, it seems the '-t' parameter for NetBSD requires a parameter, while on most other systems it is just a flag. This makes the order of the parameters effect the outcome.

In './tests/init.sh', the function 'mktempd_' has code like this:
  # First, try to use mktemp.
  d=`unset TMPDIR; { mktemp -d -t -p "$destdir_" "$template_"; } 2>/dev/null` \
    || fail=1

Trying it on NetBSD yields unexpected results:

    $ mktemp -d -t -p destdir template.XXXXXX  
    /tmp/-p.xHVlMf3f
    destdir
    template.xUCbsO

While if we just change the order, and make '-t' the last parameter, it would "just work" (assuming TMPDIR is unset).		

    $ mktemp -d -p destdir -t template.XXXXXX
    destdir/template.XXXXXX.SEvvNSKE


An attempt to make a portable version:

  d=`unset TMPDIR; { mktemp -d -p "$destdir_" -t "$template_"; } 2>/dev/null` \
    || e=`TMPDIR="$destdir_" mktemp -d -t "$template_" 2>/dev/null` \
    || fail=1

Running the attached script on various systems gives:

===
Linux:       with-p    foo/bar.nNBIHo   (=alpine/busybox)
Linux:       with-p    foo/bar.Da60cG   (=debian/gnu-coreutils)
FreeBSD:     no-p      foo/bar.XXXXXX.60xAkfa8
NetBSD:      with-p    foo/bar.XXXXXX.myeDmTST
Darwin:      no-p      foo/bar.XXXXXX.xKNmTTzN
OpenBSD:     with-p    foo/bar.zVf8w1
SunOS:       with-p    foo/bar.rtai1S    (=OpenSolaris 5.10)
SunOS:       with-p    foo/bar.rXa4bY    (=OpenSolaris 5.11)
===

[make-temp-dir.sh (application/octet-stream, attachment)]

[Message part 5 (text/plain, inline)]

Hope this helps, and happy to test further,
regards,
 - assaf

Message #28 received at 23107 <at> debbugs.gnu.org (full text, mbox):

From: Assaf Gordon <assafgordon <at> gmail.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: Jim Meyering <jim <at> meyering.net>, 23107 <at> debbugs.gnu.org
Subject: Re: bug#23107: new snapshot available: gzip-1.6.56-9190
Date: Fri, 25 Mar 2016 23:31:08 -0400

Sorry, I was wrong about the failure causes on Alpine Linux 3.3.3:

> On Mar 25, 2016, at 23:17, Assaf Gordon <assafgordon <at> gmail.com> wrote:
> 
> <gzip-alpine-3.3.3-test-suite.log>
> 

'help-version' failed due to using 'less -V', and 'zdiff' failed due to something else (which I'm looking into). 

FAIL: help-version
==================
[...]
+ env zless zin.gz
less: unrecognized option: V


FAIL: zdiff
===========
[...]
+ diff -u exp out
--- exp
+++ out
@@ -1,4 +1,5 @@
-1c1
-< a
----
-> b
+--- /dev/fd/5
++++ -
+@@ -1 +1 @@
+-a
++b
+ fail=1



regards,
 - assaf

Message #31 received at 23107 <at> debbugs.gnu.org (full text, mbox):

From: Assaf Gordon <assafgordon <at> gmail.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: Jim Meyering <jim <at> meyering.net>, 23107 <at> debbugs.gnu.org
Subject: Re: bug#23107: new snapshot available: gzip-1.6.56-9190
Date: Sat, 26 Mar 2016 00:17:09 -0400

[Message part 1 (text/plain, inline)]

> FAIL: zdiff
> ===========

On this system (Alpine linux + busybox), 'diff' always produces unified output (regardless of using '-u').
It does not support 'normal' output ( https://busybox.net/downloads/BusyBox.html#diff ).

The attached patch is a suggestion to always test with unified diff.
It solves the problem on this system, but might introduce problems on other systems which don't support '-u' (or don't have sed).

[0001-tests-zdiff-use-unified-diff.patch (application/octet-stream, attachment)]

[Message part 3 (text/plain, inline)]

comments welcomed.
- assaf

Message #34 received at 23107 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Assaf Gordon <assafgordon <at> gmail.com>
Cc: Jim Meyering <jim <at> meyering.net>, 23107 <at> debbugs.gnu.org
Subject: Re: bug#23107: new snapshot available: gzip-1.6.56-9190
Date: Sat, 26 Mar 2016 01:15:31 -0700

Assaf Gordon wrote:
> It solves the problem on this system, but might introduce problems on other systems which don't support '-u' (or don't have sed).

Yes, this is a problem, as older Unixes don't have 'diff -u'.

Perhaps we should wait until a later gzip version before worrying too much about 
Busybox. It is close to a release, and clearly these odds and ends of gzip have 
never worked with Busybox.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.